Skip to main content

What’s your historical error rate?

Historical error rate & backtesting

B
Written by Brooke Schuyler

Prices

In our most recent evaluation across 70+ country–commodity–variety cash-price series, our models delivered a pooled MAPE (Mean Absolute Percentage Error) = 13% on monthly forecasts, meaning that on average, our monthly price predictions were about 13% off from the actual weekly prices over the course of 1 year.

Forecasts span 1–12 months ahead and are refreshed weekly; as you’d expect, near-term horizons are more accurate with lower MAPEs, while the range widens in longer horizons. We benchmark against simple naïve/seasonality and OLS baselines.

Climate

Our weather forecasts are also extremely performant. Across a 1-year test period, we found single-digit MAPE values across a majority of the key variables, like precipitation, max temp, min temp, and average temp. This is reflected in our platform's effectiveness in detecting climate-related supply disruption, where our error rate (missing a disruption) ranged from just 6 to 13% depending on the commodity.

Can I backtest your data?

Yes. Two options: self-serve and custom. The platform already supports self-backtesting of our bullish/bearish/neutral signals against the past year of climate and price context, and quantitative teams can pull longer histories via API/CSV for programmatic testing. The dataset is rebuilt nightly, which keeps histories stable for reproducible runs; you can also compare rule variants (e.g., threshold or percentile triggers) and benchmark against simple baselines. If you want us to set up a starter template—or to replicate your in-house rules on our side—email hello@helios.sc and we’ll get you going.

We can also perform backtests as a custom delivery—e.g., your entry/exit logic, lookback windows, rebalance cadence, target markets, and evaluation metrics. We’ll return results with the full methodology, parameter settings, and exports (CSV/PDF) you can audit internally.

Did this answer your question?