What forecast accuracy measures
Forecast accuracy is the forecast's track record: how far Tightly's forecast missed what actually sold, week by week, set beside the misses of doing it by hand. It is measured across the lines on view, not line by line; how firmly to hold one line is Confidence as a word. It reads the same on every plan.
When the forecast is scored
The accuracy pass runs every Monday at 06:30 UTC and scores the seven days that ended the day before. It compares the forecast for each managed variant with what that variant sold, on your own direct-to-consumer channels; the Evidence in the drawer names that population as your direct-to-consumer sales. A new tenant has no record until its first full week has been scored. Scores move only when a Monday pass lands: nothing you press on a page changes one.
WMAPE and how the error is worked out
The headline figure is WMAPE, the weighted absolute percentage error: the units the forecast missed by, over or under, added across every selling line, divided by the units sold. It reads as units of error per 100 units sold, and lower is better. Because it adds misses in both directions, a line forecast high and another forecast low do not cancel. Over / under is the other figure: the total forecast against total sales, as a percentage above or below, where opposite misses can cancel.
The by-hand methods Tightly is compared with
Every week the same lines are also scored by four methods a planner could run in a spreadsheet, on the same weeks and the same sales:
| Method | What it predicts |
|---|---|
| Last week carried forward | last week's sales again |
| 4-week moving average | the average of the 28 days before the week |
| 8-week moving average | the average of the 56 days before the week |
| Same week last year | the same week 52 weeks earlier |
The comparison in the head of Demand forecasting draws two bars, Tightly and the by-hand method, and says how much less or more error Tightly had, over WMAPE · Lower is better. It opens on the 8-week moving average; Compare with, in the drawer, picks another.
Backtested and Actual results
The comparison carries one of two words. Actual results is the forecast as it was served, scored week after week. Backtested appears after the engine itself has changed: today's forecasting engine is re-run over the same closed weeks and scored against the same by-hand methods, so the record describes the engine you are on rather than one that no longer runs. Either way the weeks and lines are the ones sold.
The Forecast performance drawer
Pressing the comparison opens the Forecast performance drawer. It carries:
- the record's word, Backtested or Actual results, and how many weeks were scored;
- WMAPE, the bars against the method picked in Compare with, and a sentence of units of error per 100 sold;
- Over / under, and a sentence on what it means;
- Evidence: the forecast, the comparison, the period and the population;
- All methods: the weekly line of the error ratio, then the method table, with Tightly first and every by-hand method after it. Its columns are Method, Error per 100 sold, Units missed, Over / under and Error ratio. An Error ratio below 1 means Tightly missed less than that method; Tightly's own row reads Not compared;
- Forecast status, the engine's own sentence, only where the forecast is behind.
The record follows the page's window and filters, so it scores the weeks and lines on view.
When there is no record
Where nothing can be compared, the head reads the reason instead of the bars: Not measured, Awaiting first score, No scored weeks, or Not read where the read did not answer. The drawer says why in one line:
No week has been scored yet.
No scored weeks in this date range. Choose a different date range.
No full week measured yet.
Where the read failed it says Performance unavailable, with Try again beside it.
Accuracy is not confidence
Accuracy is the whole forecast's error against what sold, measured once a week. Confidence grades each line from its own sales history on every sync, before the week is sold. A good accuracy record and a Watch closely line can sit on the same page. See The forecasting engine, Forecast methods and the Monday re-score and Demand forecasting.
Glossary line
Forecast accuracy is the forecast's weekly track record, WMAPE against what sold, beside the error of four by-hand methods.