Laws of Forecasting

Correct Forecasts are Not Always Good Ones

A correct forecast doesn't mean that the forecast method is valid.

To borrow from a general saying, "you're only as good as your last forecast." This implies that if your forecast is wrong, your methodology must be bad -- and if the forecast is correct, you must be doing a good job. Not necessarily. A forecast could be right once in a while just by luck. If you wanted a forecast of whether sales would increase next month, and you based it purely on a coin flip, you would be right about half of the time. Is it a good forecast method? Probably not.

What if you had a forecast method that was perfectly sound, but it ignored an important seasonal factor that only comes up once a year? Then you could go through 11 months of good forecasts, and then have a huge miss.

What if you have two factors in the business that usually offset each other, but sometimes amplify each other? If these are not included in the forecast methodology, the forecast will still be OK during those months when they offset each other.

The point is, it is dangerous to judge the quality of a forecasting method based only on the accuracy of the forecasting, especially over a short period of time. Look under the hood, and see if the methodology makes sense for your business. A consistently bad forecast indicates that something is probably wrong, but a string of good forecasts does not necessarily mean that the forecast method is correct.

By the way, a good forecast methodology will produce bad forecasts from time to time, so don't throw out the methodology the first time it misses. In both cases, judge the process, not the results.

Return to The Laws of Forecasting