# Evaluating Forecasting Algorithms

## Introduction

In order to determine when and how much of an item to order, it is necessary to have a forecast of customer demands.

The human brain is not as good as appropriate statistical analysis at interpreting statistical data. Forecasting of customer demands using demand history alone is, therefore, better done by a computer using an appropriate algorithm than by a person. If people can do a better job than the computer in that regard then the algorithm needs to be replaced with a more suitable one. Only when there is relevant market knowledge should people need to be involved in the forecasting.

It should not be necessary for purchasing officers to modify the computer’s reordering recommendations for Category C items unless they know something which the computer doesn’t. This requires the use of suitable forecasting and reordering algorithms. The purchasing officers should only need to look at Category B items to check to see if the items concerned are ones for which market knowledge should be taken into account. They should pay more attention to Category A and B items than to Category C items. However, reordering decisions need to be made reasonably quickly because the time thus spent is part of the lead time and therefore increases the required overall investment in inventory. Ensuring that the computer’s reordering recommendations for Category C items is appropriate is of considerable assistance in this regard. Care should be taken in relation to ordering Category A items but the purchasing officer should not need to look at demand history or sales history.

In an inventory management environment, the desirable characteristics of forecasting algorithms are considerably different from those of forecasting algorithms used for other purposes.

As far as possible, customer demands, rather than sales, should be used for forecasting purposes.

Much of this article was written under the assumption that demand rate estimates are updated monthly using monthly demand or sales aggregates. If weekly data is used then the word “week” should be substituted for “month”.

## Effects of Relatively High Demands

Items are more likely to come up for ordering after relatively high demand than at other times. There are two reasons for this. One is that the item is likely to have fallen below its reorder point as a result of the high demand. The other is that, for most forecasting algorithms, the forecast demand will increase at that time. This will increase the reorder point if it is based on the demand forecast. The substantial reduction in stock and increase in the reorder point are likely to result in reordering being triggered. The resulting ordering is then carried out on the basis of a demand forecast which is too high, thereby resulting in a inappropriately early ordering of a quantity which is too high. For these reasons, the quality of an algorithm used for forecasting demands should be judged, primarily, on the basis of the accuracy and bias of the forecasts after relatively high demands. The problem is illustrated in the following graph:

For the purpose of producing the above graph, the reorder review period was treated as being one month and the supplier lead time was treated as being two months. Ordering was treated as taking place at month end if the item was below its reorder point at that time.

Notice the relatively high demands in months 4, 8, 13 and 16 and the high inventory levels two months later (in months 6,10, 15 and 18).

In order to deal with this problem, it is best for forecasts to be frozen when a relatively high demand occurs. This eliminates the above-mentioned problem. That does not mean to say that the high demands should not be taken into account in forecasting demands. They can and should be taken into account when there is evidence that the relatively high demand was not a rare or one off event or simply the result of random fluctuations.

If the demand rate (mean monthly demand) is not changing then the demand changes from month to month are just random fluctuations. In the absence of evidence that the mean (average) demand per month is changing, the forecasts are just estimates of the mean demand per month and should get closer and closer to the mean.

## Rapid Demand Rate Changes

The forecasts should increase when there is substantial evidence that the demand rate has increased but they should not be sensitive to random fluctuations. If there is evidence that the demand rate has dropped then the forecast demands should diminish immediately to reduce the likelihood of dead inventory resulting from continued ordering. However, if subsequent demands indicate that the demand rate has not really dropped (i.e. that the low demands had been the result of random fluctuations), then the estimated demand rate should increase to about the level which it had been previously.

## Performance Measurement

For the reason given above and illustrated by the graph, measurement of the performances of demand forecasting algorithms should not be carried out using methods used in relation to forecasts used for other purposes. One approach is to compare the effects on both investment in inventory and on customer service. Unfortunately, the results of the performance measurement will then depend on the reordering algorithm used. A simpler approach which doesn’t have this problem is as follows: The forecasts whose accuracy and bias are measured should only be those which are made immediately after relatively high demands. Suppose that monthly demand or sales totals are used for forecasting. Then, for a forecast to have its performance measured, it needs to be made immediately after a month in which the demand or sales total was greater than in the preceding month and not less than in the following month. The forecasts need to be made at least two months ahead.

Comparing forecasting algorithms using the demand history of a single item is not normally a good idea. This is because the results of the comparison tend to be determined largely by random fluctuations in demand unless the item is fast moving and a long period of history is used. One forecasting algorithm, called “Focus Forecasting (TM)”, uses the result of such comparison using recent history to select a forecasting algorithm to be used subsequently. That technique has been found to perform poorly. The main reason for this is that the selection of the forecasting algorithm tends to be unduly influenced by random fluctuations in demand. I strongly advise against the use of Focus Forecasting for inventory management purposes.

That does not mean that selection of a forecasting algorithm on the basis of performance comparison with other algorithms is not a good idea. It does, however, require the use of sufficient data so that random fluctuations in demand do not influence the result of the comparison. In order to obtain sufficient data, it would usually be necessary to carry out the comparison for a group of items, not for individual items. Also, the comparison of the quality of the forecasts should be carried out using one of the two methods recommended in the first paragraph of this section.

## Importance of Exponential Smoothing

The forecasting algorithms used should be based on a technique called exponential smoothing. It is a very simple technique which involves storing an average which is updated each month. It uses the entire demand history or sales history of each item. Several forecasting algorithms incorporate this technique. One of the reasons for using it is that the forecast demands will be greater than zero if there have ever been any sales. Zero demand forecasts are of little use for inventory management purposes. They will cause some inventory management calculations to fail because it is not possible to divide by zero. Another reason for using such a forecasting algorithm is that it facilitates manual intervention in the forecasting process to take market knowledge into account.

## Sensitivity

The appropriate sensitivity of demand rate estimates should be less for long lead times than for short lead times. In the above graph, the sensitivity was inappropriately high for such a long lead time. Reducing the sensitivity would reduce the over-ordering problem but would not eliminate it.

## Systematic Demand Rate Changes

If the demand rate is progressive increasing or decreasing (i.e. there is a trend) then it is desirable for this to be used to adjust the demand rate estimate each month before the latest month’s demand is used to update the estimate. It is dangerous to extrapolate trends into the future because there is no guarantee that they will continue. For a new item, the demand is likely to increase rapidly initially. It will not continue to increase forever so, when the demand rate levels off, extrapolating the earlier trend into the future will result in serious over-ordering.

If there are seasonal effects then they should be taken into account in forecasting. However, a lot of data is needed to model the seasonality. Unless an item is fast moving and has been stocked for at least three years, the seasonality cannot be modelled reliably using only the demand history or sales history for the item concerned. For slower moving items or items with less than three years’ history, seasonal forecasting using only the data for the item concerned can do more harm than good. It is much safer to model the seasonal characteristics of appropriate groups of items. As an example, consider size XXXL of woollen sweaters (jumpers) of a particular style and colour. It is unlikely that there would have been sufficient demand for the item concerned to enable reliable modelling of seasonality to be carried out. However, if the total demand for all sizes, styles and colours had been used for that modelling then good seasonal forecasting could probably be done. An added complication in relation to this particular example is that clothing is subject to fashion changes. However, when dealing with seasonality for a group of items, market knowledge is likely to be useful and, also, less than three years history can be used.

Almost all demand rate estimation techniques can be modified to produce forecasts which take into account trends and seasonal effects.

## Multiple Warehouses or Stores

If there is more than one store (or warehouse) then, when forecasting the demand for an item in one store, it is usually helpful to take into account the changes in demands for the item in the other stores to the extent that the influences on those demands are relevant. This is because trends in one store might also occur in other stores and for the same reasons. The statistical analysis involved can be rather complicated but, in many cases, moderately simple techniques will suffice. The extent to which that is the case depends, to some extent, on the nature of the supply chain and the costs of inter-store transfers.

## Use of Transaction History

It is desirable to take into account more detailed demand history or sales history than just monthly totals. In decades past, this was difficult because of the high cost of disk storage. However, these days, it is normal to store the entire transaction history for each item. This information should not be wasted by using only monthly aggregates of demand.

## What is to Come

In later posts, I will describe a demand rate estimation algorithm which is reasonably simple and comes close to satisfying the criteria mentioned in this post. I will also discuss extension of it to carry out forecasting involving trends, seasonality, multiple warehouses and market knowledge.