An assessment of OECD and UK leading indicators.
Over the years economists have sought to develop a range of forecasting devices. The National Institute places a great deal of reliance on its macroeconomic models as forecasting tools. As a very minimum they provide a structural framework which forces a degree of statistical and economic coherence on the forecaster.
Independently of economic models, many statistical offices have maintained a system of leading indicators intended to provide a quite separate forecasting tool. These are constructed as attempts to provide predictors of movements in 'the business cycle' with particular attention being paid to an ability to predict turning points in the business cycle.
In the United Kingdom a system of cyclical indicators has existed for just over 20 years, based on work done by O'Dea (1975). The indicators in the United States have a longer pedigree, going back to work by Burns and Mitchell (1946). In 1987 (Nilsson, 1987) the OECD began to produce leading indicators for all its member states. This means that a reasonable backlog of experience has built up which allows us to assess the performance of these leading indicators. We attempt that in this paper.
2. The Construction of Cyclical Indicators
Cyclical indicators are constructed as an exercise in 'measurement without theory' to quote Koopmans' (1947) criticism of the exercise. Series are selected which show clear turning points well in advance of turning points in 'the cycle'. If the lead time is reasonably constant and if there are no spurious turning points, then the series in question can fulfil a role as a leading indicator.
A composite cyclical indicator can be produced by combining a number of leading indicators. Each is detrended and scaled so that its variance over the cycle corresponds to the cyclical variation of the economy. Averaging over these detrended series then leads to a composite cyclical indicator.
It is apparent from this brief description that the exercise is not in any conventional sense econometric. The forecaster is asked to use his judgement, rather than any statistical criteria, as to which series are good indicators. Not only are the usual statistical techniques absent, but, it is plain that, if one wants to fit turning points, then there is the problem that there are relatively few of them leading to a shortage of degrees of freedom. Artis, Bladen-Hovell and Zhang (1995) attempt to investigate the prediction of turning points in detail and do not conclude that the OECD indicators are generally satisfactory in this respect.
There is in any case a question which has to be addressed before the search for leading indicators can take place. What is 'the cycle' and how can it be identified.
In the United Kingdom this is answered by the idea that the cycle is represented by the coincident indicator, and this in turn is intended to represent movements in GDP. The indicator does not coincide with GDP for three reasons. First of all, it is produced monthly and therefore shows the outcome of an attempt to interpolate movements in quarterly GDP. Secondly, it is smoothed of short-term fluctuations, so as to show an underlying cyclical movement rather than erratic quarter on quarter or month on month fluctuations. Thirdly, 'trend growth' in GDP is removed so as to provide an indicator of the state of the cycle.
The OECD chooses its cyclical indicators to anticipate turning points in industrial production. This is preferred to GDP presumably because industrial production is published monthly while GDP is available only quarterly. However, the indicators are not intended just to anticipate turning points. 'Cyclical coherence' is also regarded as important.
A study by Stock and Watson (1991) suggested an alternative definition of the cycle, as a computed variable chosen so as to have to ability to explain as much variation as possible in a number of economic series which are believed to co-incide with 'the cycle'. A leading indicator can then be chosen to predict movements in this cyclical variable. However, this definition means that 'the cycle' relates to something different from movements in output and as such can be a source of confusion rather than clarification.
3. Identification or Generation of the Cycle
While economists have long been wedded to the idea that there is a regular cycle in the economy an alternative view has recently gained ground. Nelson and Plosser (1982) suggested that movements in GDP might be described by a random walk, with growth in the current period unaffected by whether the previous period has shown fast or slow growth. If this is the case, while there will be movements which may appear cyclical, there is no regular cycle to be found.
This need not, in itself matter. A means for predicting turning points may be of greater interest if these turning points appear irregularly than if there is a regular pattern to them. But in the United Kingdom the cycle is identified by measuring deviations relative to a 5-year moving average. A period of 5 years is chosen because it is believed that the cycle is about 5 years in length. It has been known for many years (Slutsky, 1937) that this procedure can generate an apparent regular cycle in the previously irregular movement represented by a random walk. Osborn (1995) stresses that such a approach exaggerates the component with a period equal to the length of the window used to calculated the moving average.
That this is not purely a theoretical curiosity is shown by the coincident cyclical indicator for the 1980s [ILLUSTRATION FOR CHART 1 OMITTED]. This shows a recession in the mid 1980s roughly five years between the more memorable recessions of 1979-81 and 1990-92. The indicator suggests, moreover that the recession was of similar magnitude. Although there was a 'growth pause' at around this time, the most reasonable inference is that the recession has been generated by the method of identifying the cycle.
There is obviously a concern that, if leading indicators are identified by their ability to anticipate spurious turning points as well as actual turning points, potentially useful indicators may be lost. Conversely, spurious indicators of genuine turning points may be selected.
A separate problem arises from measurement relative to a moving average. This is that, for the most recent data it is not possible to work out a symmetric moving average because that depends on future data as well as past data. Forecasts of some sort or other have to be used and this means that, as more data appear the estimate of the moving average and therefore of the cyclical position is liable to revision. The most recent estimates of both the state of the cycle and of the leading indicators can be liable to substantial revision for this reason. The problem is of course compounded if component data are themselves liable to revision.
The detrending approach adopted by the OECD does not suffer from the problem of phantom cycles which affects the moving average approach. However, the identification of trends in variables which are themselves random walks can also lead to substantial revision to the indicators themselves. Chart 2 shows that the magnitude of the revisions can be large.
Such revisions, compounded by the smoothing process described in the next section, can be important. A study of the United States (Diebold 1991) suggested that, while the final estimates of the leading indicators were able to predict future movements in output, the initial estimates did not have any predictive power. This, of course, defeats much of the purpose of a leading indicator.
4. Smoothing of Data
In addition to the removal of trends by means of one of the methods described above, the series used are smoothed to remove high-frequency noise. The logic of this is that, with erratic variables like the stock market (which features as a component of the leading indicator series) high-frequency movements can obscure cyclical movements. A moving-average filter is used to suppress these high-frequency movements, with the length of the filter being chosen to be long enough for the variance of the series over the cycle to exceed the variance of the series on a high-frequency basis. However the maximum length of the window is set at 6 months.
Once again, there has to be considerable concern about the application of this technique to a series which has a unit root. The removal of high frequencies from a random walk will certainly lead to a smoother series, but the process is not clarifying the signal - it is simply distorting it. Furthermore the use of a moving average again faces the problems identified above. It requires future values to be forecast and therefore leads to revisions as these future values are observed.
Seen in conventional econometric terms, the smoothed series have properties which would not make them ideal candidates for building long-term econometric relationships. The removal of high frequencies by means of smoothing leads to low frequencies remaining. Once the series consist only of low frequencies it is difficult to find satisfactory relationships between them and the series they should be predicting. However, before looking at this issue, we address the question whether the leading indicators do actually anticipate changes in the direction of the series they are meant to lead.
5. A Non-parametric Test of the OECD and UK Indicators
In this section we consider the use of the OECD indicators from the position of a user who wants to anticipate quarter on quarter movements to industrial production.
We aggregated up both variables to quarters and we looked at the ability of the leading indicator correctly to identify above-average growth in industrial production several quarters later. The indicator gives a correct signal if an above-average increase is followed by an above-average increase in industrial production, or if below-average growth signals below-average growth industrial output. An incorrect signal is given otherwise.
If the leading indicator has no information content we should expect half of the signals to be incorrect and this then offers a null hypothesis. Table 1 shows the proportion of incorrect signals for leads of 3, 4 and 5 quarters over the period 1975-1994 for the 14 OECD countries for which there are coherent data. Proportions which are significantly different from 0.5 at a 5 per cent level are shown.
Table 1. The proportions of unsuccessful signals from the OECD indicators, 1966-1994
3 quarter lead 4 quarter lead 5 quarter lead
United States **0.35 **0.36 **0.35 Japan **0.36 **0.35 0.42 Australia 0.45 0.45 0.45 Austria **0.40 **0.37 **0.41 Belgium 0.42 **0.41 **0.41 Finland **0.38 0.43 0.46 France **0.40 0.41 **0.41 Germany **0.40 0.41 0.46 Greece 0.46 0.46 0.48 Italy 0.47 **0.40 0.45 Netherlands 0.42 0.41 0.43 Spain **0.32 **0.35 0.46 Sweden 0.44 0.43 0.45 UK 0.48 0.44 0.49
** indicates significant at a 5% level.
Perhaps the most striking point to emerge from these proportions is that none of them are larger than 0.5. If the leading indicators did not inform about the future growth in industrial output, one would expect an equal number above and below 0.5. We find proportions significantly different from 0.5 for 8 of the 14 countries or for 7 of the 14 countries if we limit ourselves to the lag of three quarters (which is the most successful of the three lags tried).
We now apply the same test to the United Kingdom indicators over the period 1959-1995; we look at the ability of the leading indicators to anticipate movements in the UK co-incident indicator rather than industrial production, because the leading indicators are designed with this in mind. Once again, we aggregate the indicators to a quarterly basis in order to look at quarterly forecasting. As with the OECD's indicators, we see that changes in direction of the UK leading indicators are more likely than not to anticipate changes in the direction of the co-incident index. Moreover they perform better than a naive assumption that the direction of change of the co-incident indicator simply persists. And the short indicator does better than the long indicator with a lead of three quarters, with the position being reversed for a lead of five quarters. However, as Hendry and Emerson (1995) point out, it has to be remembered that these tests are applied to the indicators constructed ex post. It does not automatically follow that changes in the direction of the co-incident variables are identified so precisely ex ante.
Table 2. The proportion of unsuccessful signals from the UK indicators, 1959-1995
3-qtr lead 4-qtr lead 5-qtr lead
Naive model 0.43 0.50 0.51 Shorter-leading (**)0.33 (**)0.37 0.43 Longer-leading (**)0.41 (**)0.41 (**)0.36
** indicates significantly different from 0.5 at 5% level.
These non-parametric tests indicate how likely it is that the indicators anticipate changes in the direction of the co-incident variables. We now complement these by looking at regression models. These allow us to establish how good the indicators are at predicting changes in the co-incident variables.
6. Tests of Regression Models using the OECD Indicators
In order to assess the performance of the OECD indicators we test their statistical significance in regression equations designed to predict movements in monthly industrial production. Monthly rather than quarterly data are used because the indicators are designed to provide monthly information.
To be completely satisfactory as indicators of changes in the co-incident variables, there would ideally be a clear and precise relationship between movements in the two. An efficient indicator with a lead of say, six months, would encapsulate all the information that is of any use in predicting movements of the co-incident variable. This means that values of neither the coincident variable nor the leading indicator with a lag of more than six months would have significant explanatory power in a regression used to predict movements in the co-incident indicator. If the indicator is efficient it is also the case that there will be no other obvious economic variables which are statistically significant with lags of more than six months in such a regression equation.
Here we test the first, but not the second form of efficiency. For the OECD indicators we estimate vector autoregressions for both industrial production and for the leading indicator itself. The latter equation is useful when assessing the forecasting performance of the industrial production series and the leading indicator taken together. Since both industrial production and the leading indicators have unit roots, we estimate equations in logarithmic first differences. However, we allow for the possibility that there is a long-term relationship in levels between the indicator and the industrial production series. Indeed for the indicator to be a valid predictor one might expect this to be the case (Pain 1994).
The equations we estimate for each OECD country are
[Delta]log(Ind Prod) = [summation of] [[Alpha].sub.1i][Delta] log[(IndProd).sub.t - i] where 1 to 15 + [summation of] [[Beta].sub.1i][Delta]log[(Leading Indicator).sub.t - i] where 1 to 15 + [[Gamma].sub.1a](log [Leading Indicator.sub.t - 16] - [[Gamma].sub.1b] log [Ind Prod.sub.t - 16]) (1)
[Delta]log(Leading Indicator) = [summation of] [[Alpha].sub.2i][Delta]log[(Ind Prod).sub.t - i] + [summation of] [[Beta].sub.2i][Delta]log[(Leading Indicator).sub.t - i] where 1 to 15 + [[Gamma].sub.2a](log [Leading Indicator.sub.t - 16] - [[Gamma].sub.2b]log [Ind Prod.sub.t - 16]) (2)
This structure makes it possible to carry out a number of tests on the leading indicators. First of all we can test for the significance of the terms in first differences. By using 15 monthly lags we are able to test both for the significance of the first 6 lag terms ([[Beta].sub.1, 1].. [[Beta].sub.1, 6]) and for the last 9 lag terms ([[Beta].sub.1, 7].. [[Beta].sub.1, 15]). When the first 6 terms are significant, we say that the indicator has a short lead, while when the last nine are significant, we say that it has a long lead. Plainly it is possible for an indicator to function [TABULAR DATA FOR TABLE 3 OMITTED] both as a shorter-leading and as a longer-leading indicator.
Secondly, we can construct a mis-specification test. If the model is valid there should be (at most) a single co-integrating vector in both equations, implying that [[Gamma].sub.1b] = [[Gamma].sub.2b]. If this test is accepted, we can then test for the significance of [[Gamma].sub.1a]. This is a test for the presence of a co-integrating relationship between the indicators and industrial production.
A second misspecification test can be conducted on equation (2). A desirable property of the leading indicator is that it should lead industrial production, but that industrial production should not lead the indicator. Otherwise industrial production and composite indicator function as leading indicators of each other. Thus we should expect that the parameters [[Alpha].sub.2i] and [[Gamma].sub.2a] should be jointly insignificant.
Thirdly, we can assess the forecast performance of the models. We do this by constructing forecasts of industrial production up to six months ahead. We then use the equation standard errors to calculate standard errors for the change in quarterly industrial production for the quarter ending six months ahead relative to the quarter ending three months ahead. This then serves to indicate how good the leading indicator actually is at forecasting quarterly movements in industrial production and provides a measure of the reliability which should be attached to any forecast generated by the leading indicator.
The first column of Table 3 shows the result of the test that the cointegrating vector should be the same in both regression equations. This is passed in all cases. However we can see from column 2 that not all the leading indicators are co-integrated with industrial production. The test is failed by France, Germany, Greece and Italy In these countries the leading indicator does not evolve in the long term in the same way as industrial production. The third column identifies the coefficient, [[Gamma].sub.1b] which shows the percentage change in industrial production associated with a 1 per cent change in the leading indicator in the long run. Results are presented for those countries where the co-integrating vector is statistically significant. It can be seen that these vary a great deal, even for the countries for which there is a valid long-term relationship. The range stretches from 0.76 for the United Kingdom to 1.56 for the Netherlands. Finally the last test shows the statistical significance of the terms in industrial production in the equation for the leading indicator. In France and Germany industrial production functions as a 'leading indicator' of the OECD's indicator.
Our next test concerns the significance of the terms in log differences of the OECD leading indicators, and indicators their ability to predict changes in industrial production. These tests are carried out after the cross-equation restriction, [[Gamma].sub.1b] = [[Gamma].sub.2b] has been imposed and the co-integrating vectors have been removed where they are not statistically significant. The first column shows the significance of the lags of up to six months, so that a significant entry here indicates that the indicator functions as a short leading indicator. The second column shows the significance of the last nine months and indicates whether the variable functions as a long leading indicator. In particular, a significant F-statistic in the [TABULAR DATA FOR TABLE 4 OMITTED] [TABULAR DATA FOR TABLE 5 OMITTED] second column indicates that the indicator is not an efficient short-run leading indicator.
Finally, in Table 5, we can address the question how well the restricted VAR equations using the indicators actually forecast. Looking at quarter on quarter growth rates as described above, we can compare those derived from the VARs including the leading indicators from two simpler models. The first is a monovariate VAR with a 15-month lag using only industrial production: a pure autoregression. The second is a naive 'model' which assumes that industrial production evolves as a random walk, so that the forecast increase in industrial production is simply the average quarterly growth rate. The standard error of this turns out to be [square root of 19] times the standard error of month-on-month changes to industrial production.
This table shows that, in all cases except the Netherlands, the inclusion of the leading indicator in the VAR results in a reduction of forecast errors. The reduction in the standard error is, however, small compared with that achieved by using a monovariate VAR instead of a naive model to predict growth in industrial production.
There is one specific case which merits special attention. The use of a monovariate VAR to predict changes to United States' industrial production results in an increase in the standard error over the naive model. The explanation is that, although the monovariate VAR is stable, in the short term it is extrapolative. This means that, for making short-term projections, it performs worse than the naive model.
In general, while the regression models using the leading indicators do improve the forecasts of movements to industrial production, the quality of the forecasts that they produce must be described as poor in absolute terms. This is true even though they are calculated from a VAR regression and not from a rigid lag length between indicator movements and movements of industrial production. It follows that any attempt to anticipate movements to industrial production by relying on a fixed lag from movements in the leading indicator is going to be more unreliable.
7. Regression Results for the UK Indicators
The construction of regression models for the UK is complicated by the fact that, although the indicators are plainly stationary, the filtering out of high frequencies is so effective that, using conventional (but rather low-powered) tests, they appear to have unit roots. But the process of measurement relative to a moving average means that non-stationary terms are subtracted out; the question of handling unit roots does not arise.
This means that the regression equations should be constructed to explain the log level of the coincident indicator by means of the shorter and longer leading indicators. The question of co-integration does not arise.
[TABULAR DATA FOR TABLE 6 OMITTED]
The equations which we estimate are
log(Coinc Indicator) = [summation of] [[Alpha].sub.1i]log[(Coinc Indicator).sub.t - i] where 1 to 15 + [summation of] [[Beta].sub.1i]log[(Leading Indicator).sub.t - i] where 1 to 15 + [[Gamma].sub.1] (3)
log(Leading Indicator) = [summation of] [[Alpha].sub.2i]log[(Coinc Indicator).sub.t - i] where 1 to 15 + [summation of] [[Beta].sub.2i]log[(Leading Indicator).sub.t - i] where 1 to 15 + [[Gamma].sub.2] (4)
We estimate the system twice, first with the longer leading indicator and secondly with the shorter leading indicator. We do not present a single combined regression because the purpose of this paper is to assess the performance of the indicators on their own terms and not as part of composite models. It should however be noted that Britton and Pain (1992, p. 66) find that, in a regression equation designed to predict movements in output, the composite leading indicators are not significant when other variables such as the real equity price and the slope of the yield curve are introduced.
Hendry and Emerson (1995) find that the component variables of the leading indicators are significant in such regressions and Yeend (1996) discusses the forecasts generated by such an indicator. All these observations cast doubt on whether the existing leading indicators are good indicators of future movements in the coincident indicator.
Once again we can test the significance of the six shorter or the nine longer lags of each of the shorter and longer leading indicators in the regression equations. One might hope that the shorter leading indicator will not have significant long lags, while the longer leading indicator will not have significant short lags. We also test to see whether the co-incident indicator functions as a leading indicator of the official indicators by looking at the significance of [[Alpha].sub.2i] in equation (4).
The results of the tests are shown in Table 6 and it can be seen that both longer and shorter indicators function only as short leading indicators, with no significant explanatory power being added by the inclusion of lags longer than six months. The shorter leading indicator is efficient in that the inclusion of longer lags in the regression equation does not enhance its explantory power significantly. But the coincident indicator functions as a leading indicator of both official leading indicators.
The forecast performance is not more encouraging. As Table 7 shows, the standard errors calculated from the forecasting models are larger than those calculated from a naive model. The latter assumes that a forecast of no change in the coincident indicator is made. Once again, this is because the models are extrapolative in the short term. But it suggests that the variables are not well suited to their roles as leading indicators.
These results indicate first of all that the leading indicators analyzed generally have some predictive power, although even this conclusion is based on an ex post analysis reflecting the fact that the indicators have been chosen with the benefit of hindsight. It has to be said that, on the basis of the non-parametric test, this predictive power is not very strong. Typically on about 40 per cent of occasions growth in the leading indicator on one side of the trend rate is followed by growth in industrial production on the other side of the trend growth rate. Thus the performance is not very much better than random.
[TABULAR DATA FOR TABLE 7 OMITTED]
The indicators do typically play a role in regression equations which can be used to predict growth in industrial production. However, the reliability of forecasts generated from such equations is typically low; it cannot be said that they are a very helpful tool for predicting future economic performance.
The UK indicators are constructed relative to a 5-year moving average, and there is real concern that this technique may create regular cycles where none exist in the original data. There is, again, some evidence that these indicators do have predictive power, although not that the longer leading indicator actually has a longer lead than the shorter indicator.
The OECD indicators generally appear to have a longer lead than the UK indicators, although it should be noted that the OECD indicators of the UK are not significant in the relevant regressions; thus one cannot conclude that the OECD have a better approach than the UK Office of National Statistics.
All the indicators are unsatisfactory in that they are not clear predictors of movements in the 'coincident variable', either industrial production or, in the UK, the coincident indicator. However, the source of the problem may be a belief that 'the state of the cycle' is something which should be measured separately from output. If this belief is abandoned, then it seems clear that leading indicators should simply be forecasts of movements in output, to be constructed using the techniques which are generally believed to lead to good forecasting models. Such a change of approach is needed if efficient leading indicators are to be produced.
Artis, M., Bladen-Hovell, R. and Zhang, W. (1995), 'Turning points in the international business cycle: an analysis of the OECD leading indicators for the G7 countries', OECD Economic Studies, 24, pp. 125-165.
Britton, A., and Pain, N. (1992), Economic Forecasting in Britain, National Institute Report no. 4.
Burns, A. and Mitchell, W. (1946), Measuring Business Cycles, NBER, New York.
Diebold, F. (1991), 'Forecast output with the composite leading index: a real time analysis', Journal of the American Statistical Association, 86, pp. 603-610.
Hendry, D. and Emerson, R. (1995), 'An evaluation of forecasting using leading indicators', Royal Economic Society Conferences, Canterbury.
Koopmans, T. (1947), 'Measurement without theory', Review of Economics and Statistics, vol. 29.
Nelson, A. and Plosser, A. (1982), 'Trends and random walks in macroeconomic series', Journal of Monetary Economics, 10, pp. 139-162.
Nilsson, R. (1987), 'OECD leading indicators', OECD Economic Studies, no. 9, pp. 105-146.
O'Dea, D. (1975), Cyclical Indicators for the Post-War British Economy, Cambridge University Press.
Osborn, D.R. (1995), 'Moving average detrending and the analysis of business cycles', Oxford Bulletin of Economics and Statistics, no. 57(4), pp. 547-558.
Pain, N. (1994), 'Cointegration and forecast evaluation: some lessons from National Institute forecasts', Journal of Forecasting, no. 13, pp. 481-493.
Slutsky, E. (1937), 'The summation of random causes as a source of cyclical processes', Econometrica, no. 5, pp. 105-146.
Stock, J. and Watson, M. (1991), 'A probability model of the co-incident indicators', in G. Moore and K. Lahiri, eds, Leading Economic Indicators, Cambridge University Press, pp. 63-90.
Yeend, C. (1996), 'Cyclical indicators for the UK economy', Economic Trends, 509, pp. 34-38.
|Printer friendly Cite/link Email Feedback|
|Publication:||National Institute Economic Review|
|Date:||May 1, 1996|
|Previous Article:||The performance of National Institute economic forecasts.|
|Next Article:||Forecast error bounds by stochastic stimulation.|