Printer Friendly

Machine Learning Approaches to Macroeconomic Forecasting.

Forecasting macroeconomic conditions can be challenging. Accurate forecasts require an approach complex enough to incorporate relevant economic data but focused enough to exclude irrelevant data. Most forecasters attempt to balance focus and complexity with one of two approaches: aggregating forecasts from professional forecasters into a consensus prediction or using statistical modeling techniques typically specialized for time-series data. However, both approaches involve discretionary choices about data and methods. Although forecasters underpin the choices they make about models and complexity with economic intuition and judgement, these assumptions can be flawed.

Machine learning approaches, on the other hand, automate as many of those choices as possible in a manner that is not subject to the discretion of the forecaster. Indeed, optimizing a model without assumptions or judgement is a central issue of study in machine learning. In this article, I apply machine learning techniques to find an optimal forecasting model for the unemployment rate. After identifying a model that is not subject to discretion or assumptions, I compare the forecasts obtained from the machine learning model to forecasts obtained through traditional forecasting approaches. My experiments show that when supplied with diverse and complex data, the machine learning model can outperform simpler time-series models as well as a consensus of professional forecasters, with better performance at shorter horizons. In particular, experiments show that a machine learning model can identify turning points in the unemployment rate earlier than competing methods, suggesting that it can provide forecasters with more guidance about cyclical fluctuations than a consensus or autoregressive forecast.

Section I reviews the statistical and consensus approaches for macroeconomic forecasting. Section II discusses how machine learning can be used to optimize model complexity. Section III shows that one machine learning model, the Elastic Net, can outperform traditional models at all horizons, detect turning points earlier, and identify variables that can help predict unemployment from among a broad set.

I. Popular Approaches to Unemployment Forecasting

The unemployment rate is a challenging variable to forecast because it can be influenced by a wide variety of other variables. In the past, forecasters have attempted to overcome these challenges through statistical modeling or consensus forecasting. Statistical modeling benefits from strong mathematical theory that can lead to less biased forecasts and more consistent predictions. One of the most common statistical models for time-series forecasting is an autoregressive (AR) model, which assumes future observations are predicted primarily by recent observations. The unemployment rate is largely consistent with that assumption. In addition to AR models, forecasters often use a random walk model as a control or baseline. In a random walk, predictions for tomorrow's unemployment rate are based on the unemployment rate today plus an additional random shock. If a model cannot perform better than a random walk, it is unlikely it will be useful in making predictions.

Some forecasters might assume that a wide variety of variables, in addition to the past unemployment rate, may be useful in forecasting unemployment and therefore incorporate additional information in their forecasting models (Barnichon and Nekarda 2013). Still, they may overlook other important information that might influence the unemployment rate, including info on housing markets, consumer sentiment, and interest rates.

Consensus forecasting, contrasted with statistical models, leverages the wisdom of the crowd and benefits from combining as many different kinds of models as there are forecasters. These models range from sophisticated empirical systems of equations to simple gut instinct. Consensus forecasts are available for the unemployment rate from both Blue Chip Economic Indicators and the Survey of Professional Forecasters (SPF). By drawing from a variety of techniques and methods, these consensus forecasts can be more accurate than any particular professional forecast (Clemen 1989).

Both statistical modeling and consensus forecasting require a range of different choices that affect model complexity and forecasting performance. The AR model requires some very clear choices--specifically, the number of lagged variables to include. This choice can have a significant effect on AR forecasting performance. Although many researchers use some metric such as the Bayesian information criterion to determine the number of lags, they still must choose a metric and the maximum number of lags. There are not many models that do not require any choices, but the random walk model is one example; the model is purely a function of random movements based on past observations of the unemployment rate.

On the surface, the consensus forecast may not seem to involve many choices, as all the choices are being made by the forecasters themselves. This is a desirable quality, which is why the Blue Chip forecasts are commonly used as a competitive benchmark. However, researchers using these forecasts must still make choices about how to use them. Some may decide to use the median forecast instead of the average forecast. Others may choose to average only top-performing forecasters.

Rather than make restrictive modeling assumptions, researchers might alternatively consider a model that can admit any number of variables. Here, the field of choices begins to explode. For example, the FRED economic data platform, housed at the Federal Reserve Bank of St. Louis, advertises 527,000 economic series--which of these should researchers choose to include?

Machine learning techniques and models can simplify this set of choices tremendously. Models such as Elastic Net allow researchers to make a single choice about how to constrain the model's complexity rather than considering the inclusion or exclusion of all potential series. Moreover, other machine learning techniques can help researchers discover the optimal model complexity, eliminating that choice as well.

II. Introducing Machine Learning and Forecasting

The field of machine learning encompasses a wide variety of models. One way to identify the optimal forecasting model is to decompose the source of forecast errors into two parts--bias and variance. The bias of a model describes errors due to inaccurate assumptions about a problem or data. The variance of a model describes errors due to a model's sensitivity to small perturbations in the underlying data.

Figure 1 shows the effects of bias and variance on model predictions. In particular the figure shows four bullseyes arising from the combination of two different conditions: high or low bias and high or low variance. The orange center of the bullseye represents the correct prediction, and the blue dots represent model predictions. Bias indicates how close to the center of the bullseye a model's predictions are, while variance indicates how tightly or loosely the predictions are clustered. Models with high bias and low variance yield observations that are tightly clustered but far from the correct prediction. Models with low bias and high variance, on the other hand, yield observations that are only loosely clustered around the correct prediction. Both of these sources of errors are related to the complexity of a model. Understanding the tradeoff between bias and variance can help forecasters optimize model complexity and thus minimize the total error in their predictions.

The complexity of a model can be assessed in many different ways, which are often dependent on the nature of the model itself. Sources of model complexity include the number of variables in a model, the number of parameters a model "learns," the number of parameters a user defines, or the number of variable relationships a model captures internally.

The relationship between model complexity, bias, and variance is often illustrated as a trade-off. Figure 2 shows that as the complexity of a model increases, the model typically exhibits less bias but more variance. The dashed line indicates optimum model complexity, where the total error due to both bias and variance is minimized and thus the corresponding forecast error for new observations (called the generalization error) is minimized.

To further illustrate this trade-off, consider a forecaster using a linear regression model estimated using ordinary least squares (OLS) to forecast the unemployment rate. In this case, the complexity of the model refers to the number of explanatory variables the forecaster chooses to include. If the forecaster uses a simple model with a single variable for the previous period's observed unemployment rate, then the model will have large forecast errors from bias due to the likely incorrect assumption that future unemployment is only dependent on current unemployment. However, the model will have small errors due to variance because the model is robust to small perturbations in the underlying data (that is, small changes in the observed variable translate to small changes in the forecast, proportional to the estimated model coefficient).

The forecaster can increase the complexity of their model by adding variables. In doing so, the forecast error from bias will decline because the assumption that only current unemployment determines future unemployment has been relaxed. However, the forecast error from variance will increase because small perturbations in lots of variables (including those that may be irrelevant to the unemployment rate) will now translate to disproportionally large changes in the forecast.

The Elastic Net model, which has become popular in machine learning, can precisely balance errors due to bias and variance through regularization. Regularization effectively "disciplines" a model by introducing penalties for overfitting the data. The Elastic Net model blends two different kinds of regularization by penalizing both the number of variables in the model and the extent to which any given variable contributes to the model's forecast (the magnitude of the variable's coefficient). By applying these penalties, Elastic Net learns which variables are most important, eliminating the need for researchers to make discretionary choices about which variables to include (further details on how the penalties are assessed are available in the appendix). In this way, the model is able to pare complex data down to only what is needed to capture the complexity of the problem at hand.

III. Comparing Machine Learning to Consensus and Statistical Forecasts

Machine learning provides a strong set of tools for finding the optimal complexity of a model, freeing forecasters from the need to use strong assumptions or judgement to simplify their models. To assess whether these advantages yield more accurate forecasts, I compare the results from one regularized machine learning model, the Elastic Net, to typical consensus and statistical forecasts of the U.S. civilian unemployment rate at a monthly frequency. I also assess these methods' utility in identify turning points in the business cycle as seen in the unemployment rate.

The benchmark models used for comparison are the consensus forecast from Blue Chip, an AR model, and a random walk. I use Blue Chip forecasts rather than forecasts from the SPF, as the SPF does not forecast at a monthly frequency. (1) The AR model is an integrated model that uses monthly changes in the unemployment rate to forecast the future unemployment rate. I choose the lag length using the Bayesian information criterion, with a maximum lag length of six months. By definition, the random walk and AR models only use past observations of the unemployment rate to forecast future outcomes.

In contrast, the Elastic Net model forecasts the future unemployment rate using FRED-MD, a diverse variable set of 138 macroeconomic variables drawn from a number of economic categories (McCracken and Ng 2015). Adding these variables allows the Elastic Net model more potential complexity but allows overfitting as well. The combined data set spans March 1959 through April 2017 for a total of 698 monthly observations. I use the first half of this sample (March 1959 through June 1988) to train the initial models and fit the coefficients, and the second half of the sample (July 1988 through April 2017) for the forecast comparison. I use a rolling forecast framework, in which I train a model using data observed up to current period t to fit the coefficients, make a forecast for period t + 1, then advance one period and repeat until the end of the forecast span. I consider forecasting horizons of up to two years ahead at three-month intervals (three, six, nine, 12, and 24 months).

Forecast accuracy

Table 1 shows that Elastic Net yields the most accurate forecasts as measured by the mean absolute error (MAE) for all horizons. Elastic Net outperforms even Blue Chip forecasting, indicating that the model is effectively learning the underlying relationships between economic series. Table 2 shows that Elastic Net outperforms Blue Chip by 0.07 percent on average. The advantage is more pronounced at shorter horizons, and the learned relationships are less useful at horizons above 12 months. The performance advantage of Elastic Net over Blue Chip is statistically significant for most forecast horizons, as shown in Table 3. The p-values in this table indicate a strong, statistically significant difference at forecast horizons of 12 months or fewer. (2)

Forecast turning points

While comparing the errors of different models provides one way to evaluate their predictive performance, the raw errors themselves do not tell the entire story. One hallmark of the unemployment rate is its cyclical movement: the unemployment rate falls in an expansion and rises in a recession. Training a model to capture and forecast this cyclical pattern has advantages beyond statistically lowering errors. Forecasting turning points in the unemployment rate may provide useful information for policymakers.

I identify turning points in the forecast and actual unemployment rate using the following steps: first, I smooth the forecast and actual unemployment rates using a three-month centered moving window. Second, I find the months of highest and lowest unemployment for each business cycle (spanning peak to trough). Third, I calculate the difference in months between the turning point in the actual unemployment rate and the turning point in the forecast unemployment rate.

Table 4 shows that Elastic Net is also able to identify unemployment rate turning points earlier than Blue Chip and other methods. The advantage for Elastic Net is more pronounced at shorter horizons, meaning it identifies upcoming turning points in the near term earlier than other methods. On average, Elastic Net identifies turning points 5.1 months earlier than Blue Chip, as shown in Table 5. The advantage in the number of months across horizons appears to be fairly consistent, without strong variation between short and long horizons.

The earlier turning points for Elastic Net can be identified visually at both short and long horizons, as shown in Charts 1 and 2. Panels A and B of Chart 1 shows that at a three-month horizon, the differences from the actual unemployment rate are relatively minor--specifically, the blue (actual) and orange (predicted) lines often overlap. However, the Elastic Net forecast appears to track (overlap) the actual rate track more closely than the Blue Chip forecast. Indeed, the Blue Chip forecast appears to lag the actual rate by a consistent amount, especially during downward trends.

Panels A and B of Chart 2 show that at a 12-month horizon, the forecasts deviate more from the actual rates, as expected. The blue and orange lines in Panels A and B show very little overlap, instead crossing over in the periods between turning points. Again, however, the Blue Chip forecast appears to lag the actual rate by a consistent amount throughout the range of the forecast. The Elastic Net model, on the other hand, appears to respond to turning points earlier to change the forecast trend. These charts provide qualitative support for the differences shown in performance statistics.

Variables identified by Elastic Net

In addition to forecast improvements, Elastic Net can also identify variables potentially important to predicting the unemployment rate. By using regularization to control the size of variable coefficients in the model, Elastic Net can drive some coefficients to exactly zero and thus select a more parsimonious number of variables than an unregularized linear model. Inspecting the model to see how it is making forecasts may uncover new relationships in the data that have been previously overlooked.

The variables identified by Elastic Net cover mostly labor market and housing, with some additional variables covering consumption, output and income, and interest and exchange rates. The coefficients for these variables are shown in Tables 6 and 7, for forecast horizons of three and 12 months, respectively. At a three- and 12-month horizon, variables from housing, manufacturing, and interest rates have the largest coefficients, indicating they have the largest effect on the unemployment rate forecast. The coefficients listed in these tables are averaged over all forecast steps. Only variables with coefficients greater than two standard errors from zero are included.

Panels A and B of Figure 4 show that the strength of these coefficients can vary over time for some variables while staying constant for others. Examining the coefficients allows us to discover how the learned model changes in response to different economic conditions over time. For example, Panel A shows that while many of the coefficient series are stable at the three-month horizon (shown by the solid white bands), the housing and interest rate variables change throughout the forecast period (shown by the shifts in color over time). Prior to 1993-94, the coefficients on the housing and interest rate variables were near zero or only briefly below zero. After 1993-94, however, the coefficients were consistently strong (indicated by the more saturated orange bands).

This period of inconsistency broadly centers around the 1990-91 recession, but extends both ahead and behind. The coefficients on these variables diminished near the 2001 recession as well. Interestingly, only the coefficient on the federal funds rate variable diminished after the 2007-09 recession, indicating that this variable has diverged from the others and may no longer be relevant.

Panel B shows several other interesting patterns in variable coefficients for the 12-month forecast horizons. The conspicuous band of color change on the left side of the heat map shows that coefficients on the labor, housing, and interest variables strengthened. This band corresponds to the late-1989 to mid-1991 recessionary period of negative or slow GDP growth, suggesting the model is picking up disruptions in economic relationships associated with the recession. Another pattern of change across variables is seen in the 1996-98 period and again in 1999-2000. These years correspond to a relatively stable expansion before the 2001 recession, suggesting that the influence of manufacturing, housing, and interest rate variables strengthened after the recession. In contrast, the coefficients on these variables weakened after the 2007-09 recession; however, the coefficients on other variables weakened during this period as well, suggesting these changes may be due to broader conditions.

IV. Conclusions

Forecasting is challenging, and the wealth of new and accessible data describing economic conditions presents an opportunity to explore more complex models that can capture more of the economic data. The field of machine learning provides a number of methods to address and capitalize on this complexity, both through increasingly complex models as well as methods to control and optimize that complexity.

I compare the performance of consensus, statistical, and machine learning methods for forecasting the monthly U.S. unemployment rate. My analysis shows that a more complex model, when properly controlled and provided with enough data from which to learn, can significantly outperform consensus and simpler statistical forecasting methods. The key to this result is the control of model complexity through regularization, a machine learning technique that yields a model complex enough to avoid underfitting the data but not so complex as to overfit it.


Details of the Elastic Net Model

This appendix discusses the technical details of the Elastic Net model and how it is estimated. For completeness, I build the Elastic Net model piecewise as I add regularization components. The objective function for estimating coefficients [//] ={[[beta].sub.1], [[beta].sub.2], ..., [[beta].sub.m]} in an ordinary least squares (OLS) model is given as:

[mathematical expression not reproducible]

Here, I omit the intercept without loss of generality, and represent the set of coefficients [beta] and independent variables x as vectors with the i subscript indicating the ith of n total samples, with [y.sub.i] as the response for the ith sample. Next, an [L.sub.1] regularization term is added, giving the LASSO objective:

[mathematical expression not reproducible]

Then, an [L.sub.2] regularization term is added as in a ridge regression, giving the Elastic Net objective:

[mathematical expression not reproducible]

In this objective, the penalty weights [[lambda].sub.1] and [[lambda].sub.2] are chosen as positive but unbounded, so a re-parameterization is often desirable. The revised objective is:

[mathematical expression not reproducible]

which gives two alternative parameters, [alpha] and [rho], where [rho] is bounded between 0 and 1 and controls how the regularization strength is balanced between the [L.sub.1] and [L.sub.2] terms. The parameter [alpha] controls the overall strength of regularization and is positive and unbounded. The final objective function is convex and so can be minimized using any convex optimization method such as gradient or coordinate descent.


(1) One challenge with using Blue Chip forecasts for comparison is that some of the historical data used by models in my forecasts have been revised, while the Blue Chip forecasts were made before revised data were available.

(2) Statistical significance is computed using a two-sample Kolmogorov-Smirnov test to compute the probability that errors from two models are drawn from the same distribution.


Barnichon, Regis, and Christopher J. Nekarda. 2012. "The Ins and Ours of Forecasting Unemployment: Using Labor Force Flows to Forecast the Labor Market." Board of Governors of the Federal Reserve System, Finance and Economics Discussion Series 2013-19, November.

Clemen, Robert T. 1989 "Combining Forecasts: A Review and Annotated Bibliography." International Journal of Forecasting, vol. 5, no. 54, pp. 559-583. Available at

Cook, Thomas R., and Aaron Smaller Hall 2017. "Macroeconomic Indicator Forecasting with Deep Neural Networks." Federal Reserve Bank of Kansas City, Research Working Paper no. 17-11, September. Available at

Diebold, Francis X. 1997. "The Past, Present, and Future of Macroeconomic Forecasting." NBER Working Paper no. 6290, November. Available at

Lucas, Robert E., Jr. 1976. "Econometric Policy Evaluation: A Critique." Carnegie Rochester Conference Series on Public Policy, vol. 1, pp. 19-46. Available at

McCracken, Michael W., and Serena Ng. 2015. "FRED-MD: Monthly Database for Macroeconomic Research." Federal Reserve Bank of St. Louis, working paper, August. Available at

Pescatori, Andrea, and Saeed Zaman. 2011. "Macroeconomic Models, Forecasting, and Policymaking." Federal Reserve Bank of Cleveland, Economic Commentary, vol. 19, October.

Sims, Christopher A. 1980. "Macroeconomics and Reality." Econometrica, vol. 48, no. 1, pp. 1-48. Available at

Ye, Jianming. 1998. "On Measuring and Correcting the Effects of Data Mining and Model Selection." Journal of the American Statistical Association, vol. 93, no. 441, pp. 120-131. Available at

Zou, Hui, and Trevor Hastie. 2005. "Regularization and Variable Selection via the Elastic Net." Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no. 2, pp. 301-320. Available at

By Aaron Smalter Hall

Aaron Smalter Hall is a senior data scientist at the Federal Reserve Bank of Kansas City. This article is on the bank's website at

Caption: Figure 1 Errors of a Model Due to Bias and Variance

Caption: Figure 2 Trade-Off between Errors Due to Variance and Errors Due to Bias

Caption: Chart 1 Unemployment Forecasts at the Three-Month Horizon

Caption: Chart 2 Unemployment Forecasts at the 12-Month Horizon

Caption: Figure 4 Coefficients over 1988-2017 for Elastic Net
Table 1
MAE Results

Horizon   Random walk     AR      Blue Chip  Elastic Net
(months)   (percent)   (percent)  (percent)  (percent)

3            0.35        0.23       0.26       0.17
6            0.53        0.38       0.39       0.27
9            0.79        0.58       0.52       0.42
12           1.00        0.82       0.65       0.58
15           1.22        1.04       0.76       0.74
18           1.49        1.30       0.92       0.88
21           1.69        1.55        --        1.02
24           1.91        1.78        --        1.20
Average      1.12        0.96        --        0.66

Note: Forecasting results shown for all horizons with each of the four
Sources: Blue Chip Economic Indicators, Bureau of Labor Statistics,
Federal Reserve Bank of St. Louis FRED, and author's calculations.

Table 2
Differences between MAE for Elastic Net and Other Forecasts

Horizon   Random walk     AR      Blue Chip
(months)   (percent)   (percent)  (percent)

3            0.18        0.06       0.09
6            0.26        0.11       0.12
9            0.37        0.16       0.10
12           0.42        0.24       0.07
15           0.48        0.30       0.02
18           0.61        0.42       0.04
21           0.67        0.53        --
24           0.71        0.58        --
Average      0.46        0.30       0.07

Note: Positive values indicate better performance for Elastic Net.
Sources: Blue Chip Economic Indicators, Bureau of Labor Statistics,
Federal Reserve Bank of St. Louis FRED, and author's calculations.

Table 3
P-Values for Statistical Significance

Horizon (months)  Random walk   AR    Blue Chip

 3                   0.008     0.000    0.000
 6                   0.006     0.000    0.000
 9                   0.171     0.000    0.000
12                   0.006     0.000    0.001
15                   0.000     0.000    0.021
18                   0.005     0.000    0.199
21                   0.003     0.000     --
24                   0.043     0.000     --

Note: Significance computed for MAE with respect to Elastic Net
compared with AR, random walk, and Blue Chip.
Sources: Blue Chip Economic Indicators, Bureau of Labor Statistics,
Federal Reserve Bank of St. Louis FRED, and author's calculations.

Table 4
Forecast Error in Predicting Turning Points (in Months)

Horizon  Random walk   AR   Blue Chip  Elastic Net

 3            6.2      4.0     4.2       0.6
 6            8.4      6.0     7.0       1.4
 9           10.8      8.8     9.2       4.0
12           15.8     10.8    11.8       6.4
15           17.8     12.8    14.0       7.0
18           18.6     13.4    12.8       8.8
21           20.8     14.2     --        7.0
24           19.8     13.8     --       10.0
Average      14.7     10.4     --        5.65

Note: Average distances are calculated from predicted turning points to
actual turning points for forecasting methods and variable sets for all
Sources: Blue Chip Economic Indicators, Bureau of Labor Statistics,
Federal Reserve Bank of St. Louis FRED, and author's calculations.

Table 5
Difference in Turning Point Distances from Elastic Net (in Months)

Horizon  Random walk  AR   Blue Chip

 3           5.6      3.4    3.6
 6           7.0      4.6    5.6
 9           6.8      4.8    5.2
12           9.4      4.4    5.4
15          10.8      5.8    7.0
18           9.8      4.6    4.0
21          13.8      7.2    --
24           9.8      3.8    --
Average      9.0      4.8    5.1

Note: Positive values indicate better performance for Elastic Net.
Sources: Blue Chip Economic Indicators, Bureau of Labor Statistics,
Federal Reserve Bank of St. Louis Federal FRED, and author's

Table 6
Coefficients for Elastic Net Model for Three-Month Horizon

Variable                                  Coefficient  Category

Consumer sentiment index                  -0.005       Consumption,
                                                       orders, and
                                          (0.001)      inventories
Help-wanted index for United States       -0.025       Labor market
Average duration of unemployment (weeks)   0.032       Labor market
Capacity utilization: manufacturing       -0.043       Output and income
Effective federal funds rate              -0.100       Interest and
                                                       exchange rates
New private housing permits, Midwest      -0.110       Mousing
New private housing permits, Northeast    -0.115       Housing
Three-month Treasury minus federal funds  -0.148       Interest and
                                                       exchange rates
Average weekly hours: manufacturing       -0.164       Labor market
Civilian unemployment rate                 0.874       Labor market

Note: Standard errors are in parentheses.
Sources: Bureau of Labor Statistics, Federal Reserve Bank of St. Louis
FRED, and author's calculations.

Table 7
Coefficients for Elastic Net Model for 12-Month Horizon

Description                   Coefficient  Category

Consumer sentiment index      -0.015       Consumption, orders, and
                              (0.006)      inventories
Help-wanted index for
United States                 -0.036       Labor market
New private housing permits,
Northeast (SAAR)              -0.165       Housing
Six-month Treasury minus
federal funds                 -0.205       Interest and exchange rates
Three-month Treasury
minus federal Rinds           -0.216       Interest and exchange rates
Average weekly hours:
manufacturing                 -0.237       Labor market
Civilian unemployment rate     0.590       Labor market

Note: Standard errors are in parentheses.
Sources: Bureau of Labor Statistics, Federal Reserve Bank of St. Louis
FRED, and author's calculations.

Please Note: Illustration(s) are not available due to copyright restrictions.
COPYRIGHT 2018 Federal Reserve Bank of Kansas City
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Hall, Aaron Smalter
Publication:Economic Review (Kansas City)
Article Type:Abstract
Geographic Code:1USA
Date:Sep 22, 2018
Previous Article:The Response of U.S. Investment to Oil Price Shocks: Does the Shale Boom Matter?

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters