Real-time forecasting revisited: letting the data decide.
Business Economics (2013) 48, 8-28.
Keywords: real-time forecasting, real-time estimates, nowcasting, GDP estimates, economic indicators
The decisions of business planners and policymakers often hinge on the current performance and future outlook for the economy. Private and public forecasters therefore have great interest in estimating and forecasting key economic variables, with a particular focus on aggregate economic activity as measured by real GDP growth. The breadth of private sector involvement in economic forecasting in the United States is illustrated by the well-known Blue Chip Economic Indicators publication, which presents the economic forecasts for 55 private-sector forecasters, and the public policy forecasts of the Office of Management and Budget, the Congressional Budget Office, and of the Federal Reserve that underpin the fiscal and monetary policy outlooks. Most short-run forecasts (one to two years) and intermediate- to longer-range projections (5-10 years and beyond) are model-based forecasts, and they "jump off- from historical data and current-quarter values. The contemporaneous, current-quarter performance of the economy is of particular importance for current policy purposes and also as the base for the future projections. Various alternative approaches exist for making current-quarter estimates for real GDP growth. They include:
* behavioral equation and model-based forecasts with judgmental add factors;
* specific-component accounting (also known as "bean counting"), as illustrated by the Bureau of Economic Analysis' (BEA) "Key source data and assumptions" supplemental estimates;
* indicator-based estimates, including those based on coincident activity indices;
* Real-time or "nowcasting" data-based estimation approaches.
Real-time forecasts produce estimates for current-quarter real GDP growth, typically based on a centered value from a set of estimates from incoming indicators; they are usually intended to be data-based and to not include forecaster judgment or add factors.
This paper presents a new estimating system we have developed for making real-time GDP fore-casts--describing the methodology, estimation process, and software code. This system is more flexible than previous real-time forecasting approaches and allows the data to decide the "best" real-time GDP forecast for varying sample horizons and varying specifications by indicator through time. Estimation methodologies in this research area--and prior versions of the system we use--typically have been constrained by using various ex ante or predetermined fixed relationships; for example a fixed historical sample horizon and fixed empirical specifications for the indicator variables. As in earlier real-time estimating analyses, our system uses the data on key indicators as they become available (and in uneven timing during the quarter) to generate an estimate of current-quarter real GDP growth, based on the estimated historical indicator-GDP relationships, as well as for the proper data-availability point during the quarter. The weights we use for combining the indicator estimates are determined by the fits of the historical relationships.
An important innovation in the system we present is that the improved system now searches across a variety of specifications and across sample horizons--choosing the "best" specification by indicator for a given sample. This is done by using a minimum Schwarz criterion, while also searching for the best sample horizon for minimizing the mean absolute error for a recent prediction period. Hence, the system is now much more flexible in determining the specifications and sample periods for the indicator-specific estimating equations, both within a given quarter for varying data availability and across quarters through time as the system is used in ongoing real-time estimating efforts. And, while research in the area of real-time forecasting and nowcasting has been conducted at varying levels of theoretical and mathematical rigor and sophistication--and in many cases with a focus on academic audiences--we believe our approach is one that can be readily understood and useful for the applied and ongoing analyses that are typical for most business and public sector economists and forecasters.
The intent of the paper is to be organized in a manner and to present information such that interested forecasters and researchers can gain useful information on how the system works and better understand how it can be used in practice. Following this introduction, Section 1 of the paper addresses real-time estimation methodology and some other literature in this research area. Section 2 describes the modeling and estimation process for the system and discusses some of the EViews software code used to implement the system. Section 3 presents illustrative results for the application of the system for a specific recent quarter and the evolution of real-time GDP estimates during that quarter. Section 4 examines the prediction properties for the system estimate and for the individual indicator-specific forecasts. Section 5 discusses other potential applications for our real-time estimation system, with a specific application to show how the system can be used in an analogous approach for real-time predictions of the monthly change in payroll jobs. Section 6 further considers what useful information can be gleaned from these types of estimates. Section 7 presents concluding observations.
1. General Methodology and Other Research and Literature
The research and literature for "real time" forecasting and "nowcasting" have been expanding rapidly over the past decade. (1) Stock and Watson  surveyed the "theoretical and empirical research on methods for forecasting economic time series variables with many predictors," describing how that "provides the opportunity to exploit a much richer base of information than is conventionally used for time series forecasting." Kitchen and Monaco  described an early version of a real-time forecasting system "adopted at the U.S. Treasury to use the broad variety of incoming data to construct 'real-time' estimates of quarterly real GDP growth." Evans  used a comprehensive analysis to calculate daily real-time estimates by "modeling the growth in GDP as the quarterly aggregate of an unobserved daily process for real economy-wide activity" with "model parameters ... estimated by (quasi) maximum likelihood using the Kalman filter algorithm." Giannone, Reichlin, and Small  presented an estimating methodology for producing current-quarter forecasts by adapting a common factors model, combining "the idea of 'bridging' monthly information with the nowcast of quarterly GDP with the idea of using a large number of data releases within a single statistical framework." In particular, they clearly addressed the challenge from the evolving nature of the incoming data of the current quarter:
In real time, some data series have observations through the current period, whereas for others the most recent observations may be available only for a month or quarter earlier. Consequently, the underlying data sets are unbalanced. Appropriately dealing with this "jagged edge" feature of the data is key for producing a nowcast that, by exploiting information in the most recent releases, has a chance to compete with judgmental forecasts. [Gian none, Reichlin, and Small 2008, p. 666] (2)
Their approach allows for the nowcast to be conditioned on a large number of variables. A variety of studies have examined real-time forecasting for euro area GDP and activity [Golinelli and Parigi, 2008; Giannone, Reichlin, and Simonelli 2009; Runstler and others 2009; Bulligan, Golinelli, and Parigi 2010; Angelini and others 2011; Drechsel and Maurin, 2011]. (3)
The system that we present and use for the analysis of this paper is a descendent of the earlier model system described in Kitchen and Monaco . The Kitchen and Monaco system estimated the indicator-specific historical relationships between monthly indicators and real GDP growth (while properly accounting for indicators' intraquarter data availability), produced current-quarter indicator-specific forecasts from the estimated relationships, and then combined the individual indicator forecasts by weighting according to the strength of the indicators' historical relationships to real GDP growth. The software code for the system we use in this system, while maintaining much of the general methodology, was a complete rewriting relative to the prior system in order to incorporate more estimation flexibility and an inherent decision process within the system for choosing the best estimations by specifications and sample horizons. The prior methodology was primarily a constrained "estimation system" that ran the various estimating equations for largely ex ante user-specified and predetermined specifications and sample periods, which then generated the system estimate from those estimations. The current system is more of an "estimation and decision system" in that it iteratively runs through various alternative specifications and samples for the estimating equations, and within that process it makes the decisions for the "best" specifications and samples to use in the final estimation. (4) The choices in the system are based on the minimum Schwarz criterion to determine the best specification for a given sample size, then using the minimum absolute error for the prediction for real GDP growth for a recent period of time to determine the best sample (and for the best specification for that sample). Hence, this approach is much more in the spirit of "letting the data decide," allowing for the estimation process to be determined in an ongoing and evolving real-time analysis, largely independent of subjective ex ante user-specified relationships. A significant part of the software code needed to implement both the prior system and the current system involves addressing the "jagged edge" problem of the availability of data in the current quarter, and assuring the proper estimation given that problem. The methodology and system operation are described in more detail below.
2. Modeling, Estimation, and Software Code Specifics
This section of the paper provides a description of the operation of the system we use and how it is implemented through software code. Our real-time forecasting system is written in EViews software code--in 612 lines of code (653 lines if comments are included). We are currently running the system in EViews 7.1. The following discussion of this section describes how the system:
* reads in the data and generates quarterly based variables from the monthly data set, specifically accounting for the different numbers of months of data available by indicator during the quarter;
* estimates alternative empirical specifications for the indicator-real GDP growth relationships and searches across those specifications for the "best" specification by indicator for a given sample size according to the Schwartz criterion statistic (and while properly accounting for the given amount of data available during the quarter, that is, one, two or three months of data);
* iterates through rolling sample sizes for up to 101 prior quarters of history and chooses the preferred specification-sample equation based on the minimum mean absolute error for predicted GDP growth for an eight-quarter period prior to the current quarter;
* generates a "centered value" real-time estimator from the collection of indicator-based estimates according to relative weightings based on the R-squared values of the indicator estimated equations;
* writes the output table for the real GDP growth prediction by indicator, along with the specification and sample size used by indicator.
The data and differing frequencies
We use data on major macroeconomic and financial data series as key indicators from which to construct the estimates of current-quarter real GDP growth. Most data series are of monthly frequency--for example, data on payroll jobs, the unemployment rate, industrial production, retail sales, orders and shipments, and others. Other data series are weekly (unemployment insurance claims) or even daily or continuous (stock market indices). The system as currently maintained uses data on 24 indicators--generally well-recognized series (and that are presented in detail in tables for examples discussed below).
We maintain a data base of monthly frequency, updating the data series on an ongoing basis as new data become available--reflecting the "jagged edge" of the current period data as described above. For the higher frequency series such as unemployment insurance claims and the stock market indices, we update the monthly average values for available data and use a random walk assumption that any not-yet-observed values for a given month are equal to the last observed value. That allows us to use a variable with initial observations early in a month to provide information for the beginning of that period. A second database transforms the monthly time-series vectors of the first database into quarterly data series with monthly elements (Table I). For example, in the second database, three separate payroll jobs series exist for a given quarter: for the first month of the quarter; the second month of the quarter; and for the third month of the quarter. That quarterly database allows for an easier way to read the data into the EViews computer program and to manage and transform the data once it is in the program. This approach assures the proper accounting for the availability of data by series (the proper part of the jagged edge)--and correct estimation over time for that given real-time availability--yet within a quarterly frequency that aligns with the dependent variable of interest, real GDP growth. Although the system has the monthly and quarterly frequency aspects, it could potentially be run at any time--even on a daily or intradaily basis--to update estimates as new data arrive.
The EViews program begins by creating a work file and then reading in the (quarterly frequency) data from the second database described above. Initial transformations of variables are then made through GENR statements--for example, transforming nominal series to real series and creating the proper quarterly average series for the cases of varying data availability for differing monthly stages of the quarter.
Choosing from alternative specifications
The estimations within the modeling framework identify the relationship between a specific indicator variable and real GDP growth during the current quarter. The specifications therefore use the dependent variable of [y.sub.t] = real GDP percentage change during the quarter at an annual rate, and right-hand-side variables as various transformed forms of [x.sub.it] = level of the ith indicator variable. The EViews computer code uses sequential FOR-NEXT loops to run through the alternative specifications by indicator variable (and separate sets of the statements by the stage of the data availability of the "jagged edge" for the quarter) and tests for the best specification for a given sample size by comparing the Schwarz criterion statistics. The specifications currently considered in the program (and by numerical value denoting the specification in the output) are:(5)
1. First difference specification:
[y.sub.t] = [alpha] + [beta]([x.sub.i,t]--[x.sub.i,t--1]) + [e.sub.t].
2. First and lagged differences specification:
[y.sub.t] = [alpha] + [[beta].sub.1]([x.sub.i,t]--[x.sub.i,t--1]) + [[beta].sub.1]([x.sub.i,t--1]--[x.sub.i,t--2]) + [e.sub.t].
3. Percentage change specification:
[y.sub.t] = [alpha] + [beta][[x.sub.i,t]/[x.sub.i,t--1]]--1] + [e.sub.t].
4. Lagged percentage change specification:
[y.sub.t] = [alpha] + [beta][[x.sub.i,t--1]/[x.sub.i,t--1]]--2] + [e.sub.t].
5. Current level and first difference:
[y.sub.t] = [alpha] + [beta][x.sub.i,t] + [[beta].sub.2]([x.sub.i,t]--[x.sub.i,t--1]) + [e.sub.t].
6. Current and lagged percentage changes specification:
Table 1. Monthly and Quarterly Data Bases Monthly Quarterly Database database Date Variable Variable2 ... Date GDP 1 YearMonth YearMonth YearMonth ... YearQuarter YearQuarter ... ... ... ... Y(t--1)M(10) V1(t--l, V2(t--l, ... 10) 10) Y(t--1)M(11) V1(r--1, V2(t--1, ... 11) 11) Y(t--1)M(12) V1(t--l, V2(t--l, ... ... ... 12) 12) Y(t)M(1) V1(t, 1) V2(M) ... Y(t--1)Q(4) GDP(t--1, 4) Y(t)M(2) V1(t, 2) V2(t, 2) ... Y(t)Q(l) GDP(t, l) Y(t)M(3) V1(t, 3) V2(t, 3) ... ... ... ... ... ... ... Monthly Database Date Variable Variable Variable Variable2 ... 1, Month 1, Month2 1, Month3 Month 1 1 YearMonth YearMonth YearMonth YearMonth YearMonth ... Y(t--1)M(10) Y(t--1)M(11) Y(t--1)M(12) ... ... ... Y(t)M(1) V1(t--1, V1(t--1, V1(t--1, V2(t--1, ... 10) 11) 12) 10) Y(t)M(2) V1(t, 1) V1(t, 2) V1 (t, 3) V2(t, l) ... Y(t)M(3) ... ... ... ...
[y.sub.t] = [alpha] + [[beta].sub.1][[x.sub.i,t]/[x.sub.i,t--1]]--1 + [[beta].sub.2]([x.sub.i,t--1]--[x.sub.i,t--2]) + [e.sub.t].
Although the above specifications that we have included in the system code provide a relatively good range of alternatives, the system obviously could consider an even greater number and range of alternative specifications and thereby potentially make the system even more encompassing in terms of letting the data decide.(6) Nonetheless, we are comfortable that the range of specifications we have included for the analysis and presentation in this paper provides a good illustration of the potential for alternative specification choices and the way the system works, and also is consistent with a principle of parsimony.
Choosing the sample: Iterations through rolling sample sizes
The system uses nested IF-THEN-ELSE statements by variable to compare the prediction performance for specifications across sample sizes, and then chooses the best equation by sample and by specification. The iterations across sample sizes start from a minimum sample of at least 28 quarters, up to a maximum of 101 quarters; while these parameters are somewhat subjective, the minimum sample size contains data from across a business cycle at least and the maximum covers up to 25 years of data. The best relative prediction performance is determined by the lowest mean absolute error for the predicted values of real GDP growth for an eight-quarter period prior to the current quarter. Further, because in real time, estimates for real GDP growth in recent quarters are typically "preliminary" estimates, the most-recent prior quarter is not included in the eight-quarter prediction comparison period. For example, if the system is generating a predicted value for the second quarters of 2012, the eight-quarter period for prediction comparison ends two quarters previously, the fourth quarter of 2011.
Generating the centered value--Relative [R.sup.2] weighting
After the "best" individual indicator-specific estimates are determined, the system determines the centered value for the estimate from the real-time system by weighting individual indicator estimates according to a relative [R.sup.2] weighting method, using the [R.sup.2] values from the chosen indicator-specific estimating equations. The weight for the ith indicator estimate for period t, [w.sub.i,t] for k indicators in the system, is then given by:
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1)
and the system estimate is given by:
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (2)
where the subscript s represents the "system" value and the [^.y.sub.i,t] term is the real GDP growth forecast for indicator [x.sub.i,t] determined from the search across the alternative specifications as described above. Note that the [^.y.sub.i,t] estimates and the [w.sub.i,t] weights will vary through time and also can vary within a period as the data availability varies across (the jagged edge of) the current quarter (and as the program-determined "best" specification potentially changes as well). This weighting methodology gives greater weighting to the indicator estimates that have a stronger historical relationship in predicting real GDP growth and also assures the sum of the weights will be equal to 1.0. Note that a potential alternative weighting methodology could use the inverse of the mean absolute errors from the recent prediction comparison period that was used in the system to determine the best sample-specification combination. Such an approach would determine the weights based on the recent prediction performance rather than the fit over the historical sample. Our initial examination of results from the two approaches indicates that there is little difference between the weighted predictions of those two approaches.(7)
The output table
The system code finishes by writing the output table listing the individual indicators and the system-generated predicted value for current-quarter real GDP growth. To provide additional information by indicator, the output includes the R-squared for the estimation equation used; a number assigned to identify the specification employed; and the sample size over which the prediction equation was estimated. These results can be seen in Tables 2-7 and are discussed in the next section.
3. Illustrative Results--and Evolution of Estimates during a Quarter
To illustrate the performance of the system, Tables 2-7 present a collection of six different system output tables at varying times for making estimates for real GDP growth for the second quarter of 2012, incorporating the annual NIPA revisions of July 2012. The 24 indicator variables in the system can be seen in the tables.
Table 2 shows an initial system estimate at the end of April 2012--that is, at the end of the first month of the second quarter, when only very initial observations on any data for the quarter could be observed. At that point, the only indicator data available for the quarter were: initial and continuing claims for unemployment insurance for the first several weeks of April; consumer confidence and consumer sentiment for April; the Philadelphia Fed business activity index for April; and the S&P500 stock index for most of April. The data thus represented only a very initial snapshot for the beginning of the quarter. That is why it is also very important that the estimating and forecasting equations from the system only use historical data for the first month of the historical quarters and not full three-month quarterly data, properly accounting for the "jagged edge." Although one would need to be cautious in interpreting results from such a limited, initial set of data, the first result from the system at that time showed a weighted estimate of about 2.1 percent for real GDP growth in the second quarter. Such a result could be interpreted as the very initial data indicating that real GDP growth was on track to be positive and at a moderate rate.
Table 2. Real-time Forecast Results for 2012.6)2, April 30 Indicator Prediction [R,sup.2] Specification Sample (%pts) S&P 500 Index 3.90 0.368 3 55 Philadelphia 2.71 0.351 5 68 Fed Index Continuing UI 2.60 0.599 2 29 Claims Consumer 1.94 0.121 1 48 Confidence Consumer 1.62 0.212 5 82 Sentiment Unemp. Ins. 0.83 0.787 6 35 Claims Housing Starts -- -- -- -- ISM Non-Mfg. -- -- -- -- Index ISM Purchasing -- -- -- -- Mgrs. Index Unemployment -- -- -- -- Rate Aggregate -- -- -- -- Worker Hours Industrial -- -- -- -- Production Payroll jobs -- -- -- -- Shipments, --- -- -- -- Durables Real -- -- -- -- Consumption Expenditures Real Retail -- -- -- -- Sales NAHB HMI -- -- -- -- Work Week -- -- -- -- Real Exports -- -- -- -- Payroll Jobs -- -- -- -- Diffusion Real --- -- -- -- Residential Construction NFIB Good Time -- -- -- -- Durable Goods -- -- -- -- Orders Real -- -- -- -- Nonresidential Construction Real-time Forecast, RTFU, Current Quarter GDP Growth(1) Median Forecast Indicator Weighted Contribution (%pts) S&P 500 Index 0.589 Philadelphia 0.390 Fed Index Continuing UI 0.639 Claims Consumer 0.096 Confidence Consumer 0.141 Sentiment Unemp. Ins. 0.268 Claims Housing Starts -- ISM Non-Mfg. -- Index ISM Purchasing -- Mgrs. Index Unemployment -- Rate Aggregate -- Worker Hours Industrial -- Production Payroll jobs Shipments, -- Durables Real -- Consumption Expenditures Real Retail -- Sales NAHB HMI -- Work Week -- Real Exports -- Payroll Jobs -- Diffusion Real -- Residential Construction NFIB Good Time -- Durable Goods -- Orders Real -- Nonresidential Construction Real-time 2.12 Forecast, RTFU, Current Quarter GDP Growth(1) Median 2.27 Forecast (1.) Weights determined by relative [R.sup.2] weighting. Specification reference for templar: 1 is first difference; 2 is first and lagged differences; 3 is percent charm; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
Table 3 shows how the system estimates changed as more data came in over the subsequent weeks, through the middle of May--and notably now including several of the more-recognized data reports on the economy for April, including the Employment Situation release from the Bureau of Labor Statistics (BLS), industrial production, and retail sales. With that additional data (but again only primarily for the first month of the quarter), the system estimate for real GDP growth was at 2.4 percent. The individual indicators generated a fairly large range of estimates, from 1.1 percent for the housing market index from the National Association of Home Builders up to 4.05 percent from the S&P500 stock index.
Table 3. Real-time Forecast Results for 2012.Q2, May 17 Indicator Prediction [R.sup.2] Specification Sample (%pts) S&P 500 Index 4.05 0.313 3 101 ISM Purchasing 3.44 0.236 5 85 Mgrs. Index Industrial 3.04 0.669 1 35 Production Real Retail 2.92 0.587 3 46 Sales Housing Starts 2.87 0.326 4 29 Unemployment 2.85 0.450 2 47 Rate Continuing UI 2.70 0.684 2 29 Claims Payroll jobs 2.48 0.475 5 58 Payroll Jobs 2 14 0.082 4 49 Diffusion Work Week 2.13 0.208 5 55 Philadelphia 1.96 0.398 5 96 Fed Index Consumer 1.94 0.121 1 48 Confidence Unemp. Ins. 1.91 0.831 6 30 Claims Aggregate 1.81 0.566 5 29 Worker Hours Consumer 1.77 0.249 5 83 Sentiment NFIB Good Time 1.62 0.055 4 31 ISM Non-Mfg. 1.42 0.329 5 45 Index NAHB HM1 1.10 0.260 5 36 Shipments, -- -- -- -- Durables Real -- -- -- -- Consumption Expenditures Real Exports -- -- -- -- Real -- -- -- -- Residential Construction Durable Goods -- -- -- -- Orders Real -- -- -- -- Nonresidential Construction Real-time Forecast, RTFU. Current Quarter GDP Growth(1) Median Forecast Indicator Weighted Contribution (%pts) S&P 500 Index 0.185 ISM Purchasing 0.119 Mgrs. Index Industrial 0.297 Production Real Retail 0.251 Sales Housing Starts 0.137 Unemployment 0.188 Rate Continuing UI 0.270 Claims Payroll jobs 0.172 Payroll Jobs 0.026 Diffusion Work Week 0.065 Philadelphia 0.114 Fed Index Consumer 0.034 Confidence Unemp. Ins. 0.232 Claims Aggregate 0.150 Worker Hours Consumer 0.064 Sentiment NFIB Good Time 0.013 ISM Non-Mfg. 0.068 Index NAHB HM1 0.042 Shipments, -- Durables Real -- Consumption Expenditures Real Exports -- Real -- Residential Construction Durable Goods -- Orders Real -- Nonresidential Construction Real-time 2.43 Forecast, RTFU. Current Quarter GDP Growth(1) Median 2.14 Forecast (1.) Weights determined by relative R-squared weighting. Specification reference for expl var: 1 is first difference; 2 is first and lagged differences; 3 is percent change; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
By the beginning of June (Table 4) initial information for at least the first month was available for most indicators, and several had begun to have a second month as well. The system estimate for real GDP growth was at about 2.2 percent, with a range across the indicators of 0.3 to 3.7 percent. By the end of June (Table 5)--the actual end of the second quarter, but still before all data for the quarter were available--most of the indicator variables had two months of data for the quarter, some had three months (unemployment claims, consumer confidence and sentiment, and S&P500 index), and a couple still had only one month (construction and exports). The system at that time showed a weighted estimate of 1.97 percent.
Table 4. Real-time Forecast Results for 2012.Q2, June 1 Indicator Prediction R2 Specification Sample Weighted (%pts) Contribution (%pts) S&P 500 Index 3.72 0.332 3 99 0.123 Real 3.53 0.321 6 85 0.113 Residential Construction Industrial 3.02 0.665 1 35 0.201 Production Real 2.99 0.618 3 37 0.185 Consumption Expenditures Real Retail 2.91 0.584 3 46 0.170 Sales Housing Starts 2.84 0.319 4 29 0.090 Continuing UI 2.70 0.682 2 29 0.184 Claims Unemployment 2.60 0.516 2 48 0.134 Rate Real 2.53 0.324 5 29 0.082 Nonresidential Construction ISM Purchasing 2.47 0.463 5 48 0.114 Mgrs. Index Payroll jobs 2.31 0.503 5 65 0.116 Shipments. 1.98 0.602 3 34 0119 Durables Philadelphia 1.96 0.398 5 96 0.078 Ked Index Consumer 1.86 0.250 5 85 0.046 Sentiment Unerap. Ins. 1.83 0.828 6 30 0.151 Claims Consumer 1.78 0.207 5 83 0.037 Confidence NFIB Good Time 1.62 0.058 4 31 0.009 Payroll Jobs 1.60 0.231 5 48 0.037 Diffusion ISM Non-Mfg. 1.42 0.327 5 45 0.046 Index Aggregate 1.29 0.622 5 35 0.080 Worker Hours NAHB HMI 1.09 0.259 5 36 0.028 Work Week 0.62 0.226 1 46 0.014 Durable Goods 0.32 0.680 3 29 0.022 Orders Real Exports -- -- -- -- 0.000 Real-time 2.18 Forecast, RTFU, Current Quarter GDP Growth Median 1.98 Forecast (1.) Weights determined by relative [R.sup.2] weighting. Specification reference for expl var: 1 is first difference; 2 is first and lagged dilTerences; 3 is percent change; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change. Table 5. Real-time Forecast Results for 2012.Q2, June 28 Indicator Prediction [R.sup.2] Specification Sample (%pts) S&P 500 Index 3.93 0.332 3 99 Real 3.53 0.321 6 85 Residential Construction Consumer 3.39 0.228 3 94 Confidence Real 2.99 0.618 3 37 Consumption Expenditures Housing Starts 2.98 0.362 6 50 Unemployment 2.60 0.516 2 48 Rate Real 2.53 0.324 5 29 Nonresidential Construction Continuing Ul 2.52 0.747 5 29 Claims ISM Purchasing 2.47 0.463 5 48 Mgrs. Index Payroll jobs 2.31 0.503 5 65 Industrial 2.24 0.701 I 29 Production Shipments, 2.18 0.684 1 34 Durables Real Retail 2.16 0.669 3 44 Sales Unemp. Ins. 1.73 0.825 6 29 Claims Payroll Jobs 1.60 0.231 5 48 Diffusion Consumer 1.53 0.262 5 85 Sentiment NFIB Good Time 1.44 0.058 4 29 ISM Non-Mfg. 1.38 0.413 5 59 Index Philadelphia 1.29 0.426 5 101 Fed Index Aggregate 1.29 0.622 5 35 Worker Hours NAHB HMI 0.85 0.242 5 36 Work Week 0.62 0.226 1 46 Durable Goods 0.32 0.768 3 29 Orders Real Exports --0.26 0.473 5 80 Real-time Forecast. RTFU, Current Quarter GDP Growth(1) Median Forecast Indicator Weighted Contribution (%pts) S&P 500 Index 0.118 Real 0.103 Residential Construction Consumer 0.070 Confidence Real 0.168 Consumption Expenditures Housing Starts 0.098 Unemployment 0.122 Rate Real 0.074 Nonresidential Construction Continuing Ul 0.171 Claims ISM Purchasing 0.104 Mgrs. Index Payroll jobs 0.105 Industrial 0.143 Production Shipments, 0.135 Durables Real Retail 0.131 Sales Unemp. Ins. 0.130 Claims Payroll Jobs 0.034 Diffusion Consumer 0.036 Sentiment NFIB Good Time 0.008 ISM Non-Mfg. 0.052 Index Philadelphia 0.050 Fed Index Aggregate 0.073 Worker Hours NAHB HMI 0.019 Work Week 0.013 Durable Goods 0.022 Orders Real Exports -0.011 Real-time 1.97 Forecast. RTFU, Current Quarter GDP Growth(1) Median 2.17 Forecast (1.) Weights determined by relative [R.sup.2] weighting. Specification reference for expl var: I is first difference; 2 is first and lagged differences; 3 is percent change; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
The final two tables (Tables 6 and 7) for the estimates for the quarter show the system estimates at the end of July-1.8 percent--and at the end of August-1.8 percent. At the end of July, the BEA released the advance estimate of real GDP growth for the second quarter, at 1.5 percent--compared with the real-time system estimate of 1.8 percent at that time and the market consensus for 1.2 percent. By the end of August, the BEA released its second estimate, at 1.7 percent, compared with the real-time system estimate of 1.8 percent and a market consensus of 1.7 percent. By the end of September, the BEA released its final estimate of 1.3 percent for real GDP growth for the second quarter of 2012.
Table 6. Real-time Forecast Results for 2012.Q2, July 26 Indicator Prediction [R.sup.2] Specification Sample (%pts) S&P 500 Index 3.94 0.332 3 99 Housing Starts 3.91 0.314 6 101 Consumer 3.39 0.228 94 Confidence Real 3.36 0.388 6 89 Residential Construction Payroll Jobs 2.70 0.267 5 43 Diffusion Shipments, 2.65 0.655 I 47 Durables Continuing UI 2.49 0.747 2 29 Claims Real 2.34 0.312 5 29 Nonresidential Construction Unemployment 2.21 0.673 2 29 Rate ISM Purchasing 2.14 0.476 5 47 Mgrs. Index Industrial 1.87 0.742 1 29 Production Unemp. Ins. 1.73 0.825 6 29 Claims Real 1.72 0.537 6 101 Consumption Expenditures NFIB Good Time 1.55 0.055 4 30 Consumer 1.50 0.262 1 85 Sentiment Payroll jobs 1.45 0.498 3 45 Work Week 1.32 0.291 1 50 Philadelphia 1.29 0.426 5 101 Fed Index ISM Non-Mfg- 1.19 0.552 45 Index Aggregate 1.15 0.469 3 49 Worker Hours Real Retail 1.12 0.772 3 29 Sales NAHB HMI 0.85 0.242 5 36 Real Exports 0.65 0.570 1 30 Durable Goods 0.49 0.799 3 29 Orders Real-time Forecast, RTFU, Current Quarter GDP Growth(1) Median Forecast Indicator Weighted Contribution (%pts) S&P 500 Index 0.114 Housing Starts 0.107 Consumer 0.068 Confidence Real 0.114 Residential Construction Payroll Jobs 0.063 Diffusion Shipments, 0.152 Durables Continuing UI 0.163 Claims Real 0.064 Nonresidential Construction Unemployment 0.130 Rate ISM Purchasing 0.089 Mgrs. Index Industrial 0.121 Production Unemp. Ins. 0.125 Claims Real 0.081 Consumption Expenditures NFIB Good Time 0.007 Consumer 0.034 Sentiment Payroll jobs 0.063 Work Week 0.034 Philadelphia 0.048 Fed Index ISM Non-Mfg- 0.057 Index Aggregate 0.047 Worker Hours Real Retail 0.076 Sales NAHB HMI 0.018 Real Exports 0.032 Durable Goods 0.034 Orders Real-time 1.84 Forecast, RTFU, Current Quarter GDP Growth(1) Median 1.73 Forecast (1.) Weights determined by relative [R.sup.2] weighting. Specification reference for expl var: 1 is first difference; 2 is first and lagged differences; 3 is percent change; 4 is lagged percent change: 5 is current level and first difference: 6 is current and lagged percent change. Table 7. Real-time Forecast Results for 2012.Q2, August 30 Indicator Prediction [R.sup.2] Specification Sample (%pts) Real 3.98 0.324 2 84 Residential Construction Housing Starts 3.03 0.434 6 29 Payroll Jobs 2.92 0.229 5 48 Diffusion S&P 500 Index 2.68 0.501 3 48 Unemployment 2.42 0.505 2 48 Rate Real 2.36 0.313 5 29 Nonresidential Construction Continuing Ul 2.34 0.676 2 29 Claims Real Exports 2.12 0.522 1 48 ISM Purchasing 2.10 0.439 5 48 Mgrs. Index Shipments. 1.98 0.695 1 29 Durables Industrial 1.89 0.664 1 29 Production Real 1.77 0.551 6 101 Consumption Expenditures Unemp. Ins. 1.65 0.770 6 29 Claims Consumer 1.63 0.131 1 50 Confidence Consumer 1.54 0.264 5 85 Sentiment NFIB Good Time 1.46 0.065 4 29 Philadelphia 1.35 0.404 5 98 Fed Index ISM Non-Mfg. 1.27 0.530 5 45 Index Payroll jobs 1.23 0.569 3 33 Work Week 1.21 0.299 5 64 Real Retail 1.00 0.758 3 29 Sales Aggregate 0.92 0.444 3 50 Wtirker Hours NAHB HMI 0.68 0.191 5 48 Durable Goods 0.59 0.688 3 29 Orders Real-time Forecast. RTFU, Current Quarter GDP Growth(1) Median Forecast Indicator Weighted Contribution (%pts) Real 0.118 Residential Construction Housing Starts 0.120 Payroll Jobs 0.061 Diffusion S&P 500 Index 0.122 Unemployment 0.111 Rate Real 0.067 Nonresidential Construction Continuing Ul 0.144 Claims Real Exports 0.101 ISM Purchasing 0.084 Mgrs. Index Shipments. 0.125 Durables Industrial 0.114 Production Real 0.089 Consumption Expenditures Unemp. Ins. 0.116 Claims Consumer 0.019 Confidence Consumer 0.037 Sentiment NFIB Good Time 0.009 Philadelphia 0.050 Fed Index ISM Non-Mfg. 0.061 Index Payroll jobs 0.064 Work Week 0.033 Real Retail 0.069 Sales Aggregate 0.037 Wtirker Hours NAHB HMI 0.012 Durable Goods 0.037 Orders Real-time 1.80 Forecast. RTFU, Current Quarter GDP Growth(1) Median 1.71 Forecast (1.) Weights determined by relative [R.sup.2] weighting. Specification reference for expl var: 1 is first difference; 2 is first and lagged differences; 3 is percent change; 4 is lagged percent change: 5 is current level and first difference; 6 is current and lagged percent change.
In summary for this section, the results and comparisons for the evolution of system estimates for the second quarter of 2012 illustrate how the system performed in providing information about real GDP growth, from a beginning of a very limited set of information on few indicators through the accumulation of additional information across indicators until information for all months for all indicators was available--and until final estimates from BEA were made.
4. Properties of the Estimators Using Current-Vintage Data
This section presents information on the properties of the estimates using current-vintage data; we use the current-vintage data in the absence of a long time series of estimates based on the contemporaneously available real-time data at the rolling historical data vintages at the initial observation points. Although some researchers may have a preference for using historical real-time data for evaluating the estimates, note that there is rarely (and sometimes never) a pure apples-to-apples comparison for data and analysis through time, and especially relative to the observed data at another point in time historically. Data definitions, methodologies, sources, and so on, all change through time, and that is particularly true for an indicator such as real GDP growth, which is the focus of much of the attention in real-time analysis. Hence, for generating current estimates in real time for the current quarter, there are strong arguments that it is best to use relationships as identified in the current vintage of data that reflects current in-place methods and construction.
To examine the properties of the system and indicator estimates, we ran the system to generate estimates over the historical period from 1997:Q1 through 2011:Q4 using current-vintage data. Table 8 presents the root-mean-square errors (RMSEs) for the estimates, showing various results that will be discussed in turn. Of particular note, the results in the left-hand side of Table 8 show the results for the flexible estimation system that is the primary focus of this paper; those on the right-hand side of Table 8 provide a comparison to a system in which the specifications and sample horizons are fixed. The top of the table presents results for system estimates; the bottom of the table shows results for indicator-specific estimates.
Table 8. Root Mean Square Errors by Predictor Flexible System Fixed System Predictor or RMSE Predictor or RMSE indicator (%pts) indicator (%pts) System estimator 1.59 System estimator 1.76 System estimator, 1.38 System estimator, 1.52 adjusted adjusted Autoregressive 2.32 Autoregressive 2.39 (AR(2)) estimator (AR(2)) estimator Top 5 Indicators, 1.53 Top 5 Indicators, 1.71 equal weighting equal weighting Top 5. adjusted 1.47 Top 5. adjusted 1.54 Continuing Ul Claims 1.70 Industrial 1.88 Production Shipments. Durables 1.75 Unemp. Ins. Claims 1.89 Real Retail Sales 1.75 ISM Non-Mfg. Index 2.00 Industrial 1.76 Shipments. Durables 2.01 Production Durable Goods Orders 1.77 Real Consumption 2.03 Expenditures Unemp. Ins. Claims 1.80 Continuing Ul Claims 2.09 Aggregate Worker 1.83 Unemploymeni Rate 2.11 Hours Unemploymeni Rate 1.85 Durable Goods Orders 2.11 Real Consumption 1.90 Aggregate Worker 2.15 Expenditures Hours ISM Non-Mfg. Index 1.91 Real Exports 2.15 Real Exports 1.91 Philadelphia Fed 2.16 Index Philadelphia Fed 1.97 Payroll jobs 2.16 Index S&P 500 Index 1.99 Work Week 2.20 Payroll jobs 2.06 Real Retail Sales 2.25 ISM Purchasing Mgrs. 2.09 Payroll Jobs 2.31 Index DilTusion Housing Starts 2.27 Real Residential 2.33 Construction Real Residential 2.29 ISM Purchasing Mgrs. 2.41 Construction Index Payroll Jobs 2.29 Consumer Confidence 2.43 DilTusion NFIB Good Time 2.34 Consumer Sentiment 2.45 NAHB HM1 2.35 NFIB Good Time 2.49 Work Week 2.36 NAHB HMI 2.50 Consumer Sentiment 2.40 S&P 500 Index 2.64 Real Nonresidential 2.41 Housing Starts 2.93 Construction Consumer Confidence 2.48 Real Nonresidential 2.98 Construction
Comparison for system and indicator-specific indicators--and a subset of the best indicators
The first line for the flexible system results (left side) shows the RMSE for the system estimate at 1.59 percentage points, which compares favorably with the individual indicator estimates in the bottom of the table, where the lowest RMSE is at 1.70 for continuing unemployment insurance claims. The system estimate also compares favorably with a naive rolling autoregressive model of order 2 [AR(2)], estimation for which the RMSE is 2.32 percentage points as shown in the third line.
An interesting question is the extent to which the full system is better than a small subset of indicators in making the real-time estimate. The fourth line shows the RMSE for an estimate using an equally weighted combination of the "top 5" indicator estimates, (8) based on using the five indicators with the lowest RMSE over the estimation horizon--for the flexible system those "best" indicators are (real) durable goods orders: continuing unemployment insurance claims; (real) shipments of durable goods; industrial production; and real retail sales. An estimate based on those five indicators alone actually slightly outperforms the full system estimate, with an RMSE of 1.53 compared with the 1.59 of the system.
Tests regarding the properties of the system estimate
Questions remain about other properties of the estimates, notably unbiasedness, and in the sense discussed in Diebold and Lopez [1994 "The key property of optimal forecast errors from which all others follow ... is unforecastability on the basis of information available at the time the forecast was made." To examine these questions, we conducted some simple regression tests; the results are presented in Table 9. The first equation of Table 9 shows the results for regressing actual real GDP growth on the system estimate (using ordinary least squares); if the system estimate is an unbiased predictor then the joint hypothesis that the constant coefficient is zero and the slope coefficient is 1.0 should not be rejected. The estimated constant coefficient is negative and significantly less than zero (at the 5 percent level) and the slope coefficient is significantly greater than one; the F-statistic from a Wald test for testing the joint (0,1) hypothesis is significant at 7.33, rejecting that joint (0,1) hypothesis. The second estimated equation of Table 9 goes further, regressing the fitted error from the first equation on the one-quarter-lagged prediction error for the system estimate. If the system estimate misses the prediction in a given quarter, does that carry information for making a prediction in the next quarter? That is, real GDP growth at times exhibits a "saw tooth" pattern--if real GDP growth is abnormally large one quarter, does real GDP growth tend to be lower in the subsequent quarter? The estimation results show that the estimated coefficient on the prior quarter prediction error is significantly different from zero at the 10 percent level. Further, including the lagged prediction error in the estimated equation (equation (3) of Table 9) results in an estimated coefficient significant at the 5 percent level and the F statistic of 6.77 shows that the joint hypothesis of (0,1,0) is rejected. Although the saw-tooth pattern does not always dominate, when it does the additional role for the lagged predictor can be important, so we include that for making an adjusted estimate.
Table 9. Regression Results for Testing Properties of the Estimators Sample data: 1997:Q/-2011:Q4 Equation Dependent Constant System System F Variable Estimate Estimate Statistic Prediction for Error Testing (t-1) (0,1) 1. Real GDP --1.26 1.37 ** -- 7.33 ** growth ** (0.33) (0.10) -- 2. Error from 0.07 -- --0.23 * -- equation (1) (0.19) -- (0.12) -- 3. Real GDP --1.64 1.48 ** 0.29 ** -- growth ** (0.37) (0.11) (0.13) -- Equation Dependent F Variable Statistic for Testing (0.1,0) 1. Real GDP -- growth -- 2. Error from -- equation (1) -- 3. Real GDP 6.77 ** growth -- *Significant at the 0.10 level; ** signficant at the 0.05 level. Standard errors in parentheses.
Figure 1 shows the historical values for the alternative series--actual real GDP growth, the (flexible) real-time (RT) system estimate, and for the "adjusted" series that accounts for the bias and the information from the prior quarter prediction error. Figure 1 shows that the real-time system estimate tends to underpredict (absolutely) the more-extreme values for real GDP growth, whereas the adjusted measure does a better job of capturing those deviations from trend performance. During periods when real GDP growth is largely per- forming near trend such as occurred during the late-1990s and the mid-2000s--there is little difference in the estimates. In particular, during recession and low-growth periods the RT system estimate understates the extent of GDP growth declines to a greater degree than the adjusted estimates.
Looking back at Table 8 and the RMSEs for the alternative estimates, the second line shows the lower RMSE for the adjusted system estimator, at 1.38, compared with the 1.59 of the unadjusted system estimate. We made an analogous adjustment for the top 5 indicators estimate (results not shown); the fifth line of Table 8 shows that the RMSE is lower for the adjusted estimate for the top 5 case, as well, but with less relative improvement. The lower relative improvement apparently is because the top 5 indicators tend to be more volatile to begin with, and tend to do a somewhat better job during the more extreme growth periods.
In general, the adjusted system estimate yields the lowest RMSE, followed closely by the top 5 adjusted estimate. In the example presented above in Section 3 for the second quarter of 2012, the adjusted estimate would be somewhat lower at 1.2 percent compared with the final unadjusted system estimate of 1.8 percent--and compared with BEA's final 1.3 percent estimate.
Comparison to fixed system estimates
Finally, the right-hand side of Table 8 presents the equivalent set of RMSE numbers for running a version of the system in which the system is constrained to having fixed specifications and fixed sample sizes (80 quarters).(9) The fixed system results almost uniformly have higher RMSE across the measures, illustrating that the flexible estimation process yields better results for the system estimates and for the individual indicator-specific estimates (although there are a couple exceptions among the "worst" performing indicators in the bottom of the table).
5. Other Potential Applications and a Monthly Payroll Jobs Estimation Example
The methodology and system employed here have many potential applications beyond real-time estimates of real GDP growth. We consider applications to a restricted "best set of indicators" version of our system, an alternative approach with separate system estimates by GDP component, specific indicators or sectors, and an illustration of a version of the system as applied to the monthly payroll jobs estimate from the BLS.
An alternative methodology: Using a subset of the best indicators predictions
As observed in the analysis of the prior section, a potential refinement to the methodology would be to restrict the construction of the system estimate to a small subset of the indicators, choosing to use a set of the indicator estimates that had the better prediction performance. As observed, such a change may not yield estimates that are much different from those of the full system. In practice, our view is that we would tend to use the full system estimate, and use the top 5 estimates to help inform our views during particularly volatile growth periods and notably near business cycle turning points. Yet for many analysts and forecasters, the results indicate that tracking the performance of those key indicators may yield most of the information that could be gleaned from a larger, more comprehensive set of indicators. But using a flexible estimation approach such as in our system is also apparently important for best tracking the information from those indicators.
An alternative methodology: GDP by components
Another alternative methodology would be to apply the system estimation procedures to an estimation-by-components approach for the components of the accounting identity for GDP of: C-F /+ G +(X--M). Such a methodology could be promising, especially given the challenges we have observed in repeated use over time of the system estimation, notably that the system may do particularly well in estimating an aggregate indicator such as the change in private domestic real final sales. That is, the system does less well in predicting real GDP growth when there are substantial changes in more-exogenous components such as changes in inventories, government spending, or net exports. The first quarter of 2011 is a notable example; our system estimate overstated real growth in that period as government spending registered a particularly large decline. A components approach would move more toward the "bean counting" accounting approach described earlier; a forecaster could use system-based estimates by component independently or to augment or serve as a check for the bean-counting estimates for current-quarter GDP growth. Although we have not conducted formal investigation of this approach, we have observed that making real-time estimates of some of the components--and notably government spending--is difficult in practice and could limit the ability to use the system in such an application.
Potential applications to specific industries, indices, custom measures
The real-time system we describe in this paper can potentially be applied to almost any series for which sufficient historical data exist to estimate relationships to other indicators. Obvious candidates are the major monthly macroeconomic variables such as: industrial production, durable goods orders and shipments, payroll jobs and the unemployment rate, and so on. For example, Parigi, Golinelli, and Bodo  use a real-time, nowcasting approach for short-run predictions of industrial production for Italy. In fact, in the next section, we illustrate just such an application to the monthly payroll jobs estimate from the BLS. Before turning to that, we note that the potential also exists to construct special indices or measures for a given industry or industries, or even a given firm, and then use the system to regularly update estimated performance/activity based on the system's estimated relationships, and that would then evolve automatically through time. One can even imagine using an applied version of the system to make estimates of a performance measure for a firm or industry that is unobservable in real time, and that may not be directly observable until after a substantial lag of time (for example, potentially even a year or more).
An application to monthly payroll jobs estimation
To illustrate how our real-time forecast system can be applied to an alternative economic indicator other than real GDP growth, we rewrote the EViews code to apply it to one of the more-important and well-known economic indicators, the monthly payroll jobs estimate from the BLS. Given the closer matching of monthly frequencies for most of the explanatory indicators used and the dependent variable of payroll jobs, the main differences in the EViews code were to eliminate the nested loops for making sure the data for the proper month(s) of the quarter were used and making the code apply to a single month rather than multiple months and quarterly averages. Also, we used a subset of key economic indicators that are typically or often available prior to the release of the Employment Situation report and have a sufficiently long time series. Hence, we use data on initial and continuing unemployment insurance claims, the employment measure from the Philadelphia Fed's business outlook survey, consumer confidence and sentiment, the S&P500 index, indices on business activity and employment from the Institute for Supply Management (ISM), and the ADP employment report. Some of these series are available in the weeks of the month prior to and leading up to the Employment Situation release (unemployment insurance claims, the S&P 500 index, consumer sentiment, Philadelphia Fed index), others are only available closer or immediately before (consumer confidence, revised consumer sentiment, ADP), and sometimes some of these indicators are released just before or potentially not released prior to the Employment Situation (ISM survey measures). This group of indicators is generally well-known as the key set of monthly indicators available prior to the Employment Situation, and many are used by analysts and forecasters to inform their personal estimates for payroll jobs growth in advance of the release of the estimates. Again, one of the key aspects of our analysis is the system's computer program and code that allow the flexible and (almost) instantaneous updating and data-determined choices of empirical specifications and sample horizons and for an ongoing process of making payroll jobs estimates from month to month.(10)
Similar to the analysis for the evolution of the system estimate for real GDP, we present the evolution of estimates across a month for payroll jobs for May 2012 (Tables 10-15). Note that the "jagged edge" phenomenon and duration are greatly reduced in this application, spanning one month rather than the three months of the quarter for the real GDP application. Table 10 shows an initial system estimate in the middle of May (May 18) when only very initial observations on any data for the month were observed: data on initial and continuing claims for unemployment insurance for the beginning weeks of May; the preliminary estimate for consumer sentiment for May; the Philadelphia Fed index employment measure; and the S&P500 stock index for the first half of May. The data thus represented an initial snapshot for May, but with key data on unemployment insurance claims available by that time. The system's initial estimate at that time was for payroll jobs growth of 93,000 jobs.
Table 10. Real-time Forecast for Payroll Jobs for May 2012, May 18 Indicator Prediction [R.sup.2] Specification Sample Weighted (jobs) Contribution (jobs) UI CLAIMS 141 0.599 5 252 36 Philly Fed 99 0.659 5 165 28 Empl. Index Continuing 92 0.687 5 99 27 UI Claims S&P500 35 0.095 3 300 I Sentiment 9 0.305 5 246 1 ISM -- -- -- -- -- Non-Mfg Confidence -- -- -- -- -- ISM -- -- -- -- -- Employment ISM Non -- -- -- -- -- Mfg. Empl ADP -- -- -- -- -- Real-time 93 Forecast, Current Month Payroll Jobs Change(1) Median 92 Forecast (1.)Weights determined by relative [R.sup.2] weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
Table 11 shows how the system estimates changed over the following week (through May 25) as more data came in, although not substantial changes, with the system estimate rising only slightly to 95,000. By May 29 (Table 12) consumer confidence was available (and yielding a very low, negative estimate), pulling down the system estimate to 84,000. By the end of May (May 31), Table 13 shows that the addition of the ADP estimate for May (at + 133,000) yielded the highest indicator-specific estimate at 145,000 jobs, and raised the system estimate to 99,000. Note this was a month when the ISM data were not available prior to the Employment Situation release. The market consensus during this period prior to the ADP release was for an increase in payroll jobs of 150,000, with a range of 95,000 to 206,000. On June 1, the BLS released its first estimate for payroll jobs growth in May at 69,000. Hence, in the comparison of this example for May 2012, the real-time system estimates of job growth lower than 100,000--and well below the consensus--were "correct" and relatively better than the market consensus. However, even the relatively low estimates of 84,000 (pre ADP) and 99,000 (post ADP) from the system were high compared with the BLS' initially reported 69,000.
Table 11. Real-time Forecast for Payroll Jobs for May 2012, May 25 Indicator Prediction [R.sup.2] Specification Sample Weighted Gobs) Contribution (jobs) UI CLAIMS 141 0.599 5 252 36 Philly Fed 99 0.659 5 165 28 Empt. Index Continuing 91 0.687 5 99 27 UI Claims S&P500 56 0.095 3 300 2 Sentiment 20 0.305 5 246 3 ISM -- -- -- -- -- Non-Mfg Confidence -- -- -- -- -- ISM -- -- -- -- _ Employment ISM Non -- -- -- -- -- Mfg. Empl ADP -- -- -- -- -- Real-time 95 Forecast. Current Month Payroll Jobs Change (1) Median 91 Forecast (1.) Weights determined by relative R2 weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change. Table 12. Real-time Forecast for Payroll Jobs for May 2012, May 29 Indicator Prediction [R.sup.2] Specification Sample Weighted (jobs) Contribution (jobs) UICLAIMS 141 0.599 5 252 33 Philly Fed 99 0.659 5 165 25 Empl. Index Continuing 91 0.687 5 99 24 UI Claims S&P500 56 0.095 3 300 2 Sentiment 20 0.305 5 246 2 Confidence -19 0.253 5 249 -2 ISM -- -- -- -- _ Non-Mfg ISM -- . -- -- -- -- Employment ISM Non -- -- -- -- -- Mfg. Empl ADP -- -- -- -- -- Real-time 84 Forecast. Current Month Payroll Jobs Change(1) Median 74 Forecast (1.) Weights determined by relative R2 weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change. Table 13. Real-time Forecast for Payroll Jobs for May 2012, May 31 Indicator Prediction [R.sup.2] Specification Sample Weighted (jobs) Contribution (jobs) A DP 145 0.859 1 137 36 UI CLAIMS 131 0.599 5 252 23 Continuing 100 0.687 5 99 20 UI Claims Philly Fed 99 0.659 5 165 19 Empl. Index S&P500 55 0.095 3 300 2 Sentiment 20 0.305 5 246 2 Confidence -19 0.253 5 249 -1 ISM -- -- -- -- -- Non-Mfg ISM -- -- -- -- -- Employment ISM Non -- -- -- -- -- Mfg. Empl Real-time 99 Forecast, Current Month Payroll Jobs Change(1) Median 99 Forecast (1.) Weights determined by relative R2 weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
The system estimates in Tables 14 and 15 include the data on the ISM manufacturing employment index (Table 14) released on June 1 after the Employment Situation release, and the ISM nonmanufacturing business activity and employment indices (Table 15) released on June 5. The estimate from the system with all indicators included is for 105,000 jobs for May, still above the 69,000 initial BLS estimate, but in the bottom of the range of the private market predictions cited above and much closer than the consensus. With subsequent revisions in the June and July reports, the payroll jobs change for May was at 87,000, closer to the system estimate.
Table 14. Real-time Forecast for Payroll Jobs for May 2012, June 1 Indicator Prediction [R.sup.2] Specification Sample Weighted (jobs) Contribution (jobs) ISM 197 0.605 5 137 29 Employment ADP 145 0.859 1 137 31 UI CLAIMS 131 0.599 5 252 19 Continuing 100 0.687 5 99 17 UI Claims Philly Fed 99 0.659 5 165 16 Empl. Index S&P500 55 0.095 3 300 1 Sentiment 20 0.305 5 246 2 Confidence -19 0.253 5 249 -1 ISM -- -- -- -- -- Non-Mfg ISM Non -- -- -- -- -- Mfg. Empl Real-time 114 Forecast. Current Month Payroll Jobs Change(1) Median 100 Forecast (1.) Weights determined by relative R2 weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change. Table 15. Real-time Forecast for Payroll Jobs for May 2012, June 5 Indicator Prediction [R.sup.2] Specification Sample Weighted (jobs) Contribution (jobs) ISM 196 0.603 5 137 25 Employment ADP 144 0.858 1 137 26 UI CLAIMS 131 0.599 5 252 16 Philly Fed 100 0.657 5 165 14 Empl. Index Continuing 99 0.686 5 99 14 UI Claims ISM Non 65 0.390 5 180 5 Mfg. Empl S&P500 55 0.095 3 300 1 ISM 51 0.315 5 180 3 Non-Mfg Sentiment 20 0.306 5 246 1 Confidence -19 0.253 5 249 -1 Real-time 105 Forecast. Current Month Payroll Jobs Change(1) Median 82 Forecast (1.) Weights determined by relative R2 weighting. Specification reference for expl var: 1 is first difference; 2 is first percent change; 3 is first and lagged differences; 4 is lagged percent change; 5 is current level and first difference; 6 is current and lagged percent change.
6. What is the Useful Information in these Estimates?
For the real-time GDP estimates, the estimates are probably best viewed as measuring the underlying performance of GDP growth, of underlying growth in the economy. Often very special, even idiosyncratic, changes to components will yield a real GDP growth number for a specific quarter that is quite different from the overall underlying performance. While there is potentially limited value in correctly predicting a one-time low or high value when the economy is growing at a fundamentally different rate, nonetheless it would still be of value to understand if there were special factors that resulted in an estimate being particularly high or low.
The lamp post problem and private domestic activity
The behavior of temporarily high or low quarterly growth estimates in real time also highlights some of the challenges of making such estimates in our framework and methodology. The effective weighting is determined by the relative performance of the fits of the equations, the R-squares, but in practice that may result in lower weighting for important variables for constructing the real GDP growth estimate. This is a version of the "lamp post" problem, that the data and resulting equations being used are the ones that can be readily observed and through time, while idiosyncrasies and special factors that are more difficult to observe could move the current-quarter estimate relative to what is the typical relationship over history.(11) The observable information may potentially be better in many periods at measuring private domestic activity--sudden shifts in net exports and government are particularly difficult to capture. However, this is not a universal generalization, as various quarters historically had outsized changes in real private domestic final sales that were difficult to predict (notably around the deepest parts of the recent recession). Nonetheless, being able to identify when a particularly low value or high value for a quarter is a fundamental change in underlying performance would be of particular value for business, information, and policy purposes.
Information by sector
The detailed indicator-specific estimates from the real-time estimation system provide information on how specific industries or sectors are performing given their historical relationship to real growth in the economy. For example, what are the estimates and the relative magnitudes from "real economy" indicators, or from labor market indicators, or from sentiment and survey data? Those estimates and relationships can provide potentially useful information on the relative performance of differing parts of the economy. Additional research could examine, for example, whether sentiment vs. real activity disparities carry additional useful information about the current or expected future performance of the economy.(12)
Reliability of the estimates and the data
The preliminary nature of initial and subsequent estimates for real GDP growth--and the relatively long time before such estimates go through annual or comprehensive revisions--highlights some of the challenges of forecasting real GDP growth. What confidence do we have that the reported values of real GDP growth are properly capturing the true performance of the economy in real time? Or, could it be that the real-time estimates from a system such as ours could in fact give a more useful measure of underlying performance because of the direct tie to the broad set of underlying data rather than being tied to the accounting framework and associated initial data limitations for the formal estimates? The broader questions of such challenges are beyond the scope of this paper, but we will note that the BEA publishes a set of comparisons for revisions to GDP in its initial (advance) estimates for a quarter. For example, in its January 2012 release for its estimates of GDP for the fourth quarter of 2011, the BEA reported that for real GDP the average deviation without regard to sign of the "advance" estimate to the "final" (post annual and comprehensive revision) estimate for the percent change was 1.3 percentage points, and with a standard deviation of 1.0 percentage points (for comparisons for the period 1983 to 2008).(13) Hence, substantial uncertainty exists in real time regarding an actual point estimate even for the government's formal estimate of the rate of real GDP growth. A separate, broad-set-of-indicators-based performance measure based on historical relationships to real GDP growth can therefore potentially provide additional useful information in real time regarding the underlying performance of the economy. An analogous challenge exists for the formal BLS estimates for payroll jobs growth in real time, as the jobs growth estimates are subject to initial monthly revisions for two months and then, later, annual benchmark revisions that can lead to substantial changes in the month-to-month estimated changes.
Other high frequency and social media data
With the increased use of internet search engines and social media over the past decade and the availability of information regarding their use, the prospects for using high frequency and high volume data from those sources appear attractive in terms of gaining additional information in real time about the behavior of the economy and key actors, such as consumers and investors. Choi and Varian , for example, examine data on search engine searches and related economic data and claim that "Google Trends may help in predicting the present," and Bollen, Mao, and Zeng  -analyze the text content of daily Twitter feeds" and find that that "the accuracy of [Dow Jones industrial average] predictions can be significantly improved by the inclusion of specific public mood dimensions."
We have made efforts to use various high frequency data in the real-time system but have not observed much success. In general, the information gained from such sources is very noisy, and it is difficult to identify a strong enough signal for generating significant explanatory power for the aggregate economic variables of interest--espe-cially in terms of providing additional explanatory information beyond the variables we already include in the system. Further, our analysis requires a sufficiently long historical time series to estimate the relationship of the series to the aggregate economic variable of interest. While there may be useful information in high frequency and social media series, at this time we have not been able to reliably identify or incorporate additional robust and significant explanatory information in our system.
7. Concluding Observations
The flexible real-time forecasting system presented in this paper provides a useful analytical tool for generating ongoing, data-based "nowcasts" of real GDP growth. The prediction errors for the system estimates over a historical period compare favorably with those of individual indicators, and the flexible nature of the system--allowing the data to decide the specifications and sample horizons for estimations--yields improved prediction performance relative to estimations with fixed specifications and sample horizons. Also, while the full system estimates adjusted for observed bias and inefficiency produced the lowest root mean square errors, the evidence suggests that a small subset of key indicators does almost as well as the full system estimate, particularly for periods of more extreme real GDP growth. The analysis indicates that forecasters can potentially glean useful information from applying such an approach in practice for making real GDP growth estimates; further, they may be able to do so relatively well with a small set of key indicators. The analysis also demonstrated the opportunities for applying the methodology to other indicators by converting the system to examine real-time estimates for the monthly change in payroll jobs. In the end, although challenges exist for applying the system for making predictions in practice, the analysis and evidence presented in this paper indicate that the system can provide potentially useful real-time information about real GDP growth and other key economic variables.
Angelini, Elena, Gonzalo Camba-Mendez, Domenico Giannone, Lucrezia Reichlin, and Gerhard Riinstler. 2011. "Short-Term Forecasts of Euro Area GDP Growth." The Econometrics Journal, 14(1): C25-44.
Blue Chip Economic Indicators, 2012 Aspen Publishers, 37(5), May 10.
Bollen, Johan, Huina Mao, and Xiao-Jun Zeng. 2011. "Twitter Mood Predicts the Stock Market." Journal of Computational Science, 2(1): 1-8.
Bulligan, Guido, Roberto Golinelli, and Giuseppe Parigi. 2010. "Forecasting Monthly Industrial Production in Real-Time: From Single Equations to Factor-Based Models." Empirical Economics, 39(2): 303-36.
Choi, Hyunyoung, and Hal Varian. 2009. "Predicting the Present with Google Trends," unpublished manuscript, Google, Inc., April 10.
Diebold, Francis X., and Jose A. Lopez. 1996. "Forecast Evaluation and Combination," NBER Techincal Working Paper No.192, March 1996, published in Handbook of Statistics 14: Statistical Methods in Finance, edited by G.S. Maddala and C.R. Rao, North-Holland.
Drechsel, Katja, and Laurent Maurin. 2011. "Flow of Conjunctural Information and Forecast of Euro Area Economic Activity." Journal of Forecasting, 30(3): 336-54.
Evans, Martin D.D. 2005. "Where Are We Now? Real-Time Estimates of the Macroeconomy." International Journal of Central Banking, 1(2): 127-75.
Ferrara, Laurent, Dominique Guegan, and Patrick Rako-tomarolahy. 2010. "GDP Nowcasting with Ragged-Edge Data: A Semi-Parametric Modeling." Journal of Forecasting, 29(1-2): 186-99.
Giannone, Domenico, Lucrezia Reichlin, and Saverio Simonelli. 2009. "Nowcasting Euro Area Economic Activity in Real-Time: The Role of Confidence Indicators." National Institute Economic Review, 210(1): 90-7.
Giannone, Domenico, Lucrezia Reichlin, and David Small. 2008. "Nowcasting: The Real-Time Informational Content of Macroeconomic Data." Journal of Monetary Economics, 55(4): 665-76.
Golinelli, Roberto, and Giuseppe Parigi. 2008. "Real-Time Squared: A Real-Time Data Set for Real-Time GDP Forecasting." International Journal of Forecasting, 24(3): 368-85.
Kitchen, John, and Ralph Monaco. 2003. "Real-Time Forecasting in Practice: The U.S. Treasury Staff's Real-Time GDP Forecast System." Business Economics, 38(4): 10-9.
Parigi, Giuseppe, Roberto Golinelli, and Giorgio Bodo. 2010. "Forecasting Industrial Production in the Euro Area." Empirical Economics, 25(4): 541-61.
Riinstler, Gerhard, Karim Barhoumi, Szilard Benk, Ric-dardo Cristadoro, Ard Den Reijer, Audrone Jakaitiene, Piotr Jelonek, Antonio Rua, Karsten Ruth, and Christophe Van Nieuwenhuyze. 2009. "Short-Term Forecasting of GDP Using Large Datasets: A Pseudo Real-Time Forecast Evaluation Exercise." Journal of Forecasting, 28(7): 595-611.
Stock, James H., and Mark W. Watson. 2006. "Forecasting with Many Predictors," in Handbook of Economic Forecasting, Vol. 1, Chapter 10, edited by G. Elliott, C. Granger, and A. Timmermann. Elsevier.
Wagner, Neal, Zbigniew Michalewicz, Moutaz Khouja, and Rob Roy McGregor. 2005. "Forecasting with a Dynamic Window of Time: The DyFor Genetic Program Model.- Lecture Notes in Computer Science, 3490: 205-15.
Paper prepared for the National Association for Business Economics 2012 Mennis Award for presentation at the NABE 2012 Annual Meeting. Views expressed in the paper are the authors' and do not represent the views of the U.S. Department of the Treasury or any other institution.
(1.) "Real-time" research and analyses can have two different definitions: one definition concerns examining the role of the ex post use of the contemporaneously available vintage of data during historical periods relative to the current vintage of data; the second definition is the one used in this paper, the use of currently available data in real time to produce estimates of the variable of interest as it is being formed and prior to the release of the formal estimate.
(2.) Ferrara, Guegan, and Rakotomarolahy  refer to "ragged edge" data; both "jagged" and "ragged" refer to the same phenomenon. We use "jagged" in our discussion.
(3.) Private firms have also produced proprietary and client-restricted estimators for current real GDP growth, including Goldman Sachs current activity index and Moody's Analytics high frequency model.
(4.) In an interesting application that was observed as this paper was being written, Wagner and others  illustrated a "dynamic forecasting genetic program" modeling approach for forecasting GDP in a nonstatic environment.
(5.) The notation as presented is simplified by not using differing notations for the alpha and beta coefficients across the specifications even though those would obviously differ across the specifications.
(6.) 0ther specifications, including varying forms with lagged dependent variable or ARMA models, could be considered; the specifications we used reflect our ex ante determination to focus on the role of individual indicators and the estimated indicator-specific relationships for making predictions of the dependent variable.
(7.) Stock and Watson  discuss alternative "forecast combining methods" including relative historical and forecast performance weightings as shown and discussed here.
(8.) We arbitrarily chose to use the top five to illustrate the small subset.
(9.) The specifications were predetermined by subjective ex ante analysis by indicator.
(10) Because of the large number of iterations and loops to compare alternative specifications and samples, the EViews program for the real-time system for monthly payroll jobs estimates requires just over 3 minutes to run on a Dell T3500 computer with an Intel Xeon 3.20 GHz processor and a Windows 7 operating system. Analogously for the real GDP system, the time required for the system to run is about seven and a half minutes.
(11.) The "lamp post problem" is derived from an old joke: A drunk is looking for his car keys under a lamp post. A passerby asks if he is sure that he lost them there. The drunk replies that he lost the keys over there, in the dark. "So why are you looking for them here?" asks the passer-by. "Because the light is better," replies the drunk.
(12.) Related to this, Giannone. Reichlin, and Simonelli  address the role of confidence indicators in real-time estimation for euro area real activity.
(13.) The RMSE of 1.3 percentage points for our adjusted system estimate compares relatively well with those BEA deviations.
JACKSON KITCHEN and JOHN KITCHEN *
*Jackson Kitchen is a recent graduate in economics from The College of William and Mary has worked at the American Road & Transportation Builders Association in Washington, D.C. and at Competition Dynamics in Salem, Massachusetts.
John Kitchen is an economist with the U.S. Department of the Treasury, with previous positions at the Office of Management and Budget, the House Budget Committee, and the Council of Economic Advisers. He has also taught in graduate programs at the University of Maryland and Wake Forest University. He received his Ph.D. from the University of Pittsburgh.
|Printer friendly Cite/link Email Feedback|
|Comment:||Real-time forecasting revisited: letting the data decide.|
|Author:||Kitchen, Jackson; Kitchen, John|
|Date:||Jan 1, 2013|
|Previous Article:||In search of system understanding and control.|
|Next Article:||Is productivity growth too strong for our own good?|