Printer Friendly

Real-time forecasting revisited: letting the data decide.

Real-time GDP forecasting, also often known as "nowcasting," produces estimates for current-quarter real GDP growth, typically based on a centered value from a set of estimates from incoming indicators. These real-time measures are usually intended to be data-based and to not be based on forecaster judgment or add factors. Even so, estimation methodologies in this research area--and prior versions of the system we use--typically have been constrained by using various "fixed" relationships, such as a fixed historical sample horizon and fixed empirical specifications for the indicator variables. This paper describes the methodology, estimation, and software code for a more flexible real-time GDP system that allows the data to decide the best real-time GDP forecast for varying sample horizons and varying specifications for each indicator variable through time. Our system uses data on key indicators as they become available (accounting for the "jagged-edge" nature of the data in the current quarter) to generate an estimate of current-quarter real GDP growth, with weights for combining the indicator-specific estimates as determined by the strength of the indicators' historical relationships to GDP growth. The improved system searches across a variety of specifications and across sample horizons to choose the best specification as determined by a minimum Schwarz criterion test while also searching for the best sample horizon for minimizing the mean absolute error for a recent prediction period. We illustrate the operation of the system for real-time estimates of real GDP growth over a specific quarter, and examine the properties of the estimates and the implications for predictions. We also discuss potential additional applications and demonstrate a specific application for real-time predictions of the monthly change in payroll jobs.

Business Economics (2013) 48, 8-28.

Keywords: real-time forecasting, real-time estimates, nowcasting, GDP estimates, economic indicators

The decisions of business planners and policymakers often hinge on the current performance and future outlook for the economy. Private and public forecasters therefore have great interest in estimating and forecasting key economic variables, with a particular focus on aggregate economic activity as measured by real GDP growth. The breadth of private sector involvement in economic forecasting in the United States is illustrated by the well-known Blue Chip Economic Indicators publication, which presents the economic forecasts for 55 private-sector forecasters, and the public policy forecasts of the Office of Management and Budget, the Congressional Budget Office, and of the Federal Reserve that underpin the fiscal and monetary policy outlooks. Most short-run forecasts (one to two years) and intermediate- to longer-range projections (5-10 years and beyond) are model-based forecasts, and they "jump off- from historical data and current-quarter values. The contemporaneous, current-quarter performance of the economy is of particular importance for current policy purposes and also as the base for the future projections. Various alternative approaches exist for making current-quarter estimates for real GDP growth. They include:

* behavioral equation and model-based forecasts with judgmental add factors;

* specific-component accounting (also known as "bean counting"), as illustrated by the Bureau of Economic Analysis' (BEA) "Key source data and assumptions" supplemental estimates;

* indicator-based estimates, including those based on coincident activity indices;

* Real-time or "nowcasting" data-based estimation approaches.

Real-time forecasts produce estimates for current-quarter real GDP growth, typically based on a centered value from a set of estimates from incoming indicators; they are usually intended to be data-based and to not include forecaster judgment or add factors.

This paper presents a new estimating system we have developed for making real-time GDP fore-casts--describing the methodology, estimation process, and software code. This system is more flexible than previous real-time forecasting approaches and allows the data to decide the "best" real-time GDP forecast for varying sample horizons and varying specifications by indicator through time. Estimation methodologies in this research area--and prior versions of the system we use--typically have been constrained by using various ex ante or predetermined fixed relationships; for example a fixed historical sample horizon and fixed empirical specifications for the indicator variables. As in earlier real-time estimating analyses, our system uses the data on key indicators as they become available (and in uneven timing during the quarter) to generate an estimate of current-quarter real GDP growth, based on the estimated historical indicator-GDP relationships, as well as for the proper data-availability point during the quarter. The weights we use for combining the indicator estimates are determined by the fits of the historical relationships.

An important innovation in the system we present is that the improved system now searches across a variety of specifications and across sample horizons--choosing the "best" specification by indicator for a given sample. This is done by using a minimum Schwarz criterion, while also searching for the best sample horizon for minimizing the mean absolute error for a recent prediction period. Hence, the system is now much more flexible in determining the specifications and sample periods for the indicator-specific estimating equations, both within a given quarter for varying data availability and across quarters through time as the system is used in ongoing real-time estimating efforts. And, while research in the area of real-time forecasting and nowcasting has been conducted at varying levels of theoretical and mathematical rigor and sophistication--and in many cases with a focus on academic audiences--we believe our approach is one that can be readily understood and useful for the applied and ongoing analyses that are typical for most business and public sector economists and forecasters.

The intent of the paper is to be organized in a manner and to present information such that interested forecasters and researchers can gain useful information on how the system works and better understand how it can be used in practice. Following this introduction, Section 1 of the paper addresses real-time estimation methodology and some other literature in this research area. Section 2 describes the modeling and estimation process for the system and discusses some of the EViews software code used to implement the system. Section 3 presents illustrative results for the application of the system for a specific recent quarter and the evolution of real-time GDP estimates during that quarter. Section 4 examines the prediction properties for the system estimate and for the individual indicator-specific forecasts. Section 5 discusses other potential applications for our real-time estimation system, with a specific application to show how the system can be used in an analogous approach for real-time predictions of the monthly change in payroll jobs. Section 6 further considers what useful information can be gleaned from these types of estimates. Section 7 presents concluding observations.

1. General Methodology and Other Research and Literature

The research and literature for "real time" forecasting and "nowcasting" have been expanding rapidly over the past decade. (1) Stock and Watson [2006] surveyed the "theoretical and empirical research on methods for forecasting economic time series variables with many predictors," describing how that "provides the opportunity to exploit a much richer base of information than is conventionally used for time series forecasting." Kitchen and Monaco [2003] described an early version of a real-time forecasting system "adopted at the U.S. Treasury to use the broad variety of incoming data to construct 'real-time' estimates of quarterly real GDP growth." Evans [2005] used a comprehensive analysis to calculate daily real-time estimates by "modeling the growth in GDP as the quarterly aggregate of an unobserved daily process for real economy-wide activity" with "model parameters ... estimated by (quasi) maximum likelihood using the Kalman filter algorithm." Giannone, Reichlin, and Small [2008] presented an estimating methodology for producing current-quarter forecasts by adapting a common factors model, combining "the idea of 'bridging' monthly information with the nowcast of quarterly GDP with the idea of using a large number of data releases within a single statistical framework." In particular, they clearly addressed the challenge from the evolving nature of the incoming data of the current quarter:
  In real time, some data series have observations
  through the current period, whereas for others the
  most recent observations may be available only for a
  month or quarter earlier. Consequently, the
  underlying data sets are unbalanced. Appropriately
  dealing with this "jagged edge" feature of the data
  is key for producing a nowcast that, by exploiting
  information in the most recent releases, has a chance
  to compete with judgmental forecasts. [Gian none,
  Reichlin, and Small 2008, p. 666] (2)


Their approach allows for the nowcast to be conditioned on a large number of variables. A variety of studies have examined real-time forecasting for euro area GDP and activity [Golinelli and Parigi, 2008; Giannone, Reichlin, and Simonelli 2009; Runstler and others 2009; Bulligan, Golinelli, and Parigi 2010; Angelini and others 2011; Drechsel and Maurin, 2011]. (3)

The system that we present and use for the analysis of this paper is a descendent of the earlier model system described in Kitchen and Monaco [2003]. The Kitchen and Monaco system estimated the indicator-specific historical relationships between monthly indicators and real GDP growth (while properly accounting for indicators' intraquarter data availability), produced current-quarter indicator-specific forecasts from the estimated relationships, and then combined the individual indicator forecasts by weighting according to the strength of the indicators' historical relationships to real GDP growth. The software code for the system we use in this system, while maintaining much of the general methodology, was a complete rewriting relative to the prior system in order to incorporate more estimation flexibility and an inherent decision process within the system for choosing the best estimations by specifications and sample horizons. The prior methodology was primarily a constrained "estimation system" that ran the various estimating equations for largely ex ante user-specified and predetermined specifications and sample periods, which then generated the system estimate from those estimations. The current system is more of an "estimation and decision system" in that it iteratively runs through various alternative specifications and samples for the estimating equations, and within that process it makes the decisions for the "best" specifications and samples to use in the final estimation. (4) The choices in the system are based on the minimum Schwarz criterion to determine the best specification for a given sample size, then using the minimum absolute error for the prediction for real GDP growth for a recent period of time to determine the best sample (and for the best specification for that sample). Hence, this approach is much more in the spirit of "letting the data decide," allowing for the estimation process to be determined in an ongoing and evolving real-time analysis, largely independent of subjective ex ante user-specified relationships. A significant part of the software code needed to implement both the prior system and the current system involves addressing the "jagged edge" problem of the availability of data in the current quarter, and assuring the proper estimation given that problem. The methodology and system operation are described in more detail below.

2. Modeling, Estimation, and Software Code Specifics

This section of the paper provides a description of the operation of the system we use and how it is implemented through software code. Our real-time forecasting system is written in EViews software code--in 612 lines of code (653 lines if comments are included). We are currently running the system in EViews 7.1. The following discussion of this section describes how the system:

* reads in the data and generates quarterly based variables from the monthly data set, specifically accounting for the different numbers of months of data available by indicator during the quarter;

* estimates alternative empirical specifications for the indicator-real GDP growth relationships and searches across those specifications for the "best" specification by indicator for a given sample size according to the Schwartz criterion statistic (and while properly accounting for the given amount of data available during the quarter, that is, one, two or three months of data);

* iterates through rolling sample sizes for up to 101 prior quarters of history and chooses the preferred specification-sample equation based on the minimum mean absolute error for predicted GDP growth for an eight-quarter period prior to the current quarter;

* generates a "centered value" real-time estimator from the collection of indicator-based estimates according to relative weightings based on the R-squared values of the indicator estimated equations;

* writes the output table for the real GDP growth prediction by indicator, along with the specification and sample size used by indicator.

The data and differing frequencies

We use data on major macroeconomic and financial data series as key indicators from which to construct the estimates of current-quarter real GDP growth. Most data series are of monthly frequency--for example, data on payroll jobs, the unemployment rate, industrial production, retail sales, orders and shipments, and others. Other data series are weekly (unemployment insurance claims) or even daily or continuous (stock market indices). The system as currently maintained uses data on 24 indicators--generally well-recognized series (and that are presented in detail in tables for examples discussed below).

We maintain a data base of monthly frequency, updating the data series on an ongoing basis as new data become available--reflecting the "jagged edge" of the current period data as described above. For the higher frequency series such as unemployment insurance claims and the stock market indices, we update the monthly average values for available data and use a random walk assumption that any not-yet-observed values for a given month are equal to the last observed value. That allows us to use a variable with initial observations early in a month to provide information for the beginning of that period. A second database transforms the monthly time-series vectors of the first database into quarterly data series with monthly elements (Table I). For example, in the second database, three separate payroll jobs series exist for a given quarter: for the first month of the quarter; the second month of the quarter; and for the third month of the quarter. That quarterly database allows for an easier way to read the data into the EViews computer program and to manage and transform the data once it is in the program. This approach assures the proper accounting for the availability of data by series (the proper part of the jagged edge)--and correct estimation over time for that given real-time availability--yet within a quarterly frequency that aligns with the dependent variable of interest, real GDP growth. Although the system has the monthly and quarterly frequency aspects, it could potentially be run at any time--even on a daily or intradaily basis--to update estimates as new data arrive.

The EViews program begins by creating a work file and then reading in the (quarterly frequency) data from the second database described above. Initial transformations of variables are then made through GENR statements--for example, transforming nominal series to real series and creating the proper quarterly average series for the cases of varying data availability for differing monthly stages of the quarter.

Choosing from alternative specifications

The estimations within the modeling framework identify the relationship between a specific indicator variable and real GDP growth during the current quarter. The specifications therefore use the dependent variable of [y.sub.t] = real GDP percentage change during the quarter at an annual rate, and right-hand-side variables as various transformed forms of [x.sub.it] = level of the ith indicator variable. The EViews computer code uses sequential FOR-NEXT loops to run through the alternative specifications by indicator variable (and separate sets of the statements by the stage of the data availability of the "jagged edge" for the quarter) and tests for the best specification for a given sample size by comparing the Schwarz criterion statistics. The specifications currently considered in the program (and by numerical value denoting the specification in the output) are:(5)

1. First difference specification:

[y.sub.t] = [alpha] + [beta]([x.sub.i,t]--[x.sub.i,t--1]) + [e.sub.t].

2. First and lagged differences specification:

[y.sub.t] = [alpha] + [[beta].sub.1]([x.sub.i,t]--[x.sub.i,t--1]) + [[beta].sub.1]([x.sub.i,t--1]--[x.sub.i,t--2]) + [e.sub.t].

3. Percentage change specification:

[y.sub.t] = [alpha] + [beta][[x.sub.i,t]/[x.sub.i,t--1]]--1] + [e.sub.t].

4. Lagged percentage change specification:

[y.sub.t] = [alpha] + [beta][[x.sub.i,t--1]/[x.sub.i,t--1]]--2] + [e.sub.t].

5. Current level and first difference:

[y.sub.t] = [alpha] + [beta][x.sub.i,t] + [[beta].sub.2]([x.sub.i,t]--[x.sub.i,t--1]) + [e.sub.t].

6. Current and lagged percentage changes specification:
Table 1. Monthly and Quarterly Data Bases

Monthly                                Quarterly
Database                               database

Date          Variable   Variable2  ...  Date         GDP
              1

YearMonth     YearMonth  YearMonth  ...  YearQuarter  YearQuarter

...             ...          ...          ...

Y(t--1)M(10)  V1(t--l,   V2(t--l,   ...
              10)        10)

Y(t--1)M(11)  V1(r--1,   V2(t--1,   ...
              11)        11)

Y(t--1)M(12)  V1(t--l,   V2(t--l,   ...  ...          ...
              12)        12)

Y(t)M(1)      V1(t, 1)   V2(M)      ...  Y(t--1)Q(4)  GDP(t--1,
                                                      4)

Y(t)M(2)      V1(t, 2)   V2(t, 2)   ...  Y(t)Q(l)     GDP(t, l)

Y(t)M(3)      V1(t, 3)   V2(t, 3)   ...  ...           ...

...           ...        ...        ...
Monthly
Database

Date          Variable   Variable   Variable   Variable2  ...
              1, Month   1, Month2  1, Month3  Month 1
              1

YearMonth     YearMonth  YearMonth  YearMonth  YearMonth  ...

Y(t--1)M(10)

Y(t--1)M(11)

Y(t--1)M(12)  ...          ...          ...

Y(t)M(1)      V1(t--1,   V1(t--1,   V1(t--1,   V2(t--1,   ...
              10)        11)        12)        10)

Y(t)M(2)      V1(t, 1)   V1(t, 2)   V1 (t, 3)  V2(t, l)   ...

Y(t)M(3)      ...        ...        ...        ...


[y.sub.t] = [alpha] + [[beta].sub.1][[x.sub.i,t]/[x.sub.i,t--1]]--1 + [[beta].sub.2]([x.sub.i,t--1]--[x.sub.i,t--2]) + [e.sub.t].

Although the above specifications that we have included in the system code provide a relatively good range of alternatives, the system obviously could consider an even greater number and range of alternative specifications and thereby potentially make the system even more encompassing in terms of letting the data decide.(6) Nonetheless, we are comfortable that the range of specifications we have included for the analysis and presentation in this paper provides a good illustration of the potential for alternative specification choices and the way the system works, and also is consistent with a principle of parsimony.

Choosing the sample: Iterations through rolling sample sizes

The system uses nested IF-THEN-ELSE statements by variable to compare the prediction performance for specifications across sample sizes, and then chooses the best equation by sample and by specification. The iterations across sample sizes start from a minimum sample of at least 28 quarters, up to a maximum of 101 quarters; while these parameters are somewhat subjective, the minimum sample size contains data from across a business cycle at least and the maximum covers up to 25 years of data. The best relative prediction performance is determined by the lowest mean absolute error for the predicted values of real GDP growth for an eight-quarter period prior to the current quarter. Further, because in real time, estimates for real GDP growth in recent quarters are typically "preliminary" estimates, the most-recent prior quarter is not included in the eight-quarter prediction comparison period. For example, if the system is generating a predicted value for the second quarters of 2012, the eight-quarter period for prediction comparison ends two quarters previously, the fourth quarter of 2011.

Generating the centered value--Relative [R.sup.2] weighting

After the "best" individual indicator-specific estimates are determined, the system determines the centered value for the estimate from the real-time system by weighting individual indicator estimates according to a relative [R.sup.2] weighting method, using the [R.sup.2] values from the chosen indicator-specific estimating equations. The weight for the ith indicator estimate for period t, [w.sub.i,t] for k indicators in the system, is then given by:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1)

and the system estimate is given by:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (2)

where the subscript s represents the "system" value and the [^.y.sub.i,t] term is the real GDP growth forecast for indicator [x.sub.i,t] determined from the search across the alternative specifications as described above. Note that the [^.y.sub.i,t] estimates and the [w.sub.i,t] weights will vary through time and also can vary within a period as the data availability varies across (the jagged edge of) the current quarter (and as the program-determined "best" specification potentially changes as well). This weighting methodology gives greater weighting to the indicator estimates that have a stronger historical relationship in predicting real GDP growth and also assures the sum of the weights will be equal to 1.0. Note that a potential alternative weighting methodology could use the inverse of the mean absolute errors from the recent prediction comparison period that was used in the system to determine the best sample-specification combination. Such an approach would determine the weights based on the recent prediction performance rather than the fit over the historical sample. Our initial examination of results from the two approaches indicates that there is little difference between the weighted predictions of those two approaches.(7)

The output table

The system code finishes by writing the output table listing the individual indicators and the system-generated predicted value for current-quarter real GDP growth. To provide additional information by indicator, the output includes the R-squared for the estimation equation used; a number assigned to identify the specification employed; and the sample size over which the prediction equation was estimated. These results can be seen in Tables 2-7 and are discussed in the next section.

3. Illustrative Results--and Evolution of Estimates during a Quarter

To illustrate the performance of the system, Tables 2-7 present a collection of six different system output tables at varying times for making estimates for real GDP growth for the second quarter of 2012, incorporating the annual NIPA revisions of July 2012. The 24 indicator variables in the system can be seen in the tables.

Table 2 shows an initial system estimate at the end of April 2012--that is, at the end of the first month of the second quarter, when only very initial observations on any data for the quarter could be observed. At that point, the only indicator data available for the quarter were: initial and continuing claims for unemployment insurance for the first several weeks of April; consumer confidence and consumer sentiment for April; the Philadelphia Fed business activity index for April; and the S&P500 stock index for most of April. The data thus represented only a very initial snapshot for the beginning of the quarter. That is why it is also very important that the estimating and forecasting equations from the system only use historical data for the first month of the historical quarters and not full three-month quarterly data, properly accounting for the "jagged edge." Although one would need to be cautious in interpreting results from such a limited, initial set of data, the first result from the system at that time showed a weighted estimate of about 2.1 percent for real GDP growth in the second quarter. Such a result could be interpreted as the very initial data indicating that real GDP growth was on track to be positive and at a moderate rate.
Table 2. Real-time Forecast Results for 2012.6)2, April 30

Indicator        Prediction  [R,sup.2]  Specification  Sample
                   (%pts)

S&P 500 Index          3.90      0.368              3      55

Philadelphia           2.71      0.351              5      68
Fed Index

Continuing UI          2.60      0.599              2      29
Claims

Consumer               1.94      0.121              1      48
Confidence

Consumer               1.62      0.212              5      82
Sentiment

Unemp. Ins.            0.83      0.787              6      35
Claims

Housing Starts           --         --             --      --

ISM Non-Mfg.             --         --             --      --
Index

ISM Purchasing           --         --             --      --
Mgrs. Index

Unemployment             --         --             --      --
Rate

Aggregate                --         --             --      --
Worker Hours

Industrial               --         --             --      --
Production

Payroll jobs             --         --             --      --

Shipments,              ---         --             --      --
Durables

Real                     --         --             --      --
Consumption
Expenditures

Real Retail              --         --             --      --
Sales

NAHB HMI                 --         --             --      --

Work Week                --         --             --      --

Real Exports             --         --             --      --

Payroll Jobs             --         --             --      --
Diffusion

Real                    ---         --             --      --
Residential
Construction

NFIB Good Time           --         --             --      --

Durable Goods            --         --             --      --
Orders

Real                     --         --             --      --
Nonresidential
Construction

Real-time
Forecast, RTFU,
Current Quarter
GDP Growth(1)

Median
Forecast

Indicator          Weighted
                 Contribution
                    (%pts)

S&P 500 Index           0.589

Philadelphia            0.390
Fed Index

Continuing UI           0.639
Claims

Consumer                0.096
Confidence

Consumer                0.141
Sentiment

Unemp. Ins.             0.268
Claims

Housing Starts             --

ISM Non-Mfg.               --
Index

ISM Purchasing             --
Mgrs. Index

Unemployment               --
Rate

Aggregate                  --
Worker Hours

Industrial                 --
Production

Payroll jobs

Shipments,                 --
Durables

Real                       --
Consumption
Expenditures

Real Retail                --
Sales

NAHB HMI                   --

Work Week                  --

Real Exports               --

Payroll Jobs               --
Diffusion

Real                       --
Residential
Construction

NFIB Good Time             --

Durable Goods              --
Orders

Real                       --
Nonresidential
Construction

Real-time                2.12
Forecast, RTFU,
Current Quarter
GDP Growth(1)

Median                   2.27
Forecast

(1.) Weights determined by relative [R.sup.2] weighting.
Specification reference for templar: 1 is first difference;
2 is first and lagged differences; 3 is percent charm; 4 is
lagged percent change; 5 is current level and first
difference; 6 is current and lagged percent change.


Table 3 shows how the system estimates changed as more data came in over the subsequent weeks, through the middle of May--and notably now including several of the more-recognized data reports on the economy for April, including the Employment Situation release from the Bureau of Labor Statistics (BLS), industrial production, and retail sales. With that additional data (but again only primarily for the first month of the quarter), the system estimate for real GDP growth was at 2.4 percent. The individual indicators generated a fairly large range of estimates, from 1.1 percent for the housing market index from the National Association of Home Builders up to 4.05 percent from the S&P500 stock index.
Table 3. Real-time Forecast Results for 2012.Q2, May 17

Indicator        Prediction  [R.sup.2]  Specification  Sample
                   (%pts)

S&P 500 Index          4.05      0.313              3     101

ISM Purchasing         3.44      0.236              5      85
Mgrs. Index

Industrial             3.04      0.669              1      35
Production

Real Retail            2.92      0.587              3      46
Sales

Housing Starts         2.87      0.326              4      29

Unemployment           2.85      0.450              2      47
Rate

Continuing UI          2.70      0.684              2      29
Claims

Payroll jobs           2.48      0.475              5      58

Payroll Jobs           2 14      0.082              4      49
Diffusion

Work Week              2.13      0.208              5      55

Philadelphia           1.96      0.398              5      96
Fed Index

Consumer               1.94      0.121              1      48
Confidence

Unemp. Ins.            1.91      0.831              6      30
Claims

Aggregate              1.81      0.566              5      29
Worker Hours

Consumer               1.77      0.249              5      83
Sentiment

NFIB Good Time         1.62      0.055              4      31

ISM Non-Mfg.           1.42      0.329              5      45
Index

NAHB HM1               1.10      0.260              5      36

Shipments,               --         --             --      --
Durables

Real                     --         --             --      --
Consumption
Expenditures

Real Exports             --         --             --      --

Real                     --         --             --      --
Residential
Construction

Durable Goods            --         --             --      --
Orders

Real                     --         --             --      --
Nonresidential
Construction

Real-time
Forecast, RTFU.
Current Quarter
GDP Growth(1)

Median
Forecast

Indicator          Weighted
                 Contribution
                    (%pts)

S&P 500 Index           0.185

ISM Purchasing          0.119
Mgrs. Index

Industrial              0.297
Production

Real Retail             0.251
Sales

Housing Starts          0.137

Unemployment            0.188
Rate

Continuing UI           0.270
Claims

Payroll jobs            0.172

Payroll Jobs            0.026
Diffusion

Work Week               0.065

Philadelphia            0.114
Fed Index

Consumer                0.034
Confidence

Unemp. Ins.             0.232
Claims

Aggregate               0.150
Worker Hours

Consumer                0.064
Sentiment

NFIB Good Time          0.013

ISM Non-Mfg.            0.068
Index

NAHB HM1                0.042

Shipments,                 --
Durables

Real                       --
Consumption
Expenditures

Real Exports               --

Real                       --
Residential
Construction

Durable Goods              --
Orders

Real                       --
Nonresidential
Construction

Real-time                2.43
Forecast, RTFU.
Current Quarter
GDP Growth(1)

Median                   2.14
Forecast

(1.) Weights determined by relative R-squared weighting.
Specification reference for expl var: 1 is first difference;
2 is first and lagged differences; 3 is percent change; 4 is
lagged percent change; 5 is current level and first difference;
6 is current and lagged percent change.


By the beginning of June (Table 4) initial information for at least the first month was available for most indicators, and several had begun to have a second month as well. The system estimate for real GDP growth was at about 2.2 percent, with a range across the indicators of 0.3 to 3.7 percent. By the end of June (Table 5)--the actual end of the second quarter, but still before all data for the quarter were available--most of the indicator variables had two months of data for the quarter, some had three months (unemployment claims, consumer confidence and sentiment, and S&P500 index), and a couple still had only one month (construction and exports). The system at that time showed a weighted estimate of 1.97 percent.
Table 4. Real-time Forecast Results for 2012.Q2, June 1

Indicator        Prediction     R2  Specification  Sample      Weighted
                     (%pts)                                Contribution
                                                                 (%pts)
S&P 500 Index          3.72  0.332              3      99         0.123
Real                   3.53  0.321              6      85         0.113
Residential
Construction
Industrial             3.02  0.665              1      35         0.201
Production
Real                   2.99  0.618              3      37         0.185
Consumption
Expenditures
Real Retail            2.91  0.584              3      46         0.170
Sales
Housing Starts         2.84  0.319              4      29         0.090
Continuing UI          2.70  0.682              2      29         0.184
Claims
Unemployment           2.60  0.516              2      48         0.134
Rate
Real                   2.53  0.324              5      29         0.082
Nonresidential
Construction
ISM Purchasing         2.47  0.463              5      48         0.114
Mgrs. Index
Payroll jobs           2.31  0.503              5      65         0.116
Shipments.             1.98  0.602              3      34          0119
Durables
Philadelphia           1.96  0.398              5      96         0.078
Ked Index
Consumer               1.86  0.250              5      85         0.046
Sentiment
Unerap. Ins.           1.83  0.828              6      30         0.151
Claims
Consumer               1.78  0.207              5      83         0.037
Confidence
NFIB Good Time         1.62  0.058              4      31         0.009
Payroll Jobs           1.60  0.231              5      48         0.037
Diffusion
ISM Non-Mfg.           1.42  0.327              5      45         0.046
Index
Aggregate              1.29  0.622              5      35         0.080
Worker Hours
NAHB HMI               1.09  0.259              5      36         0.028
Work Week              0.62  0.226              1      46         0.014
Durable Goods          0.32  0.680              3      29         0.022
Orders
Real Exports             --     --             --      --         0.000
Real-time                                                          2.18
Forecast, RTFU,
Current Quarter
GDP Growth
Median                                                             1.98
Forecast

(1.) Weights determined by relative [R.sup.2] weighting.

Specification reference for expl var: 1 is first difference; 2 is
first and lagged dilTerences; 3 is percent change; 4 is lagged
percent change; 5 is current level and first
difference; 6 is current and lagged percent change.

Table 5. Real-time Forecast Results for 2012.Q2, June 28

Indicator        Prediction  [R.sup.2]  Specification  Sample
                   (%pts)

S&P 500 Index          3.93      0.332              3      99

Real                   3.53      0.321              6      85
Residential
Construction

Consumer               3.39      0.228              3      94
Confidence

Real                   2.99      0.618              3      37
Consumption
Expenditures

Housing Starts         2.98      0.362              6      50

Unemployment           2.60      0.516              2      48
Rate

Real                   2.53      0.324              5      29
Nonresidential
Construction

Continuing Ul          2.52      0.747              5      29
Claims

ISM Purchasing         2.47      0.463              5      48
Mgrs. Index

Payroll jobs           2.31      0.503              5      65

Industrial             2.24      0.701              I      29
Production

Shipments,             2.18      0.684              1      34
Durables

Real Retail            2.16      0.669              3      44
Sales

Unemp. Ins.            1.73      0.825              6      29
Claims

Payroll Jobs           1.60      0.231              5      48
Diffusion

Consumer               1.53      0.262              5      85
Sentiment

NFIB Good Time         1.44      0.058              4      29

ISM Non-Mfg.           1.38      0.413              5      59
Index

Philadelphia           1.29      0.426              5     101
Fed Index

Aggregate              1.29      0.622              5      35
Worker Hours

NAHB HMI               0.85      0.242              5      36

Work Week              0.62      0.226              1      46

Durable Goods          0.32      0.768              3      29
Orders

Real Exports         --0.26      0.473              5      80

Real-time
Forecast. RTFU,
Current Quarter
GDP Growth(1)

Median
Forecast

Indicator          Weighted
                 Contribution
                    (%pts)

S&P 500 Index           0.118

Real                    0.103
Residential
Construction

Consumer                0.070
Confidence

Real                    0.168
Consumption
Expenditures

Housing Starts          0.098

Unemployment            0.122
Rate

Real                    0.074
Nonresidential
Construction

Continuing Ul           0.171
Claims

ISM Purchasing          0.104
Mgrs. Index

Payroll jobs            0.105

Industrial              0.143
Production

Shipments,              0.135
Durables

Real Retail             0.131
Sales

Unemp. Ins.             0.130
Claims

Payroll Jobs            0.034
Diffusion

Consumer                0.036
Sentiment

NFIB Good Time          0.008

ISM Non-Mfg.            0.052
Index

Philadelphia            0.050
Fed Index

Aggregate               0.073
Worker Hours

NAHB HMI                0.019

Work Week               0.013

Durable Goods           0.022
Orders

Real Exports           -0.011

Real-time                1.97
Forecast. RTFU,
Current Quarter
GDP Growth(1)

Median                   2.17
Forecast

(1.) Weights determined by relative [R.sup.2] weighting.

Specification reference for expl var: I is first difference;
2 is first and lagged differences; 3 is percent change; 4
is lagged percent change; 5 is current level and
first difference; 6 is current and lagged percent change.


The final two tables (Tables 6 and 7) for the estimates for the quarter show the system estimates at the end of July-1.8 percent--and at the end of August-1.8 percent. At the end of July, the BEA released the advance estimate of real GDP growth for the second quarter, at 1.5 percent--compared with the real-time system estimate of 1.8 percent at that time and the market consensus for 1.2 percent. By the end of August, the BEA released its second estimate, at 1.7 percent, compared with the real-time system estimate of 1.8 percent and a market consensus of 1.7 percent. By the end of September, the BEA released its final estimate of 1.3 percent for real GDP growth for the second quarter of 2012.
Table 6. Real-time Forecast Results for 2012.Q2, July 26

Indicator        Prediction  [R.sup.2]  Specification  Sample
                   (%pts)

S&P 500 Index          3.94      0.332              3      99

Housing Starts         3.91      0.314              6     101

Consumer               3.39      0.228                     94
Confidence

Real                   3.36      0.388              6      89
Residential
Construction

Payroll Jobs           2.70      0.267              5      43
Diffusion

Shipments,             2.65      0.655              I      47
Durables

Continuing UI          2.49      0.747              2      29
Claims

Real                   2.34      0.312              5      29
Nonresidential
Construction

Unemployment           2.21      0.673              2      29
Rate

ISM Purchasing         2.14      0.476              5      47
Mgrs. Index

Industrial             1.87      0.742              1      29
Production

Unemp. Ins.            1.73      0.825              6      29
Claims

Real                   1.72      0.537              6     101
Consumption
Expenditures

NFIB Good Time         1.55      0.055              4      30

Consumer               1.50      0.262              1      85
Sentiment

Payroll jobs           1.45      0.498              3      45

Work Week              1.32      0.291              1      50

Philadelphia           1.29      0.426              5     101
Fed Index

ISM Non-Mfg-           1.19      0.552                     45
Index

Aggregate              1.15      0.469              3      49
Worker Hours

Real Retail            1.12      0.772              3      29
Sales

NAHB HMI               0.85      0.242              5      36

Real Exports           0.65      0.570              1      30

Durable Goods          0.49      0.799              3      29
Orders

Real-time
Forecast, RTFU,
Current Quarter
GDP Growth(1)

Median
Forecast

Indicator          Weighted
                 Contribution
                    (%pts)

S&P 500 Index           0.114

Housing Starts          0.107

Consumer                0.068
Confidence

Real                    0.114
Residential
Construction

Payroll Jobs            0.063
Diffusion

Shipments,              0.152
Durables

Continuing UI           0.163
Claims

Real                    0.064
Nonresidential
Construction

Unemployment            0.130
Rate

ISM Purchasing          0.089
Mgrs. Index

Industrial              0.121
Production

Unemp. Ins.             0.125
Claims

Real                    0.081
Consumption
Expenditures

NFIB Good Time          0.007

Consumer                0.034
Sentiment

Payroll jobs            0.063

Work Week               0.034

Philadelphia            0.048
Fed Index

ISM Non-Mfg-            0.057
Index

Aggregate               0.047
Worker Hours

Real Retail             0.076
Sales

NAHB HMI                0.018

Real Exports            0.032

Durable Goods           0.034
Orders

Real-time                1.84
Forecast, RTFU,
Current Quarter
GDP Growth(1)

Median                   1.73
Forecast

(1.) Weights determined by relative [R.sup.2] weighting.

Specification reference for expl var: 1 is first difference;
2 is first and lagged differences; 3 is percent change; 4
is lagged percent change: 5 is current level and first
difference: 6 is current and lagged percent change.

Table 7. Real-time Forecast Results for 2012.Q2, August 30

Indicator        Prediction  [R.sup.2]  Specification  Sample
                   (%pts)

Real                   3.98      0.324              2      84
Residential
Construction

Housing Starts         3.03      0.434              6      29

Payroll Jobs           2.92      0.229              5      48
Diffusion

S&P 500 Index          2.68      0.501              3      48

Unemployment           2.42      0.505              2      48
Rate

Real                   2.36      0.313              5      29
Nonresidential
Construction

Continuing Ul          2.34      0.676              2      29
Claims

Real Exports           2.12      0.522              1      48

ISM Purchasing         2.10      0.439              5      48
Mgrs. Index

Shipments.             1.98      0.695              1      29
Durables

Industrial             1.89      0.664              1      29
Production

Real                   1.77      0.551              6     101
Consumption
Expenditures

Unemp. Ins.            1.65      0.770              6      29
Claims

Consumer               1.63      0.131              1      50
Confidence

Consumer               1.54      0.264              5      85
Sentiment

NFIB Good Time         1.46      0.065              4      29

Philadelphia           1.35      0.404              5      98
Fed Index

ISM Non-Mfg.           1.27      0.530              5      45
Index

Payroll jobs           1.23      0.569              3      33

Work Week              1.21      0.299              5      64

Real Retail            1.00      0.758              3      29
Sales

Aggregate              0.92      0.444              3      50
Wtirker Hours

NAHB HMI               0.68      0.191              5      48

Durable Goods          0.59      0.688              3      29
Orders

Real-time
Forecast. RTFU,
Current Quarter
GDP Growth(1)

Median
Forecast

Indicator          Weighted
                 Contribution
                    (%pts)

Real                    0.118
Residential
Construction

Housing Starts          0.120

Payroll Jobs            0.061
Diffusion

S&P 500 Index           0.122

Unemployment            0.111
Rate

Real                    0.067
Nonresidential
Construction

Continuing Ul           0.144
Claims

Real Exports            0.101

ISM Purchasing          0.084
Mgrs. Index

Shipments.              0.125
Durables

Industrial              0.114
Production

Real                    0.089
Consumption
Expenditures

Unemp. Ins.             0.116
Claims

Consumer                0.019
Confidence

Consumer                0.037
Sentiment

NFIB Good Time          0.009

Philadelphia            0.050
Fed Index

ISM Non-Mfg.            0.061
Index

Payroll jobs            0.064

Work Week               0.033

Real Retail             0.069
Sales

Aggregate               0.037
Wtirker Hours

NAHB HMI                0.012

Durable Goods           0.037
Orders

Real-time                1.80
Forecast. RTFU,
Current Quarter
GDP Growth(1)

Median                   1.71
Forecast

(1.) Weights determined by relative [R.sup.2] weighting.

Specification reference for expl var: 1 is first difference;
2 is first and lagged differences; 3 is percent change;
4 is lagged percent change: 5 is current
level and first difference; 6 is current and lagged percent change.


In summary for this section, the results and comparisons for the evolution of system estimates for the second quarter of 2012 illustrate how the system performed in providing information about real GDP growth, from a beginning of a very limited set of information on few indicators through the accumulation of additional information across indicators until information for all months for all indicators was available--and until final estimates from BEA were made.

4. Properties of the Estimators Using Current-Vintage Data

This section presents information on the properties of the estimates using current-vintage data; we use the current-vintage data in the absence of a long time series of estimates based on the contemporaneously available real-time data at the rolling historical data vintages at the initial observation points. Although some researchers may have a preference for using historical real-time data for evaluating the estimates, note that there is rarely (and sometimes never) a pure apples-to-apples comparison for data and analysis through time, and especially relative to the observed data at another point in time historically. Data definitions, methodologies, sources, and so on, all change through time, and that is particularly true for an indicator such as real GDP growth, which is the focus of much of the attention in real-time analysis. Hence, for generating current estimates in real time for the current quarter, there are strong arguments that it is best to use relationships as identified in the current vintage of data that reflects current in-place methods and construction.

To examine the properties of the system and indicator estimates, we ran the system to generate estimates over the historical period from 1997:Q1 through 2011:Q4 using current-vintage data. Table 8 presents the root-mean-square errors (RMSEs) for the estimates, showing various results that will be discussed in turn. Of particular note, the results in the left-hand side of Table 8 show the results for the flexible estimation system that is the primary focus of this paper; those on the right-hand side of Table 8 provide a comparison to a system in which the specifications and sample horizons are fixed. The top of the table presents results for system estimates; the bottom of the table shows results for indicator-specific estimates.
Table 8. Root Mean Square Errors by Predictor

Flexible System               Fixed System

Predictor or           RMSE   Predictor or           RMSE
indicator             (%pts)  indicator             (%pts)

System estimator        1.59  System estimator        1.76

System estimator,       1.38  System estimator,       1.52
adjusted                      adjusted

Autoregressive          2.32  Autoregressive          2.39
(AR(2)) estimator             (AR(2)) estimator

Top 5 Indicators,       1.53  Top 5 Indicators,       1.71
equal weighting               equal weighting

Top 5. adjusted         1.47  Top 5. adjusted         1.54

Continuing Ul Claims    1.70  Industrial              1.88
                              Production

Shipments. Durables     1.75  Unemp. Ins. Claims      1.89

Real Retail Sales       1.75  ISM Non-Mfg. Index      2.00

Industrial              1.76  Shipments. Durables     2.01
Production

Durable Goods Orders    1.77  Real Consumption        2.03
                              Expenditures

Unemp. Ins. Claims      1.80  Continuing Ul Claims    2.09

Aggregate Worker        1.83  Unemploymeni Rate       2.11
Hours

Unemploymeni Rate       1.85  Durable Goods Orders    2.11

Real Consumption        1.90  Aggregate Worker        2.15
Expenditures                  Hours

ISM Non-Mfg. Index      1.91  Real Exports            2.15

Real Exports            1.91  Philadelphia Fed        2.16
                              Index

Philadelphia Fed        1.97  Payroll jobs            2.16
Index

S&P 500 Index           1.99  Work Week               2.20

Payroll jobs            2.06  Real Retail Sales       2.25

ISM Purchasing Mgrs.    2.09  Payroll Jobs            2.31
Index                         DilTusion

Housing Starts          2.27  Real Residential        2.33
                              Construction

Real Residential        2.29  ISM Purchasing Mgrs.    2.41
Construction                  Index

Payroll Jobs            2.29  Consumer Confidence     2.43
DilTusion

NFIB Good Time          2.34  Consumer Sentiment      2.45

NAHB HM1                2.35  NFIB Good Time          2.49

Work Week               2.36  NAHB HMI                2.50

Consumer Sentiment      2.40  S&P 500 Index           2.64

Real Nonresidential     2.41  Housing Starts          2.93
Construction

Consumer Confidence     2.48  Real Nonresidential     2.98
                              Construction


Comparison for system and indicator-specific indicators--and a subset of the best indicators

The first line for the flexible system results (left side) shows the RMSE for the system estimate at 1.59 percentage points, which compares favorably with the individual indicator estimates in the bottom of the table, where the lowest RMSE is at 1.70 for continuing unemployment insurance claims. The system estimate also compares favorably with a naive rolling autoregressive model of order 2 [AR(2)], estimation for which the RMSE is 2.32 percentage points as shown in the third line.

An interesting question is the extent to which the full system is better than a small subset of indicators in making the real-time estimate. The fourth line shows the RMSE for an estimate using an equally weighted combination of the "top 5" indicator estimates, (8) based on using the five indicators with the lowest RMSE over the estimation horizon--for the flexible system those "best" indicators are (real) durable goods orders: continuing unemployment insurance claims; (real) shipments of durable goods; industrial production; and real retail sales. An estimate based on those five indicators alone actually slightly outperforms the full system estimate, with an RMSE of 1.53 compared with the 1.59 of the system.

Tests regarding the properties of the system estimate

Questions remain about other properties of the estimates, notably unbiasedness, and in the sense discussed in Diebold and Lopez [1994 "The key property of optimal forecast errors from which all others follow ... is unforecastability on the basis of information available at the time the forecast was made." To examine these questions, we conducted some simple regression tests; the results are presented in Table 9. The first equation of Table 9 shows the results for regressing actual real GDP growth on the system estimate (using ordinary least squares); if the system estimate is an unbiased predictor then the joint hypothesis that the constant coefficient is zero and the slope coefficient is 1.0 should not be rejected. The estimated constant coefficient is negative and significantly less than zero (at the 5 percent level) and the slope coefficient is significantly greater than one; the F-statistic from a Wald test for testing the joint (0,1) hypothesis is significant at 7.33, rejecting that joint (0,1) hypothesis. The second estimated equation of Table 9 goes further, regressing the fitted error from the first equation on the one-quarter-lagged prediction error for the system estimate. If the system estimate misses the prediction in a given quarter, does that carry information for making a prediction in the next quarter? That is, real GDP growth at times exhibits a "saw tooth" pattern--if real GDP growth is abnormally large one quarter, does real GDP growth tend to be lower in the subsequent quarter? The estimation results show that the estimated coefficient on the prior quarter prediction error is significantly different from zero at the 10 percent level. Further, including the lagged prediction error in the estimated equation (equation (3) of Table 9) results in an estimated coefficient significant at the 5 percent level and the F statistic of 6.77 shows that the joint hypothesis of (0,1,0) is rejected. Although the saw-tooth pattern does not always dominate, when it does the additional role for the lagged predictor can be important, so we include that for making an adjusted estimate.
Table 9. Regression Results for Testing Properties of
the Estimators Sample data: 1997:Q/-2011:Q4

Equation  Dependent   Constant   System     System        F
          Variable              Estimate   Estimate   Statistic
                                          Prediction     for
                                            Error      Testing
                                            (t-1)       (0,1)

1.        Real GDP      --1.26   1.37 **          --    7.33 **
          growth            **

                        (0.33)    (0.10)          --

2.        Error from      0.07        --    --0.23 *         --
          equation
          (1)

                        (0.19)        --      (0.12)         --

3.        Real GDP      --1.64   1.48 **     0.29 **         --
          growth            **

                        (0.37)    (0.11)      (0.13)         --

Equation  Dependent       F
          Variable    Statistic
                         for
                       Testing
                       (0.1,0)

1.        Real GDP           --
          growth

                             --

2.        Error from         --
          equation
          (1)

                             --

3.        Real GDP      6.77 **
          growth

                             --

*Significant at the 0.10 level; ** signficant at the 0.05 level.
Standard errors in parentheses.


Figure 1 shows the historical values for the alternative series--actual real GDP growth, the (flexible) real-time (RT) system estimate, and for the "adjusted" series that accounts for the bias and the information from the prior quarter prediction error. Figure 1 shows that the real-time system estimate tends to underpredict (absolutely) the more-extreme values for real GDP growth, whereas the adjusted measure does a better job of capturing those deviations from trend performance. During periods when real GDP growth is largely per- forming near trend such as occurred during the late-1990s and the mid-2000s--there is little difference in the estimates. In particular, during recession and low-growth periods the RT system estimate understates the extent of GDP growth declines to a greater degree than the adjusted estimates.

Looking back at Table 8 and the RMSEs for the alternative estimates, the second line shows the lower RMSE for the adjusted system estimator, at 1.38, compared with the 1.59 of the unadjusted system estimate. We made an analogous adjustment for the top 5 indicators estimate (results not shown); the fifth line of Table 8 shows that the RMSE is lower for the adjusted estimate for the top 5 case, as well, but with less relative improvement. The lower relative improvement apparently is because the top 5 indicators tend to be more volatile to begin with, and tend to do a somewhat better job during the more extreme growth periods.

In general, the adjusted system estimate yields the lowest RMSE, followed closely by the top 5 adjusted estimate. In the example presented above in Section 3 for the second quarter of 2012, the adjusted estimate would be somewhat lower at 1.2 percent compared with the final unadjusted system estimate of 1.8 percent--and compared with BEA's final 1.3 percent estimate.

Comparison to fixed system estimates

Finally, the right-hand side of Table 8 presents the equivalent set of RMSE numbers for running a version of the system in which the system is constrained to having fixed specifications and fixed sample sizes (80 quarters).(9) The fixed system results almost uniformly have higher RMSE across the measures, illustrating that the flexible estimation process yields better results for the system estimates and for the individual indicator-specific estimates (although there are a couple exceptions among the "worst" performing indicators in the bottom of the table).

5. Other Potential Applications and a Monthly Payroll Jobs Estimation Example

The methodology and system employed here have many potential applications beyond real-time estimates of real GDP growth. We consider applications to a restricted "best set of indicators" version of our system, an alternative approach with separate system estimates by GDP component, specific indicators or sectors, and an illustration of a version of the system as applied to the monthly payroll jobs estimate from the BLS.

An alternative methodology: Using a subset of the best indicators predictions

As observed in the analysis of the prior section, a potential refinement to the methodology would be to restrict the construction of the system estimate to a small subset of the indicators, choosing to use a set of the indicator estimates that had the better prediction performance. As observed, such a change may not yield estimates that are much different from those of the full system. In practice, our view is that we would tend to use the full system estimate, and use the top 5 estimates to help inform our views during particularly volatile growth periods and notably near business cycle turning points. Yet for many analysts and forecasters, the results indicate that tracking the performance of those key indicators may yield most of the information that could be gleaned from a larger, more comprehensive set of indicators. But using a flexible estimation approach such as in our system is also apparently important for best tracking the information from those indicators.

An alternative methodology: GDP by components

Another alternative methodology would be to apply the system estimation procedures to an estimation-by-components approach for the components of the accounting identity for GDP of: C-F /+ G +(X--M). Such a methodology could be promising, especially given the challenges we have observed in repeated use over time of the system estimation, notably that the system may do particularly well in estimating an aggregate indicator such as the change in private domestic real final sales. That is, the system does less well in predicting real GDP growth when there are substantial changes in more-exogenous components such as changes in inventories, government spending, or net exports. The first quarter of 2011 is a notable example; our system estimate overstated real growth in that period as government spending registered a particularly large decline. A components approach would move more toward the "bean counting" accounting approach described earlier; a forecaster could use system-based estimates by component independently or to augment or serve as a check for the bean-counting estimates for current-quarter GDP growth. Although we have not conducted formal investigation of this approach, we have observed that making real-time estimates of some of the components--and notably government spending--is difficult in practice and could limit the ability to use the system in such an application.

Potential applications to specific industries, indices, custom measures

The real-time system we describe in this paper can potentially be applied to almost any series for which sufficient historical data exist to estimate relationships to other indicators. Obvious candidates are the major monthly macroeconomic variables such as: industrial production, durable goods orders and shipments, payroll jobs and the unemployment rate, and so on. For example, Parigi, Golinelli, and Bodo [2010] use a real-time, nowcasting approach for short-run predictions of industrial production for Italy. In fact, in the next section, we illustrate just such an application to the monthly payroll jobs estimate from the BLS. Before turning to that, we note that the potential also exists to construct special indices or measures for a given industry or industries, or even a given firm, and then use the system to regularly update estimated performance/activity based on the system's estimated relationships, and that would then evolve automatically through time. One can even imagine using an applied version of the system to make estimates of a performance measure for a firm or industry that is unobservable in real time, and that may not be directly observable until after a substantial lag of time (for example, potentially even a year or more).

An application to monthly payroll jobs estimation

To illustrate how our real-time forecast system can be applied to an alternative economic indicator other than real GDP growth, we rewrote the EViews code to apply it to one of the more-important and well-known economic indicators, the monthly payroll jobs estimate from the BLS. Given the closer matching of monthly frequencies for most of the explanatory indicators used and the dependent variable of payroll jobs, the main differences in the EViews code were to eliminate the nested loops for making sure the data for the proper month(s) of the quarter were used and making the code apply to a single month rather than multiple months and quarterly averages. Also, we used a subset of key economic indicators that are typically or often available prior to the release of the Employment Situation report and have a sufficiently long time series. Hence, we use data on initial and continuing unemployment insurance claims, the employment measure from the Philadelphia Fed's business outlook survey, consumer confidence and sentiment, the S&P500 index, indices on business activity and employment from the Institute for Supply Management (ISM), and the ADP employment report. Some of these series are available in the weeks of the month prior to and leading up to the Employment Situation release (unemployment insurance claims, the S&P 500 index, consumer sentiment, Philadelphia Fed index), others are only available closer or immediately before (consumer confidence, revised consumer sentiment, ADP), and sometimes some of these indicators are released just before or potentially not released prior to the Employment Situation (ISM survey measures). This group of indicators is generally well-known as the key set of monthly indicators available prior to the Employment Situation, and many are used by analysts and forecasters to inform their personal estimates for payroll jobs growth in advance of the release of the estimates. Again, one of the key aspects of our analysis is the system's computer program and code that allow the flexible and (almost) instantaneous updating and data-determined choices of empirical specifications and sample horizons and for an ongoing process of making payroll jobs estimates from month to month.(10)

Similar to the analysis for the evolution of the system estimate for real GDP, we present the evolution of estimates across a month for payroll jobs for May 2012 (Tables 10-15). Note that the "jagged edge" phenomenon and duration are greatly reduced in this application, spanning one month rather than the three months of the quarter for the real GDP application. Table 10 shows an initial system estimate in the middle of May (May 18) when only very initial observations on any data for the month were observed: data on initial and continuing claims for unemployment insurance for the beginning weeks of May; the preliminary estimate for consumer sentiment for May; the Philadelphia Fed index employment measure; and the S&P500 stock index for the first half of May. The data thus represented an initial snapshot for May, but with key data on unemployment insurance claims available by that time. The system's initial estimate at that time was for payroll jobs growth of 93,000 jobs.
Table 10. Real-time Forecast for Payroll Jobs for May 2012, May 18

Indicator   Prediction  [R.sup.2]  Specification  Sample    Weighted
              (jobs)                                      Contribution
                                                             (jobs)

UI CLAIMS          141      0.599              5     252            36

Philly Fed          99      0.659              5     165            28
Empl.
Index

Continuing          92      0.687              5      99            27
UI Claims

S&P500              35      0.095              3     300             I

Sentiment            9      0.305              5     246             1

ISM                 --         --             --      --            --
Non-Mfg

Confidence          --         --             --      --            --

ISM                 --         --             --      --            --
Employment

ISM Non             --         --             --      --            --
Mfg. Empl

ADP                 --         --             --      --            --

Real-time                                                           93
Forecast,
Current
Month
Payroll
Jobs
Change(1)

Median                                                              92
Forecast

(1.)Weights determined by relative [R.sup.2] weighting.
Specification reference for expl var: 1 is first difference; 2 is first
percent change; 3 is first and lagged differences; 4 is lagged percent
change; 5 is current level and first difference; 6 is current and lagged
percent change.


Table 11 shows how the system estimates changed over the following week (through May 25) as more data came in, although not substantial changes, with the system estimate rising only slightly to 95,000. By May 29 (Table 12) consumer confidence was available (and yielding a very low, negative estimate), pulling down the system estimate to 84,000. By the end of May (May 31), Table 13 shows that the addition of the ADP estimate for May (at + 133,000) yielded the highest indicator-specific estimate at 145,000 jobs, and raised the system estimate to 99,000. Note this was a month when the ISM data were not available prior to the Employment Situation release. The market consensus during this period prior to the ADP release was for an increase in payroll jobs of 150,000, with a range of 95,000 to 206,000. On June 1, the BLS released its first estimate for payroll jobs growth in May at 69,000. Hence, in the comparison of this example for May 2012, the real-time system estimates of job growth lower than 100,000--and well below the consensus--were "correct" and relatively better than the market consensus. However, even the relatively low estimates of 84,000 (pre ADP) and 99,000 (post ADP) from the system were high compared with the BLS' initially reported 69,000.
Table 11. Real-time Forecast for Payroll Jobs for May 2012, May 25

Indicator   Prediction  [R.sup.2]  Specification  Sample      Weighted
                 Gobs)                                    Contribution
                                                                (jobs)

UI CLAIMS          141      0.599              5     252            36

Philly Fed          99      0.659              5     165            28
Empt.
Index

Continuing          91      0.687              5      99            27
UI Claims

S&P500              56      0.095              3     300             2

Sentiment           20      0.305              5     246             3

ISM                 --         --             --      --            --
Non-Mfg

Confidence          --         --             --      --            --

ISM                 --         --             --      --             _
Employment

ISM Non             --         --             --      --            --
Mfg. Empl

ADP                 --         --             --      --            --

Real-time                                                           95
Forecast.
Current
Month
Payroll
Jobs
Change (1)

Median                                                              91
Forecast

(1.) Weights determined by relative R2 weighting.

Specification reference for expl var: 1 is first difference;
2 is first percent change; 3 is first and lagged differences;
4 is lagged percent change; 5 is current level and first difference;
6 is current and lagged percent change.

Table 12. Real-time Forecast for Payroll Jobs for May 2012, May 29

Indicator   Prediction  [R.sup.2]  Specification  Sample      Weighted
                (jobs)                                    Contribution
                                                                (jobs)

UICLAIMS           141      0.599              5     252            33

Philly Fed          99      0.659              5     165            25
Empl.
Index

Continuing          91      0.687              5      99            24
UI Claims

S&P500              56      0.095              3     300             2

Sentiment           20      0.305              5     246             2

Confidence         -19      0.253              5     249            -2

ISM                 --         --             --      --             _
Non-Mfg

ISM                 --       . --             --      --            --
Employment

ISM Non             --         --             --      --            --
Mfg. Empl

ADP                 --         --             --      --            --

Real-time                                                           84
Forecast.
Current
Month
Payroll
Jobs
Change(1)

Median                                                              74
Forecast

(1.) Weights determined by relative R2 weighting.

Specification reference for expl var: 1 is first difference;
2 is first percent change; 3 is first and lagged differences;
4 is lagged percent change; 5 is current level and first
difference; 6 is current and lagged percent change.

Table 13. Real-time Forecast for Payroll Jobs for May 2012, May 31

Indicator   Prediction  [R.sup.2]  Specification  Sample      Weighted
                (jobs)                                    Contribution
                                                                (jobs)

A DP               145      0.859              1     137            36

UI CLAIMS          131      0.599              5     252            23

Continuing         100      0.687              5      99            20
UI Claims

Philly Fed          99      0.659              5     165            19
Empl.
Index

S&P500              55      0.095              3     300             2

Sentiment           20      0.305              5     246             2

Confidence         -19      0.253              5     249            -1

ISM                 --         --             --      --            --
Non-Mfg

ISM                 --         --             --      --            --
Employment

ISM Non             --         --             --      --            --
Mfg. Empl

Real-time                                                           99
Forecast,
Current
Month
Payroll
Jobs
Change(1)

Median                                                              99
Forecast

(1.) Weights determined by relative R2 weighting.

Specification reference for expl var: 1 is first difference;
2 is first percent change; 3 is first and lagged differences;
4 is lagged percent change; 5 is current level and first
difference; 6 is current and lagged percent change.


The system estimates in Tables 14 and 15 include the data on the ISM manufacturing employment index (Table 14) released on June 1 after the Employment Situation release, and the ISM nonmanufacturing business activity and employment indices (Table 15) released on June 5. The estimate from the system with all indicators included is for 105,000 jobs for May, still above the 69,000 initial BLS estimate, but in the bottom of the range of the private market predictions cited above and much closer than the consensus. With subsequent revisions in the June and July reports, the payroll jobs change for May was at 87,000, closer to the system estimate.
Table 14. Real-time Forecast for Payroll Jobs for May 2012, June 1

Indicator   Prediction  [R.sup.2]  Specification  Sample      Weighted
                (jobs)                                    Contribution
                                                                (jobs)

ISM                197      0.605              5     137            29
Employment

ADP                145      0.859              1     137            31

UI CLAIMS          131      0.599              5     252            19

Continuing         100      0.687              5      99            17
UI Claims

Philly Fed          99      0.659              5     165            16
Empl.
Index

S&P500              55      0.095              3     300             1

Sentiment           20      0.305              5     246             2

Confidence         -19      0.253              5     249            -1

ISM                 --         --             --      --            --
Non-Mfg

ISM Non             --         --             --      --            --
Mfg. Empl

Real-time                                                          114
Forecast.
Current
Month
Payroll
Jobs
Change(1)

Median                                                             100
Forecast

(1.) Weights determined by relative R2 weighting.

Specification reference for expl var: 1 is first difference;
2 is first percent change; 3 is first and lagged differences;
4 is lagged percent change; 5 is current level and first
difference; 6 is current and lagged percent change.

Table 15. Real-time Forecast for Payroll Jobs for May 2012, June 5

Indicator   Prediction  [R.sup.2]  Specification  Sample      Weighted
                (jobs)                                    Contribution
                                                                (jobs)

ISM                196      0.603              5     137            25
Employment

ADP                144      0.858              1     137            26

UI CLAIMS          131      0.599              5     252            16

Philly Fed         100      0.657              5     165            14
Empl.
Index

Continuing          99      0.686              5      99            14
UI Claims

ISM Non             65      0.390              5     180             5
Mfg. Empl

S&P500              55      0.095              3     300             1

ISM                 51      0.315              5     180             3
Non-Mfg

Sentiment           20      0.306              5     246             1

Confidence         -19      0.253              5     249            -1

Real-time                                                          105
Forecast.
Current
Month
Payroll
Jobs
Change(1)

Median                                                              82
Forecast

(1.) Weights determined by relative R2 weighting.

Specification reference for expl var: 1 is first difference;
2 is first percent change; 3 is first and lagged differences;
4 is lagged percent change; 5 is current level and first
difference; 6 is current and lagged percent change.


6. What is the Useful Information in these Estimates?

For the real-time GDP estimates, the estimates are probably best viewed as measuring the underlying performance of GDP growth, of underlying growth in the economy. Often very special, even idiosyncratic, changes to components will yield a real GDP growth number for a specific quarter that is quite different from the overall underlying performance. While there is potentially limited value in correctly predicting a one-time low or high value when the economy is growing at a fundamentally different rate, nonetheless it would still be of value to understand if there were special factors that resulted in an estimate being particularly high or low.

The lamp post problem and private domestic activity

The behavior of temporarily high or low quarterly growth estimates in real time also highlights some of the challenges of making such estimates in our framework and methodology. The effective weighting is determined by the relative performance of the fits of the equations, the R-squares, but in practice that may result in lower weighting for important variables for constructing the real GDP growth estimate. This is a version of the "lamp post" problem, that the data and resulting equations being used are the ones that can be readily observed and through time, while idiosyncrasies and special factors that are more difficult to observe could move the current-quarter estimate relative to what is the typical relationship over history.(11) The observable information may potentially be better in many periods at measuring private domestic activity--sudden shifts in net exports and government are particularly difficult to capture. However, this is not a universal generalization, as various quarters historically had outsized changes in real private domestic final sales that were difficult to predict (notably around the deepest parts of the recent recession). Nonetheless, being able to identify when a particularly low value or high value for a quarter is a fundamental change in underlying performance would be of particular value for business, information, and policy purposes.

Information by sector

The detailed indicator-specific estimates from the real-time estimation system provide information on how specific industries or sectors are performing given their historical relationship to real growth in the economy. For example, what are the estimates and the relative magnitudes from "real economy" indicators, or from labor market indicators, or from sentiment and survey data? Those estimates and relationships can provide potentially useful information on the relative performance of differing parts of the economy. Additional research could examine, for example, whether sentiment vs. real activity disparities carry additional useful information about the current or expected future performance of the economy.(12)

Reliability of the estimates and the data

The preliminary nature of initial and subsequent estimates for real GDP growth--and the relatively long time before such estimates go through annual or comprehensive revisions--highlights some of the challenges of forecasting real GDP growth. What confidence do we have that the reported values of real GDP growth are properly capturing the true performance of the economy in real time? Or, could it be that the real-time estimates from a system such as ours could in fact give a more useful measure of underlying performance because of the direct tie to the broad set of underlying data rather than being tied to the accounting framework and associated initial data limitations for the formal estimates? The broader questions of such challenges are beyond the scope of this paper, but we will note that the BEA publishes a set of comparisons for revisions to GDP in its initial (advance) estimates for a quarter. For example, in its January 2012 release for its estimates of GDP for the fourth quarter of 2011, the BEA reported that for real GDP the average deviation without regard to sign of the "advance" estimate to the "final" (post annual and comprehensive revision) estimate for the percent change was 1.3 percentage points, and with a standard deviation of 1.0 percentage points (for comparisons for the period 1983 to 2008).(13) Hence, substantial uncertainty exists in real time regarding an actual point estimate even for the government's formal estimate of the rate of real GDP growth. A separate, broad-set-of-indicators-based performance measure based on historical relationships to real GDP growth can therefore potentially provide additional useful information in real time regarding the underlying performance of the economy. An analogous challenge exists for the formal BLS estimates for payroll jobs growth in real time, as the jobs growth estimates are subject to initial monthly revisions for two months and then, later, annual benchmark revisions that can lead to substantial changes in the month-to-month estimated changes.

Other high frequency and social media data

With the increased use of internet search engines and social media over the past decade and the availability of information regarding their use, the prospects for using high frequency and high volume data from those sources appear attractive in terms of gaining additional information in real time about the behavior of the economy and key actors, such as consumers and investors. Choi and Varian [2009], for example, examine data on search engine searches and related economic data and claim that "Google Trends may help in predicting the present," and Bollen, Mao, and Zeng [2011] -analyze the text content of daily Twitter feeds" and find that that "the accuracy of [Dow Jones industrial average] predictions can be significantly improved by the inclusion of specific public mood dimensions."

We have made efforts to use various high frequency data in the real-time system but have not observed much success. In general, the information gained from such sources is very noisy, and it is difficult to identify a strong enough signal for generating significant explanatory power for the aggregate economic variables of interest--espe-cially in terms of providing additional explanatory information beyond the variables we already include in the system. Further, our analysis requires a sufficiently long historical time series to estimate the relationship of the series to the aggregate economic variable of interest. While there may be useful information in high frequency and social media series, at this time we have not been able to reliably identify or incorporate additional robust and significant explanatory information in our system.

7. Concluding Observations

The flexible real-time forecasting system presented in this paper provides a useful analytical tool for generating ongoing, data-based "nowcasts" of real GDP growth. The prediction errors for the system estimates over a historical period compare favorably with those of individual indicators, and the flexible nature of the system--allowing the data to decide the specifications and sample horizons for estimations--yields improved prediction performance relative to estimations with fixed specifications and sample horizons. Also, while the full system estimates adjusted for observed bias and inefficiency produced the lowest root mean square errors, the evidence suggests that a small subset of key indicators does almost as well as the full system estimate, particularly for periods of more extreme real GDP growth. The analysis indicates that forecasters can potentially glean useful information from applying such an approach in practice for making real GDP growth estimates; further, they may be able to do so relatively well with a small set of key indicators. The analysis also demonstrated the opportunities for applying the methodology to other indicators by converting the system to examine real-time estimates for the monthly change in payroll jobs. In the end, although challenges exist for applying the system for making predictions in practice, the analysis and evidence presented in this paper indicate that the system can provide potentially useful real-time information about real GDP growth and other key economic variables.

REFERENCES

Angelini, Elena, Gonzalo Camba-Mendez, Domenico Giannone, Lucrezia Reichlin, and Gerhard Riinstler. 2011. "Short-Term Forecasts of Euro Area GDP Growth." The Econometrics Journal, 14(1): C25-44.

Blue Chip Economic Indicators, 2012 Aspen Publishers, 37(5), May 10.

Bollen, Johan, Huina Mao, and Xiao-Jun Zeng. 2011. "Twitter Mood Predicts the Stock Market." Journal of Computational Science, 2(1): 1-8.

Bulligan, Guido, Roberto Golinelli, and Giuseppe Parigi. 2010. "Forecasting Monthly Industrial Production in Real-Time: From Single Equations to Factor-Based Models." Empirical Economics, 39(2): 303-36.

Choi, Hyunyoung, and Hal Varian. 2009. "Predicting the Present with Google Trends," unpublished manuscript, Google, Inc., April 10.

Diebold, Francis X., and Jose A. Lopez. 1996. "Forecast Evaluation and Combination," NBER Techincal Working Paper No.192, March 1996, published in Handbook of Statistics 14: Statistical Methods in Finance, edited by G.S. Maddala and C.R. Rao, North-Holland.

Drechsel, Katja, and Laurent Maurin. 2011. "Flow of Conjunctural Information and Forecast of Euro Area Economic Activity." Journal of Forecasting, 30(3): 336-54.

Evans, Martin D.D. 2005. "Where Are We Now? Real-Time Estimates of the Macroeconomy." International Journal of Central Banking, 1(2): 127-75.

Ferrara, Laurent, Dominique Guegan, and Patrick Rako-tomarolahy. 2010. "GDP Nowcasting with Ragged-Edge Data: A Semi-Parametric Modeling." Journal of Forecasting, 29(1-2): 186-99.

Giannone, Domenico, Lucrezia Reichlin, and Saverio Simonelli. 2009. "Nowcasting Euro Area Economic Activity in Real-Time: The Role of Confidence Indicators." National Institute Economic Review, 210(1): 90-7.

Giannone, Domenico, Lucrezia Reichlin, and David Small. 2008. "Nowcasting: The Real-Time Informational Content of Macroeconomic Data." Journal of Monetary Economics, 55(4): 665-76.

Golinelli, Roberto, and Giuseppe Parigi. 2008. "Real-Time Squared: A Real-Time Data Set for Real-Time GDP Forecasting." International Journal of Forecasting, 24(3): 368-85.

Kitchen, John, and Ralph Monaco. 2003. "Real-Time Forecasting in Practice: The U.S. Treasury Staff's Real-Time GDP Forecast System." Business Economics, 38(4): 10-9.

Parigi, Giuseppe, Roberto Golinelli, and Giorgio Bodo. 2010. "Forecasting Industrial Production in the Euro Area." Empirical Economics, 25(4): 541-61.

Riinstler, Gerhard, Karim Barhoumi, Szilard Benk, Ric-dardo Cristadoro, Ard Den Reijer, Audrone Jakaitiene, Piotr Jelonek, Antonio Rua, Karsten Ruth, and Christophe Van Nieuwenhuyze. 2009. "Short-Term Forecasting of GDP Using Large Datasets: A Pseudo Real-Time Forecast Evaluation Exercise." Journal of Forecasting, 28(7): 595-611.

Stock, James H., and Mark W. Watson. 2006. "Forecasting with Many Predictors," in Handbook of Economic Forecasting, Vol. 1, Chapter 10, edited by G. Elliott, C. Granger, and A. Timmermann. Elsevier.

Wagner, Neal, Zbigniew Michalewicz, Moutaz Khouja, and Rob Roy McGregor. 2005. "Forecasting with a Dynamic Window of Time: The DyFor Genetic Program Model.- Lecture Notes in Computer Science, 3490: 205-15.

Paper prepared for the National Association for Business Economics 2012 Mennis Award for presentation at the NABE 2012 Annual Meeting. Views expressed in the paper are the authors' and do not represent the views of the U.S. Department of the Treasury or any other institution.

doi:10.1057/be.2012.36

(1.) "Real-time" research and analyses can have two different definitions: one definition concerns examining the role of the ex post use of the contemporaneously available vintage of data during historical periods relative to the current vintage of data; the second definition is the one used in this paper, the use of currently available data in real time to produce estimates of the variable of interest as it is being formed and prior to the release of the formal estimate.

(2.) Ferrara, Guegan, and Rakotomarolahy [2010] refer to "ragged edge" data; both "jagged" and "ragged" refer to the same phenomenon. We use "jagged" in our discussion.

(3.) Private firms have also produced proprietary and client-restricted estimators for current real GDP growth, including Goldman Sachs current activity index and Moody's Analytics high frequency model.

(4.) In an interesting application that was observed as this paper was being written, Wagner and others [2005] illustrated a "dynamic forecasting genetic program" modeling approach for forecasting GDP in a nonstatic environment.

(5.) The notation as presented is simplified by not using differing notations for the alpha and beta coefficients across the specifications even though those would obviously differ across the specifications.

(6.) 0ther specifications, including varying forms with lagged dependent variable or ARMA models, could be considered; the specifications we used reflect our ex ante determination to focus on the role of individual indicators and the estimated indicator-specific relationships for making predictions of the dependent variable.

(7.) Stock and Watson [2006] discuss alternative "forecast combining methods" including relative historical and forecast performance weightings as shown and discussed here.

(8.) We arbitrarily chose to use the top five to illustrate the small subset.

(9.) The specifications were predetermined by subjective ex ante analysis by indicator.

(10) Because of the large number of iterations and loops to compare alternative specifications and samples, the EViews program for the real-time system for monthly payroll jobs estimates requires just over 3 minutes to run on a Dell T3500 computer with an Intel Xeon 3.20 GHz processor and a Windows 7 operating system. Analogously for the real GDP system, the time required for the system to run is about seven and a half minutes.

(11.) The "lamp post problem" is derived from an old joke: A drunk is looking for his car keys under a lamp post. A passerby asks if he is sure that he lost them there. The drunk replies that he lost the keys over there, in the dark. "So why are you looking for them here?" asks the passer-by. "Because the light is better," replies the drunk.

(12.) Related to this, Giannone. Reichlin, and Simonelli [2009] address the role of confidence indicators in real-time estimation for euro area real activity.

(13.) The RMSE of 1.3 percentage points for our adjusted system estimate compares relatively well with those BEA deviations.

JACKSON KITCHEN and JOHN KITCHEN *

*Jackson Kitchen is a recent graduate in economics from The College of William and Mary has worked at the American Road & Transportation Builders Association in Washington, D.C. and at Competition Dynamics in Salem, Massachusetts.

John Kitchen is an economist with the U.S. Department of the Treasury, with previous positions at the Office of Management and Budget, the House Budget Committee, and the Council of Economic Advisers. He has also taught in graduate programs at the University of Maryland and Wake Forest University. He received his Ph.D. from the University of Pittsburgh.
COPYRIGHT 2013 The National Association for Business Economists
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Comment:Real-time forecasting revisited: letting the data decide.
Author:Kitchen, Jackson; Kitchen, John
Publication:Business Economics
Geographic Code:1USA
Date:Jan 1, 2013
Words:12856
Previous Article:In search of system understanding and control.
Next Article:Is productivity growth too strong for our own good?
Topics:

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters