Interpreting the performance of business economists during the great recession.Ii is generally believed that the recession of 2007-09 was not foreseen fore·see
tr.v. fore·saw , fore·seen , fore·see·ing, fore·sees
To see or know beforehand: foresaw the rapid increase in unemployment. by business economists. Is this perceived view accurate? We explore this issue by examining business economists' published statements about economic conditions. We compare these qualitative forecasts with the Beige Book Beige Book
A commonly used name for the Fed report entitled "Summary of Commentary on Current Economic Conditions by Federal Reserve District." It is published just before the FOMC meeting on interest rates and is used to inform the members on changes in the economy since the last . We conclude that both sets of data are similar and that business economists are responsive to information about the economy and adjust their predictions quickly Business Economics (2012) 47, 148-154.
Keywords: business forecasts, great recession, beige book, payroll data
It is generally believed that the recession of 2007 09 was not foreseen by 1110sf business economists. While Nouriel Roubini Nouriel Roubini born on March 29, 1958 in Istanbul, Turkey, is a professor of economics at New York University. He is also the chairman of Roubini Global Economics. did forecast the recession in advance he had been predicting such a decline for years and his timing was obviously not correct. Is the perceived view about the accuracy of other business forecasters accurate? We explore this issue by examining business economists published statements about economic conditions.
These published statements were usually ot a qualitative rather than numerical nature and had to be converted into quantitative data by a method described below in order to be evaluated. The forecasts covering the period January 2007 through December 2008 were obtained from the Wall Street Journal Eastern Edition by searching the ProQuest ABI Inform Complete article database. (1) This search uncovered articles ranging from opinion pieces regarding the world economy to summaries of the Federal Open Market Committee (FOMC See Federal Open Market Committee.
See Federal Open Market Committee (FOMC). ) meetings. Only those articles with direct statements made by businesses or Wall Street Journal authors concerning the current or future state of the U.S. economy were included in our database. We excluded statements made by officials of the Federal Reserve System. Other U .S. government and international institution forecasts were included only if discussed by business or Wall Street Journal authors in the context of the business environment. Altogether we obtained 231 forecasts.
1. Evaluation Methodology
Because a Large majority of the forecast statements in our sample were qualitative in nature. it was necessary to convert them into quantitative data in order to evaluate them. (2) We used a method for scoring qualitative in Formation employed by Goldfarb and others  to evaluate qualitative forecasts from the Great Depression. This proCedure converts qualitative statements into quantitative data.
The individual forecasts provided qualitative assessments about both the current state and the future direction of the economy. These two types of assessments were analyzed an·a·lyze
tr.v. an·a·lyzed, an·a·lyz·ing, an·a·lyz·es
1. To examine methodically by separating into parts and studying their interrelations.
2. Chemistry To make a chemical analysis of.
3. separately. We scored each statement using an optimism-pessimism scale that ranged from - 1 to + 1 with gradations or steps of 1/4
For example% + 1 was assigned to any quotation indicating that the economy was doing very well and would expand strongly. Statements that the economy was already in a deep recession, or would be in a particularly had recession, were given a score of - 1. Any citation indicating the economy's direction was unclear was given a 0. Table 1 presents the relationship between the scores and the paraphrased statements.
Table 1, Criteria for Scoring Qualitative Forecasts General Mindset Score Condition Diagnosed or Forecast Optimism 1 The economy is strong or will expand very strongly 3/4 The economy is growing normally or will definitely continue to grow 1/2 The economy is growing at a "modest" pace or will do well barring unforeseen events 1/4 There is some risk of a recession or downturn < 30%, or the economy will still grow but slower than usual Neutral 0 It is unclear where the economy is or where it will go because the signs are mixed Pessimism 1/4 The economy is visibly slowing, "decelerating," or there is quite a bit of risk of a recession, >30% but <60% -1/2 The economy is "sluggish'7, barely growing, or there is >60% risk of recession -3/4 The economy is declining, will contract, or there are mild recession conditions -l Recession conditions are here or imminent and it is worse than any recession in recent history
Separate scores were assigned to statements about the current situation and to projections about the future condition of the economy. We thus could separately examine the economists' assessment of the current situation and their forecasts of the future on the aforementioned a·fore·men·tioned
The one or ones mentioned previously.
Adj. 1. scale. Two time series of these scores were then constructed by averaging the scores of the individual forecasts made in each month--one time series for the statements about the present and another about the future.
Validation of the scoring procedure
Goldfarb and others  validated their methodology by comparing their scores for the forecasts made with the monthly analyses of economic conditions published in the 1929-30 issues of the Federal Reserve Bulletin. Similarly. we verify the appropriateness of our scoring of the business economists' qualitative statements by comparing our results with an analogous analogous /anal·o·gous/ (ah-nal´ah-gus) resembling or similar in some respects, as in function or appearance, but not in origin or development.
adj. scoring of the qualitative statements in various issues of the Federal Reserve Beige Book. Compiled periodically by each of the 12 Federal Reserve Banks, the Beige Book is a qualitative analysis Qualitative Analysis
Securities analysis that uses subjective judgment based on nonquantifiable information, such as management expertise, industry cycles, strength of research and development, and labor relations. of economic conditions within each Banks region. (3) Released two weeks before each FOMC meeting, the Beige Book contains information about the state of the US. economy that the FOMC considers in setting monetary policy. The Beige Book is important in this decision context because it becomes available before certain official U.S. macroeconomic mac·ro·ec·o·nom·ics
n. (used with a sing. verb)
The study of the overall aspects and workings of a national economy, such as income, output, and the interrelationship among diverse economic sectors. data are released.4 Although this information is anecdotal anecdotal /an·ec·do·tal/ (an?ek-do´t'l) based on case histories rather than on controlled clinical trials.
anecdotal adjective Unsubstantiated; occurring as single or isolated event. , previous research showed that the Beige Book generally provides valid information about the current direction of the U.S. economy [see Fettig and others 1 999; Balke and Yucel 2000; Ginther and Zavodny 2001; Balke and Petersen 2002; and Armesto and others 2009].
A comparison of the Beige Book information with our scores poses several possible technical problems. The first involves timing. While our data constitute a time series of monthly observations, the Beige Book is not published monthly because the FOMC meets on'ly eight times a year. Thus, there are missing observations for some months.
Second, economic conditions are described differently across the Federal Reserve districts. Some branches tend to be much more conservative in their descriptions while others use more extreme language to describe similar events. Thus, the scores assigned to each Banks statements may not be consistent. Finally, it has also been noted that some districts are better than others at accurately gauging the state of their regional economy: some seem to focus only on the city where the district bank is located [Armesto and others 20091. Despite these potential discrepancies, most analyses that scored the Beige Book statements and compared them with actual macroeconomic data fOUnd that this source generally tracked the U.S. economy well. Therefore, we can be relatively confident thai by using the same scoring scale and adjusting for missing dates. the beige Book should be an excellent benchmark for evaluating the business economists statements about the U.S. economy.
The closing date ['or collecting data for the Beige Book was used to determine the date to which the average scores was attributed. For example February 26, 2007 was the closing date for collecting data that were in the March 2007 report. Consequently, the date assigned for the inFormation in the March Beige Book was February 2007.
In scoring the statements in every Beige Book, particular attention was paid to the primary sentence that described the state of each regional economy. The scorings were identical to those applied to the Wall (5) freer Journal articles, but with particular attention paid to key words, such as "modest growth and 'sluggish growth.' The Beige Book score assigned to each month was the average of the scores obtained from the statements of 12 Regional Banks.
The evaluation procedure compared the scores ot the business statements with the Beige Book scores to determine whether they were consistent. A distinction was made between current and future period Forecasts. All Wall Street Journal statements referring to the current situation were assumed to be assessments of the economic conditions prevailing in the previous month: all statements about the future were assumed to concern the very near term. For example. a statement made in April 2007 about current conditions would be assigned to March 2007. It was compared with the Beige Book score, which, presumably pre·sum·a·ble
That can be presumed or taken for granted; reasonable as a supposition: presumable causes of the disaster. measured the actual economic conditions that prevailed in that month.
It was assumed that a business forecast made in April 2007 referred to conditions expected in April. May. and June of' 2007. Given this assumption the Wall Street Journal forecast score for month I should be compared with the actual conditions that prevailed in those three months. Consequently, the scores assigned to forecasts made in month t were compared with the average of the Beige Book scores for months t, t + 1, and t + 2. (6), (7)
Assessment of current conditions
Table 2 presents the comparison of the Wall Street Journal current assessments with the Beige Book. On average throughout 2007 and 2008 the Wall Street Journal current assessments are more pessimistic pes·si·mism
1. A tendency to stress the negative or unfavorable or to take the gloomiest possible view: "We have seen too much defeatism, too much pessimism, too much of a negative approach" than the comparable Beige Book assessments. Although the business current assessments of the economy are more pessimistic, both series have strong downward trends (see Figure 1) and the majority of scores for the Beige Book and the Wall Street Journal have the same sign. However. the business scores show more month-to-month volatility. Overall, it appears that the business economists had a reasonable understanding of the actual conditions of the U.S. economy as it evolved throughout 2007 and 2008.
This view is confirmed when movements in the business scores are compared with a series considered a coincident indicator Coincident Indicator
An economic factor that varies directly and simultaneously with the business cycle, thus indicating the current state of the economy.
Some examples include nonagricultural employment, personal income, and industrial production. of turning points in the U.S. economy: Nonfarm payroll employment (PAYEMS). Figure 2 presents the movements of the business scores and the growth rates Growth Rates
The compounded annualized rate of growth of a company's revenues, earnings, dividends, or other figures.
Remember, historically high growth rates don't always mean a high rate of growth looking into the future. ol the payroll data. (8) Both series show similar trends, hut there is more volatility in the Wail Street Journal assessments. (9)
Table 2. Assessment of Conditions: Scores of Forecasters and Beige Book, December 2006-November 2008 Dee-06 Jan-07 Feb-07 Mai-07 Apf-07 May-07 Jun-07 Score of 0.38 0.07 0.04 0.38 0.13 -0.50 -0.25 Representative Forecaster Score of Beige 0.38 NA 0.38 NA 0.54. 0.60 NA Book Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Score of -0.25 0.54 -0.61 -0.68 -0.38 -0.63 -0.88 Representative Forecaster Score of Beige 0.17 NA -0.15 -0.35 NA -0.17 NA Book InJ-07 Aug-07 Sep-07 Oct-07 Nov-07 Score of 0.45 -0.08 0.13 -0.17 -0.26 Representative Forecaster Score of Beige 0.44 0.33 0.25 NA 0.13 Book Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Score of -0.07 -0.38 -0.63 -0.75 -0.92 Representative Forecaster Score of Beige -0.10 -0.15 -0.40 NA -0.60 Book Note: The assessment in row 1 of conditions prevailing in a particular month is based on the individual's statement in the subsequent month (that is--a statement made in a May report about current conditions refers to the situation prevailing in April).
The Wall Street Journal forecasts have a very similar relationship to both the average Beige Book scores and to nonfarm payrolls Nonfarm payrolls is an economic employment report released monthly.
It is a compiled name for goods-producing, construction and manufacturing companies. The data is released at 1:30pm BST on the first Friday of every month, or according to the U.S. as in the case oF the current. assessments. Using our scoring procedures. the forecasts of the business economists are more pessimistic than the Beige Book assessments of actual conditions that prevailed in the periods being Forecasts as shown in Table 3. The differences between the Wall Streek Journal forecasts and the average Beige Book scores in Table 3 are similar to those in Table 2 although slightly larger and even more pessimistic. Throughout 2007. the Wall Street Journal data posted negative or near zero forecast scores while the average Beige Book score remained positive up until February 2008, the first report alter December 20O7 which the National Bureau of' Economic Research has identified as the beginning oF the 2008-09 recession. Again both series exhibit strong negative trends as seen in Figure 3. The much higher degree of volatility in the Wall Street Journal forecasts is also replicated in the PAYEMS comparison in Figure 4. (10)
Table 3. Evaluation of Forecasts: Scores of Forecasters and Beige Book, January 2007-December 2008 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07 Jul-07 Score of 0.13 -0.07 0.15 -0.08 -0.25 -0.06 0.00 Representative Forecaster Average Score NA 0.46 NA 0.57 0.52 NA 0.39 of Beige Book for t, t+1, t+ 2 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Jul-08 Score of -0.53 -0.30 -0.88 -0.10 -0.53 -0.33 -1.00 Representative Forecaster Average Score NA -0.25 -0.26 NA -0.14 NA -0.13 of Beige Book for t, t +1, t +2 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Score of 0.04 -0.13 -0.15 -0.19 -0.37 Representative Forecaster Average Score 0.24 0.19 NA 0.15 0.01 of Beige Book for t, t+1, t+ 2 Aug-08 Scp-08 Oct-08 Nov-08 Dec-08 Score of -0.40 -0.63 -0.82 -1.00 -0.90 Representative Forecaster Average Score -0.27 -0.50 NA -0.55 -0.5 of Beige Book for t, t +1, t +2
There may be an explanation for some of this volatility in the business scores. For examples the July 2007 scores show a j imp in optimism and interrupt A signal that gets the attention of the CPU and is usually generated when I/O is required. For example, hardware interrupts are generated when a key is pressed or when the mouse is moved. Software interrupts are generated by a program requiring disk input or output. a generally negative trend. The GDP GDP (guanosine diphosphate): see guanine. data for the second quarter of 2007 had just been released and showed that U.S. GDP had increased by 4 percent. a very strong growth rate. The U.S. financial market was also relatively calm in the summer of 2007. Although some investment banks The following is a list of investment banks Financial conglomerates
Large financial-services conglomerates combine commercial banking and investment banking, and sometimes insurance. continued to have serious problems, these seemed to be manageable without government assistance.
There was also more optimism in the July 2008 business SCOreS. At that point, it appeared that the economic decline would be short-lived and that the Fannie Mae Fannie Mae: see Federal National Mortgage Association. and Freddie Mac Freddie Mac: see Federal Home Loan Mortgage Corporation. bailouts would be all that was necessary to move the economy forward. (11) Even though the job problem was recognized. federal stimulus money from the tax rebate rebate, partial refund of the total price paid for goods or services. In the United States, rebates were historically given by railroads to favored shippers as a return on transportation charges. was working through the economy and was expected to stabilize stabilize
See peg. the situation. This was probably the reason that the Beige Book and the Wall Street Journal had such similar scores in July 2008 and why these scores were optimistic op·ti·mist
1. One who usually expects a favorable outcome.
2. A believer in philosophical optimism.
op relative to PAY EMS.
Finally, the relative volatility Relative volatility is a measure comparing the vapor pressures of the components in a liquid mixture of chemicals. This quantity is widely used in designing large industrial distillation processes. and pessimism pessimism, philosophical opinion or doctrine that evil predominates over good; the opposite of optimism. Systematic forms of pessimism may be found in philosophy and religion. in the Wall Street Journal Scores may also be due to the greater variety of people making statements. as well as variations in the number of statements collected per month from the Wall Street Journal articles. Statements were made by sources ranging from financial businesses, to professional forecasting firms to shipping companies. This variety of statements may also partially explain the greater degree of pessimism in the Wall Street Journal scores. Whereas the language in the various Beige beige
1. A light grayish brown or yellowish brown to grayish yellow.
2. A soft fabric of undyed, unbleached wool.
Light grayish-brown or yellowish-brown to grayish-yellow. Books was relatively conservative, the Wall Street Journal sources may have used more extreme words or more exaggerated statements. In any event, it is possible to draw some conclusions from this analysis of the business economists statements.
In order to evaluate the predictive accuracy of the business economists' qualitative statements it was necessary to convert them into quantitative scores. They were then compared with the scores that were assigned to the qualitative statements in the Beige Book, which served as the benchmark for this evaluation. Both sets of scores had similar time trends. Both the business current assessments and the forecasts became more and more pessimistic as the financial crisis progressed in 2007 and 2008. Both tracked the scores of the Beige Book statements. Based on this evidence, we conclude that the current assessments and forecasts of business economists captured the underlying trend of the economy during 2007-08.
We now return to the question that was posed originally. While the economists understood the underlying trends, there is no evidence that they predicted the Great Recession in advance. (12) While the economists scores became negative in mid-2007, a negative score does not necessarily indicate a prediction of a recession. The scores assigned to the economists' statements are based on an optimism/pessimism scale with a negative score measuring a pessimistic outlook and not a recessionary state, as described in Table 1. Based on the criteria in Table 1, a value exceeding -0.50 would indicate an assessment or forecast of a recession. This level was first exceeded in January 2008. only one month after the date identified by' the National Bureau of Economic Research as the beginning of' the Great Recession.
Thus, this sample of forecasts suggests that. business economists are very responsive to the latest information about the state of the economy and adjust their predictions quickly. On the other hand, there is also evidence that the forecasts are more volatile than warranted because the economists sometimes react too quickly to information that reverses their previous views.
We wish to thank Robert Goldfarb Robert Goldfarb serves as President and CEO of Ruane, Cunniff, and Goldfarb, the value investing firm founded in 1970 by William J. Ruane and Rick Cunniff. Goldfarb is a close friend and disciple of Warren Buffett, the CEO of Berkshire Hathaway. Mr. and Tara Sinclair for their comments on an earlier draft. All remaining errors are the responsibility of the authors.
Armesto, M.T. R. Hernandez-Murillo. M.T. Owyang. and J. Piger. 2009. "Measuring the Information Content of the Beige Book: A Mixed Data Sampling Approach." Journal of Money, Credit and Banking. 41(1): 35-55.
Balke, N.S., and D. Petersen. 2002. "Flow Well Does the Beige Book Reflect Economic Activity4? Evaluating Qualitative Information Quantitatively. Journal of Money, Credit and Banking. 23(1): 114-36.
Balke. N.S., and M. K. Yucel 20OO. Evaluating the Eleventh District's Beige Book Federal Reserve Bank of Dallas The Federal Reserve Bank of Dallas covers the Eleventh Federal Reserve District, which includes Texas, northern Louisiana and southern New Mexico. It has branch offices in El Paso, Houston, and San Antonio. Economic and Financial Review. 2000(4): 2-9.
Evans. K.. A. Trolanovski. and S. Reddy. 2008. "U.S. News: Exports, Stimulus Plan Prop up the Economy: GDP Rose 1.9% But Job Woes Loom" Wall Street Journal (Eastern Edition) (August 1): A:3.
Fettig. D., A.J. Rolnick, and D.E. Runkle. 1999. The Federil Reserve 's Beige Book: A Better Mirror than Crystal Ball. Minneapolis Federal Reserve The Region. http://www.minneapolisfed.org/publications_papers/pub_displav.cfm?id=3568 (accessed March 22, 2010).
Ginther. D.K.. and M. Zavodn . 2001. "The Beige Book: Timely Information on the Regional Economy." Federal Reserve Bank of Atlanta The Federal Reserve Bank of Atlanta is responsible for the 6th District of the Federal Reserve, which covers Alabama, Florida, Georgia, and parts of Louisiana, Mississippi, and Tennessee. Economic Review, 2001(3): 19-28.
Goldfarh. R.S., H.O. Stekler. and D. Joel. 2005. "Methodological Issues in Forecasting: Insights From the Egregious e·gre·gious
Conspicuously bad or offensive. See Synonyms at flagrant.
[From Latin Business Forecast Errors of Late 1930." .Journal of nomic Methodology. 12(4): 517-42.
Stekler. H.O, and R. Talwar A talwar, talwaar, or tulwar (Devanagari: तलवार) is a type of sword prevalent in medieval India dating back to at least the 13th century. It bears a resemblance to the Persian shamshir and the Turkish kilic. . 2011. "An Analysis of' Quantitative Forecasts During the Great Recession."Mimeo
(1.) The search term "recession" was used for the year 2007 and the term "economic forecast" was used for both 2007 and 2008
(2.) The forecasts that are examined here do not include any quantitative forecasts from professional consulting firms.
(3.) The Beige Book has been published since 1970.
(4.) The real GDP estimates are not available until a month after tile tile, one of the ceramic products used in building, to which group brick and terra-cotta also belong. The term designates the finished baked clay—the material of a wide variety of units used in architecture and engineering, such as wall slabs or blocks, floor quarter ends: state employment statistics are only available with a month delay; and gross state product is not available until two years later [Balke and Yuccl 2000; Balke and Petersen 2002; and Ginther and Zavodny 2001].
(5.) Each of the district Federal Reserve branches has their own methods for putting together their reports. but they generally comprise surveys that include, but are not limited to, local businesses, banks, and trade associations [Balke and Petersen 2002; Armesto and others 2009].
(6.) It is important to note that the Beige Book is never released in three consecutive months. Since this is the case, the scores were averaged for the months available over a three month period. For example, in early 2007 data were collected and released on February 26 and April 1 6 hut not in March. Therefore the scores for these two dates were averaged and assigned to February 2 as an average three-month score. March was given no score ill this case.
(7.) To compare the Beige Book scores to the Wall Street Journal scores, the Beige Book dates needed to he adjusted because the [J'all Street Journal scores were all assigned to the last date of every month. The Beige Book scores were originally assigned to the date when their data collection ended. hut the data collection end date is not consistent from survey to survey. If that date was on or before the 15th of a month that score was considered to apply to the last date of the previous month and if it was after the 15th, that score was assigned to the end of the same month.
(8.) The unsealed, continuously compounded annual rates of change for the series 'AIl Employees: Total nonfarm (PAY EMS were used. Available at http://alfred.stlouisfed.org/serics'?seid=PAYEMS. The Wall Street Journal assessments were compared to the unrevised Adj. 1. unrevised - not improved or brought up to date; "the book is still unrevised"
unaltered, unchanged - remaining in an original state; "persisting unaltered through time" growth rates first assigned to the months under study.
(9.) We only found three business current assessments for May 2007, which may account for the differences between the business scores and both the Beige Book scores and the payroll movements.
(10.) Forecasts were compared to the revised estimates for the continuously compouded annual growth rate of nonfarm payrolls (PAYEMS).
(11.) Evans and others ; July 2008. http://timeline.stlouisfed.org/index.efm?p=timeline
(12.) A study of' business economists' quantitative forecasts from 2007 and early 2008 showed that they predicted that the economy Would slow down but not decline [Stekler and Talwar 2011].
* Kathryn Lundquist graduated in May 2011 from the George Washington University with a master's degree in International Trade and Investment Policy and a strong interest in Forecasting. She currently works as an international trade analyst covering aluminum at the L .S. International Trade CommisSion.
Herman 0. Stekler is a Research Professor of Economics at the George Washington University. His specialty is economic Forecasting economic forecasting
Prediction of future economic activity and developments. Economic forecasts, which range from a few weeks to many years, are widely used in business and government to help formulate policy and strategy. with an emphasis on evaluation. He has written extensively on that subject.