Printer Friendly

Quantifying uncertainty: risk analysis for forecasting and strategic planning.

In many respects public decision making has grown more complex in recent years. Public agencies find themselves increasingly influenced by the impacts of national and state legislation, budget constraints on operations, new regulations and growing demands for resources. As this changing environment becomes more complex, it requires the adoption of systematic approaches for evaluating the consequences of alternative management policies and external events. A more strategic orientation is becoming critical, particularly in the formulation of policy, both in the public and private sectors.

Strategic planning, as distinct from traditional approaches, is characterized by:

* a longer timeframe,

* an accounting for contingent developments

and uncertainties,

* linkages between planning and budgeting,


* a dynamic and continuing process.

Forecasts are the cornerstone of strategic planning. Budgets, revenues from taxes and other sources, the costs of projects--all are subject to some kind of forecasting process. There is one thing certain about any forecast: to some degree it will be wrong.

Because forecasts both form the basis for planning and are subject to uncertainty, there are two areas in which conventional forecasting may promote a negative effect in the smooth functioning of government. First, from the perspective of sound management, decision makers may be ill-served by their forecasts: while a forecast may represent a best guess, it does not inform as to the likelihood of the forecasted outcome being achieved nor does it provide insight into the probabilities of alternative outcomes occurring. This is crucial information for almost any management decision in government because it enables the manager to behave strategically. Managers can prepare for various contingencies and provide balanced reporting to both administration and constituents if equipped with the additional results cited above.

Secondly, forecasts can evoke public distrust and erode consensus. When there is no accounting for the uncertainties in a forecast result, this feeds the tendency to view forecasts as being biased in favor of the decisions which are based upon them. This tendency adversely affects the consensus for a public agency's chosen course of action. Not infrequently, that consensus is a key component for successful implementation. Moreover, if the forecasting process and its underlying assumptions remain undisclosed and obscure to the public, then consensus is undermined even further.

The techniques of risk analysis combined with a structured process of expert and public review have proven to be powerful tools which enable decision makers to successfully manage situations subject to uncertainty while building public confidence. Risk analysis modeling and the stages involved in expert and public participation in the forecasting process are described below. An example is presented illustrating how the Arizona Department of Transportation with Hickling Corporation and its Risk Analysis Process have applied these techniques in the forecasting of revenue from an excise tax for the finance of a new freeway system in Maricopa County.

Forecasting and Risk Analysis

Forecasting always involves uncertainty, and the farther into the future projections are made, the more uncertainty there is and the greater the risk of producing forecasts that deviate greatly from actual outcomes. Forecasts traditionally take one of two forms: first, a single "expected outcome," or second, one in which the expected outcome is supplemented by alternative scenarios, often termed "high" and "low" cases. Both approaches fail to provide adequate perspective with regard to probable vs. improbable outcomes.

The limitation of a forecast having a single expected outcome is clear--while it may provide the single best guess, it offers no information about the range of probable outcomes. The problem becomes acute when uncertainty surrounding the underlying assumptions of the forecast is especially high. The high-case/low-case approach can actually exacerbate this problem because it gives no indication of how likely it is that the high and low cases will actually materialize: the high case usually assumes that most underlying assumptions deviate in the same direction from the expected value; and likewise for the low case. In reality, the likelihood that all underlying factors shift in the same direction simultaneously may be as remote as everything turning out exactly as expected.

A common approach to providing added perspective is through sensitivity analysis, whereby key forecast assumptions are varied one at a time in order to assess their relative impact on the expected outcome. A problem here is that the assumptions often are varied by arbitrary amounts. But a more serious flaw in this approach is that in the real world assumptions do not veer from actual outcomes one at a time; it is the impact of simultaneous differences between assumptions and actual outcomes that would provide true perspective on a forecast.

Risk analysis provides a way around the problems outlined above. It helps avoid the lack of perspective in high and low cases by measuring the probability, or odds, that an outcome actually will materialize. This is accomplished by attaching ranges (probability distributions) to the forecasts of each input variable. The approach allows all inputs to be varied simultaneously within their distributions, thus avoiding the problems inherent in conventional sensitivity analysis. The approach also recognizes interrelationships between variables and their associated probability distributions.

The result of a risk analysis is both a forecast and a quantification of the probability that the forecast will be achieved. This takes the form of probabilities that result values will not be exceeded. For instance, in a tax revenue forecast the median result may be $12 million. This means that there is a 50 percent probability that the actual revenue will exceed this value and a 50 percent probability that the actual value will lie below it. However, the results also will reveal a 20 percent probability that the revenue will not exceed, for example, $10.2 million. With this information, representing a possible outcome and an assessment of its likelihood, the decision maker can prepare for contingencies, avoid overpromising, and can report and plan with greater confidence.

Developing a Risk Analysis

The development of a forecast using risk analysis has steps common to conducting any forecast. As a first stage, result variables and factors which are believed to affect them are identified. Existing forecasting models can be accommodated and incorporated into a risk analysis.

In a departure from traditional forecasting, the relationships between inputs factors and results are mapped out diagrammatically to illustrate the structure and logic of the forecasting model. This is done in order to facilitate public involvement and to subject the forecasting process to expert scrutiny. Diagrams help where the presentation of formulas may impede the understanding and participation of nonexperts in forecasting models. The diagrams of the model's structure and logic also serve in the structuring of panel sessions and elicit expert and public response in assigning ranges of uncertainty.

Once the inputs and results are defined and the relations between them are specified, a forecasting model can be developed and integrated with risk analysis software. For each input factor identified, ranges for the values which describe a probability distribution are assigned. For persons less familiar with probability and statistics, this task may seem cumbersome. The manager typically will have an intuitive feel for the uncertainty surrounding some of the inputs, however, and the assignment of ranges is not very different from the way in which the manager usually thinks about uncertainty. For instance, a budget forecast may depend on the number of new jobs created in subsequent years. The manager may expect the number of new jobs in 1994 to be 35. He may also be 90 percent confident that the number will not exceed 42. Likewise, he may be 90 percent confident that the number will not fall below 30.

For other factors, about which the manager may not possess intuition regarding uncertainties, the requisite information usually will be easily accessible. An example may be a program cost forecast which is linked to the price of petroleum. An energy analyst can provide the following input: "The expected price of a barrel of imported crude oil is $21. With 90 percent confidence the price will not exceed S27 nor will it fall below $18."

The above examples illustrate that the assignment of ranges (probability distributions) can be accomplished in a way which is consistent with the manner by which managers and decision makers are used to thinking about them. Risk analysis software packages provide numerous probability distributions which the manager can apply to the input factors; some can automatically generate a probability distribution from a median, upper 10 percent limit and a lower 10 percent limit as in the examples given above and illustrated in Exhibit 1.


The manager, having completed the modeling and the assignment of initial ranges, could use the risk analysis software at this stage to run simulations, thus generating a risk analysis forecast. Through a process called Monte Carlo simulation in which the results are forecasted hundreds or thousands of times for different combinations of input values, risk analysis takes into account all of the uncertainty in the underlying factors and generates all possible outcomes and their related probabilities. Each result variable would be assigned a probability distribution and the additional information would undoubtedly enhance the decision-making capabilities of the manager. To build public confidence in the forecast, the additional phase of panel sessions is required.

Expert and Public Involvement

A panel of experts is convened comprised of individuals from a wide range of backgrounds, tailored to the specific needs of the forecast: government, industry, services, economics and forecasting professions. No single member of a panel is expected to possess expertise in all areas relating to the forecast. The panel session may be open to the public and press, or, may be limited to invited representatives.

Panelists are presented with background material including structure-and-logic diagrams depicting the models involved and the initial ranges for the input variables. It is stressed that despite the employment of aspects of statistical probability, no one in the panel is expected to be a statistician; rather, input is sought reflecting the experience and expertise of the panelists.

During the session a detailed panel review of the forecasting process is presented by a consulting team. Participants can critique the model and forecasting methods, and the forecasting model can be adapted to reflect suggested changes. Panelists are given sufficient information to evaluate the initial quantitative assessment of uncertainty in each variable and assumption. With the aid of on-site computer simulation, the panel helps shape the forecasts reflecting collective risks associated with all underlying judgments and assumptions.

Two elements of the process contribute to consensus building. First, through disclosure and public scrutiny of the forecasting process there is a tendency to mitigate concerns which otherwise may go unanswered. Secondly, discussion by experts and the public on probable ranges of each input variable promotes the reconciliation of disparate views. In many cases, opposing sides find that radically different assumptions on certain variables will have minimal impacts on the forecast values. In this manner, wide agreement is achieved in areas which previously may have been perceived as unbridgeable gaps.

Strategic Behavior

Having developed a risk analysis, the decision maker or agency has a forecast range of possible outcomes together with an assessment of the likelihood of their occurring. This additional information strengthens the capacity for strategic behavior on the part of the agency. For instance, to avoid a situation of "overpromising," a level of confidence greater than the mean forecast can be chosen as the "officially adopted" forecast. Likewise, to prepare for various contingent developments the agency, having assessed the likelihood of these developments, can plan accordingly.

By enabling the public agency to quantify its risks, on the one hand, and through promoting public consensus, on the other, risk analysis offers the potential of substantially improving public agency decision making and strategic planning.


The Arizona Department of Transportation (ADOT) conducts its strategic planning and decision making for the Maricopa Freeway/ Expressway Program with the assistance of forecasting models. This program has significant visibility, being central to the continuing development of the metropolitan Phoenix and surrounding areas and one of the largest new urban highway programs in the U.S. Moreover, public expectations are high, as the program is financed primarily through a voter approved one-half-cent sales tax described below.

Funding for Freeways

In 1985, comprehensive transportation financing legislation was enacted in Arizona whose most significant feature, perhaps, was the provision for a new source of revenue for financing transportation facilities in the state. The legislation gave each Arizona county the option to enact, by voter approval, up to a 1 0 percent increase in the state's existing transaction privilege tax rates, an increase designated as a transportation excise tax. In a special election, Maricopa County voters approved the 1 0 percent increase in the transaction privilege tax effective January 1, 1986. The Maricopa County tax, referred to as the "1/2 cent sales" tax, will be in effect until December 31, 2005. Tax revenues flow into the Maricopa County Regional Area Road Fund (RARF) which is administered by ADOT.

The Maricopa transportation excise tax revenues are the principal source of funding for the Maricopa Association of Governments (MAG) Freeway/Expressway Plan. These revenues are dedicated by statute to the design, right-of-way purchase and construction of controlled-access highways in the MAG Plan. In addition, the funds may be pledged to repay principal and interest costs for bonds issued by the State Transportation Board.

Forecasting Models and Risk Analysis

Revenues expected from the 1/2 cent sales tax, which is imposed at varying rates on 16 components of economic activity in the county, have a critical bearing on program planning and implementation, on bond ratings, on the state's cost of capital for highway development and on the value-for-money to be obtained from ongoing highway expenditures. Recognizing the need to develop formal, well-documented and realistic forecasts of Maricopa transportation excise taxes in light of these program requirements and the historical volatility of this revenue source, ADOT engaged economic consultants from Arizona State University (ASU) to develop a long-range forecasting model for the transportation excise tax revenues.

The model, developed in 1986 and revised in 1989, is a structural econometric model which is dynamic in nature and enables the user to consider the implications of changing economic and demographic conditions on future revenues. Major classes of business activity subject to the tax are modeled separately, since the economic forces behind each activity are different. This tax revenue stream is very strongly influenced by economic cycles and population growth, given that retail sales comprise more than half of annual collections and contracting the second largest share.

ADOT, with the aid of a consultant, developed a risk analysis from the econometric forecasting model described above. An expert panel was formed to shape assumptions and was convened on October 18, 1990. The panel included 1 1 economists representing local banks, the Federal Reserve, the Economics Department of ASU and others. Twenty-five observers from the public at large also participated in the session. The panel session was conducted as a structured workshop: members received a briefing book in advance, the model design was reviewed and consensus ranges on the model's variables were established. Panel members provided insights regarding the local economy and its prospects, concentrating on events and trends likely to have an impact on the transportation excise tax revenue growth.

Risk Analysis Output

The final product of the excise tax risk analysis is a forecast of revenue yield for each revenue category and for total revenues, together with estimates of the probability of achieving alternate yields, given the uncertainty in the underlying assumptions. As illustrated in the accompanying "Risk Analysis Output Table," for each year, the forecast values are shown for a specified confidence interval, which is the probability that the tax yield will exceed the indicated value.
(millions, current dollars)
interval(1) 1992 1993 1994 1995 1996 1997
 90% 83 92 103 100 115 120
 80% 98 106 118 117 129 139
 70% 108 116 127 129 139 152
 60% 116 125 135 140 148 163
 50% 123 132 143 148 157 173
 40% 130 140 151 157 166 183
 30% 137 147 160 167 176 194
 20% 146 156 169 178 188 208
 10% 158 168 180 195 208 228

(1) Represents the probability that tax yields will exceed the indicated

As part of its 1991 forecast update process, ADOT developed an official forecast that has a higher level of confidence of attainment than in the past with less risk of falling below expected levels. The new forecast, based on risk analysis results that include the probability-based inputs of the expert panel of economists, responds to the department's need for realistic, conservative and accurate revenue forecasts.

The forecast reflects probability levels ranging from 60 percent in 1992 to 70 percent in 1995. Given long-term uncertainties relative to the economy, ADOT is forecasting annual increases in the excise tax beginning in 1996 which would have approximately an 80 percent probability of occurrence. These probability levels translate into an average annual growth rate of 5.29 percent growth rate over the FY 1996-2006 period.

This more conservative forecast recognizes the impact that the continuing slowdown in the Maricopa County economy has had on the revenue growth and the uncertainty associated with future revenue growth. The end result is a forecast that provides a more realistic perspective on the likelihood of achieving future revenue yields, thereby increasing public confidence. This risk analysis forecast is a fundamental feature of ADOT's strategic planning process and facilitates decision making under conditions of uncertainty The risk analysis process has been extended recently to include the forecasting of right-of-way acquisition and construction costs for the Maricopa Freeway/Expressway Plan.

Daniel is a senior consultant with Hickling Corporation and manager of professional studies in the firm's Washington, D. C., office. His specialties include risk analysis, benefit-cost analysis, applied economic and policy analysis, and microcomputer applications. He holds a master's degree in economics from Tel Aviv University and B. Sc. in mathematics from the Illinois Institute of technology. The author is indebted to Suzanne Sale, director of administrative services of the Arizona Department of Transportation, who contributed major segments of his article. The author also would like to acknowledge ADOT staff members, in particular John McGee and Herbert Uphoff.
COPYRIGHT 1992 Government Finance Officers Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1992 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:includes related article
Author:Brod, Daniel
Publication:Government Finance Review
Date:Jun 1, 1992
Previous Article:Workforce diversity: the challenge to financial managers.
Next Article:Reducing service delivery costs through public/private partnerships.

Related Articles
THE COMPLETE Debt Risk Puzzle.
Ensuring Better Business Forecasting: Forecasts are always educated guesses, but too many companies set off on a vain quest for precision. Better...
Valuation essentials for CFOs. (Valuation).
IOM: the Economics of better Environmental Health.

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters