# Forecasting foreign exchange rates using objective composite models.

Key Results

No single forecasting method was according to the statistical results of this study the best under all circumstances.

Introduction

Foreign exchange rate forecasting has become increasingly important since the dissolution of the Bretton Woods system and the advent of floating exchange rates in 1973. The substantial increase in exchange rate volatility has concomitantly placed a priority on the managerial function of foreign exchange risk management.

Exchange rate forecasts are used by multinational corporations in many important areas of financial management. For example, managers use foreign exchange forecasts to convert future foreign cash flows into domestic currency units; foreign and domestic costs of capital (or returns on investment) can then be compared when making a foreign financing or investment decision. Likewise, forecasts are required for deciding whether or not a foreign currency exposure should be hedged or to what extent the exposure needs to be hedged. Monthly projections of foreign subidiaries' expenses and revenues, which are included in annual budgets, also require foreign exchange rate forecasts. Furthermore, when formulating long-range strategic plans such as a subsidiary's asset and liability structure, pricing policy, or product mix, foreign exchange rate forecasts are again needed.

Currently Used Foreign Exchange Forecasting Methods

The foreign exchange rate forecasting methods in use today by both commercial services and corporate forecasting departments are primarily econometric, judgmental, or technical methods, as summarized by Levich (1983). The forward rate, which is considered an unbiased predictor of the future spot rate by some scholars (Kohlhagen 1979 and Levich 1979), may also be used to forecast foreign exchange rates in lieu of either purchasing a commercial forecast or incurring the expense of forecasting in-house. The forward rate is the rate of exchange at which ony may contract to buy or sell a foreign currency at a designated future date. As such, forward rates can be used as forecasts of future spot exchange rates. Forward contracts may be of different maturities; the more common ones are one-month, three-month, six-month, and one-year contracts.

Econometric methods usually employ a single multiple regression equation. The independent variables are economic in nature while the dependent variable is the foreign exchange rate to be forecasted. The specification of independent variables may be prompted by various economic theories such as purchasing power parity, monetary theory, portfolio balance theory, or the asset approach (Levich 1982). Single-equation models are often an oversimplification of the real world and, for this reason, some econometric models are comprised of systems of equations.

Another foreign exchange rate forecasting method is the judgmental approach. The strength of this method is that both quantifiable variables (e.g., inflation rate and money supply) and qualitative variables (e.g., capital controls, changes in tax policy, and overselling or underselling of currencies) can be incorporated. The forecasters view these factors in the light of past experience and may canvass experts in the field before agreeing upon a forecasted exchange rate.

A technical analysis is made by tracing the recent movements of the foreign exchange rate in order to predict future exchange rates; other variables are not employed in this analysis. Buy or sell signals are produced to advise clients to hold long or short positions in currencies. For example, a company advised to hold a short position in U.S. dollars and a long position in British pounds would sell its U.S. dollars in exchange for British pounds.

The Use of Composite Forecasting

When several forecasts of the same event exist, the problem of which one to choose quickly arises. Unless one forecasting method is clearly superior to the others, it is difficult to justify a selection. A standard procedure for eliminating the confusion innate in multiple forecasts is to blend them into a single or composite forecast either by subjective (judgmental) or objective (mathematical) means (Newbold and Granger 1974, Makridakis and Winkler 1983, and Mahmoud 1984). Composite forecasts are also called consensus, combined, blended or aggregate forecasts. Some composite forecasts have been shown to improve forecast accuracy and reduce error variance (Makridakis and Winkler 1983, Granger and Ramanathan 1984, Ashton and Ashton 1985, Holden and Peel 1986, and Lawrence et al. 1986).

Nevertheless, very few of the objective combining models currently used in the field of forecasting have been applied to foreign exchange forecasting. In fact, Levich (1982) reported that only two commercial forecasting services provided combined forecasts. Moreover, Goodman and Jacobs (1983) reported that the best performance exhibited by the commercial forecasting services resulted from the use of an informal combination of two technical forecasts. This procedure recommended that a currency be bought or sold only when both forecasting techniques were in agreement.

Criteria for Evaluating Foreign Exchange Forecasts

The evaluation criteria used to judge the performance of foreign exchange rate forecasts fall into two major categories: accuracy and correctness. Accuracy is the primary performance criterion used in forecasting. Absolute accuracy is a measure of the difference between the forecasted value and the actual occurrence. It may be expressed, for example, as a mean absolute deviation, mean squared error, standard deviation, or mean forecast error. Relative accuracy measures the difference between the forecast and the actual occurrence in terms of a percentage and may be expressed, for example, as a mean percentage error, mean absolute percentage error, relative forecasting error, or coefficient of variation.

A correct forecast (i.e. one that predicts the true direction of exchange rate changes relative to the forward rate) may be more important than an accurate forecast in some currency hedging situations (Dufey and Mirus 1981, Bilson 1983, Levich 1983, and Kwok and Lubecke 1990). A forecast is correct if it is on the same side of the forward rate as the actual exchange rate. Therefore, the "correctness" criterion is also known as the "right side of the market" approach (RSOM). A correct forecast may not be very accurate (has a large deviation from the actual spot rate) but it may lead the manager to make the right decision regarding whether or not he or she should hedge a foreign exchange exposure with a forward contract.(1)

The Research Problem

This research has two main purposes. The first is to examine and compare the performance of ten objective (mathematical), composite forecasting models on a real world data set using both accuracy and correctness as evaluation criteria. The second purpose is to introduce these composite models to the field of foreign exchange rate forecasting, a field where moderate improvements in forecasting can generate savings or profits of a substantial magnitude.

Ten mathematical models are used to combine one-month exchange rate forecasts of up to four different sources (econometric, judgmental, technical and forward rate) to form composite forecasts. Because of space limitations, each model is only briefly described below. Some of the models (e.g. the constrained multiple objective linear programming model) are quite complex, and we refer the reader to alternative sources for the complete methodological details.

1. The Simple Average

A simple arithmetic average of individual forecasts has been used extensively in the literature as a model for providing a composite forecast (Doyle and Fenwick 1976, Makridakis and Winkler 1983, and Gupta and Wilton 1987). Its performance has been remarkably good for such a simple and inexpensive technique. It has become commonplace to use the composite forecast produced by the simple average as a standard against which the performance of other models is compared. The simple average forecast computed in this research is the average of the four different sources noted above.

2. The Odds Matrix Method

This method was formulated by Gupta and Wilton (1987). This approach uses a matrix of pairwise odds on relative forecast performance to derive a required set of weights. Gupta and Wilton showed that the method performed well against the simple average, a constrained linear model and an unconstrained linear model using simulated data. In this research, the odds matrix method results in a composite forecast that is a weighted average of the four source forecasts.

3. Historical Weightings

The method of historical weightings was proposed by Doyle and Fenwick (1976) and applied to sales forecasting. Under this approach, forecasting accuracy (measured in terms of the fit of each forecasting technique to past data) is used to assign weights to the different forecasting methods to form a composite forecast. The normalized weights assigned to the n forecasts are denoted as [w.sub.1], [w.sub.2], . . ., [w.sub.n], where [w.sub.i] = 1 - ([MSE.sub.i])/([MSE.sub.1], + [MSE.sub.2] + . . . + [MSE.sub.n]) and [MSE.sub.i] is the historical mean squared error of the [i.sup.th] forecast. In our research, the weights for n = 4 individual forecasts are used to generate the historical weighting composite forecast.

4. Constrained Linear Combination

Constrained linear combinations have been used-by Nelson (1972) for predicting the growth of the U.S. economy, by Clemen (1986) for studying the effect of linear constraints on the efficiency of combined forecasts, and by Holmen (1987) for combining forecasts of short-term earnings. Weights are estimated using a goal programming model and then assigned to each of the individual forecasts. The constrained model restricts the weights to be non-negative and sum to one, and it does not permit the inclusion of a constant term in the combination equation.

5. Unconstrained Linear Combination

Unconstrained models have been used by Nelson (1972) for modeling and predicting the growth of the U.S. economy, by Makridakis et al. (1982) for studying the accuracy of time-series methods, by Granger and Ramanathan (1984) for comparing the performance of constrained and unconstrained linear combinations of forecasts, and by Holden and Peel (1986) for combining economic forecasts. Under this approach, weights that have been estimated using regression analysis are assigned to the individual forecasts. Unlike the constrained version, the unconstrained model does not restrict the weights to be nonnegative nor sum to one, and the constant term is permitted in the combination equation.

6. Weighting Based Upon Actual Forecast Errors

This method was proposed by Russell and Adam (1987) in a comparative study of nine combination models used to form composite forecasts from naive single-method forecasting models. Under this approach, the weights assigned to the forecasts are based on inverse proportions of their individual accuracy as measured by mean absolute error (MAE). The normalized weight assigned to the [i.sup.th] forecast is [w.sub.i] = 1 - (1/[MAE.sub.i])/(1/[MAE.sub.1] + 1/ [MAE.sub.2] +... + 1/[MAE.sub.n]), where MAE; is the historical mean absolute error of the [i.sup.th] forecast. In our research, this composite forecasting model applies normalized weights to the individual econometric, judgmental, technical and forward rate forecasts.(2)

7. Constrained Multiple Objective Linear Programming

The constrained multiple objective linear programming model permits the articulation of multiple objectives and generates on ordered list of optimum solutions. This model has been employed by Reeves and Lawrence (1982, 1991) in studies of combining forecasts produced by multiple regression models, harmonic smoothing models, and exponential smoothing models. Gulledge et al. (1986) applied this model to combine forecasts for a number of economic policy variables and Kwok and Lubecke (1988) used this methodology to form composite forecasts for foreign exchange rates. This research employs a lexicographic solution procedure that first determines the ideal point, then computes a single efficient weighting using the L(2) norm, or Euclidean distance, as an objective function as proposed by Yu (1973) and Zeleny (1974, pp. 174-176, 486-489).

8. Unconstrained Multiple Objective Linear Programming

This model is similar to the constrained multiple objective linear programming model except that it does not restrict the weights to be nonnegative nor sum to one, and a constant term is permitted in the combination equation. The authors are unaware of any previous attempt to apply this model to forecasting. This research may be the first attempt.

9. Historical Record of the Most Accurate Forecast in All Previous Periods

This method was first suggested and employed by Bunn and Kappos (1982). Ringuest and Tang (1987a) found that of the five composite models they studied, this model had the best performance. The weight assigned to the ith forecasting method is a function of [m.sub.1], the number of times that method yielded the minimum absolute forecast error in the past. Letting T denote the total number of periods in the estimation sample, the weight assigned to this particular forecasting method is then [w.sub.i] = [m.sub.i]/T.

10. Focus Forecasting

Focus forecasting, introduced by Smith and Wright (1978), might be viewed as the simplest possible method of combining forecasts. Ringuest and Tang (1987b) indicated that this method performed well against other more complicated techniques. The method requires that a weight of one be assigned to the forecast having the minimum absolute error in the previous forecasting period and a weight of zero be applied to all other forecasts. Therefore, this method does not actually combine forecasts, but instead chooses the forecasting method that performed the best in the last period as the sole forecast of the next period.

Table 1 provides a summary of these ten composite models and the four single-method models. Each of the ten blending models is used to produce a single exchange rate forecast by combining four different types of forecasts (judgmental, econometric, technical and forward rate). The number of forecasts being blended is limited to four since it has been found that the combination of three forecasts is better than any combination of a pair (Granger and Ramanathan 1984, Ashton and Ashton 1985, and Lawrence et al. 1986) but the impact of adding another method reaches a plateau after three or four methods have been combined (Makridakis and Winkler 1983). Distinctly different types of forecasts are chosen in order to obtain the maximum improvement in the combined forecast. The objective is to examine how effective these models are at producing composite forecasts that provide either greater accuracy or greater correctness than do any of the individual forecasting techniques.(3)

Research Design

Data Acquisition

Econometric and judgmental forecasts were obtained from commercial forecasting agencies. The forecasts provided by these agencies were one-month forecasts for five major trading currencies (British pound, French franc, Swiss franc, Japanese yen, and Deutsche mark) for the last day of each month beginning on January 1, 1986 and ending on December 31, 1989. For each currency forecast, therefore, we obtained 48 forecasts for the econometric and judgmental single-method models.

Technical forecasts for the five currencies during the same time period were generated using a simple time-series model of the spot rates that we obtained from The Wall Street Journal. The procedure was suggested by Bilson (1982) and improved by Little and Sall (1984).

Forward rate forecasts were obtained from the financial section of The Wall Street Journal. The inclusion of the forward rate as one of the foreign exchange rate forecasting methods to be blended by the composite models is important in that it provides a fourth single-method forecast that is distinctly different from the other three methods. Its inclusion may contribute to the correctness (Kwok and Lubecke 1990) and/or the accuracy (Bilson 1982) of the composite forecasts.

Methodology

Ex Ante Rolling Forecasts

Results and conclusions presented in this paper are based on ex ante rolling forecasts. The forecasts are ex ante since weights determined from previous periods were applied to forecasts offuture periods. After extensive conversations with the commercial services and other researchers in the field, the authors felt confident that economic events occurring more than one year prior to the forecast period had little or no effect on the exchange rate other than that which was already incorporated in the data during the ensuing time period. It was therefore decided that the subsample used for weights estimation consisted of the most recent twelve months. For example, the first twelve months were used to form the composite forecast for the thirteenth month and the second through the thirteenth months were used to form the composite forecast for the four-teenth month. This rolling process continued until thirty-six ex ante composite forecasts were formed.

Criteria of Evaluation -- Accuracy and Correctness

The performance of these ten composite models was evaluated according to two major criteria -- accuracy and correctness. Accuracy is a measure of difference between the forecasted and the actual exchange rates. In this study, accuracy was measured by mean absolute deviations. Correctness was measured as the percentage of forecasts that were on the right side of the market (guiding the user to make the right hedging decision). Note that by random guessing, there is a 50% chance that a forecast will be on the right side. Consequently, for a forecast model to be evaluated as superior, its percentage of correctness must be substantially higher than 50%.

Tukey's Procedure for Multiple Comparisons

There were altogether fourteen forecasting approaches examined in this study -- four single-method models and ten composite models. In order to ascertain whether or not there were significant differences in the models' performance, we employed Tukey's (1949) procedure for multiple comparisons. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The Studentized range is defined as q = [(maximum sample mean) -- (minimum sample mean)]/[s/[(m).sup.1/2], where s is the standard deviation for the analysis of variance (equal to the square root of the error mean square) and m is the number of observations used to compute each sample mean.

The logic behind this multiple comparison method is that if we determine a critical value for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. In this research, the sample means correspond to the means of the 36 forecasts produced by each of the 14 forecasting methods.(4)

Statistical Test Results and Analyses

Results ot the Japanese Yen Forecasts

To illustrate the results of our analyses, let us first examine the findings for the one-month forecast of the Japanese yen. Table 2 reports the percentages of correctness for the thirteen forecasting models.(5) The first column shows the ranking of the models. In this case, the constrained linear combination model and the constrained multiple objective programming model rank tie for the first place. Both models show a very high correctness percentage of correctness, 80.6%, which is significantly larger than the 50% that would be obtained by chance (the significance results reported in the fifth column are tests of individual models against the benchmark 50%). These results indicate that if one follows the forecast in deciding whether to hedge a foreign exchange exposure with a forward contract, one would be right about 81% of the time. In fact, out of the thirteen models, five have percentages significantly higher than 50%. Interestingly, the technical forecast, which we constructed using a simple time-series model, also performs well. The other two single-method models perform poorly.

Table 2. Performance of the Forecasting Models According to the Correctness Criterion (Japanese Yen)

Notes: Tukey's procedure for multiple comparisons is used to ascertain whether or not there were significant differences in the models. performance. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The logic behind this multiple comparison method is that if a critical value is determined for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. There is mild evidence against the analysis of variance null hypothesis of no difference between the correctness of each of the forecasting methods (p-value = 0.0815). There were 36 observations for each forecasting model in the test sample.

To compare the relative performance of the models, we need to examine the results of the Tukey tests. The Tukey procedure allows for the simultaneous comparison of multiple models. The Tukey groupings in the last column of Table 2 indicate that two significantly different groups of forecasting models exist. For models falling into the same Tukey group, one cannot say that there is significant difference in their performance. For instance, one cannot say that the constrained linear combination method has significantly higher percentage of correctness than the unconstrained linear combination method. Nonetheless, one can conclude that the constrained linear combination method significantly outperforms both the judgmental and the focus forecasting models since they belong to two different Tukey groups.(6) For the analysis of variance, there is mild evidence against the null hypothesis of no difference between the correctness of each of the forecasting methods (p-value = 0.0815).

Alternatively, Table 3 presents the accuracy results of the fourteen models for the one-month Japanese yen forecast. The fourth column reports the mean absolute deviations under different models. Obviously, the smaller the deviation, the more accurate the forecast. Interestingly, the single-method technical model is ranked number one, with the smallest mean absolute deviation of 5.048. The composite focus forecasting method, which performed poorly in terms of correctness, occupies the second place. The constrained linear combination model and the constrained multiple objective linear programming model perform quite well (being ranked the third and fourth, respectively). The Tukey groupings of Table 3 indicate that two significantly different groups exist. For instance, the judgmental method is significantly outperformed by the technical, focus forecasting, and constrained linear combination models. The analysis of variance null hypothesis of no difference between the accuracy of each of the forecasting methods is rejected for the Japanese yen (p - value = 0.0289).

Table 3. Performance of the Forecasting Models According to the Accuracy Criterion (Japanese Yen)

Notes: Tukey's procedure for multiple comparisons is used to ascertain whether or not there were significant differences in the models' performance. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The logic behind this multiple comparison method is that if a critical value is determined for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. The analysis of variance null hypothesis of no difference between the accuracy of each of the forecasting methods is rejected (p-value = 0.0289). There were 36 observations for each forecasting model in the test sample.

Summary of Empirical Results

To conserve space, we do not present the correctness and accuracy results for the other four currencies. Instead, the findings are summarized in Tables 4 and 5. Table 4 lists the best and worst forecasting models under different currency forecasts when correctness is the evaluation criterion. With the exception of the French franc, all currencies have at least one forecasting model with percentage of correctness significantly larger than the 50% possible by random guessing. Across the five currencies, the unconstrained multiple objective linear programming model and the unconstrained linear combination model have the highest average percentages of correctness (70.55% and 70.00% respectively). This is in sharp contrast to the performance of the single-method models. The econometric, judgmental and technical models are on the right side of the market only 54.38%, 53.87% and 67.22% of the time, respectively.

(*) According to the ANOVA results at [Alpha] = 0.15, the null hypothesis that there is no difference between the percentage of correctness of each of the forecasting methods is rejected.

(*) According to the ANOVA results at [Alpha] = 0.15, the null hypothesis that there is no difference between the mean absolute deviation of each of the forecasting methods is rejected.

Alternatively, Table 5 lists the best and worst forecasting models when accuracy is the evaluation criterion. The best performers in terms of correctness are not the best performers in terms of accuracy. Here, the focus forecasting model or the technical model usually has the lowest mean absolute deviation. The single-method, judgmental model appears to be the least accurate model of all.

Conclusions

The statistical results presented in the previous section indicate that no single forecasting method was the best under all circumstances. For instance, in the case of the French franc, the focus forecasting model generated the most accurate forecast in this study. However, when the forecast was used for hedging decisions, the percentage of correctness was only 47.22%, even worse than guessing. Instead of relying on one single method to perform all currency forecasts, the authors recommended that the best model for a specific currency be used for forecasting the exchange rate of that currency (Tables 4 and 5). However, if a forecaster must use no more than one or two models for all currencies, either the unconstrained multiple objective linear programming model or the unconstrained linear combination model would be a good choice for forecasting correctness. If accuracy is intended, the focus forecasting model or the technical model would be more appropriate.

The results presented in this study have some interesting practical implications. Since most commercial agencies do not prepare separate currency forecasts for correctness and accuracy, the users should be aware of how a forecast they subscribe to performs for each criterion, as well as how it performs in comparison to the composite forecasts.(7) In this study, the best composite forecasts were correct 18.33% more often than the best forecasts provided by the commercial services, when averaged across the five currencies. The mean absolute deviations of the best composite forecasts represented an average improvement in accuracy of 19.74% over the mean absolute deviations of the best forecasts provided by the commercial services. Though our data came from only two of many commercial agencies, users should still consider whether some form of composite model should be applied to the purchased forecasts in order to improve their correctness or accuracy.

The major limitation of this study lies in the size and number of samples. Since only two commercial forecasts were included in the study, the generalizability of the results is limited. Each sample, consisting of 48 monthly observations for each currency (36 observations for testing), was rather small for comparing the performance of 14 forecasting models simultaneously using the Tukey procedure. Furthermore, it is possible that under a longer or different time period (e.g. in the 1990s), the best forecasting methods may be quite different. Not many commercial agencies were willing to share their past forecast data. The data were purchased from commercial agencies and therefore subject to budgetary constraints. Further empirical tests are warranted if it becomes possible to obtain a sample that covers more commercial forecasts and a longer period of time. We hope that the findings of this study will be useful to managers who need to make better exchange rate forecasts in their decision making.

Footnotes

(1) For details regarding the differences between the accuracy and correctness criteria, please refer to Levich (1983) and Kwok and Lubecke (1990).

(2) Besides the mean absolute errors, other measures such as mean squared errors or mean absolute percentage errors are sometimes used to calculate individual accuracy.

(3) The composite models used in this study are all objective (mathematical) models. Models that are either wholly or partially subjective in nature are avoided because the procedures for choosing a blend and the ensuing results vary from decision maker to decision maker. Furthermore, empirical results from subjective blending studies are mixed. Experiments by Carbone et al. (1983) concluded that judgmental adjustments by students led to no improvement in accuracy. Armstrong (1982) demonstrated that subjective alteration of objective forecasts often reduced accuracy. Although improved forecasting results were obtained in a study by Mathews and Diamantopoulos (1986), it was concluded that situation-specific expertise may have been the key factor in the forecast improvement. Also, Makridakis (1987) indicated that managers should prepare a separate judgmental forecast, and then objectively combine it with quantitative forecasts, rather than use judgment for the blending process itself.

(4) There are several assumptions underlying the statistical analysis of the randomized block design used in this research. Among these are that the model error is normally distributed and there is no significant autocorrelation within a sample. The assumption of normality is assumed to be satisfied since we used sample sizes of 36. Tests for autocorrelation were applied to the data. There was only one instance of significance for the correctness data, but a number of significant autocorrelations were detected for the accuracy data. Forecasting methods which showed significant autocorrelations were excluded from the Tukey test for that particular currency. A significance level of [Alpha] = 0.15 was chosen since it was extremely difficult to reject a null hypothesis at [Alpha] = 0.05 when the sample size consisted of only 36 observations and the comparison was made among 14 models simultaneously.

(5) The forward rate model is not included since, in evaluating the correctness, the forward rate itself is used as the benchmark. A forecast is correct if it is on the same side of the forward rate as the actual exchange rate.

(6) The experimentwise error rate of the Tukey test, [Epsilon], is set to be 0.15 in this study.

(7) The forecasts provided by commercial agencies in this study were those of the econometric and the judgmental models. The technical forecasts were prepared by the authors using a simple time-series model.

References

Armstrong, J.S., The forecasting audit, in: Makridakis, S./Wheelwright, S. C., ed., The Handbook of Forecasting, John Wiley & Sons, Inc., New York 1982, pp. 535-552.

Ashton, A. H./Ashton, R. H., Aggregating subjective forecasts: some empirical results, Management Science, 31, 1985, pp. 1499-1508.

Bilson, J. F. O., Use of forecasts in constructing a portfolio, in: Ensor, R., ed., The Management of Foreign Exchange Risk, Euromoney Publications, 1982, pp. 155-164.

Bilson, J. F. O., The evaluation and use of foreign exchange forecasting services. In: Herring, R., ed., Managing Foreign Exchange Risk, Cambridge University Press, Cambridge 1983, pp. 149-179.

Bunn, D. W./Kappos, E., Synthesis or selection of forecasting models, European Journal of Operations Research, 9, 1982, pp. 173-180.

Carbone, R./Andersen, A./Corriveau, Y./Corson, P. P., Comparing for different time series methods the value of technical expertise, individualized analysis, and judgmental adjustment, Management Science, 29, 1983, pp. 559-565.

Clemen, R. T., Linear constraints and the efficiency of combined forecasts, Journal of Forecasting, 5, 1986, pp. 31-38.

Doyle, P./Fenwick, I. A., Sales forecasting-using a combination of approaches, Long Range Planning, 9, 1976, pp. 60-69.

Dufey, G./Mirus, R., Forecasting foreign exchange rates: a pedagogical note, Columbia Journal of World Business, Summer, 1981, pp. 53-61.

Goodman, S./Jacobs, R., How the current forecasting services rate, Euromoney, August, 1983, pp. 132-139.

Granger, C. W. J./Ramanathan, R., Improved methods of combining forecasts, Journal of Forecasting, 3, 1984, pp. 197-204.

Gulledge, T. R., Jr./Ringuest, J. L./Richardson, I A., Subjective evaluation of composite econometric policy inputs, Socio-Economic Planning Sciences, 20, 1986, pp. 51-55.

Gupta, S./Wilton, P. C., Combination of forecasts: an extension, Management Science, 33, 1987, pp. 356-372.

Holden, K./Peel, D. A., An empirical investigation of combinations of economic forecasts, Journal of Forecasting, 5, 1986, pp. 229-242.

Holmen, J. S., A note on the value of combining short-term earnings forecasts. A test of Granger and Ramanatahn, International Journal of Forecasting, 3, 1987, pp. 239-243.

Kohlhagen, S. W., The forward rate as an unbiased predictor of the future spot rate, Columbia Journal of World Business, Winter, 1979, pp. 77-85.

Kwok, C. C. Y./Lubecke, T. H., Composite foreign exchange forecasting to managers of multinational corporations, Management international Review, 4, 28, 1988, pp. 15-33.

Kwok, C. C. Y./Lubecke, T. H., Improving the correctness of foreign exchange forecasts through composite forecasting, Management international Review, 4, 30, 1990, pp. 331-352.

Lawrence, M. J./Edmundson, R. H./O'Connor, M. l, The accuracy of combining judgmental and statistical forecasts, Management Science, 32, 1986, pp. 1521-1532.

Levich, R. M., Are forward exchange rates unbiased predictors of future spot rates, Columbia Journal of World Business, Winter, 1979, pp. 49-61.

Levich, R. M., Evaluating the performance of forecasters, in: Ensor, R., ed., The Management of Foreign Exchange Risk, Second Edition. Euromoney Publications 1982, pp. 121-134.

Levich, R. M., Exchange rate forecasting techniques, in: Abraham, G./Ian, G., ed., International Financial Handbook, John Wiley & Sons, Inc., New York 1983.

Little, M. R./Sall, J., The forecast procedure, SAS/ETS User's Guide. Statistical Analysis System Institute Inc., Cary, N.C. 1984, pp. 301-342.

Mahmoud, E., Accuracy in Forecasting, Journal of Forecasting, 3, 1984, pp. 139-160.

Makridakis, S./Andersen, A./Carbone, R./Fildes, R./Hibon, M./Lewandowski, R./Newton, J./Parzen, E./Winkler, R., The accuracy of extrapolation (time series) methods, Journal of Forecasting, 12, 1982, pp. 111-154.

Makridakis, S./Winkler, R. J., Averages of forecasts: some empirical results, Management Science, 29, 1983, pp. 987-996.

Makridakis, S., The future of forecasting, in: Makridakis, S./Wheelwright, eds., The Handbook of Forecasting A Manager's Guide, Second Edition. John Wiley & Sons, Inc., New York 1987.

Mathews, B. P./Diamantopoulos, A., Managerial intervention in forecasting. An empirical investigation of forecast manipulation, International Journal of Research in Marketing, 3, 1986, pp. 3-10.

Nelson, C. R., The prediction performance of the FRB-MIT PENN model of the U.S. economy, American Economic Review, 62, 1972, pp. 121-141.

Newbold, P./Granger, C. W. J., Experience with forecasting univariate time-series and the combination of forecasts, Journal of the Royal Statistical Society, 137, 1974, pp. 131-164.

Reeves, G. R./Lawrence, K. D., Combining multiple forecasts given multiple objectives, Journal of Forecasting, 1, 1982, pp. 271-279.

Reeves, G. R./Lawrence, K. D., Combining forecasts given different types of objectives, European Journal Of Operational Reserach, 51, 1991, pp. 65-72.

Ringuest, J. L./Tang, K., An empirical comparison of five procedures for combining (or selecting) forecasts, Proceedings of the National Meeting of the Decision Sciences Institute, Boston 1987 a, p. 197.

Ringuest, J. L./Tang, K., Simple rules for combining forecasts: some empirical results, Socio-Economic Planning Sciences, 21, 1987b, pp. 239-243.

Russell, T. D./Adam, E. E., Jr., An empirical evaluation of alternative forecasting combinations, Management Science, 3, 1987, pp. 1267-1276.

Smith, B. T./Wright, O. W., Focus Forecasting: Computer Techniques for Inventory Control, CBI Publishing, Boston, MA 1978.

Tukey, J W., Comparing individual means in the analysis of variance, Biometrics, 5, 1949, pp. 99-114

Winkler, R. L./Makridakis, S., The combinations of forecasts, Journal of the Royal Statistical Society, 146,1983, pp. 150-157.

Yu, P., A class of decisions for group decision problems, Management Science, 19, 1973, p. 96.

Zeleny, M., Linear Multiple Objective Programming, Springer Verlag, Berlin and New York 1974.

Thomas H. Lubecke, formerly Assistant Professor of Management Science, Angelo State University, San Angelo, TX, U.S.A.

Robert E. Markland, Academic Program Director and Professor of Management Science, University of South Carolina at Columbia, SC, U.S.A.

Chuck C.Y. Kwok, Associate Professor of International Business, University of South Carolina et Columbia, SC, U.S.A.

Joan M. Donohue, Assistant Professor of Management Science, University of South Carolina et Columbia, SC, U.S.A.

No single forecasting method was according to the statistical results of this study the best under all circumstances.

Introduction

Foreign exchange rate forecasting has become increasingly important since the dissolution of the Bretton Woods system and the advent of floating exchange rates in 1973. The substantial increase in exchange rate volatility has concomitantly placed a priority on the managerial function of foreign exchange risk management.

Exchange rate forecasts are used by multinational corporations in many important areas of financial management. For example, managers use foreign exchange forecasts to convert future foreign cash flows into domestic currency units; foreign and domestic costs of capital (or returns on investment) can then be compared when making a foreign financing or investment decision. Likewise, forecasts are required for deciding whether or not a foreign currency exposure should be hedged or to what extent the exposure needs to be hedged. Monthly projections of foreign subidiaries' expenses and revenues, which are included in annual budgets, also require foreign exchange rate forecasts. Furthermore, when formulating long-range strategic plans such as a subsidiary's asset and liability structure, pricing policy, or product mix, foreign exchange rate forecasts are again needed.

Currently Used Foreign Exchange Forecasting Methods

The foreign exchange rate forecasting methods in use today by both commercial services and corporate forecasting departments are primarily econometric, judgmental, or technical methods, as summarized by Levich (1983). The forward rate, which is considered an unbiased predictor of the future spot rate by some scholars (Kohlhagen 1979 and Levich 1979), may also be used to forecast foreign exchange rates in lieu of either purchasing a commercial forecast or incurring the expense of forecasting in-house. The forward rate is the rate of exchange at which ony may contract to buy or sell a foreign currency at a designated future date. As such, forward rates can be used as forecasts of future spot exchange rates. Forward contracts may be of different maturities; the more common ones are one-month, three-month, six-month, and one-year contracts.

Econometric methods usually employ a single multiple regression equation. The independent variables are economic in nature while the dependent variable is the foreign exchange rate to be forecasted. The specification of independent variables may be prompted by various economic theories such as purchasing power parity, monetary theory, portfolio balance theory, or the asset approach (Levich 1982). Single-equation models are often an oversimplification of the real world and, for this reason, some econometric models are comprised of systems of equations.

Another foreign exchange rate forecasting method is the judgmental approach. The strength of this method is that both quantifiable variables (e.g., inflation rate and money supply) and qualitative variables (e.g., capital controls, changes in tax policy, and overselling or underselling of currencies) can be incorporated. The forecasters view these factors in the light of past experience and may canvass experts in the field before agreeing upon a forecasted exchange rate.

A technical analysis is made by tracing the recent movements of the foreign exchange rate in order to predict future exchange rates; other variables are not employed in this analysis. Buy or sell signals are produced to advise clients to hold long or short positions in currencies. For example, a company advised to hold a short position in U.S. dollars and a long position in British pounds would sell its U.S. dollars in exchange for British pounds.

The Use of Composite Forecasting

When several forecasts of the same event exist, the problem of which one to choose quickly arises. Unless one forecasting method is clearly superior to the others, it is difficult to justify a selection. A standard procedure for eliminating the confusion innate in multiple forecasts is to blend them into a single or composite forecast either by subjective (judgmental) or objective (mathematical) means (Newbold and Granger 1974, Makridakis and Winkler 1983, and Mahmoud 1984). Composite forecasts are also called consensus, combined, blended or aggregate forecasts. Some composite forecasts have been shown to improve forecast accuracy and reduce error variance (Makridakis and Winkler 1983, Granger and Ramanathan 1984, Ashton and Ashton 1985, Holden and Peel 1986, and Lawrence et al. 1986).

Nevertheless, very few of the objective combining models currently used in the field of forecasting have been applied to foreign exchange forecasting. In fact, Levich (1982) reported that only two commercial forecasting services provided combined forecasts. Moreover, Goodman and Jacobs (1983) reported that the best performance exhibited by the commercial forecasting services resulted from the use of an informal combination of two technical forecasts. This procedure recommended that a currency be bought or sold only when both forecasting techniques were in agreement.

Criteria for Evaluating Foreign Exchange Forecasts

The evaluation criteria used to judge the performance of foreign exchange rate forecasts fall into two major categories: accuracy and correctness. Accuracy is the primary performance criterion used in forecasting. Absolute accuracy is a measure of the difference between the forecasted value and the actual occurrence. It may be expressed, for example, as a mean absolute deviation, mean squared error, standard deviation, or mean forecast error. Relative accuracy measures the difference between the forecast and the actual occurrence in terms of a percentage and may be expressed, for example, as a mean percentage error, mean absolute percentage error, relative forecasting error, or coefficient of variation.

A correct forecast (i.e. one that predicts the true direction of exchange rate changes relative to the forward rate) may be more important than an accurate forecast in some currency hedging situations (Dufey and Mirus 1981, Bilson 1983, Levich 1983, and Kwok and Lubecke 1990). A forecast is correct if it is on the same side of the forward rate as the actual exchange rate. Therefore, the "correctness" criterion is also known as the "right side of the market" approach (RSOM). A correct forecast may not be very accurate (has a large deviation from the actual spot rate) but it may lead the manager to make the right decision regarding whether or not he or she should hedge a foreign exchange exposure with a forward contract.(1)

The Research Problem

This research has two main purposes. The first is to examine and compare the performance of ten objective (mathematical), composite forecasting models on a real world data set using both accuracy and correctness as evaluation criteria. The second purpose is to introduce these composite models to the field of foreign exchange rate forecasting, a field where moderate improvements in forecasting can generate savings or profits of a substantial magnitude.

Ten mathematical models are used to combine one-month exchange rate forecasts of up to four different sources (econometric, judgmental, technical and forward rate) to form composite forecasts. Because of space limitations, each model is only briefly described below. Some of the models (e.g. the constrained multiple objective linear programming model) are quite complex, and we refer the reader to alternative sources for the complete methodological details.

1. The Simple Average

A simple arithmetic average of individual forecasts has been used extensively in the literature as a model for providing a composite forecast (Doyle and Fenwick 1976, Makridakis and Winkler 1983, and Gupta and Wilton 1987). Its performance has been remarkably good for such a simple and inexpensive technique. It has become commonplace to use the composite forecast produced by the simple average as a standard against which the performance of other models is compared. The simple average forecast computed in this research is the average of the four different sources noted above.

2. The Odds Matrix Method

This method was formulated by Gupta and Wilton (1987). This approach uses a matrix of pairwise odds on relative forecast performance to derive a required set of weights. Gupta and Wilton showed that the method performed well against the simple average, a constrained linear model and an unconstrained linear model using simulated data. In this research, the odds matrix method results in a composite forecast that is a weighted average of the four source forecasts.

3. Historical Weightings

The method of historical weightings was proposed by Doyle and Fenwick (1976) and applied to sales forecasting. Under this approach, forecasting accuracy (measured in terms of the fit of each forecasting technique to past data) is used to assign weights to the different forecasting methods to form a composite forecast. The normalized weights assigned to the n forecasts are denoted as [w.sub.1], [w.sub.2], . . ., [w.sub.n], where [w.sub.i] = 1 - ([MSE.sub.i])/([MSE.sub.1], + [MSE.sub.2] + . . . + [MSE.sub.n]) and [MSE.sub.i] is the historical mean squared error of the [i.sup.th] forecast. In our research, the weights for n = 4 individual forecasts are used to generate the historical weighting composite forecast.

4. Constrained Linear Combination

Constrained linear combinations have been used-by Nelson (1972) for predicting the growth of the U.S. economy, by Clemen (1986) for studying the effect of linear constraints on the efficiency of combined forecasts, and by Holmen (1987) for combining forecasts of short-term earnings. Weights are estimated using a goal programming model and then assigned to each of the individual forecasts. The constrained model restricts the weights to be non-negative and sum to one, and it does not permit the inclusion of a constant term in the combination equation.

5. Unconstrained Linear Combination

Unconstrained models have been used by Nelson (1972) for modeling and predicting the growth of the U.S. economy, by Makridakis et al. (1982) for studying the accuracy of time-series methods, by Granger and Ramanathan (1984) for comparing the performance of constrained and unconstrained linear combinations of forecasts, and by Holden and Peel (1986) for combining economic forecasts. Under this approach, weights that have been estimated using regression analysis are assigned to the individual forecasts. Unlike the constrained version, the unconstrained model does not restrict the weights to be nonnegative nor sum to one, and the constant term is permitted in the combination equation.

6. Weighting Based Upon Actual Forecast Errors

This method was proposed by Russell and Adam (1987) in a comparative study of nine combination models used to form composite forecasts from naive single-method forecasting models. Under this approach, the weights assigned to the forecasts are based on inverse proportions of their individual accuracy as measured by mean absolute error (MAE). The normalized weight assigned to the [i.sup.th] forecast is [w.sub.i] = 1 - (1/[MAE.sub.i])/(1/[MAE.sub.1] + 1/ [MAE.sub.2] +... + 1/[MAE.sub.n]), where MAE; is the historical mean absolute error of the [i.sup.th] forecast. In our research, this composite forecasting model applies normalized weights to the individual econometric, judgmental, technical and forward rate forecasts.(2)

7. Constrained Multiple Objective Linear Programming

The constrained multiple objective linear programming model permits the articulation of multiple objectives and generates on ordered list of optimum solutions. This model has been employed by Reeves and Lawrence (1982, 1991) in studies of combining forecasts produced by multiple regression models, harmonic smoothing models, and exponential smoothing models. Gulledge et al. (1986) applied this model to combine forecasts for a number of economic policy variables and Kwok and Lubecke (1988) used this methodology to form composite forecasts for foreign exchange rates. This research employs a lexicographic solution procedure that first determines the ideal point, then computes a single efficient weighting using the L(2) norm, or Euclidean distance, as an objective function as proposed by Yu (1973) and Zeleny (1974, pp. 174-176, 486-489).

8. Unconstrained Multiple Objective Linear Programming

This model is similar to the constrained multiple objective linear programming model except that it does not restrict the weights to be nonnegative nor sum to one, and a constant term is permitted in the combination equation. The authors are unaware of any previous attempt to apply this model to forecasting. This research may be the first attempt.

9. Historical Record of the Most Accurate Forecast in All Previous Periods

This method was first suggested and employed by Bunn and Kappos (1982). Ringuest and Tang (1987a) found that of the five composite models they studied, this model had the best performance. The weight assigned to the ith forecasting method is a function of [m.sub.1], the number of times that method yielded the minimum absolute forecast error in the past. Letting T denote the total number of periods in the estimation sample, the weight assigned to this particular forecasting method is then [w.sub.i] = [m.sub.i]/T.

10. Focus Forecasting

Focus forecasting, introduced by Smith and Wright (1978), might be viewed as the simplest possible method of combining forecasts. Ringuest and Tang (1987b) indicated that this method performed well against other more complicated techniques. The method requires that a weight of one be assigned to the forecast having the minimum absolute error in the previous forecasting period and a weight of zero be applied to all other forecasts. Therefore, this method does not actually combine forecasts, but instead chooses the forecasting method that performed the best in the last period as the sole forecast of the next period.

Table 1 provides a summary of these ten composite models and the four single-method models. Each of the ten blending models is used to produce a single exchange rate forecast by combining four different types of forecasts (judgmental, econometric, technical and forward rate). The number of forecasts being blended is limited to four since it has been found that the combination of three forecasts is better than any combination of a pair (Granger and Ramanathan 1984, Ashton and Ashton 1985, and Lawrence et al. 1986) but the impact of adding another method reaches a plateau after three or four methods have been combined (Makridakis and Winkler 1983). Distinctly different types of forecasts are chosen in order to obtain the maximum improvement in the combined forecast. The objective is to examine how effective these models are at producing composite forecasts that provide either greater accuracy or greater correctness than do any of the individual forecasting techniques.(3)

Table 1. Single and Composite Forecasting Models Forecasting method Brief description of methodology Single-method models 1. Econometric Using economic variables to forecast exchange rates. The model may consist of one or many equations. 2. Judgmental Subjectively assigning values to exchange rate forecasts based on the information of economic and other behavioral factors. 3. Technical Tracing the recent movements of foreign exchange rates to forecast future spot rates. 4. Forward Rate The market rate at which one may contract to buy or sell a foreign currency at a designated future date. Composite models 1. Simple Average A simple arithmetic average of several forecast rates. 2. Odds Matrix Using forecasting accuracy, measured in terms of the fit of each forecasting technique to the past data, to assign weights. 3. Historical Weightings Using forecasting accuracy, measured in terms of the fit of each forecasting technique to the past data, to assign weights. 4. Constrained Linear Using a goal programming model Combination to determine weights constrained to be nonnegative and sum to one, with no constant term permitted. 5. Unconstrained Linear Using the regression analysis Combination to determine weights. There are no restrictions on the weights or the constant term. 6. Weighting Based Upon The weights based on the inverse Actual Forecast Errors proportions of the accuracy of individual forecasts as measured by mean absolute errors. 7. Constrained Multiple Allowing the articulation of Objective Linear Programming multiple objectives and generating a list of optimum weights that are averaged using the L(2) norm. 8. Unconstrained Multiple Similar to the constrained multiple Objective Linear Programming objective linear programming model but without restrictions on the weights or the constant term. 9. Historical Record of the Using the weights equal to the Most Accurate Forecast in All number of periods a forecasting Previous Periods method showing the minimum forecast error divided by the total number of periods. 10. Focus Forecasting Choosing the forecasting method which performed the best in the last period as the sole forecast of this period.

Research Design

Data Acquisition

Econometric and judgmental forecasts were obtained from commercial forecasting agencies. The forecasts provided by these agencies were one-month forecasts for five major trading currencies (British pound, French franc, Swiss franc, Japanese yen, and Deutsche mark) for the last day of each month beginning on January 1, 1986 and ending on December 31, 1989. For each currency forecast, therefore, we obtained 48 forecasts for the econometric and judgmental single-method models.

Technical forecasts for the five currencies during the same time period were generated using a simple time-series model of the spot rates that we obtained from The Wall Street Journal. The procedure was suggested by Bilson (1982) and improved by Little and Sall (1984).

Forward rate forecasts were obtained from the financial section of The Wall Street Journal. The inclusion of the forward rate as one of the foreign exchange rate forecasting methods to be blended by the composite models is important in that it provides a fourth single-method forecast that is distinctly different from the other three methods. Its inclusion may contribute to the correctness (Kwok and Lubecke 1990) and/or the accuracy (Bilson 1982) of the composite forecasts.

Methodology

Ex Ante Rolling Forecasts

Results and conclusions presented in this paper are based on ex ante rolling forecasts. The forecasts are ex ante since weights determined from previous periods were applied to forecasts offuture periods. After extensive conversations with the commercial services and other researchers in the field, the authors felt confident that economic events occurring more than one year prior to the forecast period had little or no effect on the exchange rate other than that which was already incorporated in the data during the ensuing time period. It was therefore decided that the subsample used for weights estimation consisted of the most recent twelve months. For example, the first twelve months were used to form the composite forecast for the thirteenth month and the second through the thirteenth months were used to form the composite forecast for the four-teenth month. This rolling process continued until thirty-six ex ante composite forecasts were formed.

Criteria of Evaluation -- Accuracy and Correctness

The performance of these ten composite models was evaluated according to two major criteria -- accuracy and correctness. Accuracy is a measure of difference between the forecasted and the actual exchange rates. In this study, accuracy was measured by mean absolute deviations. Correctness was measured as the percentage of forecasts that were on the right side of the market (guiding the user to make the right hedging decision). Note that by random guessing, there is a 50% chance that a forecast will be on the right side. Consequently, for a forecast model to be evaluated as superior, its percentage of correctness must be substantially higher than 50%.

Tukey's Procedure for Multiple Comparisons

There were altogether fourteen forecasting approaches examined in this study -- four single-method models and ten composite models. In order to ascertain whether or not there were significant differences in the models' performance, we employed Tukey's (1949) procedure for multiple comparisons. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The Studentized range is defined as q = [(maximum sample mean) -- (minimum sample mean)]/[s/[(m).sup.1/2], where s is the standard deviation for the analysis of variance (equal to the square root of the error mean square) and m is the number of observations used to compute each sample mean.

The logic behind this multiple comparison method is that if we determine a critical value for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. In this research, the sample means correspond to the means of the 36 forecasts produced by each of the 14 forecasting methods.(4)

Statistical Test Results and Analyses

Results ot the Japanese Yen Forecasts

To illustrate the results of our analyses, let us first examine the findings for the one-month forecast of the Japanese yen. Table 2 reports the percentages of correctness for the thirteen forecasting models.(5) The first column shows the ranking of the models. In this case, the constrained linear combination model and the constrained multiple objective programming model rank tie for the first place. Both models show a very high correctness percentage of correctness, 80.6%, which is significantly larger than the 50% that would be obtained by chance (the significance results reported in the fifth column are tests of individual models against the benchmark 50%). These results indicate that if one follows the forecast in deciding whether to hedge a foreign exchange exposure with a forward contract, one would be right about 81% of the time. In fact, out of the thirteen models, five have percentages significantly higher than 50%. Interestingly, the technical forecast, which we constructed using a simple time-series model, also performs well. The other two single-method models perform poorly.

Table 2. Performance of the Forecasting Models According to the Correctness Criterion (Japanese Yen)

Rank Forecasting method Single or % Correct composite (RSOM) 1.5 Constrained Linear Combination Composite 80.6 1.5 Constrained Multiple Objective Composite 80.6 Linear Programming 3. Unconstrained Linear Combination Composite 77.8 4. Technical Single 75.0 5. Unconstrained Multiple Objective Composite 69.4 Linear Programming 8. Simple Average Composite 66.7 8. Weighting Based Upon Actual Composite 66.7 Forecast Errors 8. Historical Weightings Composite 66.7 8. Odds Matrix Composite 66.7 8. Historical Record of the Most Composite 66.7 Accurate Forecast in All Previous Periods 11. Econometric Single 61.1 12. Judgmental Single 58.3 13. Focus Forecasting Composite 52.8 Rank Forecasting method Significantly better than 50% 1.5 Constrained Linear Combination Yes 1.5 Constrained Multiple Objective Yes Linear Programming 3. Unconstrained Linear Combination Yes 4. Technical Yes 5. Unconstrained Multiple Objective Yes Linear Programming 8. Simple Average No 8. Weighting Based Upon Actual No Forecast Errors 8. Historical Weightings No 8. Odds Matrix No 8. Historical Record of the Most No Accurate Forecast in All Previous Periods 11. Econometric No 12. Judgmental No 13. Focus Forecasting No Rank Forecasting method Tukey grouping 1.5 Constrained Linear Combination A 1.5 Constrained Multiple Objective A Linear Programming 3. Unconstrained Linear Combination A 4. Technical A B 5. Unconstrained Multiple Objective A B Linear Programming 8. Simple Average A B 8. Weighting Based Upon Actual A B Forecast Errors 8. Historical Weightings A B 8. Odds Matrix A B 8. Historical Record of the Most A B Accurate Forecast in All Previous Periods 11. Econometric A B 12. Judgmental B 13. Focus Forecasting B

Notes: Tukey's procedure for multiple comparisons is used to ascertain whether or not there were significant differences in the models. performance. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The logic behind this multiple comparison method is that if a critical value is determined for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. There is mild evidence against the analysis of variance null hypothesis of no difference between the correctness of each of the forecasting methods (p-value = 0.0815). There were 36 observations for each forecasting model in the test sample.

To compare the relative performance of the models, we need to examine the results of the Tukey tests. The Tukey procedure allows for the simultaneous comparison of multiple models. The Tukey groupings in the last column of Table 2 indicate that two significantly different groups of forecasting models exist. For models falling into the same Tukey group, one cannot say that there is significant difference in their performance. For instance, one cannot say that the constrained linear combination method has significantly higher percentage of correctness than the unconstrained linear combination method. Nonetheless, one can conclude that the constrained linear combination method significantly outperforms both the judgmental and the focus forecasting models since they belong to two different Tukey groups.(6) For the analysis of variance, there is mild evidence against the null hypothesis of no difference between the correctness of each of the forecasting methods (p-value = 0.0815).

Alternatively, Table 3 presents the accuracy results of the fourteen models for the one-month Japanese yen forecast. The fourth column reports the mean absolute deviations under different models. Obviously, the smaller the deviation, the more accurate the forecast. Interestingly, the single-method technical model is ranked number one, with the smallest mean absolute deviation of 5.048. The composite focus forecasting method, which performed poorly in terms of correctness, occupies the second place. The constrained linear combination model and the constrained multiple objective linear programming model perform quite well (being ranked the third and fourth, respectively). The Tukey groupings of Table 3 indicate that two significantly different groups exist. For instance, the judgmental method is significantly outperformed by the technical, focus forecasting, and constrained linear combination models. The analysis of variance null hypothesis of no difference between the accuracy of each of the forecasting methods is rejected for the Japanese yen (p - value = 0.0289).

Table 3. Performance of the Forecasting Models According to the Accuracy Criterion (Japanese Yen)

Rank Forecasting methods Single or composite 1. Technical Single 2. Focus Forecasting Composite 3. Constrained Linear Combination Composite 4. Constrained Multiple Objective Linear Composite Programming 5. Odds Matrix Composite 6. Simple Average Composite 7. Unconstrained Linear Combination Composite 8. Unconstrained Multiple Objective Linear Composite Programming 9. Historical Record of the Most Accurate Composite Forecast in All Previous Periods 10. Historical Weightings Composite 11. Weighting Based Upon Actual Forecast Errors Composite 12. Forward Rate Single 13. Econometric Single 14. Judgmental Single Rank Forecasting methods Mean absolute deviations 1. Technical 5.048 2. Focus Forecasting 5.212 3. Constrained Linear Combination 5.500 4. Constrained Multiple Objective Linear 5.594 Programming 5. Odds Matrix 6.144 6. Simple Average 6.205 7. Unconstrained Linear Combination 6.331 8. Unconstrained Multiple Objective Linear 6.338 Programming 9. Historical Record of the Most Accurate 6.436 Forecast in All Previous Periods 10. Historical Weightings 6.480 11. Weighting Based Upon Actual Forecast Errors 6.491 12. Forward Rate 7.389 13. Econometric 7.910 14. Judgmental 8.683 Rank Forecasting methods Tukey grouping 1. Technical A 2. Focus Forecasting A 3. Constrained Linear Combination A 4. Constrained Multiple Objective Linear A B Programming 5. Odds Matrix A B 6. Simple Average A B 7. Unconstrained Linear Combination A B 8. Unconstrained Multiple Objective Linear A B Programming 9. Historical Record of the Most Accurate A B Forecast in All Previous Periods 10. Historical Weightings A B 11. Weighting Based Upon Actual Forecast Errors A B 12. Forward Rate A B 13. Econometric A B 14. Judgmental B

Notes: Tukey's procedure for multiple comparisons is used to ascertain whether or not there were significant differences in the models' performance. Tukey's procedure utilizes the Studentized range to determine whether the difference in any pair of sample means implies a difference in the corresponding population means. The logic behind this multiple comparison method is that if a critical value is determined for the difference between the largest and smallest sample means, then any other pair of sample means that differ by as much as or more than this critical value would imply a difference in the corresponding population means. The analysis of variance null hypothesis of no difference between the accuracy of each of the forecasting methods is rejected (p-value = 0.0289). There were 36 observations for each forecasting model in the test sample.

Summary of Empirical Results

To conserve space, we do not present the correctness and accuracy results for the other four currencies. Instead, the findings are summarized in Tables 4 and 5. Table 4 lists the best and worst forecasting models under different currency forecasts when correctness is the evaluation criterion. With the exception of the French franc, all currencies have at least one forecasting model with percentage of correctness significantly larger than the 50% possible by random guessing. Across the five currencies, the unconstrained multiple objective linear programming model and the unconstrained linear combination model have the highest average percentages of correctness (70.55% and 70.00% respectively). This is in sharp contrast to the performance of the single-method models. The econometric, judgmental and technical models are on the right side of the market only 54.38%, 53.87% and 67.22% of the time, respectively.

Table 4. Best and Worst Forecasting Models According to the Correctness Criterion Currency BEST (Forecasting model forecast with highest percentage of correctness) British Constrained Multiple Objective pound Linear Programming Historical Record of Most Accurate Forecast Historical Weightings Odds Matrix Simple Average Technical Weighting Based Upon Actual Forecast Errors Deutsche Unconstrained Multiple Objective mark(*) Linear Programming French Unconstrained Linear franc Combination Japanese Constrained Linear Combination yen(*) Constrained Multiple Objective Linear Programming Swiss Unconstrained Linear franc(*) Combination Currency WORST (Forecasting model % of Significantly forecast with lowest percentage of correct- > 50% correctness) ness British 78 Yes pound 78 Yes 78 Yes 78 Yes 78 Yes 78 Yes 78 Yes Econometric 61 No Deutsche 75 Yes mark(*) Focus Forecasting 42 No French 67 No franc Constrained Linear Combination 44 No Constrained Multiple Objective 44 No Linear Programming Historical Weightings 44 No Odds Matrix 44 No Simple Average 44 No Japanese 81 Yes yen(*) 81 Yes Focus Forecasting 53 No Swiss 75 Yes franc(*) Judgmental 44 No

(*) According to the ANOVA results at [Alpha] = 0.15, the null hypothesis that there is no difference between the percentage of correctness of each of the forecasting methods is rejected.

Table 5. Best and Worst Forecasting Models According to the Accuracy Criterion Currency BEST (Forecasting WORST (Forecasting Mean forecast model with lowest model with highest absolute mean absolute mean absolute deviation deviation) deviation) British pound Technical 0.022 Forward Rate 0.033 Deutsche mark Focus Forecasting 0.083 Judgmental 0.142 French franc Focus Forecasting 0.268 Judgmental 0.420 Japanese yen(*) Technical 5.048 Judgmental 8.683 Swiss franc Focus Forecasting 0.076 Judgmental 0.168

(*) According to the ANOVA results at [Alpha] = 0.15, the null hypothesis that there is no difference between the mean absolute deviation of each of the forecasting methods is rejected.

Alternatively, Table 5 lists the best and worst forecasting models when accuracy is the evaluation criterion. The best performers in terms of correctness are not the best performers in terms of accuracy. Here, the focus forecasting model or the technical model usually has the lowest mean absolute deviation. The single-method, judgmental model appears to be the least accurate model of all.

Conclusions

The statistical results presented in the previous section indicate that no single forecasting method was the best under all circumstances. For instance, in the case of the French franc, the focus forecasting model generated the most accurate forecast in this study. However, when the forecast was used for hedging decisions, the percentage of correctness was only 47.22%, even worse than guessing. Instead of relying on one single method to perform all currency forecasts, the authors recommended that the best model for a specific currency be used for forecasting the exchange rate of that currency (Tables 4 and 5). However, if a forecaster must use no more than one or two models for all currencies, either the unconstrained multiple objective linear programming model or the unconstrained linear combination model would be a good choice for forecasting correctness. If accuracy is intended, the focus forecasting model or the technical model would be more appropriate.

The results presented in this study have some interesting practical implications. Since most commercial agencies do not prepare separate currency forecasts for correctness and accuracy, the users should be aware of how a forecast they subscribe to performs for each criterion, as well as how it performs in comparison to the composite forecasts.(7) In this study, the best composite forecasts were correct 18.33% more often than the best forecasts provided by the commercial services, when averaged across the five currencies. The mean absolute deviations of the best composite forecasts represented an average improvement in accuracy of 19.74% over the mean absolute deviations of the best forecasts provided by the commercial services. Though our data came from only two of many commercial agencies, users should still consider whether some form of composite model should be applied to the purchased forecasts in order to improve their correctness or accuracy.

The major limitation of this study lies in the size and number of samples. Since only two commercial forecasts were included in the study, the generalizability of the results is limited. Each sample, consisting of 48 monthly observations for each currency (36 observations for testing), was rather small for comparing the performance of 14 forecasting models simultaneously using the Tukey procedure. Furthermore, it is possible that under a longer or different time period (e.g. in the 1990s), the best forecasting methods may be quite different. Not many commercial agencies were willing to share their past forecast data. The data were purchased from commercial agencies and therefore subject to budgetary constraints. Further empirical tests are warranted if it becomes possible to obtain a sample that covers more commercial forecasts and a longer period of time. We hope that the findings of this study will be useful to managers who need to make better exchange rate forecasts in their decision making.

Footnotes

(1) For details regarding the differences between the accuracy and correctness criteria, please refer to Levich (1983) and Kwok and Lubecke (1990).

(2) Besides the mean absolute errors, other measures such as mean squared errors or mean absolute percentage errors are sometimes used to calculate individual accuracy.

(3) The composite models used in this study are all objective (mathematical) models. Models that are either wholly or partially subjective in nature are avoided because the procedures for choosing a blend and the ensuing results vary from decision maker to decision maker. Furthermore, empirical results from subjective blending studies are mixed. Experiments by Carbone et al. (1983) concluded that judgmental adjustments by students led to no improvement in accuracy. Armstrong (1982) demonstrated that subjective alteration of objective forecasts often reduced accuracy. Although improved forecasting results were obtained in a study by Mathews and Diamantopoulos (1986), it was concluded that situation-specific expertise may have been the key factor in the forecast improvement. Also, Makridakis (1987) indicated that managers should prepare a separate judgmental forecast, and then objectively combine it with quantitative forecasts, rather than use judgment for the blending process itself.

(4) There are several assumptions underlying the statistical analysis of the randomized block design used in this research. Among these are that the model error is normally distributed and there is no significant autocorrelation within a sample. The assumption of normality is assumed to be satisfied since we used sample sizes of 36. Tests for autocorrelation were applied to the data. There was only one instance of significance for the correctness data, but a number of significant autocorrelations were detected for the accuracy data. Forecasting methods which showed significant autocorrelations were excluded from the Tukey test for that particular currency. A significance level of [Alpha] = 0.15 was chosen since it was extremely difficult to reject a null hypothesis at [Alpha] = 0.05 when the sample size consisted of only 36 observations and the comparison was made among 14 models simultaneously.

(5) The forward rate model is not included since, in evaluating the correctness, the forward rate itself is used as the benchmark. A forecast is correct if it is on the same side of the forward rate as the actual exchange rate.

(6) The experimentwise error rate of the Tukey test, [Epsilon], is set to be 0.15 in this study.

(7) The forecasts provided by commercial agencies in this study were those of the econometric and the judgmental models. The technical forecasts were prepared by the authors using a simple time-series model.

References

Armstrong, J.S., The forecasting audit, in: Makridakis, S./Wheelwright, S. C., ed., The Handbook of Forecasting, John Wiley & Sons, Inc., New York 1982, pp. 535-552.

Ashton, A. H./Ashton, R. H., Aggregating subjective forecasts: some empirical results, Management Science, 31, 1985, pp. 1499-1508.

Bilson, J. F. O., Use of forecasts in constructing a portfolio, in: Ensor, R., ed., The Management of Foreign Exchange Risk, Euromoney Publications, 1982, pp. 155-164.

Bilson, J. F. O., The evaluation and use of foreign exchange forecasting services. In: Herring, R., ed., Managing Foreign Exchange Risk, Cambridge University Press, Cambridge 1983, pp. 149-179.

Bunn, D. W./Kappos, E., Synthesis or selection of forecasting models, European Journal of Operations Research, 9, 1982, pp. 173-180.

Carbone, R./Andersen, A./Corriveau, Y./Corson, P. P., Comparing for different time series methods the value of technical expertise, individualized analysis, and judgmental adjustment, Management Science, 29, 1983, pp. 559-565.

Clemen, R. T., Linear constraints and the efficiency of combined forecasts, Journal of Forecasting, 5, 1986, pp. 31-38.

Doyle, P./Fenwick, I. A., Sales forecasting-using a combination of approaches, Long Range Planning, 9, 1976, pp. 60-69.

Dufey, G./Mirus, R., Forecasting foreign exchange rates: a pedagogical note, Columbia Journal of World Business, Summer, 1981, pp. 53-61.

Goodman, S./Jacobs, R., How the current forecasting services rate, Euromoney, August, 1983, pp. 132-139.

Granger, C. W. J./Ramanathan, R., Improved methods of combining forecasts, Journal of Forecasting, 3, 1984, pp. 197-204.

Gulledge, T. R., Jr./Ringuest, J. L./Richardson, I A., Subjective evaluation of composite econometric policy inputs, Socio-Economic Planning Sciences, 20, 1986, pp. 51-55.

Gupta, S./Wilton, P. C., Combination of forecasts: an extension, Management Science, 33, 1987, pp. 356-372.

Holden, K./Peel, D. A., An empirical investigation of combinations of economic forecasts, Journal of Forecasting, 5, 1986, pp. 229-242.

Holmen, J. S., A note on the value of combining short-term earnings forecasts. A test of Granger and Ramanatahn, International Journal of Forecasting, 3, 1987, pp. 239-243.

Kohlhagen, S. W., The forward rate as an unbiased predictor of the future spot rate, Columbia Journal of World Business, Winter, 1979, pp. 77-85.

Kwok, C. C. Y./Lubecke, T. H., Composite foreign exchange forecasting to managers of multinational corporations, Management international Review, 4, 28, 1988, pp. 15-33.

Kwok, C. C. Y./Lubecke, T. H., Improving the correctness of foreign exchange forecasts through composite forecasting, Management international Review, 4, 30, 1990, pp. 331-352.

Lawrence, M. J./Edmundson, R. H./O'Connor, M. l, The accuracy of combining judgmental and statistical forecasts, Management Science, 32, 1986, pp. 1521-1532.

Levich, R. M., Are forward exchange rates unbiased predictors of future spot rates, Columbia Journal of World Business, Winter, 1979, pp. 49-61.

Levich, R. M., Evaluating the performance of forecasters, in: Ensor, R., ed., The Management of Foreign Exchange Risk, Second Edition. Euromoney Publications 1982, pp. 121-134.

Levich, R. M., Exchange rate forecasting techniques, in: Abraham, G./Ian, G., ed., International Financial Handbook, John Wiley & Sons, Inc., New York 1983.

Little, M. R./Sall, J., The forecast procedure, SAS/ETS User's Guide. Statistical Analysis System Institute Inc., Cary, N.C. 1984, pp. 301-342.

Mahmoud, E., Accuracy in Forecasting, Journal of Forecasting, 3, 1984, pp. 139-160.

Makridakis, S./Andersen, A./Carbone, R./Fildes, R./Hibon, M./Lewandowski, R./Newton, J./Parzen, E./Winkler, R., The accuracy of extrapolation (time series) methods, Journal of Forecasting, 12, 1982, pp. 111-154.

Makridakis, S./Winkler, R. J., Averages of forecasts: some empirical results, Management Science, 29, 1983, pp. 987-996.

Makridakis, S., The future of forecasting, in: Makridakis, S./Wheelwright, eds., The Handbook of Forecasting A Manager's Guide, Second Edition. John Wiley & Sons, Inc., New York 1987.

Mathews, B. P./Diamantopoulos, A., Managerial intervention in forecasting. An empirical investigation of forecast manipulation, International Journal of Research in Marketing, 3, 1986, pp. 3-10.

Nelson, C. R., The prediction performance of the FRB-MIT PENN model of the U.S. economy, American Economic Review, 62, 1972, pp. 121-141.

Newbold, P./Granger, C. W. J., Experience with forecasting univariate time-series and the combination of forecasts, Journal of the Royal Statistical Society, 137, 1974, pp. 131-164.

Reeves, G. R./Lawrence, K. D., Combining multiple forecasts given multiple objectives, Journal of Forecasting, 1, 1982, pp. 271-279.

Reeves, G. R./Lawrence, K. D., Combining forecasts given different types of objectives, European Journal Of Operational Reserach, 51, 1991, pp. 65-72.

Ringuest, J. L./Tang, K., An empirical comparison of five procedures for combining (or selecting) forecasts, Proceedings of the National Meeting of the Decision Sciences Institute, Boston 1987 a, p. 197.

Ringuest, J. L./Tang, K., Simple rules for combining forecasts: some empirical results, Socio-Economic Planning Sciences, 21, 1987b, pp. 239-243.

Russell, T. D./Adam, E. E., Jr., An empirical evaluation of alternative forecasting combinations, Management Science, 3, 1987, pp. 1267-1276.

Smith, B. T./Wright, O. W., Focus Forecasting: Computer Techniques for Inventory Control, CBI Publishing, Boston, MA 1978.

Tukey, J W., Comparing individual means in the analysis of variance, Biometrics, 5, 1949, pp. 99-114

Winkler, R. L./Makridakis, S., The combinations of forecasts, Journal of the Royal Statistical Society, 146,1983, pp. 150-157.

Yu, P., A class of decisions for group decision problems, Management Science, 19, 1973, p. 96.

Zeleny, M., Linear Multiple Objective Programming, Springer Verlag, Berlin and New York 1974.

Thomas H. Lubecke, formerly Assistant Professor of Management Science, Angelo State University, San Angelo, TX, U.S.A.

Robert E. Markland, Academic Program Director and Professor of Management Science, University of South Carolina at Columbia, SC, U.S.A.

Chuck C.Y. Kwok, Associate Professor of International Business, University of South Carolina et Columbia, SC, U.S.A.

Joan M. Donohue, Assistant Professor of Management Science, University of South Carolina et Columbia, SC, U.S.A.

Printer friendly Cite/link Email Feedback | |

Author: | Lubecke, Thomas H.; Markland, Robert E.; Kwok, Chuck C.Y.; Donohue, Joan M. |
---|---|

Publication: | Management International Review |

Date: | Apr 1, 1995 |

Words: | 6973 |

Previous Article: | The internationalization of the hotel industry - some new findings from a field study. |

Next Article: | Location characteristics of manufacturing investments in the U.S.: a comparison of American and foreign-based firms. |

Topics: |