Printer Friendly

Applicability of the revised Mean Absolute Percentage Errors (MAPE) approach to some popular normal and non-normal independent time series.

Abstract Commonly used Mean Absolute Percentage Errors (MAPE) and the authors' revised Mean Absolute Percentage Errors (RMAPE) are applied to measure the forecasting accuracy from different Moving Average Methods for independent time series. Simulation results show that both MAPE and RMAPE can only provide sensitive forecasting accuracy measurements on Moving Average Methods when coefficients of variation (c.v.) are smaller than 0.4 or is much greater than 4.0 for those independent time series. For independent time series with moderate c.v.'s, the complexity from the ratios of MAPE and RMAPE will mislead researchers on distinguishing the forecasting accuracies from different Moving Average Methods. The complexity from the ratios will be released only when the c.v. is very small, or when the c.v. is very large. Therefore, when data are from independent time series, the Mean Absolute Deviation (MAD) reveals valid the forecasting accuracies from various Moving Average Methods, but not from MAPE or RMAPE.

Keywords Mean Absolute Percentage Errors (MAPE) * Revised Mean Absolute Percentage Errors (RMAPE) * Forecasting accuracy * Coefficient of variation (c.v.) * Mean Absolute Deviation (MAD)

JEL C10 Econometrics * Statistics * M21 Managerial Economics * M00 Business Administration

Introduction

Measures such as Mean Absolute Percentage Error (MAPE), Mean Square Error (MSE), Mean Absolute Deviation (MAD), and Root Mean Square Error (RMSE) are used to evaluate the accuracy of forecasts by economists and others. Many researchers, such as Chatfield (1988), believe that the MSE and the MAD are not appropriate forecasting accuracy measurements, because a few large observations can dominate the measurement. Since the MAPE expresses the forecasting errors from different measurement units into percentage errors on actual observations, it is unit free. Therefore, according to Goodwin and Lawton (1999), the MAPE is probably the most widely used forecasting accuracy measurement of this kink. Makridakis et al. (1982) use it as the primary measure in the M-competition, and most forecasting textbooks recommend the use of the MAPE (i.e., Hanke and Reitsch 1995, p. 120; Bowerman et al. 2004, p. 18).

Of the many forecast accuracy measures, the MAPE is also a widely used accuracy measurement in forecasting with non-negative actual observations, for instance, on monthly or quarterly sales, tourism forecasts, and economic indicators (i.e., Chen et al. 2003; Song et al. 2003; Swanson et al. 2000; Wang and Liu 2005; Weller 1989; Weller and Kurre 1987). Although the MAPE is a popular forecast accuracy measure by practitioners when dealing with non-negative actual observations, in practical forecasts such as forecasts on profits, actual observations may end up with negative values. In this paper, a revised definition for the mean absolute percentage error (RMAPE), ([k.summation over (i = 1)] (|[[[A.sub.t] - [F.sub.t]]/[A.sup.t]]|)/k, is considered.

One common criticism of the MAPE is on its existence when the actual observation, [A.sub.t], is equal to 0. Makridakis (1993) and Makridakis et al. (1998) argued that the MAPE is asymmetric in that "equal errors above the actual value result in a greater absolute percentage error than those below the actual value." Similarly, Armstrong and Collopy (1992) stated that the MAPE put a heavier penalty on forecasts that exceed the actual than those that are less than the actual.

Our simulation results further indicate that neither the MAPE nor the RMAPE is a sensitive forecasting accuracy measurement for comparing different moving average methods with moving periods (p) of 1, 3, 5, 7, 9, and averaging periods (k) of 3, 5, 7, and 9, and on a normally distributed independent time series with the coefficients of variation between 0.2 and 2.0. Moreover, the complexity from the MAPE and the RMAPE also applies to those popular non-normal independent time series when the coefficients of variation are between 0.4 and 4.0, This paper concludes that when data are from independent time series, the Mean Absolute Deviation (MAD) reveals the forecasting accuracies from various Moving Average Methods are valid, but not the MAPE and the RMAPE. Therefore, if practitioners and researchers are using the RMAPE as a criterion for comparing different forecasting methods, the results may lead to a false conclusion.

Data Analysis

The parameters set for the data collection were: T-distribution with 3 degrees of freedom, Uniform distribution in the interval of (-[1/2], [1/2]), chi-squared distribution with 2 degrees of freedom, a mean of one, and the standard deviation set as the coefficients of variation (c.v.). With these parameters, 20,000 random observations were simulated and the following transformations were made.

1. For normal distributions, let N = 1 +N(0, [[c.v.].sup.2]), where N(0, [[sigma].sup.2]) represents a normal probability distribution, with a mean of zero, and a standard deviation of [sigma].

2. For T-distributions, let T = 1 + [[c.v.]/[square root of (3)]]T(3), where T(3) is the T-distribution of 3 degrees of freedom, with a mean of zero, and a standard deviation of [square root of (3)] To avoid an actual observation to be zero generated from the T-distribution, assign its value to be 0.000001 when it occurs in the simulation.

3. For Uniform distributions, let U = 1 + [square root of (12c.v.)] U(-[1/2], [1/2]), where U(-[l/2], [1/2]) is the Uniform probability distribution in interval (-[1/2], [1/2]), with a mean of zero, and a standard deviation of [1/[square root of (12)]].

4. For Chi-squared distributions, let K = 1 + c.v. [[[x.sup.2](2)]/2]-1], where [[chi].sup.2] (2) is the Chi-squared probability distribution of 2 degrees of freedom, with a mean of two and a standard deviation of two.

Consider those probability distributions above with a mean of one and a standard deviation of 0.1, 0.2, ..., 1.9, 2.0, 3.0, 4.0 and 5.0 (i.e., with the coefficients of variation (c.v.) of 0.1, 0.2, ..., 1.9, 2.0, 3.0, 4.0 and 5.0). Then organize the data into 1,000 groups with 20 observations each. The first nine observations in each group are treated as historical observations, and the tenth to twentieth observations are treated as the future 11 observations. Moving average methods, with moving periods of 1, 3, 5, 7, and 9, are applied to historical observations and their forecasts compared with the first future observation (the tenth observation). Absolute Percentage Deviation, |[[[A.sub.10] - [F.sub.10]]/[A.sub.10]]|, is calculated for the first future observation. Now, include the eleventh observation into the new historical group. Moving average methods, with moving periods of 1, 3, 5, 7, and 9, are applied to the most current nine historical observations counted back from the tenth observation (i.e., the first observation in the old historical data group is eliminated), and their forecasts are compared with the "new" first future observation (the eleventh observation). Absolute Percentage Deviation, |[[[A.sub.11] - [F.sub.11]]/[A.sub.11]]| is calculated. Continue the above process to calculate the eleventh Absolute Percentage Deviation, |[[[A.sub.20] - [F.sub.20]]/[A.sub.20]]|.

This paper will only show the results of analyzing RMAPE's for k=3, 5, 7, and 9. A numerical example for simulated data from a normal distribution with a c.v. of 1 is listed in Table 1 for illustration purposes. Numerical examples for simulated data from non-normal distributions with a c.v. of 4 are suppressed.
Table 1 Forecasts from Moving Average Methods with Moving Period 1, 3,
5, 7, and 9 for the Normal Probability Distribution with c.v. = 1

Time t  Actual [A.sub.t]          Forecast [F.sub.t]

                             MA(1)       MA(3)       MA(5)
1          0.6997678
2         -0.2776832
3          1.2442573
4          2.2764735
5          2.1983502
6          2.7331331
7         -1.1835876
8          0.7658188
9          2.0950225
10        -0.0867006       2.0950225   0.5590845   1.3217474
11         0.3097958      -0.0867006   0.9247135   0.8647372
12        -0.6904323       0.3097958   0.7727059   0.3800698
13        -0.8469109      -0.6904323  -0.155779    0.4787008
14         0.0223705      -0.8469109  -0.4091825   0.1561549
15         0.2264929       0.0223705  -0.5049909  -0.2583755
16        -1.1179312       0.2264929  -0.1993491  -0.1957368
17         0.4320751      -1.1179312  -0.2896893  -0.4812822
18         0.5959524       0.4320751  -0.153121   -0.2567807
19         1.1348531       0.5959524  -0.0299679   0.031792
20         0.634507        1.1348531   0.7209602   0.2542885

Time t  Actual [A.sub.t]      Forecast [F.sub.t]

                              MA(7)         MA(9)
1          0.6997678
2         -0.2776832
3          1.2442573
4          2.2764735
5          2.1983502
6          2.7331331
7         -1.1835876
8          0.7658188
9          2.0950225
10        -0.0867006       1.447066831   1.172394721
11         0.3097958       1.25692998    1.085009333
12        -0.6904323       0.975976022   1.150284778
13        -0.8469109       0.563292801   0.935319263
14         0.0223705       0.051857945   0.588276549
15         0.2264929       0.22413768    0.346501025
16        -1.1179312       0.147091135   0.067985451
17         0.4320751      -0.311902257   0.07528061
18         0.5959524      -0.237791431   0.038197984
19         1.1348531      -0.196911918  -0.12836536
20         0.634507        0.063843136   0.00736283


In Table 1, for instance, the italicized figure 0.924713 for the forecast of the eleventh period from MA(3) comes from (0.7658188 + 2.0950225 - 0.0867006)/3, and the remaining figures are obtained similarly. In Table 2, for example, the italicized figure 1.2607029 for the cumulative absolute forecasting error of the eleventh period from MA (3) comes from Table 1, |-0.0867006 - 0.5590845| + |0.3097958 - 0.9247135| = 0.6457851 + 0.6149177. The remaining figures are also obtained in a similar fashion.
Table 2 Cumulative Absolute Forecasting Errors from MA(1), MA(3),
MA(5), MA(7), MA(9) for the Normal Probability Distribution with
c.v. = 1

Time t  Actual [A.sub.t]      Cumulative Absolute Errors

                             MA(1)        MA(3)        MA(5)
1           0.6997678
2          -0.2776832
3           1.2442573
4           2.2764735
5           2.1983502
6           2.7331331
7          -1.1835876
8           0.7658188
9           2.0950225
10         -0.0867006     2.181723175  0.645785197  1.408448043
11          0.3097958     2.578219664  1.2607029    1.963389423
12         -0.6904323     3.578447831  2.723841135  3.033891517
13         -0.8469109     3.734926395  3.41497298   4.359503237
14          0.0223705     4.604207788  3.846525942  4.493287634
15          0.2264929     4.808330232  4.578009793  4.978156085
16         -1.1179312     6.152754395  5.496591863  5.900350516
17          0.4320751     7.70276074   6.218356248  6.813707842
18          0.5959524     7.866638043  6.967429727  7.666440979
19          1.1348531     8.405538665  8.132250665  8.769502074
20          0.634507      8.90588467   8.218703821  9.149720654

Time t  Actual [A.sub.t]   Cumulative Absolute Errors

                              MA(7)         MA(9)
1           0.6997678
2          -0.2776832
3           1.2442573
4           2.2764735
5           2.1983502
6           2.7331331
7          -1.1835876
8           0.7658188
9           2.0950225
10         -0.0867006      1.53376748    1.25909537
11          0.3097958      2.48090162    2.034308864
12         -0.6904323      4.14730997    3.875025969
13         -0.8469109      5.557513662   5.657256123
14          0.0223705      5.587001104   6.223162169
15          0.2264929      5.589356371   6.343170248
16         -1.1179312      6.854378723   7.529086916
17          0.4320751      7.598356108   7.885881435
18          0.5959524      8.432099971   8.443635882
19          1.1348531      9.763864942   9.706854295
20          0.634507      10.33452885   10.33399851


Following the same procedure, the cumulative absolute percentage errors in Table 3 can be obtained. For instance, the italic figure 9.433359783 in Table 3 for the cumulative absolute forecasting error of the 11th period from MA(3) comes from (|-0.0867006 - 0.5590845|/|-0.0867006|) + (|0.3097958 - 0.9247135|/|0.3097958|) = 7.4484502+1.9849129.
Table 3 Cumulative Absolute Percentage Errors from MA(I), MA(3), MA(5),
MA(7), MA(9) for the Normal Probability Distribution with c.v.=l

Time t  Actual [A.sub.t]   Cumulative Absolute Percentage Error

                              MA(1)        MA(3)         MA(5)
1           0.6997678
2          -0.2776832
3           1.2442573
4           2.2764735
5           2.1983502
6           2.7331331
7          -1.1835876
8           0.7658188
9           2.0950225
10         -0.0867006     25.16386197   7.448447055  16.24495377
11          0.3097958     26.44372579   9.433359783  18.03626698
12         -0.6904323     27.89242413  11.55252219   19.58674782
13         -0.8469109     28.07718804  12.36858425   21.15197946
14          0.0223705     66.93556361  31.65974498   27.13237244
15          0.2264929     67.83679452  34.88935484   29.27313867
16         -1.1179312     69.03939459  35.71103515   30.09805027
17          0.4320751     72.62674837  37.38149554   32.21193572
18          0.5959524     72.90173224  38.63843055   33.64281022
19          1.1348531     73.37659603  39.66483739   34.61479606
20          0.634507      74.16515465  39.80108987   35.21403066

Time t  Actual [A.sub.t]      Cumulative Absolute
                               Percentage Error

                               MA(7)        MA(9)

1           0.6997678
2          -0.2776832
3           1.2442573
4           2.2764735
5           2.1983502
6           2.7331331
7          -1.1835876
8           0.7658188
9           2.0950225
10         -0.0867006     17.69038052  14.52232916
11          0.3097958     20.74766541  17.02466585
12         -0.6904323     23.16123772  19.69070136
13         -0.8469109     24.82635233  21.79509059
14          0.0223705     26.14449179  47.09206558
15          0.2264929     26.15489064  47.62191907
16         -1.1179312     27.28646504  48.6827327
17          0.4320751     29.00833547  49.50850233
18          0.5959524     30.40734623  50.44440664
19          1.1348531     31.58085937  51.55751851
20          0.634507      32.48024089  52.5459145


From Tables 2 and 3, the Mean Absolute Deviation (MAD) and the Revised Mean Absolute Percentage Error (RMAPE) for various average periods k = 1, 2, 3, ..., 11 can be obtained. For instance, the italicized figure 0.63035 and 4.7167 in Tables 4 and 5 for MAD and RMAPE with k = 2 come from (1.2607029/2) in Table 2 and (9.433359783/2) in Table 3, respectively.
Table 4 Mean Absolute Deviation (MAD) from MA (1), MA (3), MA (5), MA
(7), MA (9) for the Normal Probability Distribution with c.v.= l

k   Time t     Actual                      MAD
             [A.sub.t]

                        MA(1,k)  MA(3,k)  MA(5,k)  MA(7,k)  MA(9,k)

    1        0.6997678
    2       -0.2776832
    3        1.2442573
    4        2.2764735
    5        2.1983502
    6        2.7331331
    7       -1.1835876
    8        0.7658188
    9        2.0950225
1   10      -0.0867006  2.18172  0.64579  1.40845  1.53377  1.2591
2   11       0.3097958  1.28911  0.63035  0.98169  1.24045  1.01715
3   12      -0.6904323  1.19282  0.90795  1.0113   1.38244  1.29168
4   13      -0.8469109  0.93373  0.85374  1.08988  1.38938  1.41431
5   14       0.0223705  0.92084  0.76931  0.89866  1.1174   1.24463
6   15       0.2264929  0.80139  0.763    0.82969  0.93156  1.0572
7   16      -1.1179312  0.87896  0.78523  0.84291  0.9792   1.07558
8   17       0.4320751  0.96285  0.77729  0.85171  0.94979  0.98574
9   18       0.5959524  0.87407  0.77416  0.85183  0.9369   0.93818
10  19       1.1348531  0.84055  0.81323  0.87695  0.97639  0.97069
11  20       0.634507   0.80963  0.74715  0.83179  0.9395   0.93945

Table 5 Revised Mean Absolute Percentage Error (RMAPE) from MA(1),
MA(3), MA(5), MA(7), MA (9) for the Normal Probability Distribution
with c.v. = l

k   Time t     Actual                     RMAPE
             [A.sub.t]

                        MA(1,k)  MA(3,k)  MA(5,k)  MA(7,k)  MA(9,k)
    1        0.6997678
    2       -0.2776832
    3        1.2442573
    4        2.2764735
    5        2.1983502
    6        2.7331331
    7       -1.1835876
    8        0.7658188
    9        2.0950225
1   10      -0.0867006  25.1639  7.4484   16.2450  17.6904  14.5223
2   11       0.3097958  13.2219  4.7167    9.0181  10.3738   8.5123
3   12      -0.6904323   9.2975  3.8508    6.5289   7.7204   6.5636
4   13      -0.8469109   7.0193  3.0921    5.2880   6.2066   5.4488
5   14       0.0223705  13.3871  6.3319    5.4265   5.2289   9.4184
6   15       0.2264929  11.3061  5.8149    4.8789   4.3591   7.9370
7   16      -1.1179312   9.8628  5.1016    4.2997   3.8981   6.9547
8   17       0.4320751   9.0783  4.6727    4.0265   3.6260   6.1886
9   18       0.5959524   8.1002  4.2932    3.7381   3.3786   5.6049
10  19       1.1348531   7.3377  3.9665    3.4615   3.1581   5.1558
11  20       0.634507    6.7423  3.6183    3.2013   2.9527   4.7769


Apply the above process to a total of 1,000 simulated time series sets (each with 20 observations) from the normal probability distribution with a mean of 1 and a standard deviation of 1 (i.e., with a coefficient of variation, c.v. = 1). Conduct pair-wise T-tests on the 1,000 independent MADs and RMAPEs to determine whether there is any difference between the means of MADs and RMAPEs from forecasting methods of MA(1), MA(3), MA(5), MA(7), and MA(9). A summary of the p-values for those pair-wise T-tests is listed in Tables 6 and 7.
Table 6 P-values from Pair-wise T-tests on Means of MAD's from
MA(1), MA(3), MA(5), MA(7), MA (9) for the Normal Probability
Distribution with c.v.= 1

k        MAD1   MAD3  MAD5   MAD7    MAD9

3  MAD1   --   0.000  0.000  0.000  0.000
   MAD3   --   --     0.000  0.000  0.000
   MADS   --   --     --     0.000  0.000
   MAD7   --   --     --     --     0.000

5  MAD1   --   0.000  0.000  0.000  0.000
   MAD3   --   --     0.000  0.000  0.000
   MAD5   --   --     --     0.000  0.000
   MAD7   --   --     --     --     0.000

7  MAD1   --   0.000  0.000  0.000  0.000
   MAD3   --   --     0.000  0.000  0.000
   MAD5   --   --     --     0.000  0.000
   MAD7   --   --     --     --     0.000

9  MADI   --   0.000  0.000  0.000  0.000
   MAD3   --   --     0.000  0.000  0.000
   MAD5   --   --     --     0.000  0.000
   MAD7   --   --     --     --     0.000

Table 7 P-values from Pair-wise T-tests on Means of RMAPE's from MA(1),
MA(3), MA(5), MA(7), MA(9) for the Normal Probability Distribution with
c.v. = 1

k          RMAPE1  RMAPE3  RMAPE5  RMAPE7  RMAPE9

3  RMAPE1    --    0.398   0.376   0.149   0.110
   RMAPE3    --    --      0.353   0.058   0.046
   RMAPE5    --    --      --      0.090   0.099
   RMAPE7    --    --      --      --      0.158

5  RMAPE1    --    0.195   0.177   0.041   0.022
   RMAPE3    --    --      0.173   0.018   0.013
   RMAPE5    --    --      --      0.077   0.058
   RMAPE7    --    --      --      --      0.055

7  RMAPEI    --    0.054   0.066   0.033   0.019
   RMAPE3    --    --      0.095   0.021   0.007
   RMAPE5    --    --      --      0.086   0.150
   RMAPE7    --    --      --      --      0.401

9  RMAPE1    --    0.029   0.045   0.018   0.011
   RMAPE3    --    --      0.097   0.017   0.009
   RMAPE5    --    --      --      0.061   0.150
   RMAPE7    --    --      --      --      0.531


The tables show that the p-values of all pair-wise T-tests for the MADs for k=3, 5, 7 and 9 are less than 0.000, while most of the p-values for pair-wise T-tests for the RMAPE for k=3, 5, and 7 are larger than 0.05 with the exception of k = 9.

Results for various other popular independent time series with the coefficient of variation of 4 are listed in Tables 8, 9, 10, 11, 12 and 13. Simulated time series for various coefficients of variation of 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 1.0, 1.5, 2.0, 2.5, 3.0, 4,0, and 5.0 are also studied. Tables for their p-values are not included in this paper.
Table 8 P-values from Pair-wise T-tests on Means of MAD's from MA(1),
MA(3), MA(5), MA(7), MA (9) for the T Distribution with c.v.=4

   k        MAD1   MAD3   MAD5   MAD7   MAD9

T  3  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.080

   5  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

   7  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

   9  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

Table 9 P-values from Pair-wise T-tests on Means of RMAPE's from MA(1),
MA(3), MA(5), MA(7), MA(9) for the T Distribution with c.v. = 4

   k          RMAPE1  RMAPE3  RMAPE5  RMAPE7  RMAPE9

T  3  RMAPE1    --    0.213   0.156   0.163   0.161
      RMAPE3    --    --      0.398   0.411   0.465
      RMAPE5    --    --      --      0.385   0.227
      RMAPE7    --    --      --      --      0.145

   5  RMAPE1    --    0.140   0.099   0.129   0.160
      RMAPE3    --    --      0.308   0.288   0.307
      RMAPE5    --    --      --      0.746   0.766
      RMAPE7    --    --      --      --      0.796

   7  RMAPE1    --    0.075   0.033   0.046   0.060
      RMAPE3    --    --      0.462   0.484   0.515
      RMAPE5    --    --      --      0.977   0.953
      RMAPE7    --    --      --      --      0.927

   9  RMAPE1    --    0.069   0.027   0.038   0.048
      RMAPE3    --    --      0.509   0.519   0.583
      RMAPE5    --    --      --      0.934   0.984
      RMAPE7    --    --      --      --      0.889

Table 10 P-values from Pair-wise T-tests on Means of MAD's from MA(1),
MA(3), MA(5), MA(7), MA (9) for the Uniform Distribution with c.v. =4

   k        MAD1   MAD3   MAD5   MAD7   MAD9

U  3  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.006  0.002
      MAD7   --   --     --     --     0.140

   5  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.148

   7  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

   9  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

Table 11 P-values from Pair-wise T-tests on Means of RMAPE's from
MA(I), MA(3), MA(5), MA(7), MA(9) for the U Distribution with c.v. =4

   k          RMAPE1  RMAPE3  RMAPE5  RMAPE7  RMAPE9

U  3  RMAPE1    --    0.000   0.000   0.000   0.000
      RMAPE3    --    --      0.821   0.117   0.003
      RMAPE5    --    --      --      0.008   0.005
      RMAPE7    --    --      --      --      0.041

   5  RMAPE1    --    0.000   0.000   0.000   0.000
      RMAPE3    --    --      0.082   0.016   0.000
      RMAPE5    --    --      --      0.255   0.001
      RMAPE7    --    --      --      --      0.051

   7  RMAPE1    --    0.000   0.000   0.000   0.000
      RMAPE3    --    --      0.066   0.067   0.003
      RMAPE5    --    --      --      0.803   0.141
      RMAPE7    --    --      --      --      0.038

   9  RMAPE1    --    0.000   0.000   0.000   0.000
      RMAPE3    --    --      0.022   0.012   0.000
      RMAPE5    --    --      --      0.579   0.084
      RMAPE7    --    --      --      --      0.048

Table 12 P-values from Pair-wise T-tests on Means of MAD's from MA(1),
MA(3), MA(5), MA(7), MA (9) for the Chi-squared Distribution with
c.v. = 4

   k        MAD1   MAD3   MAD5   MAD7   MAD9

K  3  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.082

   5  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

   7  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

   9  MAD1   --   0.000  0.000  0.000  0.000
      MAD3   --   --     0.000  0.000  0.000
      MAD5   --   --     --     0.000  0.000
      MAD7   --   --     --     --     0.000

Table 13 P-values from Pair-wise T-tests on Means of RMAPE's from MA(1),
MA(3), MA(5), MA(7), MA(9) for the Chi-squared Distribution with
c.v. = 4

   k          RMAPEI  RMAPE3  RMAPE5  RMAPE7  RMAPE9

K  3  RMAPE1    --    0.001   0.000   0.000   0.000
      RMAPE3    --    --      0.007   0.001   0.006
      RMAPE5    --    --      --      0.041   0.028
      RMAPE7    --    --      --      --      0.073

   5  RMAPE1    --    0.002   0.000   0.000   0.000
      RMAPE3    --    --      0.293   0.023   0.020
      RMAPE5    --    --      --      0.022   0.206
      RMAPE7    --    --      --      --      0.528

   7  RMAPE1    --    0.003   0.003   0.000   0.000
      RMAPE3    --    --      0.081   0.486   0.012
      RMAPE5    --    --      --      0.552   0.843
      RMAPE7    --    --      --      --      0.142

   9  RMAPE1    --    0.002   0.002   0.001   0.001
      RMAPE3    --    --      0.038   0.197   0.009
      RMAPE5    --    --      --      0.620   0.564
      RMAPE7    --    --      --      --      0.071


Tables 8 and 9 report the p-values from the pair-wise T-tests for the MADs and the RMAPEs. The p-values from all pair-wise T-tests for the MADs are less than 0.001 with one exception, when k = 3. But the p-values for the RMAPEs for all k's are larger than 0.05 with few exceptions. In Tables 11 and 13, most of the p-values from the pair-wise T-tests for the MAPEs with the various k's, when the distributions are of the uniform distribution and the Chi-squared distribution, are less than 0.001. For the MADs, Tables 10 and 11 show that all p-values with the exception of few pair-wise T-tests for various k's are less than 0.001.

Conclusion

This study demonstrates that, for independent time series with the coefficients of variation between 0.4 to 4.0, the MAD can clearly show the differences from forecasting methods of MA(1), MA(3), MA(5), MA(7), and MA(9). In contrast, the RMAPE sometimes does little to differentiate between the different forecasting methods. Therefore, from our study, the RMAPE should only be used to measure the forecasting accuracy for independent time series when its coefficient of variation is very large (says much greater than 4.0) or its coefficient of variation is very small (e.g., less than 0.4). In general, the MAD is recommended for all cases.

It is well known that for independent time series, Moving Average Methods will provide accurate forecasts, but those with larger moving average periods will provide more precise forecasts. Here, we define high forecasting precision as having a lower variance for its forecasting errors. To illustrate with a commonly known phenomenon, the Moving Average Method with moving periods of nine, MA(9), will provide a better forecast than its counterparts MA(7), MA(5), MA(3), and MA (1), in term of forecasting precision. Similarly, MA(7) will provide a better forecast than MA(5), MA(3), MA(1).

In this paper, simulation results show that the popularly used forecasting accuracy measure the Mean Absolute Deviation (MAD) has the capacity of precision measurements among the use of different Moving Average Methods to independent time series. Admittedly, the Revised Mean Absolute Percentage Errors (RMAPE) can perform as well as the MAD for those popular independent time series with small coefficients of variation as is the case when the c.v. is less than 0.4. However, as the c.[epsilon]. increases, from c.v.=0.5 up to c.v. = 4, the RMAPE will lose its capacity to distinguish the superiority of Moving Average Methods with larger moving average periods. In other words, MA(9), on the average, will not generate lower RMAPEs (for various averaging periods, k) than its counterparts MA(7), MA(5), MA(3), and MA(1) for popular independent time series. Therefore, if practitioners and researchers are using RMAPE as a criterion for comparing different forecasting methods, the results may lead to a false conclusion.

References

Armstrong, J. S., & Collopy, F. (1992). Error measures for generalizing about forecasting methods: empirical comparisons. International Journal of Forecasting, 8(1), 69-80.

Bowerman, B. L., O'Connell, R. T., & Koehler, A. H. (2004). Forecasting, time series and regression: An applied approach. Belmont, CA: Thomson Brooks/Cole.

Chatfield, C. (1988). Apples, oranges, and mean squared error. International Journal of Forecasting, 4, 515-518.

Chen, J. C, Bloomfield, P., & Fu, J. S. (2003). An evaluation of alternative forecasting methods to recreation visitation. Journal of Leisure Research, 35(4), 441-454.

Goodwin, P., & Lawton, R. (1999). On the asymmetry of the symmetric MAPE. International Journal of Forecasting, 15(4), 405-408.

Hanke, J. E., & Reitsch, A. G. (1995). Business forecasting (5th ed.). Englewood Cliffs, NJ: Prentice Hall.

Makridakis, S. G. (1993). Accuracy measures: theoretical and practical concerns. International Journal of Forecasting, 9(4), 527-529.

Makridakis, S. G., et al. (1982). The accuracy of extrapolations (time series) methods: results of a forecasting competition. Journal of Forecasting, 1, 111-153.

Makridakis, S. G., Wheelwright, S. C, & Hyndman, R. J. (1998). Forecasting: Methods and applications, 3 rd edn. Wiley.

Song, H., Witt, S. F., & Jensen, T. C. (2003). Tourism forecasting: accuracy of alternative econometric models. International Journal of Forecasting, 19(1), 123-144.

Swanson, D. A., Tayman, J., & Barr, C. F. (2000). A note on the measure of accuracy for subnational demographic estimates. Demography. 37(2), 193-201.

Wang, Z., & Liu, C. (2005). An empirical evaluation of the loop detector method for travel time delay estimation. Journal of Intelligent Transportation Systems, 9(4), 161-174.

Weller, B. R. (1989). National indicator series as quantitative predictors of small region monthly employment levels. International Journal of Forecasting, 5(2), 241-247.

Weller, B. R., & Kurre, J. A. (1987). Applicability of the transfer function approach to forecasting employment levels in small regions. Annals of Regional Science, 21(1), 34-33.

L. Ren * Y. Glasure ([??])

University of Houston-Victoria, Victoria, TX, USA

e-mail: glasurey@uhv.edu

Published online: 26 August 2009

[C] International Atlantic Economic Society 2009
COPYRIGHT 2009 Atlantic Economic Society
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2009 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Ren, Louie; Glasure, Yong
Publication:International Advances in Economic Research
Geographic Code:4E
Date:Nov 1, 2009
Words:4905
Previous Article:An analysis of EU voting procedures in the enlargement context.
Next Article:How can a cartel be detected?
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters