The determination of objective temperature forecasts using proxy stations.
KEY WORDS: temperature forecasting, New Brunswick temperatures
The National Weather Service issues objective temperature forecasts for future 12-hour periods (00-12 UTC and 12-00 UTC) twice per day using a statistical approach called Model Output Statistics (MOS) (Carter et. al., 1989). Temperature is but one element included in the MOS messages, which is disseminated on the WEB (URL: tgsv5.nws.gov/tdl). There is a separate set of forecasts for up to 600 stations, with the number varying depending on which numerical model serves to supply predictor variable values to the regression equations. For short-range (i.e., 6 to 72 hour projection) forecasts, the Nested Grid Model (NGM) (Dallavalle et. al., 1992) and Aviation Model (AVN) (Dallavalle et. al., 2000) supply input to a separate set of objective temperature forecast equations. New stations are added to the list periodically. For the state of New Jersey there are 6 (NGMMOS) to 14 (AVN-MOS) stations that have a set of MOS predictions produced. Near the border with New York and Pennsylvania there are an additional 5 to 6 stations with supplied MOS guidance. This means that there are a large number of locations, including many mid-size cities, for which there are no MOS forecasts issued. The question is then raised: How to effectively use available MOS forecast guidance to make forecasts at stations lacking specific guidance? A strongly motivated user could derive a set of equations for a city of interest using much of the same techniques that are employed to produce the equations now used operationally at nearby locations. This would be difficult. A more viable possibility is that the user could simply pick the closest station to use for forecast guidance based on some criteria such as closest distance to the point of interest or using the station with the perceived best match of microclimate characteristics. Also, the user could somehow combine forecast information from two or more nearby locations in order to produce the desired result. There is much research available that indicates that temperatures can vary significantl y over short distances especially under certain meteorological conditions at night. Therefore, simply using the forecast guidance from the spatially closest station to make a forecast might not be optimal. Typically forecasters use their own experience with the microclimatology of a given location combined with the objective MOS forecasts available for one or more surrounding stations to subjectively produce the forecast for that city. This method may not produce good forecasts since there may not be one clearly best station to use in all weather conditions.
This study proposes and demonstrates a viable option for producing objective maximum and minimum temperature forecasts for a location for which MOS forecast guidance is not available.
MATERIALS AND METHODS
There is an obvious trade-off existing between the degree of effort needed to overcome the aforementioned difficulties and the accuracy of the forecasts produced. On the extremes in terms of effort expended is the use of a single 'MOS' station closest in distance to the target station on the one hand and attempting to derive a new regression equation for the target location using the MOS approach on the other. Since there are a relatively large number of stations in close proximity in the greater New Jersey region, an intermediate approach is proposed using New Brunswick, NJ as the target location to demonstrate the method. This location does not have objective forecast guidance produced for it. In New Jersey and the surrounding areas close to the state's border with Pennsylvania and New York, there are seven NGM-MOS reporting stations: Allentown, PA (ABE), Atlantic City, NJ (ACY), Newark, NJ (EWR), LaGuardia Airport (LGA), NY, Philadelphia, PA (PHL), Teterboro, NJ (TEB), and Wrightstown, NJ (WRI). Please ref er to Fig. 1 for the station locations. Each of these places have their own microclimate and in some cases it is quite different from that of New Brunswick, which is best described as a flat suburban or semi-rural site. As mentioned this presents a problem in deciding which station is the best to use in the forecasting of minimum and maximum temperatures for New Brunswick. The suggested approach is to carry out a verification of forecasts made over a recent period of time and stratify the statistics by season and weather conditions (i.e., wind and cloud coverage). This is done in order to determine the best MOS station to use, based on season and weather conditions, for making future New Brunswick temperature forecasts.
The aforementioned seven NGM-MOS stations were used for this study. Two seasons were selected to demonstrate the utility of the method: fall (September, October, and November) and winter (December, January, and February). Three recent years of data (1997, 1998, and 1999) were obtained, which included the 12 to 24 hour (00-12 UTC) minimum and 24 to 36 hour (12-00 UTC) maximum temperature forecasts for each station. Only the 12 UTC NGM-MOS forecasts were employed. Therefore, applicable minimum temperatures are for "tonight" (hereafter referred to as the first period) and the maximum temperatures are for "tomorrow" (hereafter referred to as the second period). In order to carry out the required forecast verification the corresponding observed minimum (00-12 UTC) and maximum temperatures (12-00 UTC) for New Brunswick, NJ were obtained. The total sample consisted of 539 forecasts for each station.
The NGM-MOS forecasts for each of the seven stations were compared to the observed temperatures for New Brunswick and both the absolute and algebraic error was calculated for each day in the sample. In addition, the absolute error for a number of different sub-samples was produced, in which the samples varied seasonally, diurnally, or meteorologically. The MOS forecast data for the seven stations were obtained from the Techniques Development Laboratory (TDL) (courtesy of J. Dallavalle). The observed overnight minimum (00-12 UTC) and daytime maximum (12-00 UTC) temperatures for New Brunswick, NJ were obtained from the New Jersey State Climatology Office based at Rutgers University in New Brunswick, NJ (courtesy of Mr. Keith Arnesen).
Differentiation of sub-samples by meteorological conditions was done on the basis of forecasted wind and cloud conditions at Newark, NJ (EWR). The EWR NGM-MOS forecast of wind speed and cloud coverage at 12, 15, 18, 21, and 24-hour from initial time (12 UTC each day) were combined in order to characterize the conditions in the first period (i.e., overnight from 00 to 12 UTC). For the second period (following day from 12 to 00 UTC) the projection times used in like manner were at 24, 27, 30, 33, and 36 hours from initial time. The wind speed and cloud coverage for each 12-hour period were obtained by simple averaging of the five forecasts produced for each period. To facilitate averaging, the forecast cloud coverage was converted to a numerical value. Clear (CL) and scattered (SC) were assigned a value of one, broken (BK) was assigned a value of two, and overcast (OV) was assigned a value of three.
After the five cloud coverage values were averaged for each 12-hour period in the sample, each period was placed into one of three cloud coverage categories: category 1 for a cloud coverage average of 1.00-1.65, category 2 for a cloud coverage average of 1.66-2.32, and category 3 for a cloud coverage average of 2.33-3.00. The category limits were drawn so that each of the three cloud categories would be equally broad.
For winds, category 1 included average wind speeds of 0.00-6.49 knots, category 2 included average wind speeds of 6.50-13.49 knots, and category 3 included average wind speeds of 13.50 knots or greater. The wind category limits were chosen so that about the same number of cases fell into each category.
For all days in the sample the absolute and algebraic forecast error for each of the seven NGM-MOS stations was computed. Statistics were aggregated to produce month, season, year, and combined values. Quite possibly the most important part of the study was the forecast verification for each combination of cloud and wind speed category in order to find the station with the smallest error for each meteorological condition. To facilitate interpretation of the result tables for varying meteorological conditions, the following is presented.
WINDS OF WINDS OF WINDS OF 0.00-6.49 6.50-13.49 13.5 OR MORE Cloud cover of Clouds = 1 Clouds = 1 Clouds = 1 1.00-1.65 Winds = 1 Winds = 2 Winds = 3 Cloud cover of Clouds = 2 Clouds = 2 Clouds = 2 1.66-2.32 Winds = 1 Winds = 2 Winds = 3 Cloud cover of Clouds = 3 Clouds = 3 Clouds = 3 2.33-3.00 Winds = 1 Winds = 2 Winds = 3
This indicates that cloud coverage and wind speed increase with increasing category value.
RESULTS AND DISCUSSION
Table I shows the mean absolute error (MABE) and mean algebraic error (MALE) for NGM-MOS minimum temperature forecasts for period one, separated by station, for fall and winter. The MALE is a measure of bias, or the tendency for the forecast to be too high (i.e., positive MALE) or too low (i.e., negative MALE). In addition, the sample percent that the forecasts were too high and too low are shown. This shows a wide variation in error values by station employed, especially for the fall season. Based on the 1997-1999 sample, the forecasts for station ACY is best (MABE=2.66) for the fall season first period minimum temperature forecasts. The MALE of 0.11 indicates little bias. WRI is second best and LGA is worst. These results are a bit surprising especially since ACY is the farthest station from New Brunswick. For winter, forecasts for station WRI performed best, while station TEB was second and station ACY was third. LGA was again last. When both seasons are combined (not shown) stations ACY and WRI have simil ar MABE values (i.e., 3.09 vs. 3.11). The largest MABE is for station LGA (MABE=5.97), which also has a large warm bias. Clearly the well documented urban heat island effects, which is significant at night under low cloud/low wind conditions, is an important reason for these results. The fall season typically has more of these type nights than does winter. Other noteworthy items are: (1) MABE increases from fall to winter for the more rural stations (ABE, ACY, TEB, and WRI) and decreases for the more "urban" stations (EWR, LGA, and PHL) and (2) Except for station ACY and ABE, there is a warm bias seen in the minimum temperature forecasts. The first is attributed to a combination of the fact that day-today temperature variation is larger in winter (thus there is a tendency for forecast error to increase from fall to winter) and the fact that winter nights tend to be more windy than fall nights on average (thus there is a tendency for the more urban stations to have better forecasts when applied to New Brunswic k since wind reduces heat island effects). The second point is largely attributed to heat island effects.
Table II shows the mean absolute error (MABE) and mean algebraic error (MALE) for NGM-MOS maximum temperature forecasts for period two, separated by station, for fall and winter. The error value variation is considerably smaller than for minimum temperature forecasts. This is undoubtedly because micrometeorological factors such as local terrain and surface type affect night temperatures much more than daytime temperatures. This also implies that station proximity would be at least as important as similarity of local terrain. The overall best maximum temperature forecasts (not shown) are for stations WRI (MABE=3.21) and EWR (MABE=3.32), which are among the closest stations to New Brunswick. It is further noted that the MABE values reported in this study for the best proxy stations (i.e., 2.5 to 3.5 degrees) are comparable to the Northeast US region MABE values for the latest available month (i.e., February 2002), which is given on the WEB (http://18.104.22.168/tdl/synop1 eval.htm). This suggests that suitably chosen MOS temperature forecasts have errors that are just as low for guidance being applied to non-MOS stations as they are for stations for which the guidance is intended.
Table III shows the mean absolute error for NGM-MOS minimum temperature forecasts separated by cloud coverage-wind speed categories. Both seasons are combined in order to increase sample sizes.
This shows that station ABE tends to be best under low cloud coverage conditions (cloud category 1). As cloud coverage or wind speed increases the best station tends to be one of the stations closest to New Brunswick (i.e., WRI and EWR). It is noted that several cloud coverage-wind speed combinations had small sample sizes (i.e., less than 20), and are not reported.
Table IV shows the mean absolute error for NGM-MOS maximum temperature forecasts separated by cloud coverage-wind speed categories. There is no obvious relationship between meteorological condition and attributes of the station that has the least error. Recall that averaging over all meteorological conditions indicates that the least error overall is for station WRI which is in the interior of New Jersey, like New Brunswick, and not as distant as most of the others. The largest contribution to the relatively low error for WRI seems to come from days with higher wind speeds (wind categories 2 and 3) regardless of cloud coverage.
In summary, for the case of New Brunswick, NJ fall minimum temperature forecasts, this study suggests that NGM-MOS forecasts for station ACY (Atlantic City, NJ) or WRI (Wrightstown, NJ) should be employed as a proxy, if cloud coverage is low and wind speeds are low or moderate. It is noted that 53% of the cases used in this study were of this type. For winter minimum temperature forecasts, WRI is the winner unless the cloud coverage is high (i.e., cloud category 3), in which case EWR should be used. For maximum temperature forecasts at New Brunswick, the results are less definitive but in general using station WRI as the proxy, except at low wind speeds, would minimize forecast error. The most important conclusion from the results presented is that picking the closest station, as proxy for making objective minimum temperature forecasts at a station lacking guidance is not recommended unless the microclimates are similar. Conducting a study such as that employed here to find the best proxy is recommended in or der to reduce forecast error from that achieved from subjective application of forecast guidance available at surrounding stations.
As for using the results from this type of study in real-time, a suggestion is to obtain the current MOS forecasts for several stations surrounding the station of interest. Then adjust each of the forecasts up or down based on the error characteristics obtained from studies on the historic record. One way is to add or subtract the mean algebraic error for the station to (or from) the actual MOS forecast. From here the adjusted forecasts could be averaged or otherwise combined to yield the forecast for the target station. A more sophisticated approach would be to perform multiple linear regression analysis using past MOS forecasts for each of the surrounding stations to define the predictors and the observed temperatures at the station of interest as the predictand. The regression equation created would then be available to use in real-time to make objective temperature forecasts at the target station. In addition, there are a variety of other ways one could use to combine forecasts using known forecast errors to help optimize the forecast for a station not having objective forecasts available for it.
The mean absolute errors obtained in this study when MOS forecasts from "good" candidate proxy stations are applied to the non-MOS station New Brunswick are 2.5 to 3.5 degrees. The user must decide if this magnitude of error is acceptable and appropriately hedge his or her actions accordingly.
Table I Error statistics by season for NGM-MOS 12-24 hour minimum temperature forecasts. A positive (negative) mean algebraic error indicates that the forecast temperature was higher (lower) than the New Brunswick observed temperature. Rankings show the relative forecast performance for each station. A ranking of 1 means that station had the least mean absolute error. higher ABE ACY EWR LGA PHL TEB Fall mean abs. error 3.11 2.66 489 6.96 4.30 3.23 Fall mean algebraic error -1.81 0.11 4.01 6.49 3.44 1.15 Fall forecast too high (%) 26 44 77 89 76 58 Fall forecasts too low (%) 63 43 16 9 17 32 Fall ranking 3 1 6 7 5 4 Winter mean abs. error 3.90 3.52 3.86 4.97 3.68 3.36 Winter mean algebraic error -2.47 -0.54 2.13 3.69 3.24 0.49 Winter forecasts too high (%) 24 34 55 72 64 50 Winter forecasts too low (%) 71 51 29 22 26 41 Winter ranking 6 3 5 7 4 2 higher WRI Fall mean abs. error 3.03 Fall mean algebraic error 1.11 Fall forecast too high (%) 57 Fall forecasts too low (%) 33 Fall ranking 2 Winter mean abs. error 3.20 Winter mean algebraic error 0.43 Winter forecasts too high (%) 50 Winter forecasts too low (%) 38 Winter ranking 1 Table II Error statistics by season for NGM-MOS 24-36 hour maximum temperature forecasts. A positive (negative) mean algebraic error indicates that the forecast temperature was higher (lower) than the New Brunswick observed temperature. Rankings show the relative forecast performance for each station. A ranking of 1 means that station had the least mean absolute error. ABE ACY EWR LGA PHL TEB Fall mean abs. error 3.28 3.12 3.07 2.59 3.24 2.64 Fall mean algebraic error -1.05 1.78 1.83 0.51 2.09 -0.17 Fall forerasts too high (%) 32 61 64 45 65 39 Fall forecasts too low (%) 56 26 24 38 24 49 Fall ranking 7 5 4 1 6 2 Winter mean abs. error 4.75 3.63 3.58 4.08 3.57 4.19 Winter mean algebraic error -3.82 0.97 -0.64 -1.39 -0.01 -2.17 Winter forecasts too high (%) 17 52 39 39 41 21 Winter forecasts too low (%) 75 36 48 54 41 63 Winter ranking 7 4 3 5 2 6 WRI Fall mean abs. error 2.91 Fall mean algebraic error 1.39 Fall forerasts too high (%) 57 Fall forecasts too low (%) 35 Fall ranking 3 Winter mean abs. error 3.53 Winter mean algebraic error -0.43 Winter forecasts too high (%) 38 Winter forecasts too low (%) 50 Winter ranking 1 Table III. Error statistics for NGM-MOS 12-24 hour minimum temperature forecasts stratified by cloud coverage and wind speed. MABE indicates mean absolute error and MALE indicates mean algebraic error. 'Fx' is an abbreviation for 'Forecast'. A positive (negative) mean algebraic error indicates that the forecast temperature was higher (lower) than the New Brunswick observed temperature. See text for definition of cloud and wind categories. Bold numbers denote stations with the least mean absolute error and italicized numbers denote stations with the greatest mean absolute error. ** indicates that values are not shown due to small sample size (i.e., less than 20 cases). CLOUD, WIND # OF STATISTIC ABE ACY EWR LGA PHL CATEGORY CASES 1,1 114 MABE 2.50 2.72 6.92 9.88 5.56 MALE -0.13 -0.84 6.58 9.67 5.17 Fx too high (%) 37 43 94 96 92 Fx too low (%) 51 43 4 3 4 1,2 164 MABE 2.99 3.36 4.38 6.55 4.12 MALE -1.07 -2.02 3.46 6.18 2.89 Fx too high (%) 29 35 78 93 77 Fx too low (%) 63 50 15 6 17 1,3 18 ** ** ** ** ** 2,1 18 ** ** ** ** ** 2,2 44 MABE 2.86 3.55 3.09 4.36 2.66 MALE -0.36 -3.05 1.86 3.50 1.70 Fx too high ((%) 20 38 68 77 66 Fx too low (%) 75 48 27 18 23 2,3 3 ** ** ** ** ** 3,1 53 MABE 3.30 4.32 2.68 3.49 2.92 MALE -0.66 -3.42 1.13 2.28 1.08 Fx too high (%) 15 30 58 85 57 Fx too low (%) 79 49 15 17 34 3,2 103 MABE 4.05 4.25 3.43 3.50 3.91 MALE 1.37 -2.54 0.71 1.37 1.31 Fx too high (%) 20 49 45 55 56 Fx too low (%) 70 40 48 40 33 3.3 10 ** ** ** ** ** CLOUD, WIND TEB WRI CATEGORY 1,1 3.84 3.24 2.71 1.96 77 70 19 21 1,2 3.58 2.98 0.90 0.79 62 56 29 35 1,3 ** ** 2,1 ** ** 2,2 2.50 2.10 0.41 0.61 61 57 29 27 2,3 ** ** 3,1 2.79 3.11 -1.06 -0.74 28 36 58 49 3,2 3.57 3.52 -0.47 0.67 41 42 53 48 3.3 ** ** Table IV Error statistics for NGM-MOS 12-24 hour maximum temperature forecasts stratified by cloud coverage and wind speed. MABE indicates mean absolute error and MALE indicates mean algebraic error. 'Fx' is an abbreviation for 'Forecast'. A positive (negative) mean algebraic error indicates that the forecast temperature was higher (lower) than the New Brunswick observed temperature. See text for defination of cloud and wind categories. Bold numbers denote stations with the least mean absolute error and italicized numbers denote stations with the greatest mean absolute error. ** indicates that values are not shown due to small size (i.e., less than 20 cases). CLOUD, WIND # OF STATISTIC ABE ACY EWR LGA PHL CATEGORY CASES 1,1 75 MABE 2.73 2.64 3.07 2.57 3.05 MALE -1.08 0.99 1.71 -0.44 1.83 Fx too high (%) 24 48 68 36 68 Fx too low (%) 55 33 23 51 23 1,2 160 MABE 3.45 2.42 2.59 2.84 2.56 MALE -2.44 0.26 0.54 -0.91 0.65 Fx too high (%) 21 48 35 37 52 Fx too low (%) 69 39 31 47 31 1,3 61 MABE 3.60 2.98 2.53 2.91 2.43 MALE -2.66 1.83 0.11 -0.06 0.68 Fx too high (%) 13 74 43 40 49 Fx too low (%) 74 23 43 43 34 2,1 26 MABE 3.65 3.15 3.00 3.38 2.88 MALE -2.58 1.15 0.23 -1.35 1.42 Fx too high (%) 27 58 46 31 62 Fx too low (%) 58 27 46 54 23 2,2 61 MABE 4.67 3.33 3.49 3.57 3.54 MALE -3.75 0.77 -0.28 -0.89 0.00 Fx too high (%) 15 51 44 41 46 Fx too low (%) 72 36 48 51 46 2,3 17 ** ** ** ** ** 3,1 41 MABE 5.78 6.41 5.51 5.24 6.54 MALE -1.59 3.73 1.76 1.24 3.76 Fx too high (%) 41 49 61 63 76 Fx too low (%) 59 44 32 39 22 3,2 95 MABE 5.25 4.65 4.39 4.36 4.42 MALE -2.71 2.93 0.68 0.06 1.56 Fx too high (%) 33 65 54 55 56 Fx too low (%) 63 24 41 42 37 3,3 16 ** ** ** ** ** CLOUD, WIND TEB WRI CATEGORY 1,1 2.57 2.73 -0.60 0.79 33 51 49 32 1,2 2.96 2.58 -1.26 -0.34 31 42 54 46 1,3 3.00 2.30 -0.83 0.21 29 49 57 40 2,1 3.38 3.15 -2.23 0.15 27 42 65 54 2,2 3.85 3.52 -2.05 -.06 33 41 64 44 2,3 ** ** 3,1 5.15 5.83 -0.61 2.17 44 59 39 37 3,2 4.32 3.93 -0.91 1.55 44 57 52 33 3,3 ** **
DALLAVALLE, J. P., J. S. JENSENIUS, JR., AND S. A. GILBERT. 1992. NGM-based MOS guidance- The FOUS14/FWC message. NWS Technical Procedures Bulletin No. 408, National Oceanic and Atmospheric Administration, U.S. Department of Commerce.
CARTER, G. M., J. P. DALLAVALLE, ANO H. R. GLAHN. 1989. Statistical forecasts based on NMCs numerical weather prediction system. Wea. Forecasting, 4: 401-412.
DALLAVALLE, J. P., AND M. C. ERICKSON. 2000. AVN-based MOS guidance - The alphanumeric messages. NWS Technical Procedures Bulletin No. 463, National Oceanic and Atmospheric Administration, U.S. Department of Commerce.