Background

The core mission of the National Weather Service is the protection of life and property through its Watch and Warning program. This note will concentrate on public forecasts since these are integral to this program. The public is also one of the more difficult user groups to work with given their diverse character and needs. Forecasts for the public contrast sharply with forecasts prepared for specialized groups including agriculture, aviation, and commerce. In addition, many of these forecast responsibilities are being transferred to the private sector through the public-private partnership.

The Modernization of the National Weather Service is nearing completion at a price tag of over $4 billion. It has been an overwhelming technological success. In addition, dramatic improvements in several forecast categories have been documented, e.g., lead time for tornado warnings. On the other hand, it may be more debatable as to whether the promise of much improved public forecasts and warnings has been met. So, in the forecasters' opinion, what makes a good public forecast?

Methodology

1. What makes a good forecast? Please explain.

2. Give an example of a good forecast. Please explain.

3. Give an example of a bad forecast. Please explain.

Results

A great deal of work has already been done to evaluate or verify forecasts, both within the NWS and the academic community. Typically public forecasts, are verified against another forecast (e.g., MOS or climatology) to demonstrate some improvement over the other forecast, or skill. These are typically single point forecasts. It therefore seems logical that some of the forecasters would feel a good forecast was one that showed skill over a competing scheme. A definition might include some specific ranges of tolerance for various parameters. For example, a 24 hour maximum temperature forecast that is within two degrees of what occurs might be considered a good forecast.

Of the group of forecasters surveyed, only four focused their answer on measures of skill. Since these are region specific and have been documented objectively in many rigorous objective studies, I will not present any details of these responses. Nonetheless, there is clearly some weight given to the accuracy of the forecast in measuring its goodness.

The remainder of the forecasters placed their measure of a "good" forecast in the accuracy of its perception by the public or its societal impact. Specific remarks given by the forecasters included:

• a forecast that is honest and gives some measure of confidence
• the bottom line is accuracy from our users perspective
• a forecast that conveys something that people can visualize
• a forecast that compels the public to react
• a forecast that is read by the radio disk jockey and maintains its meaning
• a forecast that creates a picture in the user's mind

Although these are not dramatic or earth shattering they do show that the forecasters are concerned about public perception and the action the forecast instills.

When the examples of good forecasts and bad forecasts were compared, there were a few surprises. For example, one forecaster felt that good forecasts were "very short forecasts." The argument was that detail is lost to most people and by keeping the forecast simple you minimize possible confusion. The forecaster's examples were:

GOOD FORECAST: "Scattered showers and sunbreaks."

BAD FORECAST: "Rain likely, turning to scattered showers, mainly near the foothills this afternoon with isolated thunderstorms. Possible small hail and gusty winds with the thunderstorms."

In the good forecast case, the forecaster suggested most people would experience benign weather so the forecast would be accurate short and sweet for them. It would only be the few individuals who experienced a thunderstorm who would think it was a bad forecast, and to include the additional material for those few was not worth the risk of confusion.

This set of forecasts contrasts sharply with another forecaster's examples. These were:

GOOD FORECAST: "Rain ending and becoming partly sunny during the afternoon. Except mostly cloudy with occasional showers continuing north of Seattle through early afternoon."

BAD FORECAST: "Mostly cloudy with scattered showers."

It seems this contrast in forecaster opinion might be linked to the Modernization, which promised highly detailed and accurate mesoscale forecasts. The technology is in place to make those forecasts but the means to communicate that information is still under development. At this time it seems the text product is a synoptic-scale forecast system and does not do as well at communicating mesoscale information to the general public.

Additional discussion from the forecasters included external factors determining whether a forecast was considered a good forecast or a bad forecast. Two forecasters mentioned the role of credibility. If there have been a few bad forecasts then credibility goes down -- less and less attention is paid to the forecasts by the user. The forecasters also mentioned other external factors that play a role, including: several sources of forecasts resulting in confusion, paraphrasing by radio disk jockeys and/or the use of old forecasts, access to weather information, and a lack of attention by the news media and emergency managers.

I have attached statements from two windstorms that struck western Washington. The statement for each storm is the one that told of the issuance of a high wind warning. Although it is difficult to compare forecasts for an entire event (since literally dozens of products are issued) these seem as good as any. What is rather dramatic about these two storms is that even though both forecasts verified well and had about the same lead time, user perception and forecast impact were dramatically different. For case 1, the windstorm was perceived as a "surprise." Even an editorial in the Seattle Times chastised the forecasters for not getting the warning out. (The Times wrote a follow-on editorial after the time line of watches and warnings was brought to their attention acknowledging the forecasts had been good). For case 2, public reaction was dramatic. Schools were dismissed early, businesses closed early, and grocery stores were emptied. Why was one of these forecasts a good forecast and one a bad forecast?

Case 1 happened to be inauguration day and there was a lot of news. The forecast office had also issued several false-alarm forecasts during the previous month. It appears that these facts conspired against the forecast and the word never got out. For Case 2 it is a little difficult to pinpoint what made the people react. There was something that evidently was very compelling to the public and to the decision makers. One possibility is that in several statements issued prior to the storm references were made to the storm -- an historical event for the Pacific Northwest. Perhaps this personalized the storm for those who had been in Seattle for that storm and they chose to react.

Discussion

This gets to a final point. If we assume a forecast is only good if it gets the general public to react according to the impending risk, then we need to protect against those things that might interfere with this process. There are probably many and these need to be explored. However, one that was discussed in this survey is an increasing trend for the issuance of warnings by more than one group. Contradictory statements and forecasts often result in a decrease in user confidence and thus little action. It is critical that a single voice have the sole responsibility to communicate watches and warnings to the general public. At this point it is the National Weather Service and this role needs to be protected.

Case 1

Case 2

Societal Aspects of Weather

Workshop's Main Page

Table of Contents