Close Window

SUNY/Stony Brook: "Effective implementation of a mesoscale model into the operational NWS forecast process"

Final Report

1. Project Objectives and Accomplishments

The objective of this project was to run the Penn State-National Center for Atmospheric Research mesoscale model (MM5) in real-time in order for our partners at the National Weather Service (NWS) Upton, NY to learn how to use mesoscale model output in operations and to learn the strengths and weaknesses of the modeling system. The MM5 biases over the Northeast U.S. were determined objectively by verifying the 48-h forecasts using conventional observations, and then the results were compared with similar verification of the National Centers for Environmental Prediction (NCEP) Eta model. Our partners at the National Weather Service (NWS) Upton, NY also provided input on model performance in area forecast discussions, written evaluations, and local seminars.

a. Maintain MM5 system and data ingest into the NWS forecast office
The Stony Brook MM5 system was fully operational during the project period, during which simulations were completed for both the 0000 and 1200 UTC cycles at 36-,12-, and 4-km horizontal grid spacing. The simulations are displayed in real-time on a web page (, and the MM5 data is routinely ingested into the Advanced Weather Interactive Processing System (AWIPS) at the Upton, NY forecast office and the Eastern Region Headquarters of the National Weather Service. During the project there were short periods when the delivery of the data (270 Mb for each cycle) was not timely either because a poor network connection between SUNY-Stony Brook and the Upton, NY forecast office or the Eta grids to initialize the MM5 were not available or complete at NCEP.

Stony Brook collaborated with the NWS to implement additional MM5 forecast products that are more relevant to operations. For example, the AWIPS-MM5 conversion program was altered to send both the convective and explicit precipitation from the MM5. This allowed forecasters to determine how much model precipitation originated from the convective paramterization. Additional web-based MM5 graphics were constructed such as CAPE, CIN, freezing level, probability of frozen precipitation, and surface relative humidity.

During the Fall of 2001 the Stony Brook modeling system was upgraded from the version 2.12 of the MM5 to version 3.4. MM5V3.4 includes Eta soil moisture initialization that varies during the simulation rather than the climatological soil moisture used in V2.12. It was hoped that the latest version of the MM5 would improve some of the model biases as outlined below. Upgrading the MM5 required rewriting the real-time scripts to include different pre-processing MM5 programs.

b. Objective verification of the MM5 and Eta
This project has completed the most comprehensive verification of the MM5 to date over the Northeastern U.S, and the results have been compared with the NCEP Eta model over the Northeastern U.S. The MM5 was objectively verified for the winter seasons of 1999-2002 and the summers of 2000 and 2001. Programs were written to bilinearly interpolate the MM5 and Eta forecast grids to the observation sites, calculate bias and RMS errors at these sites, and construct plots of the statistics. This work has identified several biases in both the MM5 and Eta.

During the winter both models tend to be too cool (0.5 to 1.5 oC) and moist (5-20%) at low- levels over land, with the MM5 moist bias 30-50% larger than the Eta. The MM5 moist bias during the 2001-2002 cool season did not improve even though the soil moisture was allowed to vary. In contrast, over the warmer Gulf Stream during the winter the MM5 and Eta surface temperatures are 1-3 oC too warm. Since both models have winds speeds that are 0.5-1.0 m s-1 weaker than observed over the water in general, this suggests that the parameterized boundary layer heat fluxes are too large. Both models have a high (0.5 to 1.0 m s-1) surface windspeed bias over most land areas; however, the MM5 at 4-km resolution has a weak wind speed bias over the urban areas of New York City and Philadelphia. In the upper troposphere there is a weak (1.0 m s-1) and moist (10-20%) in both models. During the summers of 2000 and 2001 both models had a warm and dry bias at low-levels, with the MM5 biases slightly larger. This is likely the result of using climatological soil moisture values rather than the time varying soil moisture used in the Eta.

It was found that many of these biases slowly oscillate during the season over periods of 2-3 weeks. This suggests that there may be large scale flow patterns which favor certain biases. This will be investigated further by compositing the large-scale flow associated with certain model biases.

Special attention was also given to the MM5 precipitation forecasts. During the cool season the MM5 at 12-km resolution tends to produce too much precipitation over the windward slopes of the Appalachians and too little in the lee. In contrast, during the summer the MM5 at 12- km resolution has a dry bias over much of the Northeast except for the low- lands just inland of the coast, where there is over-prediction. Sensitivity studies as well as NWS forecaster input suggests that this coastal over-prediction is the result of an over-active Kain-Fritsch convective parameterization during conditions of weak CAPE and weak low-level convergence along a sea- breeze front or lee trough. Using the Kain-Fritsch on the 12-km domain tends to reduce the explicit precipitation in the 4-km, especially over the western half of the domain during the first 12 hours of the run. Meanwhile, the other convective parameterizations, such as Grell and Betts-Miller, tend not to have these problems. This was quantified further by completing both a Kain Fritsch and Grell MM5 simulation for each forecast cycle during the summer of 2001.

The high resolution MM5 forecasts were also verified for the 2000-2001 warm season sea breeze events. This involved objectively selecting 40 sea breeze events and verifying the timing and intensity of the sea breeze transition. The inland penetration of the sea breeze across Long Island tended to be 1.5 h early on average. The MM5 surface warmed too rapidly in the early morning, and there was a late afternoon cool bias, both of which favor an early sea breeze in the model.

c. Workshop and forecaster involvement
On 15 June, 2001 a workshop was held at the Upton, NY NWS office in order for the PIs to present the MM5 and Eta verification results, provide an update on recent model developments, and discuss strategies on how to interpret and use mesoscale model data. The forecasters expressed their concerns using mesoscale model output, such as its performance during high impact events. The MM5 verification was also presented at the southern New England weather workshop, which involved many forecasters from around the Northeast region.

The WFO forecasters routinely discuss performance of the MM5 during informal forecast discussions and shift changes. Area Forecast Discussions (AFD’s) are used to document this discussion. The AFD’s were used by the PI as well as frequent direct feedback from the SOO as input for focus of the research.

d. Case study analysis
A sea breeze event on 6 June 2001 around Long Island was investigated to diagnose the three-dimensional flow and temperature evolution using the MM5 and to compare the results with the Eta model. The workstation Eta model code was run at 4-km grid spacing around much of the Northeastern U.S. During 2001 the released version of the workstation Eta only had 5-minute landuse, which is too coarse for Long Island. Stony Brook helped NCEP test a 30-s landuse for this workstation Eta sea breeze simulation. For this particular event the Eta had better timing on the sea breeze frontal progression inland, but it was shallower than the MM5 and the observations. Overall, even though both the MM5 and Eta started with the same initial conditions and sea surface temperatures, significant model differences developed, thus suggesting that even relatively deterministic mesoscale phenomena such as the sea breeze suffer from uncertainties in model physics and would therefore benefit with the use of ensembles. Other historical cases have been studied using the workstation Eta for undergraduate research projects such as tropical storm Floyd on 16 September 1999. It was found that the Betts-Miller convective parameterization in the Eta had a much more difficulty generating the heavy rainfall over northern New Jersey and southeast New York.

2. Summary of University and NWS exchanges

a. University
The MM5 data is now being ingested not only by our NWS Partner, but also Eastern Region Headquarters, the WFO’s in Taunton, MA and Mount Holly, NJ. Other surrounding WFO’s are accessing the model runs via the SUNY web page. The MM5 versus Eta verification work has resulted in presentations at other meetings such the MM5 workshop in Boulder, CO (25-27 June 2001), and the Mesoscale/NWP conference in Fort Lauderdale, FL (30 July -2 August 2001). At both meetings there were numerous representatives from the NWS, AFWA and Navy. Overall, the verification work has helped motivate field programs such as the IMPROVE project over the Oregon Cascades in December 2001, which is designed to improve model microphysical parameterizations.

b. NWS
The integration of MM5 into WFO operations resulted in improved forecaster understanding of Numerical Weather Prediction - particularly scale considerations and parameterization schemes. A survey of the forecast staff at the completion of the project showed that the MM5 model was utilized in operational forecasting at least once a week by all but one surveyed. Interestingly, the younger and culturally more open to change intern staff all replied that they used the MM5 every shift. The main reason for lack of utilization is discussed in sections 4c and d.

When the forecaster impressions of the NCEP operational models was compared to that of the MM5 using a written survey, forecasters unanimously chose either the eta or AVN. The reason for this is discussed in sections 4c and d. In terms of specific areas:

The areas that MM5 was chosen as having “strong” capabilities were:

The areas that MM5 was chosen as having “weak” capabilities were:

The team was also able to evaluate MM5 utilization through routine examination of Area Forecast Discussions, which showed over the period of the project a gradual increase in knowledge of NWP.

3. Presentations and Publications

  1. Colle, B.A., J. B. Olson, and J. S. Tongue, 2002: Long-term verification of the MM5: Part I, comparison with the Eta over the central and eastern U.S. and impact of MM5 resolution. Submitted to Wea. Forecasting.

  2. Colle, B.A., J. B. Olson, and J. S. Tongue, 2002: Long-term verification of the MM5: Part II, Evaluation of high resolution precipitation forecasts over the Northeastern U.S. Submitted to Wea. Forecasting.

  3. Colle, B.A., J. S. Tongue, and J. B. Olson, 2002: Verification of the real-time MM5 over the Eastern U.S., The Twelfth PSU/NCAR Mesoscale Modeling System User’s Workshop. Boulder, CO.

  4. Colle, B.A., J. B. Olson, and J. S. Tongue, 2002: Objective verification of the MM5 over the Eastern U.S: comparison with the NCEP Eta and impact of high resolution. To be presented at the NWP/Weather Analysis conference at San Antonio, TX.

  5. Colle, B.A., J. B. Olson, and J. S. Tongue, 2001. Objective verification of the MM5 over the Eastern U.S. Southern New England Weather Conference, November 2001.

  6. Colle, B.A., J. S. Tongue, and J. B. Olson, 2001: Verification of the Eta and MM5 over the Eastern U.S., The Eleventh PSU/NCAR Mesoscale Modeling System User’s Workshop. Boulder, CO. 124-127.

  7. Colle, B.A., J. S. Tongue, and J. B. Olson, 2001: Verification of the high resolution MM5 and Eta over the Northeast U.S., Ninth Conference on Mesoscale Processes, Fort Lauderdale, FL, American Meteorological Society.

  8. Tongue, J. S., B.A. Colle, and N. Ray, 2001: Evaluation of high resolution MM5 forecasts for complex mesoscale sea breeze circulations. Annual Conference of the National Weather Association, Spokane, WA. October 2001.

4. Summary of benefits and problems encountered

a. University benefits
This project has sparked unprecedented collaboration between Stony Brook and NWS Upton, NY and Eastern Region, which has allowed for more interaction through internships, meetings, and seminars. Forecaster input on the Stony Brook MM5 has resulted in improvements to the modeling system, such as additional web-based graphics and the use of different parameterizations to improve known model biases. This project also exposed several students to operational numerical weather prediction within class, research, and internships. For example, the verification results were shared in the advanced weather forecasting class (ATM 347). Four undergraduates helped with this project in terms of MM5 verification and case studies related to the sea breeze and thunderstorm evolution around Long Island.

b. NWS benefits
The forecast staff is using MM5 data operationally to assist in the improvement of forecast operations. The high resolution (4 km) have been valuable to visualize mesoscale processes – in particular sea breeze circulations. There have been many examples wind shift times in Terminal Forecast have been improved through use of the MM5 runs. The project has also allowed for the forecast staff to increase their understanding and application of NWP. This has been seen most dramatically with the ability to comprehend convective parameterization processes.

c. Problems encountered
Reliability of data delivery into AWIPS was a major problem for this project. The team worked diligently to ensure the reliability of data delivery to the WFO early in the project. This was successful with a subjective estimate of reliability approaching 95%. Unfortunately, security requirements at Eastern Region Headquarters (ERH) resulted in a significant loss of bandwidth on 11 March 2002, such that no MM5 data has been ingested into the WFO’s AWIPS since. Forecasters commented that they “haven't used the MM5 very much recently due to time constraints and availability.” While the MM5 data is available to the forecasters via the Internet – the loss of data within AWIPS was viewed as a “significant impact” to their integration of the data into the forecast and atmospheric learning/comprehension process.

d. Future considerations
The timely and consistent ingestion of MM5 data is required for a successful continuation of this project. ERH and the WFO are currently working in various different avenues to address this setback.

It is important to note that introduction of new data sets to NWS forecast operations must be done in a “very delicate manner” as forecasters feel they “don't have time to look at 9 different versions of the MM5” and will quickly judge and discount the new data as “not valuable” if:

  1. It does not prove reliable.

  2. Forecasters are not shown “how” to integrate the data.

  3. Leadership is not shown by the Management Team.

Finally, the forecast staff felt that the introduction of e-bufr MM5 data for conversion and use with the BUFKIT program would result in their use of the entire data set “a lot more then presently.” They also felt that MM5 data in BUFKIT would be “very important” to understanding the MM5 in more detail. As a result, introduction of MM5 profiles within BUFKIT is a goal of our follow-on project.