Close Window

Univ. of Washington (Brett Newkirk, COMET graduate student fellow): "Evaluation of model initialization over the eastern Pacific Ocean"

Final Report

Forecasts over the ocean and nearby coastal regions are often seriously in error. The cause of many of these problematic forecasts is clear: poor initialization over data sparse oceans. While large data voids exist over the Pacific Ocean, the current generation of data assimilation systems does not use the data available to its full advantage. Originally, this work intended to address this issue in several ways: (1) the direct evaluation of the initializations used in numerical weather prediction (NWP) efforts from various agencies, (2) the determinations of the effects of initialization quality on forecast skill, and (3) the use of a real-time coastal modeling system already in place to test new approaches to data collection and assimilation over the ocean.

The first two goals did not change from the project start date. The first goal was carried out by performing an appraisal of initialization quality for a collection of operational modeling systems by a comprehensive comparison with all available data assets over the eastern Pacific Ocean east of the dateline. These data included scatterometer winds, geostationary satellite winds, rawinsonde observations, aircraft observations, and surface, ship and buoy measurements. The models involved were the National Centers for Environmental Prediction (NCEP) Aviation (AVN), ETA step coordinate model (ETA), Medium Range Forecast Model (MRF) and Nested Grid Model (NGM) models, the Canadian Meteorological Center's CMC/GEM model and the Fleet Numerical Meteorological and Oceanographic Center's NOGAPS model. The last goal however, did change. Rather than use the existing modeling system to test new approaches to data collection, a new modeling technique was used to address the issues mentioned in (1) and (2). The adjoint version of the real-time coastal modeling system mentioned in (3) was used along with its forward counterpart to determine how one can diagnose a priori the error in a model initialization and the spatial and temporal details of the propagation of error.

Nearly every tool used to analyze results for the MS Thesis was put up and maintained on a web page and maintained for all potential users (forecasters) to see. The URL of the project is From its inception, input from any end-user was taken into account when a new revision of the web page was made. Two presentations were made to the Seattle National Weather Service Forecast Office (SNWSFO) in order to introduce the web page to the forecasters and to provide a formal time for ideas to be exchanged. After these presentations, weather discussion presentations were given to better show how the web page could be incorporated into the forecasters daily routine. Real-time examples were shown to the forecaster such that a regular analysis and interpretation of the data available could be instantiated.

The data from the web page was analyzed from several different viewpoints. First, a general look at the performance of each model with respect to an error score based on observations from the data assets above was conducted. Several different spatial domains were defined over the Pacific Ocean in order to address the needs of the forecasters at the Seattle National Weather Service Forecast Office. Comparing error scores for each spatial domain, it was seen that over a 13 month average, different models had the best initialization in various levels of the atmosphere. It was generally seen that the NCEP AVN model had the best upper level initialization while the Canadian Meteorological Center CMC/GEM and NOGAPS model have the best low level initialization. A similar methodology was conducted for forecast times out to 48 hours over the Pacific and rest of North America and the results varied and can be found in the MS Thesis paper.

Aside from a direct comparison of raw error scores, attempts were made at correlating errors in initialization over the Pacific to errors that occur downstream at subsequent forecast times. While not all results were statistically significant, relationships were shown to exist between initial condition errors over the Pacific and forecast errors going east to the Rocky Mountains. Several issues relating to the experimental framework design (choice of verification domains) and what exactly was being shown were encountered. For example, it was realized that the ability to measure a model error in the Pacific might not relate to an ability to measure a model error that occurs downstream (i.e. at the West Coast of North America). Data density is the dominating factor in determining the quality of the end results. In addition, it was learned that the error quantity being measured might be located in a region whose growth rate is negative, that is, the error measured is decaying with time.

Beyond the error analyses, a new modeling technique was employed to better understand the exact relationship between initial condition and forecast errors. The student attended the NCAR MM5 Adjoint Modeling class held in July 1998. From this experience he applied this knowledge and set up an adjoint modeling system over the SNWSFO forecasting region that operated in real-time, to the direct benefit of the forecasters. By close collaboration with scientists working with adjoint models, the optimal conditions in which to conduct this experiment were learned and implemented (i.e. domain size and forecast time length). The adjoint model is directly related to its corresponding forward model via adjoint theory. The adjoint model's utility with respect to forecasting is given by the fact that the adjoint model output can be used to determine where changes in the initial conditions most impact a n-hour forecast (n=24 for this study) over a pre-defined region (chosen to be a ~500km x 500km box encapsulating the Pacific Northwest). In the MS Thesis, attempts were made to correlate actual model initial condition errors and subsequent forecast errors. Several attempts were statistically significant while some were not. Again, it was determined that poor data density (mainly in Mean Sea Level Pressure observations) was a reason for statistically insignificant results. The value of conducting this experiment in a data rich region (possibly even with a synthetic data experiment) was realized and proposed for future work.

The adjoint model data was also included on the web page mentioned above. After the adjoint implementation on the web page a presentation was given at the Pacific Northwest Weather Workshop, hosted by the SNWSFO. In this forum, forecasters from throughout the region were able to network and share their ideas on the project. After this presentation, a successful MS presentation was given at the University of Washington. A Master of Science degree was earned on June 9, 2000.