Close Window

SUNY/Albany: "The New York Integrated Weather Data Network (NYIWDN)"

Final Report

1.1 The objectives of our project are:

We have been able to make useful progress on some of our objectives. Though there has been steady encouragement and good will from the team members at NYDOT, they have lost a large number of personnel over the course of this project. As a result, the major focus of the project shifted to an effort to download data from a few sample NYDOT RWIS sites and to make an effort to audit data quality from the RWIS sites by comparison with nearby NWS sites (item d) above). This has been done largely manually (see details of data now in the ASRC archive and the method used to obtain it in Appendix A). We have learned that the program to establish the network of low-cost stations has delayed indefinitely, making task e) not achievable during the period of this grant. (We have lab-tested a DOT automatic weather station from Rainwise, Inc. We await word about a site to make a long-planned field intercomparison of different stations.. On another project SUNY is currently field testing five Onset Inc. HOBO weather stations.) With the publication of the SUNY ‘SLABS’ model paper (Wojcik et al., 2003), we are now in a position to apply this work to road surface state modeling (item g). However, for this and the remaining tasks not detailed, we cannot reasonably proceed until an archive of data from many stations is available. We had not anticipated that our major effort would be the initial data acquisition step, and thus have not been able to develop a database from a large number of stations. An example of the data quality problems is given in the accompanying time series (Figs. 2-9). Fig 3 illustrates an apparent change in calibration and change in resolution at one station in the wind sensor midway through 2002. We have had very little help from the contractor, SSI on data issues. Preliminary work from the RWIS Contractor (forwarded to us by NYDOT) is that there is no reason to question the wind sensor. We await further word on this issue. We suspect that the wind speeds reported in the early part of the year are much too large to be believable. We are certain that these data acquisition glitches are not fundamental, and that they will resolved eventually. However, this work will likely not be completed at the end of this COMET project (October 2003).

The purpose of these meetings was to discuss implementation of the New York State Mesonet and the means by which the data collected from the mesonet stations would be shared amongst the users. We held several meetings in which NYDOT, SUNY and NWS personnel were present. J. Quinlan and D. Zaff of NWS organized a teleconference, during which we learned about how the MesoWest network was put together. It was clear to some at the meeting that the most fruitful effort for our team would be to investigate the role of local topography and sheltering on the RWIS measurements.

We have made progress in determining how individual RWIS station reports differ from those regularly reported by the National Weather Service at Albany Airport. This comparison is important, since forecasters currently rely on such “Class 1” stations for information.

We have demonstrated a technique to evaluate how representative of RWIS sites is the standard meteorological station at Albany airport. In the remaining time for this project, we will quantify the relationship between the ‘exposure factor’ of individual sites, and the panoramic photos of the sites we have studied intensively. We look forward to a day when we would be able to deal with the data from many stations.

A paper published by G. Wojcik et al. details the model we plan to use to predict road surface state, once data become available.

The main problem continues to be delays in the installation of the new RWIS sites and the older RWIS sites which remain under the proprietary Surface Systems Incorporated (SSI) SCAN Software. After two years of effort, SUNY researchers still acquire data through a tedious manual process, and data quality is still not up to standard. NYDOT personnel are aware of the problems and are working with them as they can, in an environment of drastically reduced resources.

Appendix A.

The following is a description of the procedure to download and prepare data from DOT, work done by SUNY undergraduate meteorology students Kim Sutkevich and Jason Herb.

1. Reach the DOT computer via a regular telephone modem, then use a browser to reach the specified NYDOT URL address:

2. Enter your user name, password.
Once logged in, on click on "Region 1 - Albany" on the left side of the screen to see a list of DOT weather stations
in the Albany area:
Rte 5 @ I-890
Rte 5 @ VanBuren
I-88 @ Sheldon Road
I-88 @ West of Sheldon
I-88 @ East of Sheldon
Rte 29 @ Rte 147

3. For each site listed, there is a link to "History" of data obtained at that site. Click on "History." A chart appears on the screen of the past 24 hours. Right below the chart, there is a "Chart Scale" and "Chart Selections" one can choose to customize the information that you would like to obtain.

4. Click on "Update." Once it has been updated, you will need to click on "History Table" just above the chart. You are then taken to a table of the past 24 hours of data.

5. Link to "Export." You then can change the end date by typing in the date of the data that you would like in the "Change End Date:" box.

6. Once you obtain the data you wish, you must copy, paste and save this data into an editing program; we used. Microsoft Word. Having saved the file in a word processing format, you must realign the data due to the fact that the data does not stay in one line for each time interval. After a time, this step was streamlined by using a program. It takes at least 15-20 minutes to fix 24 hours worth of data that is taken every ten minutes, with a somewhat longer
time for 5-minute data.

DOT Station Download Results as of 8/19/03

Data from NWS data resources: Automatic station (ASOS) observations.

Figure 1. NYDOT RWIS stations for which manually downloaded data
have been analyzed up to now. Other stations indicated are part of the
ongoing HVAMS study of the Hudson valley being conducted by
the SUNY group.

Figure 2. Panoramic views of three of the weather stations studied.

Figure 3. An illustration of the mysterious calibration difficulties found when downloading SSI
RWIS data. Also, note the coarse resolution of wind direction. Through NYDOT personnel, SSI
indicated that this resolution fitted the ‘letter of the contract’ and refused to help us access the
finer resolution data. Hourly reports from NYDOT RWIS site at Interstate 88/Sheldon for year
2002. Top panel: Temperature; middle panel: Wind speed ; bottom panel: Wind direction,
(reported only by octant).

Figure 4. Intercomparison of weather station data with that from the
Albany airport ASOS station.

Figure 5. Details of the intercomparison regression statistics.

Figure 6. Wind direction comparison of the RWIS stations with ALB ASOS.

Figure 7. Histograms of wind by sector. Note the unreasonably persistent northerly values at the
rt29 site.

Figure 8. Comparison of wind speed in sectors with that reported at ALB ASOS.

Figure 9. Summary slide from the oral presentation.