Integrated Modeling for Road Condition Prediction — Phase 3 Evaluation Report

Chapter 3. Results

This chapter presents the analyses for the Integrated Modeling for Road Condition Prediction (IMRCP) speeds and speed forecast predictions and user satisfaction interviews.

Operational Impacts Study

The IMRCP analyses examined IMRCP-predicted speeds versus historical Kansas City Scout (KC Scout) detector speeds, IMRCP speed forecasts, and IMRCP speed forecast errors. Real-time speeds and speed predictions were examined for Traffic Estimation and Prediction System (TrEPS) and machine learning-based prediction (MLP) for seven 2019–2020 weather event days.

Appendix A displays tables that contain the absolute, relative absolute, and root mean square errors by date and location for the speed forecasts.

Table 3 describes the weather event dates and weather conditions. The data were collected through the IMRCP tool's reports page. Historical data were collected from the IMRCP tool for each of the seven days from midnight to midnight. The historical data consist of observed real-time values from the detectors and real-time predictions from the model sources.

Table 3. Description of weather events.

Date	Weather and Road Condition
11/11/2019	Light snow/snow and wind gusts
11/29/2019	Wet, frosty roads; upstream incident (at State Line Road)
12/15/2019	Ice, snow, and light wind; downstream incident (at Stadium Drive)
12/16/2019	Afternoon light snow and some haze; evening snow
12/17/2019	Morning fog; fair in the afternoon; temperatures in mid- to upper- 20s; light winds
1/29/2020	Freezing rain/light snow/wintry mixture
2/12/2020	Snow with wet roads

The following observation types were selected for exploration in this evaluation: SPDLNK (average speed of vehicles on each link), VIS (surface visibility), TPVT (pavement temperature), and PCCAT (precipitation category). Sources for these observations vary. SPDLNK has three sources: KC Scout, TrEPS, and MLP. KC Scout provides real-time data from the detectors, while TrEPS and MLP provide both real-time and forecast data predicted using different model algorithms. The two sources for VIS are Real-Time Mesoscale Analysis (RTMA) and Rapid Refresh (RAP). According to the IMRCP system user guide, RTMA supplies VIS observations collected in 3 x 3-kilometer (km) grids, whereas RAP provides VIS forecast values collected in 13 x 13-km grids. TPVT estimations are supplied by the Model of the Environment and Temperature of Roads (METRo), and PCCAT observations are provided by RAP. The different categories for PCCAT are listed in Appendix D of the IMRCP system user guide.

Interstate 435 Eastbound at State Line Road

November 11, 2019, Weather and Road Conditions

The following plots display the historical data for real-time speeds obtained for Interstate 435 (I–435) and State Line Road during November 11, 2019. The temperature was in the low 30s and dropped throughout the day to about 12 degrees Fahrenheit (°F) by midnight, with periods of icing rain, light snow, and wind gusts up to 45 miles per hour (mph) from 4 a.m. to 1 p.m. The road conditions were icy with melting snow until about 2 p.m. and were dry afterwards. The road also had an upstream incident that occurred from 6:46 to 7:11 a.m., and the movement of eastbound (EB) traffic was affected by a construction work zone on I–435 at State Line Road. The construction in the area resulted in a lane shift and a reduced speed limit, which also affected traffic speeds.

November 11, 2019, Speed Predictions

The plots in figure 6 depict, from top to bottom, VIS, SPDLNK, TPVT, and PCCAT on November 11, 2019. The x-axes in each plot represent the time of day from early morning to midnight. The four plots present a snapshot of the speeds and weather conditions for the day.

• Visibility was reduced between 7 a.m. and 1 p.m., according to the RAP and RTMA sources.
• The speed plot shows the reported speeds from KC Scout roadway detectors, TrEPS, and MLP and the fluctuation and deviations during the day.
• TrEPS speeds were unavailable until early afternoon and then reported fluctuating speeds from 5 to 8 p.m. Sometimes, the speeds were 10–15 mph above/below the speed detector data.
• MLP speeds experienced similar speed fluctuations during two periods: 3:30–6 a.m. and 8:45–10 p.m. Otherwise, the MLP speed predictions appeared to be within about 5 mph of the detector data

November 11, 2019, Speed Forecasts

The plots in figure 7 display the speed forecasts obtained for I–435 and State Line Road during November 11, 2019. The plots depict, from top to bottom, the real-time prediction (historical speeds) followed by 15-, 30-, 60-, 90-, and 115-minute (min) forecasts. The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations above and below the detector speeds. The deviations appeared to grow larger with longer forecasts.

November 29, 2019, Weather and Road Conditions

On November 29, 2019, the weather temperature was in the mid 30s, with dew and frost warnings and wind gusts up to 15 mph throughout the day and night. The road conditions were wet and frosty in the morning and wet during the evening. The road also had an upstream incident that occurred from 9:52 a.m. to 12:02 p.m., and the movement of EB traffic was affected by a construction work zone on I–435 at State Line Road. The construction in the area resulted in a lane shift and a reduced speed limit, which also affected traffic speeds.

November 29, 2019, Speed Predictions

The plots in figure 8 display the historical data obtained for I–435 EB and State Line Road during November 29, 2019. On this particular day, only KC Scout detectors and MLP were reporting speeds.

The plots show:

• There was a dip in visibility in the morning and evening, according to RAP and RTMA sources.
• The data for the TrEPS model were not available on this date.
• In general, the speed predictions by MLP are very similar to the detector speeds, even on an atypical Friday with a weather event.

November 29, 2019, Speed Forecasts

The plots in figure 9 display the forecast speeds obtained for I–435 and State Line Road during November 29, 2019.

The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• MLP speed forecasts showed periods where the forecasts appeared to have large deviations above and below the detector speeds. The deviations appeared to grow larger with longer forecasts.
• TrEPS forecasts were not available for this date.

December 15, 2019, Weather and Road Conditions

On December 15, 2019, the weather temperature was in the mid 20s with a light wind (less than 10 mph) in the morning. The road conditions were dry in the early morning but snowy by 9 a.m., slushy at noon, and icy and snowy during the evening commute. The movement of EB traffic was affected by a construction work zone on I–435 at State Line Road. The construction in the area resulted in a lane shift and a reduced speed limit, which also affected traffic speeds.

December 15, 2019, Speed Predictions

The plots in figure 10 display the historical data obtained for I–435 and State Line Road during December 15, 2019. The plots show:

• Visibility was reduced between 7 a.m. and 7 p.m.
• TrEPS speeds were higher than reported by the detectors for most of the day and were unavailable after 1 p.m. on this date.
• Speed predictions by MLP were fairly accurate during the slow down between 9 a.m. and noon.
• There was an increase in MLP predicted speeds at 6 p.m. The predictions between 6 and 9 p.m. were higher than the recorded detector speed data during this time.

December 15, 2019, Speed Forecasts

The plots in figure 11 display the forecast speeds obtained for I–435 and State Line Road during December 15, 2019. The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were consistently higher than the detector speeds.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations above and below the detector speeds. The deviations appeared to grow larger with longer forecasts.

December 16, 2019, Weather and Road Conditions

On December 16, 2019, the weather temperature was in the low 30s with no precipitation and a light wind (less than 10 mph) in the morning. There was light snow and some haze during the afternoon, and it was snowy during the evening. The road conditions were dry until noon, but had melting snow and some haze during the afternoon and ice and snow during the evening commute. The movement of EB traffic was affected by a construction work zone on I–435 at State Line Road. The construction in the area resulted in a lane shift and a reduced speed limit which, also affected traffic speeds.

December 16, 2019, Speed Predictions

The plots in figure 12 display the historical data obtained for I–435 and State Line Road during December 16, 2019.

The plots show:

• There was a reduction in visibility starting in the morning, which remained low until about 6 p.m. according to RAP and RTMA sources.
• TrEPS speeds were higher than the detector throughout the day.
• MLP speeds tracked the detector speeds fairly closely for most of the day.

December 16, 2019, Speed Forecasts

The plots in figure 13 display the forecast speeds obtained for I–435 and State Line Road during December 16, 2019.

The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were consistently higher than the detector speeds.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

December 17, 2019, Weather and Road Conditions

On December 17, 2019, the weather temperature was in the mid- to upper 20s with no precipitation and a light wind (less than 10 mph) throughout the day and night. There was no precipitation during the day. The road conditions were dry during the morning, midday, and evening times. The movement of EB traffic was affected by a construction work zone on I–435 at State Line Road. The construction in the area resulted in a lane shift and a reduced speed limit, which also affected traffic speeds.

December 17, 2019, Speed Predictions

The plots in figure 14 display the historical data obtained for I–435 and State Line Road during December 17, 2019. For this day:

• There was a reduction in visibility from early morning to midafternoon, according to RAP and RTMA sources.
• TrEPS model speeds were not very accurate on this day and were consistently higher than the KC Scout detector speeds.
• MLP speeds tracked the detector speeds fairly closely for most of the day. However, between 5 and 7 a.m., there were periods when speeds were considerably lower than the detector speeds. The MLP speeds evening rush hour effect was captured well by the speed predictions from the MLP model.

December 17, 2019, Speed Forecasts

The plots in figure 15 display the forecast speeds obtained for I–435 and State Line Road during December 17, 2019. The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were consistently higher than the detector speeds and do not reflect the decline in detected speeds from about 5 to 7 p.m.
• MLP speeds decrease similar to the the decline in detected speeds from about 5 to 7 p.m.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

Speed Forecast Error Analysis for Interstate 435 and State Line Road

Figure 16 shows the MLP absolute error for the 15–115-min speed forecasts on November 11, 29, December 15, 16, and 17. Figure 17 shows the TrEPS absolute errors for the same forecast time periods. The plots show:

• For MLP, the absolute errors show that shorter-term forecasts tended to have smaller errors than longer-term forecasts. Consequently, the shorter-term forecasts tended to be more accurate. The average absolute error for the 15-min forecast was about 6.2 mph, whereas the 115-min forecast increased to about 14.8 mph.
• For TrEPS, the absolute errors were relatively consistent across all forecast times. The average absolute error was about 26 mph for all forecast times.
• Comparing the MLP to TrEPS absolute errors, the TrEPS errors tended to be larger than the MLP absolute errors. The average TrEPS error across all forecast times was about 26 mph versus about 10.6 mph for MLP.

Figure 18 and figure 19 show the relative absolute errors for MLP and TrEPS at I–435 and State Line Road on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The relative absolute error measured the magnitude (in percent) of the absolute error as compared to the actual speed. The plots show:

• For MLP, the relative absolute error for shorter-term forecasts tended to be smaller than the errors for longer-term forecasts. In other words, the error as a percentage of the speed was smaller for shorter-term forecasts. The average relative absolute error for the 15-min forecast was about 18 percent, whereas the 115-min forecast increased to about 41 percent.
• For TrEPS, the relative absolute errors were relatively large (ranging from 40 to over 100 percent) and mostly consistent across all forecast times. The average relative absolute error was about 73 percent for all forecast times.
• Comparing the MLP to TrEPS relative absolute errors, the TrEPS errors tended to be larger than the MLP errors. The average TrEPS error across all forecast times was about 73 percent versus about 29 percent for MLP.

Figure 20 and figure 21 show the root mean square errors for MLP and TrEPS at I–435 and State Line Road on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The root mean square errors measure the deviation or spread of the forecast versus actual speeds. The plots show:

• For MLP, the root mean square error showed that the shorter-term forecasts errors tended to be smaller than longer-term forecasts. The average root mean square error for the 15-min forecast was about 6 mph versus about 12 mph for the 115-min forecast.
• For TrEPS, the root mean square errors were relatively consistent across all forecast times. The average root mean square error was about 23 mph for all forecast times.
• Comparing the MLP to TrEPS root mean square errors, the TrEPS errors were about twice as large as MLP. The average TrEPS error across all forecast times was about 23 mph versus about 10 mph for MLP.

Interstate 435 Northbound at East Stadium Drive

November 11, 2019, Weather and Road Conditions

On November 11, 2019, the weather temperature was in the low 30s and dropped throughout the day to about 12 °F by midnight, with periods of icing rain, light snow, and wind gusts up to 45 mph from 4 a.m. to 1 p.m. The road conditions had freezing rain during the morning and ice and snow at noon, and were dry in the afternoon and evening.

November 11, 2019, Speed Predictions

The plots in figure 22 display the historical data obtained for I–435 and East Stadium Drive for the seven days. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network. The plots show:

• There was a dip in visibility in the morning, according to both sources, and an increase in precipitation.
• MLP speed predictions were similar to the detector data throughout the entire day at this location. There was an incident between 7:44 a.m. and 2:09 p.m. on I–435 southbound upstream from the detector.

November 11, 2019, Speed Forecasts

The plots in figure 23 display the forecast speeds obtained for I–435 and East Stadium Drive during November 11, 2019. In the plots, speed forecasts for only MLP are available because this location is outside the TrEPS network. The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• MLP speed forecasts showed periods where the forecasts appeared to have small deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

November 29, 2019, Weather and Road Conditions

On November 29, 2019, the weather temperature was in the mid 30s, with dew and frost warnings and wind gusts up to 15 mph throughout the day and night. The road conditions were wet and frosty in the morning and wet during the afternoon and evening.

November 29, 2019, Speed Predictions

The plots in figure 24 display the historical data obtained for I–435 and East Stadium Drive during November 29, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network.

The plots show:

• Both RAP and RTMA sources indicate visibility varied considerably throughout the day.
• The rush hour effect was less on this day, possibly due to being the Friday after Thanksgiving.
• Speed predictions by MLP were very similar to the detector speeds, even on an atypical Friday with a weather event and varied visibility at this location.

November 29, 2019, Speed Forecasts

The plots in figure 25 display the forecast speeds obtained for I–435 and East Stadium Drive during November 29, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network. The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• MLP speed forecasts showed periods where the forecasts appeared to have small deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

December 15, 2019, Weather and Road Conditions

On December 15, 2019, the weather temperature was in the mid-20s with a light wind (less than 10 mph) in the morning. The road conditions were dry in the early morning but snowy by 9 a.m., slushy at noon, and icy and snowy during the evening commute. There was a downstream incident from 10:26 a.m. to 12:51 p.m.

December 15, 2019, Speed Predictions

The plots in figure 26 display the historical data obtained for the first location, I–435 and East Stadium Drive, during December 15, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network.

The plots show:

• Both RAP and RTMA sources indicate that visibility decreased in the morning and remained low until slowly improving in the evening and night.
• Detector speeds recorded a slight decrease in speeds that corresponded to the reduced visibility from the morning and into the night.
• MLP speeds were similar to the detector speeds throughout the entire day at this location.

December 15, 2019, Speed Forecasts

The plots in figure 27 display the forecast speeds obtained for I–435 and East Stadium Drive during December 15, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network. The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• MLP speed forecasts showed periods where the forecasts appeared to have small deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

December 16, 2019, Weather and Road Conditions

On December 16, 2019, the weather temperature was in the low 30s with no precipitation and a light wind (less than 10 mph) in the morning. There was light snow and some haze during the afternoon, and it was snowy during the evening. The road conditions were dry until noon, but had melting snow and some haze during the afternoon and were dry during the evening commute.

December 16, 2019, Speed Predictions

The plots in figure 28 display the historical data obtained for I–435 and East Stadium Drive during December 16, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network.

The plots show:

• Both RAP and RTMA sources indicate that visibility decreased in the morning and remained low until slowly improving at about 6 p.m.
• MLP and detector speeds showed a slight increase from about 8 a.m. to about 2 p.m.
• The MLP speed predictions appeared to well match the detector speeds on this date.

December 16, 2019, Speed Forecasts

The plots in figure 29 display the forecast speeds obtained for I–435 and East Stadium Drive during December 16, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network.

The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were not available at this location.
• MLP speed forecasts showed periods where the forecasts appeared to have small deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

December 17, 2019, Weather and Road Conditions

On December 17, 2019, the weather temperature was in the mid- to upper 20s, with no precipitation and a light wind (less than 10 mph) throughout the day and night. There was no precipitation during the day. The road conditions were dry during the morning, midday, and evening times.

December 17, 2019, Speed Predictions

The plots in figure 30 display the historical data obtained for I–435 and East Stadium Drive during December 17, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network. The plots show:

• RAP and RTMA sources indicate that visibility was low in the morning until slowly improving at about noon.
• MLP and detector speeds showed a slight increase in speeds at about 8 a.m.
• MLP speed predictions appeared to well match the detector speeds on this particular date.

December 17, 2019, Speed Forecasts

The plots in figure 31 display the forecast speeds obtained for I–435 and East Stadium Drive during December 17, 2019. In the plots, speeds for only KC Scout and MLP are available because this location is outside the TrEPS network. The plots show:

• Detector and MLP speeds fluctuated throughout the day for all forecast times.
• MLP speed forecasts showed periods where the forecasts appeared to have small deviations above and below the detector speeds. The MLP speed deviations appeared to grow larger with longer forecasts.

Speed Forecast Error Analysis for Interstate 435 and East Stadium Drive

Figure 32 shows the MLP absolute error for the 15–115-min speed forecasts on November 11, 29, December 15, 16, and 17. The plots show:

• The absolute errors show that shorter-term forecasts tended to have smaller errors than longer-term forecasts.
• The shorter-term forecasts tended to be more accurate.
• The average absolute error for the 15-min forecast was about 3.5 mph versus about 6.5 mph for the 115-min forecast.

Figure 33 shows the relative absolute errors for MLP at I–435 and East Stadium Drive on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The plots show:

• The relative absolute error for the MLP shorter-term forecasts tended to be smaller than the errors for longer-term forecasts. In other words, the error as a percentage of the speed was smaller for shorter-term forecasts.
• The relative absolute error shows the shorter-term forecasts tended to be more accurate.
• The average relative absolute error for the 15-min forecast was about 7 percent versus about 13 percent for the 115-min forecast.

Figure 34 shows the root mean square errors for MLP at I–435 and East Stadium Drive on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The plots show:

• The root mean square error showed that shorter-term forecasts errors tended to be smaller than longer-term forecasts.
• The root mean square error showed that the shorter-term forecasts tended to be more accurate.
• The average root mean square error for the 15-min forecast was about 3 mph versus about 5 mph for the 115-min forecast.

Interstate 435 Eastbound at East Antioch Road

January 29, 2020, Weather and Road Conditions

On January 29, 2020, the weather temperature was in the upper 20s to low 30s with early morning snow. The road conditions had snow until about 9 a.m. and were dry afterwards. There were two incidents—both involving a stalled vehicle—between 5:42 and 6:07 p.m. and then again between 9:37 and 10:02 p.m.

January 29, 2020, Speed Predictions

The plots in figure 35 display the historical data obtained for I–435 and East Antioch Road during January 29, 2020. The speed plot shows the reported speeds from KC Scout roadway detectors, TrEPS, and MLP, and shows the fluctuation and deviations during the day. The plots show:

• RAP and RTMA sources indicate that visibility was low in the morning and until slowly improving at about 8 a.m.
• MLP and detector speeds showed speeds decreasing until about 9 a.m., then increasing until about 11 a.m. and, with an exception at about 6:30 p.m., remained consistent thereafter.
• MLP speed predictions appeared to match the detector speeds well on this particular date, and appeared to show speeds decreasing at about the time of the stalled-vehicle incidents between 5:42 and 6:07 p.m. and 9:37 and 10:02 p.m.
• TrEPS speeds were consistently much lower than KC Scout and MLP for most of the day.

January 29, 2020, Speed Forecasts

The plots in figure 36 display the MLP and TrEPS forecast speeds obtained for I–435 and East Antioch Road during January 29, 2020. The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were consistently lower than the detector speeds and do not reflect the decline in detected speeds from about 5 to 7 p.m.
• TrEPS and MLP speeds decreased similar to the the decline in detected speeds from about 8 to 10 a.m.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations below the detector speeds. Both TrEPS and MLP speed deviations appeared to grow larger with longer forecasts.

February 12, 2020, Weather and Road Conditions

On February 12, 2020, the weather temperature was about 32 degrees throughout the day with periods of light snow, with snow from about 9 a.m. to midnight. The road conditions were dry until about 9 a.m., then became wet until and through midnight.

February 12, 2020, Speed Predictions

The plots in figure 37 display the historical data obtained for the first location, I–435 and East Antioch Road, during February 12, 2020. On this date, there were multiple incidents between 7:30 a.m. and 2 p.m. near to and downstream from this detector.

The plots show:

• RAP and RTMA sources indicate that visibility decreased at about 9 a.m. until slowly improving at about 7 p.m.
• MLP and detector speeds showed speeds decreased from about 9 to 10 a.m., then returned to normal until from about 3 to 6 p.m. when precipitation was indicated by PCCAT.
• MLP speed predictions appeared to pretty well match the detector speeds on this particular date, and appeared to show decreasing speeds between 9 and 10 a.m. and increasing variability from 3 to 6 p.m.
• TrEPS speeds deviated considerately and had frequent periods that were much lower than KC Scout.

February 12, 2020, Speed Forecasts

The plots in figure 38 display the forecast speeds obtained for I–435 and East Antioch Road during February 12, 2020. The plots show:

• Detector, MLP, and TrEPS speeds fluctuated throughout the day for all forecast times.
• TrEPS speeds were consistent until about 7 a.m., then fluxuated lower than the detector speeds until about 8 p.m.
• MLP speeds for the 15-min forecast decreased similar to the the decline in detected speeds from about 9 to 10 a.m., then had deviations that grew larger with longer-term forecasts.
• MLP and TrEPS speed forecasts both showed periods where the forecasts appeared to have large deviations from the detector speeds. Both TrEPS and MLP speed deviatioans appeared to grow larger with longer forecasts.

Speed Forecast Error Analysis for Interstate 435 and East Antioch Road

Figure 39 shows the MLP absolute error for the 15–115-min speed forecasts on November 11, 29, December 15, 16, and 17. Figure 40 shows the TrEPS absolute errors for the same forecast time periods. The plots show:

• For MLP, the absolute errors show that shorter-term forecasts tended to have smaller errors than longer-term forecasts. Consequently, the shorter-term forecasts tended to be more accurate. The average absolute error for the 15-min forecast was about 4 mph, whereas the 115-min forecast increased to about 10 mph.
• For TrEPS, the absolute errors were relatively mixed across all forecast times. On January 29, 2020, the absolute error was relatively consistent and ranged from about 35 to 42 mph. On February 12, 2020, the absolute error was about 40 mph for the 15-min forecast, steadily decreased to about 11 mph for the 75-min forecast, then increased to about 21 mph for the 105-min forecast.
• Comparing the MLP to TrEPS absolute errors, the TrEPS errors tended to be larger than the MLP absolute errors. The average TrEPS error across all forecast times was about 33 mph versus about 7 mph for MLP.

Figure 41 and figure 42 show the relative absolute errors for MLP and TrEPS at I–435 and East Antioch Road on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The plots show:

• For MLP, the relative absolute error for shorter-term forecasts tended to be smaller than the errors for longer-term forecasts with the error as a percentage of the speed smaller for shorter-term forecasts. The average relative absolute error for the 15-min forecast was about 7 percent, whereas the 115-min forecast increased to about 18 percent.
• For TrEPS, the relative absolute errors were relatively mixed across all forecast times. On January 29, 2020, the absolute error was relatively consistent and ranged from about 54 to 71 percent. On February 12, 2020, the relative absolute error was about 66 percent for the 15-min forecast, steadily decreased to about 18 percent for the 75-min forecast, then increased to about 48 percent for the 105-min forecast.
• Comparing the MLP to TrEPS relative absolute errors, the TrEPS errors tended to be larger than the MLP errors. The average TrEPS error across all forecast times was about 56 percent versus about 12 percent for MLP.

Figure 43 and figure 44 show the root mean square errors for MLP and TrEPS at I–435 and East Antioch Road on November 11, 29, December 15, 16, and 17, 2019, for 15–115-min forecasts. The root mean square errors which measure the deviation or spread of the forecast versus actual speeds. The plots show:

• For MLP, the root mean square error showed that the shorter-term forecasts errors tended to be smaller than longer-term forecasts. The average root mean square error for the 15-min forecast was about 6 mph versus about 11 mph for the 115-min forecast.
• For TrEPS, the root mean square errors were relatively consistent across all forecast times. The root mean square error ranged from 19 mph to 32 mph.
• Comparing the MLP to TrEPS root mean square errors, the TrEPS errors were about three times larger than MLP. The average TrEPS error across all forecast times was about 27 mph versus about 9 mph for MLP.

Usefulness Study

The usefulness study investigated staff perceptions, attitudes, and behaviors. The goals of the interviews and discussions with KC Scout operators and supervisors were to learn about the users' experiences and perceptions relating to:

• Usage of IMRCP prior to, during, and/or after weather events.
• Accuracy of IMRCP information and forecasts.
• Integration of IMRCP into processes.
• Perspectives of IMRCP, overall (benefits, limitations, and lessons learned).

The operators reported primarily using IMRCP prior to a weather event. This included both rain and winter weather, but more often when winter weather was expected. The operators used it less during heavy rain or flooding events. In general, the operators did not use the traffic predictions capability. The operators said they liked the precipitation/weather predictions and the pavement status predictions because they can provide some insight into what locations to monitor more closely as the weather moves in. The operators used this feature to find areas with potential issues affected by the incoming weather. They reported it helped them prioritize and focus attention on particular roadway sections or areas, which in turn may improve their communication and decisions relating to motorist assist deployments. One user reported that IMRCP has been informally used to help inform transportation management center (TMC) staffing decisions during a weather event (e.g., should additional staff be called in, or can staff be sent home?).

The operators reported using IMRCP during the weather event in a more limited capacity. They are often busy during the weather events performing priority duties and responsibilities, such as entering, monitoring, and updating incidents into their system.

The operators typically do not use IMRCP to assess an event that has already passed; however, they acknowledge it may be helpful for data analysis and evaluation to determine lessons learned. One user believed there was a benefit to using IMRCP capabilities and its weather event data to look back and assess winter maintenance response efforts and emergency operation staffing decisions. Insight gained could potentially help save the agency staffing and monetary resources.

Overall, the users reported a fair confidence in the weather-related prediction components of IMRCP. As mentioned earlier, the operators did not use the traffic prediction components. The operators also indicated that periods of IMRCP downtime negatively affected the usage of IMRCP in their daily routines, especially in the 2019–2020 winter season. The operators' responses from the interviews seemed to indicate a more frequent use of IMRCP in the 2018–2019 winter season.

previous | next