Peer Exchange Workshop on the "Perfect World of Measuring Congestion"
Workshop Summary Report

APPENDIX C: WHITE PAPERS
Paper #1: Framework for Gauging State-of-the-Practice in Performance Monitoring and Management

Prepared for:
Peer Exchange Workshop on the “Perfect World of Measuring Congestion”
FHWA Office of Operations

Prepared by:
Texas A&M Transportation Institute and Battelle

FINAL
December 10, 2013

1. Introduction

This background paper is paper number one (1) of four (4) developed in preparation for the Peer Exchange Workshop to be held on December 17-18, 2013 in Washington, D.C. The objective of this paper is to describe a framework to gauge the state-of-the-practice in performance monitoring. The framework includes examples, best practices, reporting and typical data sources. These examples discuss how current congestion measures are used to support transportation investment decisions including operational strategies.

The paper begins by presenting a summary of the proposed framework developed by the Texas A&M Transportation Institute (TTI) for gauging current best practices. The framework was informed by observation of general industry practices related to performance monitoring. The remainder of the report supports these observations with additional details and examples.

2. Highlights of Framework for Gauging State-of-the-Practice

Observation of practices for performance monitoring and management provides evidence of a varying degree of the extent that performance data are used in decision-making for transportation projects and programs. Figure 1 introduces an Operations Performance Management Capability Maturity Model (OPMCMM) to identify characteristics of the performance management maturity continuum, which is evident in practice. The OPMCMM provides a method for grading agency practices on several characteristics important for performance management to be a seamless practice in a transportation organization. The six (6) categories are 1) performance measures (content and form), 2) performance management (agency culture), 3) data, 4) modes, 5) facility and trip coverage, and 6) traveler preferences and tradeoffs. The proposed framework illustrates four (4) “ribbon categories” of bronze, silver, gold and platinum to illustrate maturity level for the given characteristics.

The capability maturity model illustrated in Figure 1 was inspired by the capability maturity model applied to systems operations and management described in SHRP 2 Report L06 (Institutional Architectures to Improve Systems Operations and Management).¹ While the SHRP 2 Report L06 used four (4) levels, here “ribbon levels” are proposed for the maturity scale. The following are general highlights about the OPMCMM and observations from selected practices:

• In the OPMCMM, performance monitoring and management evolves to interactive/automated performance results, becomes data-driven using real-time or near real-time information, and has a performance connection between project-level and system-wide performance.
• At the bronze level of the OPMCMM, data sets are “siloed” in agencies and become more transparent, accessible and connected with maturity.
• The platinum level of the OPMCMM is characterized by complete modal information and understanding not only how you traveled, but how/where you really wanted to travel.
• There are a number of external sources and transportation-related contributing factors to congestion that are not currently imbedded in typical performance practice (e.g., economy, societal factors [development patterns, housing prices], weather, incidents, work zones, connected/autonomous vehicles) – these external factors will be described in background white paper #4.

The characteristics and examples suggested in Figure 1 are described further in the remainder of this paper. Background paper #2 will further describe how the industry can get to the “perfect world” (platinum level) identified in Figure 1.

3. Current Practice in Measuring Congestion

The ultimate goal of measuring congestion is improved transportation investment decision-making. Initially, performance measures provide a baseline of conditions, and, over time, allow for trend analyses of what types of transportation investments are working and where. These performance insights inform future investment decisions for transportation projects, programs and strategies.

An evaluation of the current practice in measuring congestion provides evidence that there are varying degrees to which performance data are used in transportation investment decision-making. This varied nature of the profession is captured in Figure 1 (the OPMCMM). Current practice can be viewed on a scale of increasing performance management maturity, and from left to right in Figure 1, there is increased maturity in performance management. At the left end of the scale there is a minimal degree of performance data used in decision making, and this improves as the reader tracks their eyes to the right along the graphic at the top of Figure 1. For perspective, the graphic identifies four (4) “ribbon categories” (bronze, silver, gold, and platinum) from left to right as the degree of performance data are used in decision-making.

Just below the graphic at the top of Figure 1 are short characterizations to describe the OPMCMM continuum. While there are four (4) ribbon categories, there are six (6) general short-description observations related to data in decision-making along the maturity continuum. At the far left where there is zero performance data used in decision-making, decisions are made with engineering judgment, are highly intuitive and/or based on anecdotal evidence. Progressing to the right are specific projects, characterized by a “snapshot” of performance for a project evaluation that is not really connected to system-wide performance activities. As one progresses along the maturity scale, network performance reporting is encountered. Moving further along the maturation scale, the next step includes a feedback loop in decision-making (i.e., an evaluation process is in place after decisions are made to evaluate the decision and the process). The next steps include data-informed decisions – not entirely data-driven, but data play a key role – and are then followed by data-driven decisions from performance data (in near real-time or real-time).

This section of the paper describes the following characteristics of the performance management maturity scale highlighted in Figure 1:

• Performance measures (content and form);
• Performance management (agency culture);
• Data used;
• Transportation modes;
• Facility and trip coverage; and
• Traveler preferences and tradeoffs consideration.

Selected examples of different characteristics and ribbon levels are also provided in the following section.

Performance Measures (Content and Form) at Each Ribbon Level

All ribbon levels typically use travel time-based or delay performance measures (e.g., total delay, delay per mile, travel time index, planning time index, etc.). Measures are computed at the spatial and temporal scales of interest.

• Bronze Level: Characterized by limited project and program evaluations. Typically these project evaluations are for a specific intersection analysis or segment of road where a project or program will be implemented (before study) or to evaluate the impacts after implementation (after study). The evaluation informs the public agency if the project/program was successful and whether improvements are needed. Typically it is just a snapshot of average (rather than reliability) outcome performance measures.
• Silver Level: At this level only mandated or legislated (required) reporting is performed by the agency, and these activities are “siloed” into the divisions or groups of the agency required to report.
• Gold Level: At this level measures captured include both public sector efforts (output) and performance results from the field (outcome) in selected areas of the organization (e.g., operations section of the agency reports on the number of motorist assistance patrols on the freeway system in a particular urban area [output measure] as well as average travel time information on the urban roadway system [outcome measure]).
• Platinum Level: Full and seamless family of output and outcome measures used by the agency. There is a real-time link between project-level and system reporting that facilitates real-time (or near real-time decision-making). With continuous data readily available at this level, reliability measures can be easily produced in addition to average conditions.

Performance Management (Agency Culture) at Each Ribbon Level

Equally important to the measures themselves is the culture of the transportation agency in adapting performance management into their decision-making processes; therefore, the OPMCMM includes this characteristic.

• Bronze Level: Minimal performance management interest in the agency. Performance measurement only performed when mandated or legislated as required. Performed to “check a box.”
• Silver Level: Isolated champions emerge in the agency with strong interest in performance management. Any performance management activities still siloed.
• Gold Level: Evidence of entire portions of the agency implementing and making decisions with performance measures. There is evidence of “data-informed” processes to select projects.
• Platinum Level: Performance management is ubiquitous in agency culture. Entire agency speaks “performance management language” fluently, and it feeds management decision-making. Decisions are “data-driven” using real-time information.

Data Used

All ribbon categories along the maturity scale typically use speed, volume and roadway inventory data at spatial and temporal scales of interest. As the industry matures to the “platinum level,” real-time performance management and decision-making are possible.

• Bronze Level: “Siloed” datasets with no connections. Data are sought for the project evaluation or analysis at hand and are typically acquired from different groups within the agency.
• Silver Level: Dataset connections are possible, but it requires programming/processing.
• Gold Level: Similar to the silver level where dataset connections are possible, but it requires programming/processing. At the gold level there is evidence of some coordination of datasets across agency “silos.”
• Platinum Level: At the platinum level of the maturity scale, all database types are connected (e.g., volume, crashes, pavement quality, etc.) and available through automated methods (e.g., relational database, GIS, etc.).

Transportation Modes

This characteristic of the OPMCMM relates to an agency’s ability to understand multimodal and intermodal trip characteristics.

• Bronze Level: Mode knowledge and related data and information are “siloed” in the agency and there is limited communication across modes in the organization.
• Silver Level: Some ability to capture or estimate mode shifts within agency “silo” of interest.
• Gold Level: Ability to capture mode shifts across the entire agency.
• Platinum Level: The ability to capture mode shifts over time and across the agency, including intermodal considerations (i.e., freight and person movement).

Facility and Trip Coverage

The geographic scope of travel is captured in this characteristic.

• Bronze Level: Limited temporal and spatial coverage available for performance monitoring.
• Silver Level: Selected modes and/or facilities and related temporal coverage.
• Gold Level: All modes, all facilities, all days, and all times covered.
• Platinum Level: In addition to all gold level characteristics, also ability to capture diversion from traveler information or control strategies.

Traveler Preferences and Tradeoffs

The final characteristic identified in Figure 1 relates to an agency’s ability to capture traveler preferences and tradeoffs.

• Bronze Level: Limited ability to capture trip preferences or revealed behavior. “Revealed behavior” relates to methods or technologies to provide insights or understanding about how and where particular trips are made from start to finish.
• Silver Level: Some ability in selected agency “silos” to capture some trip preferences or revealed behavior.
• Gold Level: Technological methods identified/used to capture trip preferences.
• Platinum Level: An agency has the ability to gauge or capture traveler preferences and revealed behavior. This means an agency can identify how you traveled, and has information about how you really wanted to travel. This makes the important distinction that travelers must use the built system (and modes present), but may really want to take other routes/modes if they were available.

The platinum level as defined here is admittedly far off; however, it is important to keep an eye on the fact that as transportation professionals better understand traveler trip preferences, this will help them develop the system for all users.

4. Selected Examples of Different Characteristics and Ribbon Levels

There are several examples provided in Figure 1 to help the reader identify where some selected current practice and activities might fall on the OPMCMM framework. These examples are only intended to start a discussion about how the OPMCMM can be used to grade different types of agency performance management activities.

OPMCMM Characteristic: Performance Measures (Content and Form) – Bronze Level

There are numerous examples of performance activities at the bronze level, characterized by project-specific studies. Below are a few transportation operations examples recently highlighted in FHWA’s 2012 Urban Congestion Trends Report.² These types of evaluations are project specific and provide a snapshot of performance. Table 1 includes examples of performance measures (content and form) at the bronze level.

Table 1. Examples of Performance Measures (Content and Form) at the Bronze Level
(source: FHWA’s 2012 Urban Congestion Trends Report)

Towing and Recovery Incentive Program (TRIP) (Atlanta, Georgia)
Description	TRIP provides monetary incentives for timely clearance of crashes involving commercial vehicles. A recent Georgia Department of Transportation (GDOT) study evaluated TRIP, and found that it allowed roadway opening at least 2 hours and 45 minutes faster than in 2007 (before TRIP). Other key findings were that TRIP decreased average incident cost by 70 percent, saved over $9 million in delay, wasted fuel and emissions from inception to 2009, and results in a benefit of nearly $11 for each $1 spent.
Data	Modeling of “pre-TRIP incidents” using clearance times to estimate benefits if TRIP were in place.
Measures	Costs of delay, wasted fuel, emissions, benefit-to-cost ratio.
Traffic Light Synchronization Program (California)
Description	Caltrans recently funded the Traffic Light Synchronization Program (TLSP) at $250 million from successful passage in November 2006 of California’s Proposition 1B (Transportation). Several projects have been implemented and have either met or exceeded estimated benefits. Two examples of benefits are in San Ramon, CA where ASCTs were installed on Crow Canyon Road and Bollinger Canyon Road. Both locations resulted in reductions in travel time, fuel consumption, emissions and collisions.
Data	Data sources include travel time from travel time runs, traffic volume measurement, and database of crashes.
Measures	Travel time, total delay, stopped time, congested time, fuel consumption, and emissions. Fuel consumption and emissions were estimated from SYNCHRO software.

OPMCMM Characteristic: Performance Management (Agency Culture) – Gold Level

Table 2 includes an example of performance management (agency culture) at the gold level of the Washington State DOT’s Gray Notebook.

Table 2. Examples of Performance Management (Agency Culture) at the Gold Level
(Source: WSDOT’s Gray Notebook)³

WSDOT’s Gray Notebook
Description	The Washington DOT’s Gray Notebook (GNB) is the basis of WSDOT’s external performance reporting. It is recognized nationally for setting a high standard for accessible and accurate updates on programs and projects. The report is released quarterly with sections clearly related to the six statewide transportation policy goals of safety, preservation, mobility (congestion relief), environment, economic vitality and stewardship.
Data and Measures	The report uses a number of data sources and performance measures across the six policy goal areas. WSDOT’s Performance Dashboard (page vii of the GNB Edition 50) highlights key mobility measures as annual weekday vehicle-hours of delay, average clearance times for major incidents, percentage of ferries departing on time and percentage of Amtrak Cascades trips arriving on time.

OPMCMM Characteristic: Data – Silver Level

Table 3 includes examples of the “data” characteristic at the silver level of TTI’s Urban Mobility Report and FHWA’s Urban Congestion Reports.

Table 3. Examples of Data at the Silver Level
(Sources: TTI’s Urban Mobility Report, FHWA’s Urban Congestion Reports)⁴

TTI’s Urban Mobility Report
Description	The Urban Mobility Report reports congestion in all U.S. urban areas. In the 2012 Urban Mobility Report, trend data are available from 1982 to 2011. Detailed congestion statistics are provided for 101 urban areas, and summary statistics are provided for the 498 urban areas throughout the U.S. Statistics are presented in tables by urban areas population size. The report is widely quoted on congestion and associated costs.
Data	The Urban Mobility Report is powered by huge datasets, including: 1) FHWA Highway Performance Monitoring System (HPMS) volume data, 2) INRIX 15-minute annual average speed data (850,000 miles of road), 3) FHWA Freight Analysis Framework commodity value and tonnage for trucks.
Measures	Delay per auto commuter, total delay, travel time index, excess fuel per auto commuter (gallons), congestion cost per auto commuter (wasted time and fuel), truck congestion cost, truck delay, planning time index, carbon dioxide production, truck commodity value, commuter stress index, and total peak period travel time.
FHWA’s Urban Congestion Reports
Description	The Urban Congestion Report is produced quarterly to characterize emerging traffic congestion and reliability trends at the national and city level. The reports currently include 19 urban areas in the U.S.
Data	State DOT archived traffic operations data. Data for those roadways that are instrumented with traffic sensors for the purposes for real-time traffic management are included in the dataset.
Measures	Congested hours, travel time index, planning time index.

OPMCMM Characteristic: Data – Gold Level

Table 4 includes an example of the “data” characteristic at the gold level of a future performance activity with the I-95 Corridor Coalition.

Table 4. Examples of Data at the Gold Level
(Sources: I-95 Corridor Coalition Website)⁵

I-95 Corridor Coalition Future Performance Activities
Description	One example that incorporates some aspects of this level is the planned performance measurement activities of the I-95 Corridor Coalition. The Coalition’s website discusses the ongoing development of “a corridor-wide, web-based visual analytics monitoring system for identifying major bottlenecks, reporting travel time reliability and displaying other congestion measures using private-sector vehicle probe data fused with agency incident/event data where available. This system demonstrates how states can create a congestion monitoring program using a variety of data sources.” The system will allow the users to view both real-time and historical performance at various zoom levels
Data	Real-time INRIX speed data and agency incident/event data (where available)
Measures	Travel time index, travel time reliability, hours of congestion per mile and buffer index

OPMCMM Characteristic: Facility and Trip Coverage – Silver Level

Table 5 includes an example of the facility and trip coverage characteristic at the silver level including TxDOT’s 100 Most Congested Roadways List.

Table 5. Examples of Facility and Trip Coverage at the Silver Level
(Sources: TxDOT’s 100 Most Congested Roadways List)⁶

TxDOT’s 100 Most Congested Roadways List
Description	Since 2009, Texas DOT has sponsored TTI to produce a list of the most congested roadway sections in the state on the TxDOT website. The two agencies have developed an approach that combines annual speed archive data from private companies with basic roadway geometry, and traffic counts published in the TxDOT statewide roadway inventory file (RHiNo) to calculate congestion-related performance measures. The list is used for Texas DOT to program dollars to address the worst traffic locations across the State.
Data	Annual average traffic speeds from private-sector company and basic geometry and traffic volume from TxDOT roadway inventory.
Measures	Annual hours of delay per mile, annual hours of truck delay per mile, Texas congestion index (form of travel time index), planning time index, commuter stress index, annual congestion cost, and truck congestion cost.

The Indiana Mobility Report⁷ and the Maryland State Highway Mobility Report⁸ also represent examples of the facility and trip coverage characteristic at the silver level. Both reports use private-sector speed data to estimate mobility performance measures. Both reports include evaluations and mobility improvements due to specific projects in the state; therefore identifying the mobility benefits of specific transportation investments to further inform future decision-making.

The 2013 Maryland State Highway Mobility Report goes a step further by integrating the speed data with volume information to compute delay and associated wasted time and fuel costs due to congestion (using the same methodology as TTI’s Urban Mobility Report). Maryland’s Report also includes the planning time index reliability measure.

OPMCMM Characteristic: Traveler Preferences and Tradeoffs – Bronze Level

Google Maps® has a new preferences feature that allows you to make maps of places that matter to you, allows you to save places to find them quickly later, and allows you to rate places you know to discover new places you might like. This is just an example of the types of Internet tools that can provide information about the types of places people want to go and perhaps how they choose to get there. Market tools and technologies will continue to evolve that can help agencies better understand traveler preferences and where travelers really want to go.

You may need the Adobe® Reader® to view the PDFs on this page.