Approaches to Target Setting for PM3 Measures

Chapter 2. Targets and Performance-Based Planning and Programming

This section discusses target setting within the context of TPM and PBPP. It provides an overview of TPM and the PBPP process and discusses where and how target setting fits. After discussing the role of targets in PBPP, this section then articulates the link between target setting in PBPP and the target setting regulations as mandated by the FAST Act.

Overview of Transportation Performance Management

TPM is a strategic approach that uses system performance information to make investment and policy decisions to achieve National performance goals.⁶ TPM achieves this by helping transportation agencies determine what results (strategic goals) to pursue, then guide investments to achieve those results using information from past performance levels and forecasted conditions to select the best investments. Transportation agencies routinely measure progress toward their strategic goals and use progress reports to adjust planned expenditures to more effectively allocate available resources to meet the adopted performance goals. TPM is grounded in sound data management, usability, and analysis, as well as in effective communication and collaboration with internal and external stakeholders.

There are 10 distinct components to TPM, as shown in figure 2. These include:

1. Strategic Direction
2. Target Setting
3. Performance-Based Planning
4. Performance-Based Programming
5. Monitoring and Adjustment
6. Reporting and Communication
7. Organization and Culture
8. External Collaboration and Coordination
9. Data Management
10. Data Usability and Analysis

Figure 2 . Diagram. Transportation Performance Management framework.
(Source: Federal Highway Administration, Transportation Performance Management Guidebook.

Strategic Direction (Component 1) establishes an agency’s direction through well-defined goals and objectives and enables assessment of the agency’s progress towards meeting goals by defining a set of aligned performance measures. Strategic Direction is the critical first step in the TPM process and the foundation upon which all performance management rests. In order to be effective, the Strategic Direction should be integrated into a transportation agency’s business plan and related documents.

Target setting (Component 2) is the use of baseline data, information on possible strategies, resource constraints, and forecasting tools to collaboratively establish a quantifiable level of performance the agency wants to achieve within a specific timeframe. Importantly, target setting should be evidence-based and data-driven. Targets make the link between investment decisions and performance expectations as established in the Strategic Direction. In addition, targets help bring transparency to the transportation decision-making process.

Performance-Based Planning (Component 3) is the use of agency goals, objectives, and performance trends to drive development of strategies and priorities in the long-range transportation plan and other performance-based plans and processes. The resulting planning documents become the blueprint for how an agency intends to achieve its desired performance outcomes.

Performance-Based Programming (Component 4) is the use of strategies and priorities to guide the allocation of resources to projects that are selected by agencies to achieve goals, objectives, and targets. Performance-Based Programming establishes clear linkages between investments made and expected outputs and outcomes.

Monitoring and Adjustment (Component 5) emphasizes that what agencies do with performance information distinguishes transportation performance management from performance measurement. Management is distinguished from measurement in that upon measuring performance, a management framework insists that this information be fed back into the framework in order to adjust programming decisions. In other words, performance management encourages agencies to actively use information gained from monitoring performance data to obtain key insights into the effectiveness of decisions and identify where adjustments in programming would be most effective.⁷

Reporting and Communication (Component 6) is the use of products, techniques, and processes to communicate performance information to different audiences for maximum impact. Reporting increases accountability and transparency to external stakeholders and helps explain to both agency staff and external stakeholders how TPM is driving a data-driven approach to decision-making, and why changes to previously developed plans need to occur in order to meet the strategic goals adopted by the agency.

Organization and Culture (Component 7) refers to the institutionalization of a transportation performance management culture within the agency, as evidenced by leadership support, employee buy-in, and embedded organizational structures and processes that support TPM.

External Collaboration and Coordination (Component 8) refers to the established processes to collaborate and coordinate with agencies and stakeholders on planning/visioning, target setting, programming, data sharing, and reporting. External collaboration allows agencies to leverage agency resources and capabilities, as well as increase understanding of how activities impact and are impacted by external factors.

Data Management (Component 9) encompasses a set of coordinated activities for maximizing the value of data to an organization. It includes data collection, creation, processing, storage, backup, organization, documentation, protection, integration, dissemination, archiving, and disposal. The data management effort creates, organizes, and makes available the data resources needed for the final component.

Data Usability and Analysis (Component 10) takes the valuable data sets from the previous component and ensures both that those data are accessible and usable by the staff and stakeholders that need them, but that those individuals have the required analysis capabilities available to support both the production of the performance reports identified in Component 6, and the analytical tools needed to describe the value of alternative projects, plans, and strategies that are under consideration for achieving the desired strategic goals. While many agencies have a wealth of data, those data are often disorganized or cannot be analyzed effectively to produce useful information to support target setting, monitoring, project selection, decision-making, or other TPM practices.

As illustrated in figure 3, the target setting component touches all other TPM components. The target setting activity produces the quantifiable goals that the strategic direction is intended to achieve. This in turn should drive the planning and programming tasks, which are in turn dependent on the data analysis tasks and rely on an organizational culture and supportive external collaborations that understand and buy into the goal of achieving or striving to achieve those targets. The monitoring and reporting tasks combined then help the entire organization focus on the adopted goals.

Figure 3. Diagram. Target setting relationship to Transportation Performance Management components.
(Source: Federal Highway Administration, Transportation Performance Management Guidebook.)

Overview of Performance-Based Planning and Programming

PBPP is the application of performance management principles within the transportation planning and programming processes to achieve desired performance outcomes for the transportation system. The term performance-based planning and programming simply refers to how the overall objective of TPM is implemented, in part, through the planning process. PBPP helps transportation agencies to make infrastructure investment decisions based on their ability to meet performance goals.⁸

Figure 4 shows that performance-based planning and programming has three main components:

1. Planning
2. Programming
3. Implementation

These components are discussed in greater detail in the sections that follow.

Figure 4 . Flowchart. Framework for performance-based planning and programming.
(Source: Federal Highway Administration, Performance-Based Planning and Programming Guidebook, September 2013.)

Planning

In the context of PBPP, the planning process is designed to ask two basic questions: “where do we want to go?” and “how are we going to get there?” These questions are answered by first determining the direction in which a community wants its transportation to go, and then by establishing quantifiable measures by which progress can be assessed.

Where Do We Want to Go?

This question is answered by defining the vision that a State or a region has for its transportation system as described through goals, objectives and measures—a strategic direction. A strategic direction defines the trajectory in which a community wants its multimodal transportation system to go by shaping decisions about policies and investments. A region’s strategic direction should incorporate National goals for consistency, but likely goes beyond those in any State or region to reflect and be responsive to local issues, priorities, and values. A strategic direction includes:

• Goals and Objectives—In the context of PBPP, a goal is a broad statement that describes a desired end state. An objective is a specific, measurable statement that supports achievement of a goal. Goals address key desired outcomes, and supporting objectives play a key role in shaping planning priorities. Objectives should be based on the principles of SMART as defined in the private sector for some time:⁹
  • Specific—Outline in a clear statement precisely what is required.
  • Measurable—Include a measure that enables the monitoring of progress and recognition of when the objective has been achieved.
  • Achievable—Objectives can be designed to be challenging, but it is important that failure is not built into objectives. Stakeholders should agree to the objectives to ensure commitment to them.
  • Realistic—Focus on outcomes rather than the means of achieving them.
  • Timely (or time-bound)—The date by which the outcome must be achieved.
• Performance Measures—Support objectives and serve as a basis for comparing alternative improvement strategies and for tracking performance over time.

A key to setting “where we want to go” and thus, the targets that quantify that direction, is that transportation agencies must set those directions across multiple, often competing goals, including not just transportation system use and performance, but current and desired land uses, environmental quality and sustainability, social equity, and financial reality. These constraints are both the reason why external collaboration and cooperation are so important to TPM and PBPP and key pieces of information needed when agencies ask the next question, “how are we going to get there?”

How Are We Going to Get There?

This question is the crux of the long-range planning process. It is answered by identifying the specific strategies an agency can take to improve performance and the projects that it will undertake to implement those strategies. As part of picking those projects, the project identification, planning, and analysis processes need to consider how the potential projects being evaluated help the agency reach the performance targets defined by the strategic goals. This project identification, evaluation, selection, and prioritization process is the desired planning analysis.

The planning analysis component of PBPP identifies achievable targets and investment priorities, including operating strategies that can be carried forward into programming. It includes:

• Identify Trends and Targets—Preferred trends or targets are established for each measure to provide a basis for comparing alternative packages of strategies and measuring actual progress by tracking current conditions against the desired trend. This step starts with an initial baseline of system performance. It then computes a trend line of historical performance so that the agency understands where it stands now and where current trends are taking performance. The agency then sets desired target values for future performance, taking into account the current performance trend, forecasts of future performance given expected population and economic growth, information on possible strategies and their impacts on performance, available funding, other constraints, and knowledge of the desired level of performance the public would like to see achieved. While PBPP recognizes the utility of aspirational targets, such as “zero traffic fatalities,” to signal the broader societal importance of an issue, it also stresses that agencies should set realistic targets based on data to make material improvements in performance.
• Identify Strategies and Analyze Alternatives—Performance measures are then used to assess specific strategies and to prioritize options for system improvements. Scenario analysis may be used to assess alternative packages of strategies, to consider alternative funding levels, or to explore what level of funding would be required to achieve a certain level of performance.
• Develop Investment Priorities—Based on the results of the analyses performed above, the next step involves prioritizing strategies and making tradeoffs between investments within different goal areas. Final decisions on what projects to invest in are then made within the context of the system-level changes in performance which occur from any given mix of investments in specific technology, geographic, and modal areas. This step involves prioritizing what performance outcomes are most important to the agency, which should be reflected in the answers the agency adopted to the initial question of “where do we want to go?”

Programming

The programming component of PBPP addresses the question: “what are the schedule and budget details of the plan?” The programming component starts with the prioritized list of projects selected in the planning process and provides the details required to implement those projects. This often requires adjustments to the initial list of prioritized projects. It sets the schedules for when specific projects are implemented, ensures that the sources of funds needed to implement those projects exist, and exist at the time when those funds are required. This includes ensuring that funds or projects supplied or built by other agencies also are available and appropriately scheduled.

What Are the Schedule and Budget Details of the Plan?

This question is answered by considering tradeoffs across different programs and goals, making long-range investment plans and short-range programs of projects (i.e., transportation improvement program or statewide transportation improvement program) that all lead to a financially constrained set of proposed projects. The forecast outcome from these projects then creates an expected level of system performance over time, creating an expected trend, leading to the desired target for system performance. This relates to the target setting element of TPM. (That is, does the expected outcome of the programmed improvement plan achieve or exceed the performance target? If not, then the programmed plans describe the unfunded needs for the region. This information is then taken back to the region’s decision-makers in the next round of Strategic Decision-making.)

The programming task also results in a defined resource allocation for the agency, with that resource allocation plan often tied to the resource allocation plans of other agencies in the region.

The programming component of PBPP should produce:

• Investment Plan—An investment plan details a set of strategies or projects that have not yet been programmed into a transportation improvement program (TIP) or a statewide transportation improvement program (STIP). It may be developed to cover a midrange (e.g., 10 years) outlook to bridge long-range planning efforts (e.g., 20 or more years) to a short-term investment program.
• Resource Allocation/Program of Projects—Project prioritization or selection criteria are used to identify specific investments or strategies for a capital plan or TIP/STIP. Projects included in the TIP/STIP are selected on the basis of expected performance and show a clear link to meeting performance objectives.

Implementation and Evaluation

A key part of PBPP is that in addition to simply implementing the programmed projects monitoring and reporting the performance of the system after those strategies and projects are implemented is conducted. The outcome of this performance review is then used to 1) adjust the plan, if needed, in order to more effectively achieve the performance targets; and 2) refine the techniques used to forecast system performance to better reflect the impacts achieved from the deployment of specific techniques and strategies. Thus, the implementation phase of PBPP not only implements the projects, it asks and answers the question: “how did we do?” as a result of implementing those projects.

Consequently, after executing the selected set of strategies or projects, it is important for the agency to assess how those strategies or projects contributed towards the progress required to meet the goals set in the Strategic Decision-making effort, which are reflected in the adopted performance targets.

How Did We Do?

This question addresses the reporting and evaluation process, so that future plans and programs make use of the lessons learned from previous iterations. Here is where PBPP makes use of the reporting, accountability, and transparency elements of TPM. Implementation and evaluation activities occur continuously and include:

• Monitoring.
• Evaluation.
• Reporting.

These TPM tasks ensure that decision-makers and stakeholders: (1) understand the current state of the system’s performance; (2) can track current performance against the expected performance; (3) allow the differences between current and expected performance to be analyzed and understood; thus (4) are provided key insights into how plans or expectations may need to be adjusted moving forward.

Federal Requirements for Target Setting

While the first two subsections of the report provided overviews of TPM, PBPP, and how target setting fits within these contexts, it is important to discuss Federal requirements on target setting within the transportation planning process. The general public-sector transportation planning process is provided in 23 CFR Part 450. These regulations give States and MPOs broad responsibility for planning and programming transportation improvement projects. Statewide and metropolitan planning is required, per the U.S.C., chapter 23, sections 134 and 135, respectively.

MAP-21, passed in July 2012, mandated in 23 U.S.C. 150 that the U.S. Department of Transportation (USDOT) take a performance- and outcome-based approach to transportation planning. The objective of this approach was for States to invest resources in projects that collectively would make progress towards the achievement of National transportation goals. To this end, MAP-21 defined 7 National goals related to safety, infrastructure condition, congestion relief, system reliability, freight movement and economic vitality, environmental sustainability, and reduced project delivery delays. ¹⁰ The FAST Act, passed in December 2015, continued the performance-based transportation program of MAP-21.

FHWA adopted regulations for implementing transportation performance management in 23 CFR Part 490. regulations establish the National performance measures and detail the process for State DOTs and MPOs to establish performance targets.¹¹ These include targets for:

• The condition of pavements on the Interstate System (23 CFR 490.307(a)(1) and (2)).
• The condition of pavements on the NHS excluding the Interstate (23 CFR 490.307(a)(3) and (4)).
• The condition of bridges on the NHS (23 CFR 490.407(c)(1) and (2)).
• NHS Travel Time Reliability (23 CFR 490.507(a)(1) and (2)).
• Freight movement on the Interstate System (23 CFR 490.607).
• Traffic congestion (23 CFR 490.707(a)).
• On-road mobile source emissions (23 CFR 490.807).

The measures developed to cover NHS travel time reliability and freight movement on the Interstate System are referred to as the Third Performance Measure Rule (PM3) measures. Their definition and calculation have been documented by FHWA.¹²

Summary of Federal Requirements for Target Setting

As established in Federal legislation, the targets that State DOTs and MPOs are required to set can best be described as “realistic” within the PBPP context. As discussed in the PBPP section, while aspirational goals are useful for reflecting the importance of broader societal issues related to the transportation system, data-driven realistic targets are necessary to achieve material improvements in performance. The Federal legislation recognizes and reflects this.

Under 23 U.S.C. 150(e), State DOTs also must document performance and progress towards achieving targets via a biennial report submitted to the FHWA.¹³ The biennial report must describe:

• The condition and performance of the NHS in the State.
• The effectiveness of the investment strategy document in the State asset management plan for the NHS.
• Progress in achieving performance targets.
• The ways in which the State is addressing congestion at freight bottlenecks, including those identified in the National Freight Strategic Plan, within the State.

Benefits of Setting Targets to Agencies

Target setting should not be viewed as simply a regulatory requirement. Instead, target setting should be viewed as a tool for helping agencies to advance their own mission and goals. The TPM framework makes the case for the importance of target setting while the PBPP framework demonstrates how it may be implemented in practice.

The national performance measures are useful for achieving agency-specific goals, but target setting does not have to be limited to those measures. Expanding the practice of target setting to other performance measures, including those derived from travel time data, can yield a number of benefits to agencies. These include:

• Driving a conversation about current conditions and how to achieve future outcomes—Targets can help agencies communicate to stakeholders the resources and policies (i.e., investments) needed to meet expectations.
• Creating a method for evaluating processes currently in-place—If a current process or practice does not help an agency meet one of its stated targets, then the usefulness of the process should be reevaluated. Target setting can help reveal to agencies practices that are no longer as useful as they once were.
• Guiding the prioritization and allocation of resources—Ideally, targets are bound to a performance period and reflect what can be realistically achieved within a set of investments, policies, and strategies defined within an implementation plan.
• Enabling assessment of strategy effectiveness by focusing on linking goals, objectives, and measures to policy and investment decisions—Assessing the potential effectiveness of various strategies and investments for achieving stated goals and objectives before their implementation is central to performance management.
• Forming a powerful argument for additional or alternative investments—Targets help transportation agencies communicate to decision-makers, stakeholders, and the public the magnitude and types of investments that are needed to achieve a desired target level.
• Managing expectations by clarifying what outcomes are desired—In response to public outcry over a transportation issue, an external political body may mandate that a transportation agency improve the performance over some aspect of the transportation system, such as safety or congestion. In this scenario, targets can help communicate what is realistically achievable given technical and resource constraints.