Surface Transportation System Funding Alternatives Phase I Evaluation: Pre-Deployment Activities for a User-Based Fee Demonstration by the Minnesota Department of Transportation

Chapter 4. Independent Evaluation Methodology

This chapter summarizes the independent evaluation approach and methodology employed by the study team in coordination with staff from the Federal Highway Administration (FHWA) headquarters in the Office of Operations and the FHWA Division office representatives of the respective grantee sites. The chapter defines the evaluation framework and includes responses to key questions that the U.S. Department of Transportation (USDOT) expressed interest in knowing about road usage charge (RUC) approaches and their viability and characteristics if implemented on a national scale.

Evaluation Approach

Source: HDR Inc.
Figure 2. Diagram. Exploratory research for road usage charge technology options logic model.

As its name suggests, the fundamental concept of RUC is that users pay a direct charge for the use of a roadway. However, it is important to understand that both "use" and "user" can be defined in several different ways, and the mechanism by which a charge is levied can also vary significantly. The seven grantee agencies are all using different combinations of technologies and various paradigms and mechanisms to levy charges. Often the fundamental objective or goal of the RUC system is a significant factor in identifying the technology options, data collection, and how fees are levied. Previous research has characterized this phenomenon through the use of a RUC logic model, as illustrated in Figure 2.

One critical component of this evaluation included trying to understand the fundamental objectives of the RUC systems as deployed by the grantee sites. In particular, the team attempted to determine the primary objective of the Phase I implementation. Determining the objective provided overarching insight into more detailed assessments and evaluation of the efficacy, costs, and scalability of the systems at a regional or national level. Please see the discussion in the evaluation process section below for a summary of how the study team conducted this evaluation.

Evaluation Framework — U.S. Department of Transportation Questions

The key questions that USDOT intends to examine as part of this evaluation are presented in Table 3. To explore these questions within the context of the grantee sites' proposed activities, the evaluation team elaborated on the questions and defined relevant metrics for conducting the evaluation. While some questions were found to be highly applicable to Phase I activities, others were marginally applicable. Table 3 provides the assessment framework and Table 4 provides the system attributes relevant to the evaluation.

Table 3. Assessment framework.
	USDOT Evaluation Question	Relevant Site Question / Metrics	Applicability to Minnesota's Phase I Activities
Q1	What is the viability of road usage charge (RUC) on a nationwide scale?	Not applicable	Low
Q2	Would the fee assessment and collection mechanisms be scalable?	What considerations for scalability are you including in developing the agreements with shared mobility providers?	High
Q3	What is the efficiency of the fee assessment and collection relative to the fuel tax?	What are your initial findings on the efficiency of a mobility-as-a-service-based user fee system?	Medium
Q4	What are the system attributes and characteristics of the RUC systems with respect to privacy, security, user acceptance, ease of use, ability to audit, charging accuracy, reliability, equity, ability for a user to circumvent the charge, and other factors?	See detailed metrics in Table 4.	Medium
Q5	What is the user and stakeholder perception of mileage-based user fees in general and of pilot activities?	What are the findings of the stakeholder outreach conducted as part of Phase I?	High
Q6	What changes in institutional and financial setting, frameworks, models, and elements are required?	What are some of the lessons learned on internal processes and workflows from with the Department of Revenue and the legal department?	Moderate
Q7	What is the financial sustainability of each pilot deployment?	Have you conducted an analysis of funding needs for future phases of the demonstration?	Low

Source: FHWA

Table 4. System attributes.
Functional Parameter	Description
User-orientated parameters
Privacy	Privacy relates to the nature of the information being collected as opposed to the integrity of the information.
Equity	Equity relates to how user costs and other outcomes will impact people in different income brackets and people of different races/ethnicities, gender, English proficiency level, and travel mode.
Potential for Value-Added Services	Value-added services refer to the ability to add other transportation-related applications or software to the system to enhance system performance, reduce congestion, and improve mobility.
Ability to Audit	Extent to which an individual can contest their charges and have visibility into how those charges were accrued and assessed.
Ease of Use/Public Acceptance	The degree to which the system use is straightforward and time that a participant needs to spend interacting with the installed system is minimized; the level of acceptance by the traveling public.
Transparency	User awareness, specifically in real time, of what they are being charged.
Cost to User	Cost of equipment or installation to the end-user and cost of the per-mile (or other) charge.
System-orientated parameters
Data and Communications Security	Data source integrity and storage, transmission and access.
Charging Accuracy	The system's ability to assess the expected charge for each use of the roadway.
Charging Precision/ Repeatability	The system's ability to produce a consistent assessment of fees repeatedly for identical travel.
System Reliability	System "up-time."
Flexibility to Adapt	Ability of the technologies and systems to be upgraded or updated.
Flexibility to Expand	Ability of the system to respond to increased demand/system capacity and add technological capabilities.
Interoperability	Ability for the system to interact and exchange information across multiple jurisdictions.
Compatibility with Low Tech	Assessment based on the system's ability to accommodate users that cannot utilize the technology.
Evasion	Evaluation of how easily the system can be circumvented.
System Costs	Understanding of the full spectrum of investment costs, including initial capital, operating, and maintenance costs.
Ease of Enforcement	Ability of law enforcement to identify travelers that have evaded the system.
Cyber Security	Extent to which the system is vulnerable to a cyber-attack or release of private information.
Ability to Reallocate Revenue	Extent to which the system collects information that can be used to inform allocation of revenue.

Source: FHWA

Evaluation Process

The evaluation team devised an approach centered on periodic interfaces with the grantee agencies, including a site visit with a subset of grantees conducting pilot deployments to better understand the rationale and outcomes for Phase I activities.

Kick-Off Meeting

At the start of the evaluation, the evaluation team conducted 90-minute kick-off meetings with each of the grantee sites. The primary purpose of this call was to introduce the goal and scope of the evaluation and obtain information about the pilot's Phase I goals, scope and timeline. The evaluation team requested program documents compiled up to that point and updated project management plans.

Transportation Research Board Annual Meeting Workshops

The Surface Transportation System Funding Alternatives (STSFA) evaluation team facilitated two workshops during the Phase I evaluation. These workshops were held concurrently with the 2018 and 2019 Transportation Research Board (TRB) Annual Meeting in Washington, D.C. At the time of the 2018 workshop, most grantee sites were either just starting on their Phase I activities or planning to begin shortly. The first segment of the 2018 workshop was intended to address one or more "big questions" that each project was designed to answer. The questions in this workshop were structured to underscore a hypothesis that would be either supported or not supported as a result of the Phase I activities. Table 5 lists the specific questions posed during the 2018 TRB workshop, and Table 6 summarizes Minnesota Department of Transportation's (MnDOT's) responses.

It is important to note that, at the time of the 2018 workshop, several grantee sites were either just starting or preparing to start their Phase I activities. The information shared during this session was thus primarily based on prior RUC endeavors or based on very early activities.

Table 5. 2018 Transportation Research Board workshop questions.
Q1	What is the "one big question" that your project is best positioned to answer?
Q2	If you could tell your counterparts in fellow States looking to implement some form of road usage charge system, what would be your most important piece of advice to them?
Q3	What is the most important thing you have learned to date?
Q4	What is the biggest challenge you have faced, or expect to face with this project?

Source: FHWA

Table 6. Minnesota's Phase I summary as articulated at the 2018 workshop.
Field Deployment (Y/N) Using 2016 Funds	The "one big question" at the start of Phase I	Other Focus Areas of Phase I	Lessons and Challenges
No	Can mobility-as-a-service be an early applicant of a road usage charge?	Can we find a solution at the convergence of shared mobility and connected and autonomous vehicles and prove that, with better technology and the car-sharing platform, we have a solution that can be implemented widely?	Retain motor fuel tax on those vehicles that it makes sense to collect motor fuel tax on. New vehicles will likely have technology embedded in them providing the conditions for a scalable, efficient system that can be seamlessly integrated. We will understand this better when we undertake the demonstration.

Source: FHWA

At the time of the 2019 workshop, most grantee sites had either completed or substantially completed Phase I activities. This workshop also focused on lessons learned from Phase I and the reactions to a potential national RUC pilot. The MnDOT representative shared the following lessons:

The pilot should articulate a clear goal and remain consistent.
The fuel tax is efficient and should remain.
A distance-based user fee (DBUF) would backfill revenue lost from the motor fuel tax (MFT).
In the future, vehicle ownership is likely to decline. Consequently, Minnesota will experience reduced revenue generation from registration fees in addition to the loss in MFT revenue due to higher fuel efficiency and electric vehicles.
The equity issue has to be resolved. In any model, someone is subsidizing someone else.
A win for Minnesota is that mobility-as-a-service (MaaS) providers are onboard. The proof of concept with 70 vehicles; 4,633 unique trips; and 103,550 miles recorded showed that MnDOT can download the data and put it on a secure data repository and generate simulated invoices.

Conference Call Update

In September 2018, the evaluation team, in conjunction with Federal Highway Administration (FHWA) staff, conducted a conference call update with MnDOT to learn about the project progress, initial findings from the activities completed, and the timeline for completion of remaining activities. This conference call update was in lieu of a site visit given that MnDOT did not implement a pilot or have demonstration underway at the time of the planned site visits for Phase I evaluations.

previous | next