PRACTICAL GUIDANCE ON THE APPLICATION OF TRAFFIC SIMULATION AND ANALYSIS TOOLS FOR TRANSPORTATION-INVESTMENT DECISIONS

The remainder of this document provides insights into some common issues in the application of microsimulation tools. These issues are followed by specific actions that can be taken within a defined modeling process to either alleviate the issue or to allow practitioners to make decisions with a solid understanding of the potential impacts of the issue. These ideas were supported based on findings from interviews with local practitioners at each case study location.

Issues and Pitfalls of Implementation (So Why Didn't My Analysis Work?)

The most important concept for a user to remember is that software modeling tools are nothing more than mathematical equations that are attempting to replicate human behavior. Deterministic models try to simulate actions of aggregated drivers and micro-simulation models try to replicate individual driver movements. In either case aggregate behavior or individual behavior will vary due to a wide degree of variables including but not limited to age, driver experience, geographic area, time of day, weather conditions, congestion, and trip purpose. It requires an experienced traffic engineer and software model user to avoid the many pitfalls that will result in a traffic analysis not replicating existing or future driver behavior.

Based on the case studies, interviews with dozens of traffic engineers, and the authors' personal experiences, the most common pitfalls to avoid for a successful application of traffic operational software are as follows:

A. Overuse of Default Values (Or It's in the Model – It Has to Be Right) – The practice of using software packages without verifying the default values is very common. The tendency is for users to simply to apply the default values because of the high cost and limited project budgets to verify the values.

The odds of taking a software package out of its shrink wrap and successfully replicating existing ground conditions are very remote. Many software packages are developed outside of the United States. The software developers will use default values that are suitable for drivers of their particular countries. For example, many of the roundabout software packages overestimate the capacity by 20% when applied to US conditions (reference NCHRP Report 572 for Project 3-65, "Roundabouts in the United States"). One reason for this overestimation may be the values used in the packages are based on international drivers who are more familiar with the operations of a roundabout and this is reflected in their default values for variables such as critical gap time.

The type and range of default values may be extensive. Values such as critical gap times, follow up times, car following and lane changing algorithms, saturation flows, pedestrian walking speeds, lost time, and design vehicle characteristics such as acceleration and deceleration rates are just a few of the variables a user of traffic operations software packages needs to consider.

So What Should a User Do?

Before using a software package, it is critical for the user to determine and review the default values that are contained within the package. The review should also look at the mathematical equations which are included as the model algorithms. For example, in the Highway Capacity Manual as part of the procedure for determining the impact of parking on intersection capacity, the equation assumes it takes 18 seconds to park a car. For determining the impact of buses on the system it assumes that the buses are stopping for 14.4 seconds. The user must determine if these values are reasonable for their analysis. This determination should be done before the model is applied as part of a project analysis.

But there are so many default values! And how do you determine what the values should be? In the end, it is the experience of the user that is absolutely critical when establishing what reasonable values are. And that experience must be applied on a project by project basis. However, experience must be supported by verification, and that leads to Overcoming Pitfall B...

B. Calibration (Or Trust the Model But Verify) – Many interviewees indicated that calibrating the traffic operations model is absolutely critical. As indicated in Issue A using the default values from the software packages means that the models will most likely not simulate field conditions. All models from microsimulation to deterministic must be calibrated in order to simulate correctly.

For example, in a recent environmental study conducted in Washington, DC, as part of an operational analysis the levels of service for an unsignalized intersection were determined. The HCM based model predicted that the intersection would experience delays of up to 300 seconds on the minor road. Yet when a field check was conducted, the typical minor street delays were in the range of 30 seconds. A thorough review of the analysis indicated that the volume counts were correct and all data had been inputted correctly. So the team looked at the default values used in the model and tried to verify them in the field. As a result, it was determined that the typical critical gap times of the drivers at the intersection were 1 to 1.5 seconds lower than the gap times used in the model. Lower critical gap times mean higher capacity and less delays experienced by the drivers. As a result the critical gap times were lowered and the model replicated the 30 seconds of typical delay experienced by drivers at the unsignalized intersection.

So What Should a User Do?

Models that are being applied in a region or study area for the first time should be calibrated. If the study is for a future site, the model should be applied to nearby existing intersections or roadway sections and the user should verify that it can replicate existing traffic operations.

The real question is how we select the variable or variables that should be modified so that the model may be calibrated. Again the answer lies in having an experienced traffic engineer who is familiar with the software and the sensitivity of the model variables to conduct the calibration effort. Even with an experienced user, the calibration of the model may require a significant level of effort.

C. Queuing Analysis (Or How Far Back Will That Queue Go?) – During the interviews several users recognized that queue analysis was absolutely critical for determining the operational characteristics of the intersection or roadway. In fact several interviewees indicated that they were more concerned with the models accurately predicting the queues than the level of service.

In at least one of the case studies the major breakdown in the analysis was not accurately predicting the backup conditions downstream from the project. The existing analysis and the improvement simply moved the problem downstream and the backup interfered with the project's operation.

So What Should a User Do?

The first step the user should take is to validate and calibrate the model(s) as discussed in Pitfall (?) B. If the model variables have been reviewed and the model is still not simulating the queues accurately it very well may be that the area coverage is not large enough. The system analysis must cover the entire affected area. In some cases the project boundaries are too small and do not include areas which may influence the model results. For example, a major improvement beyond the project study area may affect the traffic volume or cause traffic to divert. The application of the model in the limited project area may not recognize the potential change in traffic patterns.

D. Unusual Geometrics and Conditions (Or This Is Above and Beyond the Call of Duty) – Many of the models are limited in their capability of analyzing unusual geometric or traffic conditions. Examples are 3 lane or 5 lane sections, rail lines through intersections, heavy pedestrian movements, complex signal controls, five or six legged intersections, oversaturated conditions, etc.

So What Should a User Do?

The various software packages all have strengths and weaknesses and different capabilities for analyzing unusual geometrics and conditions. The first step a user must take is to match the software's capabilities with the given conditions to be analyzed.

Sometimes field conditions are so complex that one software package cannot meet the analytical requirements for a study involving complex field conditions. Under these conditions, a combination of software packages might be considered. For example, in a recent corridor study in Vermont, the project involved a series of signalized intersections, arterial sections, roundabouts, unsignalized intersections, freeway ramps, multilane facilities, rail operations in the corridor, and a need for demonstrating operations to the public through animation. No one software package has the capability of meeting all the analysis requirements and therefore a combination of four different packages were selected for the analysis.

E. Inaccurate Field Data (Or What Was That Field Crew Doing Out There?) – Sometimes the models are not inaccurate, but the analysis is conducted with poor field data. The model user must always consider if the field data they have is accurate. Was the data collected on a typical day? Was there a breakdown of the counting equipment? Was there human error? The user must always remember that traffic varies from day to day which means there will always be some variation between the model simulation and typical traffic conditions.

How the data is collected could also affect the results. For example, at intersections field crews often count traffic by recording the number of vehicles entering the intersection – a reasonable technique as long as you do not have saturated conditions. However, only counting the vehicles that go through the intersection means by definition the analysis should never show volume exceeding capacity. If saturated conditions exist and you do not count the queued vehicles, the models will never replicate field conditions.

So What Should a User Do?

When conducting a traffic operations analysis not only should you look at the model parameters but a thorough analysis of how the data input was collected should be considered. Again, the most important element of an accurate project analysis are the users' experience and their ability to know what makes sense and what does not when reviewing field data.

F.Inaccurate Travel Demand Forecasting (Or the Forecast Is Right, We Just Got the Wrong Year) - The travel demand models have built-in inaccuracies which are passed on to the traffic operations software that uses their traffic volume forecasts as part of future year operations analysis. The demand models many times are in error due to land use changes or population and employment forecasts that are simply wrong.

So What Should a User Do?

When developing design year traffic, it is a good practice not to use raw traffic assignments from the traffic demand models. Rather develop growth factors based simulating existing and future traffic assignments. The growth factors should then be used to expand existing ground counts. The result is a much more accurate future traffic forecast and a more accurate operational analysis.

G. Inaccurate Origin-Destination Information (Or Tell Me Where They Are Coming From and Where They Are Going) – More advanced microsimulation models typically require the user to not only input traffic counts but also where these vehicles are coming from and where are they going. The microsimulation models use the origin/destination information to route the traffic through the system. Improvements in the transportation network may result in the model rerouting the traffic as part of its analysis. However, the original origins and destinations for the traffic entering into the system will not change. Most software packages have the origins and destinations as inputs that do not vary and do not analyze potential future changes even if within the project there are significant travel time improvements that may change the traffic patterns. This insensitivity to forecast changes in the origins and destinations may result in significant errors.

So What Should a User Do?

Microsimulation software is constantly being updated. The next generation of microsimulation software very well may be interconnected to regionwide travel demand models. The next generation of traffic software will allow the user to incorporate an iteration process where the results of an operational analysis will be reiterated back through the regional travel demand models.

However, until the software development catches up to the demand analysis requirements, for analysis of major regionwide transportation improvements the analysts could manually reiterate the results back through the regional travel demand forecasting model and rerun the microsimulation traffic operations software. A reiteration model application should be used only with a great deal of caution because it is only necessary for the largest regional transportation improvements. For smaller size improvements once is enough and the iteration process will make only an insignificant improvement in the results.

H. Sanity Checks (Or It Must Be Right It Comes From a Computer) – There is a tendency to over-rely on the results of the traffic operations models. Sanity checks should be made by the experienced user to determine if the results make sense. Too many times the user believes the model is working because it produces numbers.

So What Should a User Do?

Sometimes a user cannot see the forest because of the trees. A simple but effective way to conduct a sanity check is to get a second opinion. Look at the final results and if the model says we are handling 3000 vehicles per hour per lane and they are moving at 60 miles an hour it's time to get a new model. Look at the results and see if the traffic forecast is physically possible.

I. Demand Data (Or Do You Know How to Count Traffic?) – Most turning movement counts at signalized intersections are performed by counting vehicles as the pass through the intersection and ignoring the unmet demand. This produces unrealistic data in oversaturated conditions that do not represent the true demand at the intersection. Performing capacity analyses using data collected this way can severely underestimate the delay and back-of-queue results and yield inaccurate levels of service. For congested signals, arrival demand (not departure flows) must be used for the capacity analysis to accurately match field conditions.

So What Should a User Do?

Counting the unmet demand at the end of each period for adding to the departure flows yields the more appropriate arrival demands that should be used in these analyses. This can be done by estimating the length of the queue at the beginning of the red phase at the end of each period for all movements. These distances can be converted into vehicles per lane and added to the through volumes to produce arrival demand for each period. It is important to proportion the queued vehicles at the same turning rates as the through count for that period, and to count the unmet demand only once (subtracting it from the subsequent through count). For signalized intersection capacity analyses, congested conditions require multiple-period analyses using the unmet demand for each period as the initial queue for the subsequent period to accurately compute the third delay term (d3).

J. Different Definitions of Level of Service (Or I Can't Define the LOS F But I Know It When I See It) – Different software packages have different definitions of level of service. Although the Highway Capacity Manual defines the measures of effectiveness for various levels of service most traffic operations software packages use their own definitions of level of service. For example, when analyzing levels of service for roundabouts, some packages use the level of service criteria for signalized intersections versus unsignalized intersections. Some microsimulation packages may not even calculate density and have no direct correlation to the HCM level of service

So What Should a User Do?

When first using a software package, the user must review what and how the package is developing levels of service. The review should look at how the software is treating different types of facilities such as freeways, ramps, weaving areas, multilanes, signalized intersections, and unsignalized intersections. If the definitions of levels of service are different than the HCM definition then an equivalency should be established. Sometimes this is established by using the microsimulation to optimize the highway operations and then reanalyzing the facility by using an HCM based software package.

The user should also be careful when relying simply on a visual display of a software package to get a sense of what the level of service may be. It is critical for the user to review the evaluation output files rather than rely on the visual display to get a true understanding of the facility's level of service operations.