Procuring, Managing, and Evaluating the Performance of Contracted TMC Services
effectively display to a non-technical audience. The concept behind step 5 is to examine the list of measures and ensure that you will have information that can easily and quickly be understood by the target audience. In the terms of evaluating outsourced activities, the audience may be TMC management. It is also important to realize that in some cases, there may be multiple audiences, including such diverse groups as politicians and city leaders, the general public, agency management, planners and engineers. Each group has a different need for information and a different capacity for evaluating the information presented to them. Understanding those facets and how your performance measures support those presentations is the outcome of this step.
Step 6 – The concept, at this step in the performance measurement program, is to identify exactly what the data requirements are for any given measure. How much data? From what locations does is come? How often? Can it be used “raw,” or does it have to be processed? How must it be processed? Does the data need to be stored andfor what period of time? What is the reliability of the data? These questions and can be used to establish detailed technical requirements for the data needs to support performance measurement. This in turn, establishes reporting requirements for the contractor.
Step 7 – Following directly from Step 6, a solid plan for data collection is the result of this step. Whereas Step 6 identified the data need (e.g., 5-minute vehicle counts), this step identifies the source and mechanism for obtaining that data (e.g., automatic traffic counters at multiple locations along the freeway. Data stored in 5-minute bins in flat files and transmitted automatically, on a 24-hour cycle, to the TMC.) This step would also identify the specific tools and techniques that may be necessary to produce the final measure. These requirements may be placed on the contractor or the TMC, depending on the measure and the specific activity under evaluation.
Step 8 – Perhaps the simplest step in the program, this activity compares the actual results of the performance measure to the desired results, or goals, detailed in Step 4. An explicit categorization of the comparison results should be made, including date, time, overall result, measure, measure value, target, and difference between the value and target. This level of detail is an important input to Step 9 in the program. In terms of evaluation of contracted services, this activity would almost certainly take place at the TMC, as part of the evaluation process. It should be clear, however, to the contractor what the metrics, or performance targets, are in order to have a transparent and cooperative program.
Step 9 – Perhaps the most nebulous of all the steps in the program, Step 9 seeks to identify what (if any) remedial actions are needed to continue to advance the evaluation of contracted services via performance measures. In essence, Step 9 becomes a planning or brainstorming exercise. Is the contract achieving the cost, schedule, and performance goals? How can contractor performance be improved? Are we, as the TMC, measuring the right items? How effective is the contractor’s performance in meeting, or contributing to the agency’s program
Page 57 of 97