to be 80%, a state would be able to say that it was 90% confident that the actual result was + /- 10% of the observed result.
Increasing to 53 inspections in Vermont and to 63 inspections in Regions 4 and 9 in New York as shown in column 3, would allow them to say they were 90% certain that the actual performance of the universe was + or – 10% of the observed result if the observed performance was 50%. If the performance level was above 50%, the confidence interval would be narrower and/or the confidence level would be higher. See column 8 which shows that with one or two more inspections per state, if the observed a good performance rate was 90% states would be able to say they were 95% certain that the actual performance of the sector was + /- 7% of the observed result.
Notice that regardless of the confidence level or observed performance rate, states would need sample sizes of at least 85 (for Vermont), 95 for New Hampshire, and over 100 for all other states to obtain a confidence interval of + /
5%, regardless of the confidence level and observed performance.
Decision about sample size for benchmarking performance
In order to benchmark performance in an individual state, a minimum sample size of between 53 – 67 inspections for each state would be needed, depending on the state’s universe size. Inspecting this number would benchmark the performance of each state with a minimum level of precision (+ or – 10%) and with a reasonable level of confidence (90%) and assuming 50% observed compliance rates on each indicator.
In addition to benchmarking an individual state’s performance, sample size estimates were calculated for project states that wished to compare auto body performance results between states.
The States Common Measures Project Final Report