X hits on this document

335 views

0 shares

0 downloads

0 comments

95 / 108

5.3.2 Sample Size Needed to Compare Performance Levels between States

The chart below shows the sample sizes needed for various assumptions about confidence level, compliance rates, power and the magnitude of the differences states want to detect. The cells with between 34 – 65 inspections are highlighted. Note that unless states inspect more than 80 facilities each, the highest difference that could realistically be detected is 15%, and that would only be if both states had relatively high compliance rates.

Confidence Level: % certainty that a difference of the size listed below is not due to chance

90%

Power: % certainty that a smaller difference than the given difference IS due to chance (in other words that you 80% are not missing a true difference)

State A

50%

State B

50%

estimated compliance rate of the universe

  • #

    of facilities needed to determine that a finding that State

A's performance is

than

State B's performance is not due to chance

56

  • #

    of facilities needed to determine that a finding that State

A's performance is

than

100

State B's performance is not due to chance

  • #

    of facilities needed to determine that a finding that

State A's performance is than State B's performance is not due to chance

225

90%

50%

70%

50%

90%

90%

80%

90%

34

82

60

146

135

328

90%

90%

70%

50%

70%

70%

70%

50%

90%

95%

95%

95%

90%

80%

80%

90%

49

46

65

107

88

82

115

190

197

185

260

428

Decision on sample size for comparing performance across states

The Project Quality Assurance Officer recommended that each state inspect at least 56 facilities for the auto body sector. By inspecting at least 56 auto body facilities per state, the results would provide sufficient precision at a 90% confidence level, as well as allow the project to say that, for example, a 20% difference between two states was statistically significant and as such is not due to chance and may be due to differences in state programs.

5.4 UNIVERSE IDENTIFICATION AND RANDOM SAMPLE SELECTION RECORD

On November 6, 2007, the Project Management Team conducted a conference call to discuss the auto body universe identification methodologies proposed by participating states. The result of the call and subsequent discussions was the establishment of a performance standard for universe identification that included each state’s approach to identifying their universe. The approach for universe identification and random sample

The States Common Measures Project Final Report

95

95%

90%

70% 70%

95

137

309

Document info
Document views335
Page views335
Page last viewedWed Jan 18 16:29:00 UTC 2017
Pages108
Paragraphs3821
Words29990

Comments