3.2 UNIVERSE IDENTIFICATION AND RANDOM SAMPLING
If the facilities are randomly selected, statistical methods can be used to generalize the field observation findings to the sector as a whole and to determine if the observed differences among states are likely to be due to chance, or reflect actual differences in performance. Random samples are the foundation of all statistical analyses. Without them, bias can be introduced into the results. For example, if a state chose to target facilities that they suspected had problems (something field observers frequently do in order to maximize the likelihood of identifying and resolving environmental problems), the results from that state would likely show worse performance than the results from a state that selected facilities randomly. Selecting facilities randomly ensures a comparable analysis of results across states.
An SQG Universe Identification and Random Sampling Record was developed and distributed to states to document each state’s methodology for universe identification and random sample selection. Rather than the Project Management Team establishing a single method for universe identification, each state used and documented its own identification approach. The Quality Assurance Officer determined that the approaches used by the project states were consistent and appropriate.
The Project Management Team also required that all states use the same methodology for generating a random sample of facilities to inspect. In cases where a facility turned out to be “not applicable,” e.g., the facility closed or generated more or less than the 100-1000 kg of RCRA hazardous waste per month, all states agreed to replace this facility by inspecting the next facility on their randomized list.
The Quality Assurance Officer for the project provided step-by-step instructions to the states for generating a random list of facilities from their universe. See Appendix H for How to Generate a Random Sample.
SQG UNIVERSE IDENTIFICATION AND RANDOM SAMPLING SELECTION RECORD
RANDOM SAMPLING SELECTION
State Common Measures Project: Methodology for Generating A Random Sample, June 8, 2007
Alternative Method (describe):
The States Common Measures Project Final Report