3.3 STEPS TO ENSURE FIELD DATA QUALITY
The Project Management Team had to ensure that the project states were measuring the same activities in the same way so that the results were truly comparable from state to state. This was a challenge as there were multiple states with different regulations and different approaches to performing inspections. There also were different environmental concerns and different levels of detail regarding compliance with the different state standards. For example, a state that interpreted “good condition” for a hazardous waste container as absolutely no chipped paint, ding, or rust of any kind, would observe lower performance than a state that interpreted “good condition” as containers that were not actively leaking at the time of the inspection. Similarly, record review procedures could affect observed compliance rates. For example, states that reviewed the past 3 years of records would likely find more violations than states that reviewed the past 6 months of records. The Project Management Team had to minimize these differences as much as possible before any data collection could occur.
In order to ensure consistency with each state’s approach to data collection the Project Management Team developed a Common Measures Checklist using the final SQG indicators that were selected by the project states. States had the option to use either the Common Measure Checklist or use their own checklist so long as it included the Common Measures Checklist questions exactly as they were phrased. Alternative checklists had to be approved by the Quality Assurance Officer before use.
In order to further ensure consistency with each state’s approach to data collection, the Project Management Team presented mandatory data collection phone training on May 23, 2007. The training was provided by state field and enforcement experts with over twenty-five years of hazardous waste experience, the Project Manager and the Quality Assurance Officer. All project state leads, and as many field data collectors as possible were required to attend the training. The state lead was required to certify, when submitting SQG data, that all field observers who collected the data either participated in the training or were trained by the state lead. Attendance records were also kept to document that all field observers received the data collection training. See Appendix I for Training Attendance Log Template.
The training included:
Careful review and discussion of each indicator, specifically reviewing the wording and intent.
Refining language as necessary.
Agreeing upon procedures for interpretation.
Reviewing decision rules for determining whether or not the facility was achieving the indicator.
Minor adjustments were made to the checklist as a result of the training. See Exhibit 3.1 below for the States Common Measures Project SQG Performance Checklist.
The States Common Measures Project Final Report