X hits on this document





11 / 28

the model predictions were on track, and to provide information needed to upgrade the models to predict exposure levels for new pesticides. A single-year monitoring effort funded at the level of 7-10 million can at best produce exposure estimates for one point in time for a handful of pesticides, requiring a great deal of additional investment to address the remaining pesticides and to address changes in exposure over time.

Statistical Considerations

The SAP understands that sophisticated analyses were used to determine the sample size required to achieve a 95% confidence about the 95th percentile. These methods are used to set the data quality standard but focusing on a single percentile seems unnatural and unnecessary, similar to describing a human's body shape by specifying the size of its nose. One SAP member suggested that the Agency consider using Kolmogorov-Smirnov confidence intervals for distributions instead. These confidence intervals express the uncertainty arising from limited sampling for the distribution as a whole, rather than for just a single percentile. The method is distribution-free, so that it doesn't make any obviously untenable assumptions. The method is known to be somewhat conservative, although its conservativism may not be appropriate in this regulatory assessment context.

The Kolmogorov-Smirnov confidence limits are classically expressed as intervals on the probability, and there may be some issue in translating between uncertainty about p and uncertainty about the x-value. There might also be some issue—or perhaps some economy—in the translation from the underlying distribution of annual mean concentrations for CWSs to the 10-year extreme distribution. Perhaps the literature on extreme value theory should be consulted.

Another issue that seems insufficiently considered in designing monitoring protocols is measurement error. Although laboratory measurement error is rarely negligible, it is often ignored in subsequent statistical analysis and risk assessments, which are commonly preoccupied with sampling error. One Panel member recommended that the monitoring protocol insist that measurement error be (1) recorded, (2) reported, and (3) propagated. Primary data measurements such as chemical determinations should be recorded with error intervals that summarize the precision of laboratory protocols that produced them. These intervals should be reported in all derived data sets and summaries based on the data. Finally, any calculations that make use of the measurements should also propagate measurement errors, at least with some simple bounding analysis. Although propagating the measurement error through calculations is obviously more trouble than simply ignoring it (which is what using only best estimates does), doing so expresses the reliability of any conclusions that are based on the measurements.

There appears to be a great deal of confusion between domains and strata in the presentations and the background material. It appears that domains refer to populations of interest and that strata refer to survey design units that can be aggregated to domains of interest. Clarification should be provided in future documents clearly specifying the domains of interest and


Document info
Document views49
Page views49
Page last viewedWed Oct 26 23:55:48 UTC 2016