and patient) and the model components (or categories of variables) and the probable relationships among them. The two-headed arrows represent the possibility of interaction effects among the different program- and patient-level variables, including cross-level effects between program- and patient-level variables.
Research shows that the degree to which multilevel modeling improves statistical estimation in comparison to ordinary least squares (OLS) or logistic regression analysis depends on the extent of cross-level effects in the data and variation in the dependent variable at different levels of analyses (see Bryk and Raudenbush, 1992; Krull and MacKinnon, 1999; Heinrich and Lynn, 2001). Large “between-group” variances, defined by Kreft (1996) as more than 25 percent of variation between groups, indicate that an OLS regression is more likely to mis-estimate relationships. In preliminary analyses of the NTIES data, we found that as much as half of the total variation in post-program outcomes is between substance abuse treatment service delivery units (where the outcome is a measure of drug use in the post-program period.)
Specifying and testing alternative outcome measures will be another challenge of the next phase of analysis. Schildhaus et al. (2000) used measures of reported use/non-use and average number of days of drug use in the follow-up period, as well as a measure of criminal behavior. In a study of methadone treatment outcomes using the NTIES data, Greenfield (1999) included measures of employment, income and HIV/AIDS risk behaviors as outcomes, in addition to drug use and criminal behavior measures. Also using the NTIES data, Orwin and Ellis (2000) developed severity scales for six domains — drug use, alcohol use, criminal activity, psychiatric problems, medical problems, and employment problems — and used measures of pre- to post-program changes in the severity scales as dependent variables in multilevel models. NORC researchers are presently developing new outcome measures, described as improvement scores