USING FIDELITY SCALES FOR EVIDENCE-BASED PRACTICES
What is fidelity?
Fidelity refers to the degree of implementation of an evidence-based practice (EBP). A fidelity scale measures fidelity. Such scales have been developed for each of the six EBPs included in the Implementing EBP Project (assertive community treatment, supported employment, integrated treatment for dual disorders, illness management, family psychoeducation, and medication guidelines). Each scale assesses approximately 15 to 30 critical ingredients of the EBP, based on its underlying principles and methods. The scale items provide concrete indications that the practice is being implemented as intended. For example, one item on the Supported Employment Fidelity Scale concerns rapid job search. This item is rated as fully implemented if the consumers in a program average one month or less between admission to the supported employment program and their first job interview.
Why measure fidelity?
Several assumptions underlie the use of fidelity scales. First, a fidelity scale should adequately sample the critical ingredients of the EBP to differentiate between programs that follow the practice and those that do not. Research suggests that fidelity scales for supported employment and for assertive community treatment do accomplish this. Second, fidelity scales should be sensitive enough to detect progress in the development of a program from the start-up phase to its mature development. There is some evidence that fidelity scales achieve this goal as well. Third, high-fidelity programs are expected to have greater effectiveness than low-fidelity programs in achieving desired consumer outcomes. Several studies comparing fidelity ratings to outcomes also support this assumption.
One key use of fidelity scales is for monitoring programs over the course of their development (and even after they are fully established). Experience by implementers suggests that routine use of fidelity scales provides an objective, structured way to give feedback about program development. This is an excellent method to diagnose program weaknesses and clarify strengths for providing positive feedback on program development. Fidelity scales also provide a comparative framework for evaluating statewide trends and outliers. The strategic use of repeated evaluations of programs using fidelity scales, either on an individual program or statewide level, is based on the general principle that whatever is paid attention to is more likely to be improved.
How are fidelity scales used?
In the Implementing EBP Project we have developed fidelity scales that are simple to understand. EBP items are rated on a 5-point response format, ranging from 1 equaling no implementationto 5 equaling full implementation, with intermediate numbers representing progressively greater degrees of implementation. The response alternatives are behaviorally anchored, identifying measurable elements of the practice. Independent evaluators using multiple sources of information make the most valid ratings. Sources of information include interviews with staff, observation of team meetings, review of charts, and intervention observations. A daylong site visit is the optimal method for acquiring this information. Interviewers should be familiar with the EBP being rated. Although we recommend outside raters, fidelity scales can also be used by program managers to conduct self-ratings. The validity of self-ratings (or any ratings, for that matter) depends on the knowledge of the person making the ratings, access to accurate information pertaining to the ratings, and the objectivity
Family Psychoeducation Toolkit