measure of the goodness of the design, and an efficient design has a small variance matrix. If a design is orthogonal and balanced, then it has optimum efficiency.

The experimental design will affect the estimable functional form of the model and its statistical efficiency (i.e., the standard error around the parameter estimates). Researchers therefore, should examine and test several design approaches. A full-factorial design in which every possible profile is presented will allow for the independent estimation of all main effects and interactions (i.e., all main effects and interactions are orthogonal). However, the number of profiles required by a full-factorial design may be too numerous for subjects to feasibly evaluate. Large designs can be blocked into multiple sub-designs to limit the number of tasks each respondent must complete. This blocking approach also allows for the independent identification of all parameters and interactions, but a loss of orthogonality may occur if a proportion of the sub-designs are not returned (13). Researchers should therefore ensure that each sub-design is randomly assigned to subjects.

The fractional factorial design is an alternative approach to decreasing the number of conjoint tasks required in the data collection instrument. Fractional factorial designs are orthogonal profiles constructed from a subset of the full factorial. These designs guarantee that all attribute main effects are independently estimable and they allow for the independent estimation of some attribute interactions if such interactions are defined by the researcher a priori (13).

Fractional-factorial designs typically are generated using published “catalog designs” (29-30) or statistical programs (e.g, SAS, SPSS). In choice-based conjoint analysis, the orthogonal arrays are used as “seed” profiles and choice alternatives are generated from the seed design using techniques that enforce the design criterion of orthogonality (zero correlation between attributes), level balance (each attribute level occurs with equal frequency), and minimal level overlap (each attribute level only appears once in a given choice) (27). These criteria provide good statistical efficiency for linear models, but for non-linear probabilistic models (e.g., the conditional logit model) designs that are more statistically efficient can be constructed by using design algorithms to search for an experimental design that minimizes a summary measure of the information matrix (e.g., the D- error criterion, (27, 31).

v. Preference Elicitation

Given that the aim of a conjoint analysis is to measure preferences, it is important to ask, “Were preferences elicited credibly?”

There are multiple question formats that can be used in preference-elicitation studies. Different elicitation formats provide responses to different questions. Therefore, researchers should ensure that the elicitation format used in a conjoint study is appropriate to answer the research questions the study is designed to answer. For example, the most appropriate elicitation format may differ for choice experiments that evaluate a new drug, health service, or public health program. In addition, data generated using different question formats will require different methods of statistical analysis.

In a discrete-choice experiment or forced-choice conjoint study, each conjoint task includes two profiles, and each profile is defined by a set of attribute levels between which subjects are asked to choose. Alternative question formats include ratings or rankings. Ratings and rankings provide more information about preferences

# ISPOR Conjoint Analysis in Health Task Force Report

Page 9