understand the instructions and whether participants feel the questions are appropriate. The larger pilot allows for consistency or rationality tests and can give estimates of coefficient size and direction.
vii. Data collection
Given that conjoint analysis is an empirical method, it is important to ask, “Was the data collection plan appropriate?”
Conjoint analysis surveys can be administered in many different ways including postal surveys using a paper-and- pencil survey instrument (22), non-mediated paper-and-pencil surveys completed at a finite set of study sites (37) electronic administration at a finite set of study sites using a PC (38), or electronic administration over the Internet (39). The complexity of most conjoint questions probably precludes the use of telephone-based data collection unless the survey instrument is mailed to subjects in advance. Interviewer-led administration of the survey may improve the quality of data because the interviewer can more fully explain the task including answering some questions (obviously without leading the participant in any way), and recognise if more explanation is needed. However interviewers may lead to bias because participants feel they need to produce socially desirable answers or they may believe that their answers could influence their future care. McColl & Fayers (40) provide an interesting review of administration methods. Participants should provide consent according to human subjects regulations, and they should also be made aware that the choices they are presented with are not necessarily choices that could be available to them from their doctor.
Subject recruitment must be consistent with the study objectives. Specifically, the sample of subjects must reflect those people whose preferences we aim to elicit and to whom we wish to generalise. For example, if we wish to quantify the preferences of women with metastatic breast cancer then the sample should be drawn from this population.
However, it is often difficult or costly to generate a sufficiently large sample of respondents from a narrowly defined population. Therefore, it may be reasonable to elicit preferences from not only a sample with a specific medical condition, but also from a sample of people at risk for the specific medical condition. Using the example above, it may be reasonable to include women with localized tumors of the breast who can reasonably conceive of developing more advanced breast cancer. Where this is done it needs to be made explicit and addressed in the analysis.
International studies present their own challenges to conjoint surveys. It is very important to ensure that appropriate translation and cultural adaptation of the survey takes place if it is used in multiple countries. Methods used in the development of other patient reported outcomes instrument should be sufficient (41). It may also be necessary to undertake additional pilot work in the new countries to ensure that the attributes, levels and conjoint tasks are acceptable. If a cost attribute is included then it is not sufficient to simply use exchange rates and instead we would recommend the use of purchasing power parities.
The design and conduct of conjoint studies should also consider the participant and whether there are any issues which would affect their ability to complete the survey. Conjoint studies can be cognitively complex to complete. Some patient groups who have known cognitive function problems – such as people with neurological diseases –
ISPOR Conjoint Analysis in Health Task Force Report