Smart Survey Design
The pre-test or test pilot of the survey provides two functions. It first serves as the initial “live” test of the survey, and secondly it is the last step in finalizing the survey questions and form. The pre-test is possibly one of the most critical steps in administering a survey. By opting out of conducting a test pilot, you could jeopardize the accuracy of the data. The test pilot offers feedback on whether the survey’s wording and clarity is apparent to all survey respondents and whether the questions mean the same thing to all respondents.
The three basic goals of the pre-test are (Iraossi 2006, 89):
To evaluate the competency of the questionnaire.
To estimate the length of the survey or time to take the survey.
To determine the quality of the surveyor.
Checklist of Concerns During the Test Run:
According to Iraossi, the following is a good checklist of concerns regarding the survey and question design that, as the survey designer, you should consider during the test pilot (2006, 90-92):
Do the respondents understand the survey’s objective?
Do the respondents feel comfortable answering the questions?
Is the wording of the survey clear?
Is the time reference clear to the respondents?
Are the answer choices compatible with the respondents’ experience in the matter?
Do any of the items require the respondent to think too long or hard before responding? Which ones?
Which items produce irritation, embarrassment, or confusion?
Do any of the questions generate response bias? Which ones?
Do the answers collected reflect what you want in regards to the purpose of the survey?
Is there enough diversity in the answers received?
Is the survey too long?
According to your test audience, have any other important issues been overlooked?