usefulness may be underrated.
Consolvo et al. increased the realism of their ESM study using Palm PDAs that simulated location requests from their friends, family and colleagues at random times . The participants were asked to respond to the request assuming that it had been actually made by the specific individual. However, Consolvo et al. noted that the random simulated requests were often implausible from a social standpoint. To add even more context, Iachello et al. combined event-contingent ESM with experience prototyping , calling this technique “paratyping” . A technique similar to paratyping was developed by Roßnagel et al. in the context of IT end-user security evaluation .
In related work, Ammenwerth et al. point out that there are inherent tensions in the formative evaluation of IT security mechanisms . When testing IT end-user security, users’ reactions and performance must be evaluated on technology that does not exist, and yet the user must be familiar with the technology. Further, tests should include breakdowns that would be unacceptable if they happened in reality. Ammenwerth et al. describe how they used a simulation study to conduct this kind of evaluation. In simulation studies, a working prototype is tested by “real users [performing] realistic tasks in a real social context [and subject to] real attacks and breakdowns” . Simulation studies are more complicated and expensive than Iachello’s paratypes, because they require careful selection of “expert participants,” extensive briefing to familiarize them with the technology, and complex data collection procedures. For this reason, they are best used at later stages of design.
3.2.2 Directly Asking About Privacy versus Observation
An important issue that needs to be considered in all techniques for understanding and evaluating privacy is that there is often a difference between what people say they want and what they actually do in practice. For example, in the first part of a controlled experiment by Berendt et al. , participants indicated their privacy preferences on a questionnaire. Later, the same participants went through a web-based shopping tour and were much more likely to disclose personal information than previously stated. Their explanation is that participants were enticed in disclosing information in view of potential benefits they would receive.
Focus groups can be used to gather privacy preferences [143, 174]. The advantages and drawbacks of focus groups are well known in the HCI and Software Engineering community and are similar in this context . We have found that focus groups on privacy have unique drawbacks, including susceptibility to cross-talk between informants and the fact that conventions of social appropriateness may bias responses to questions that an informant may consider sensitive or inappropriate. The latter is especially relevant in the context of privacy. For example, when investigating personal privacy issues between different generations of a family, a focus group with both parents and children will provide poor data.
Individual interviews, especially taking appropriate precautions to strengthen informants’ trust of the researcher, will result in better information . Still, interviews have other weaknesses. First, the information that can be gained from an interview is limited by people’s familiarity with a given system. Second, interviews do not scale well.
end-user-privacy-in-human-computer-interaction-v57.docPage 26 of 85