Abstract
The development of appropriate lab-based evaluation techniques for mobile technologies requires continued research attention. In particular, experimental design needs to account for the environmental context in which such technologies will ultimately be used. This requires, in part, that relevant environmental distractions be incorporated into evaluations. This chapter reflects on different techniques that were used in three separate lab-based mobile evaluation experiments to present visual distractions to participants and to measure the participants’ cognizance of the distractions during the course of mobile evaluations of technology. The different techniques met the different needs of the three studies with respect to the fidelity of the data captured, the impact of acknowledging distractions on the evaluation task, and the typical context of use for the technology being evaluated. The results of the studies showed that the introduction of visual distractions did have an impact on the experimental task and indicate that future work is required in this area.
Key Terms in this Chapter
Interfering Distractions: Distractions incorporated within a lab-based experiment which may be passive or active and they interfere with a user’s ability to effectively interact with a mobile device.
Context: The intended context of use for a mobile application.
Field Study: The evaluation of a mobile application that takes place in the actual context of use. The advantage of such evaluations is that problems that only arise in the particular context will be detected. The disadvantages of such evaluations include difficulties in controlling the environment and capturing evaluation data.
Visual Distraction: Distractions that require participants’ visual attention.
Passive Distractions: Distractions incorporated within a lab-based experiment which distract users but require no active response.
Lab Evaluation: The evaluation of a mobile application that takes place in a laboratory. The advantages of such evaluations include ease of controlling the environment and data capture. Disadvantages include difficulties in creating an appropriately realistic evaluation setting.
Active Distractions: Distractions incorporated within a lab-based experiment which require a user to respond or react in some way (the required response varying according to the nature of the distraction).