Abstract
Gesture-based 3D interaction has been considered a relevant research topic as it has a natural application in several scenarios. Yet, it presents several challenges due to its novelty and consequential lack of systematic development methodologies, as well as to inherent usability related problems. Moreover, it is not always obvious which are the most adequate and intuitive gestures, and users may use a variety of different gestures to perform similar actions. This chapter describes how spatial freehand gesture based navigation methods were developed to be used in virtual walkthroughs meant to be experienced in large displays using a depth sensor for gesture tracking. Several iterations of design, implementation, user tests, and controlled experiments performed as formative and summative evaluation to improve, validate, and compare the methods are presented and discussed.
TopBackground
The use of gestures in human-computer interaction can be traced back to Sketchpad, developed in the sixties by Ivan Sutherland, as it used an early form of stroke-based gestures using a light pen on a display. After this first attempt, gestures have gained popularity as a means of realizing novel interaction methods, and several devices have been developed to support this possibility. Namely, manipulating virtual objects using natural hand gestures in virtual environments was made possible in the eighties through instrumented gloves (Fisher et al., 1986). In the nineties, a vision-based system (Freeman & Weissman, 1995) demonstrated a viable solution for more natural device-free gestural interfaces, and later other approaches have been used, as for instance the ones described in Boussemart et al. (2004), Malik et al. (2005), Karam (2006), and Wachs et al (2011); yet, only the recent advent of affordable depth cameras truly gave an essential momentum to the spatial freehand paradigm of gesture-based user interfaces.
Besides eliminating the need for an input device, spatial freehand gestures have several advantages as an interaction method: they are natural to humans who constantly use them to communicate and control objects in the real world from infancy, and may underpin powerful interactions due to hands’ multiple degrees of freedom, promising ease of access and naturalness also due to the absence of constrains imposed by wearable devices (Wachs et al., 2011; Ni, 2011, Ren et al. 2013b; Jankowski & Hachet, 2015).
Key Terms in this Chapter
Freehand Gesture: A gesture performed in the absence of constraints imposed by wearable devices (as gloves) or handheld tracking devices.
Navigation: A fundamental task in 3D environments allowing users to find their way and move around the environment. It presents several challenges as providing spatial awareness, and efficient ways to move between distant places. It includes travel and wayfinding (motor and cognitive components, respectively).
Virtual Walkthrough: A tour allowing users to walk through a specific place (e.g. a virtual museum, virtual library or virtual university campus) without having to travel physically.
Gesture: A form of non-verbal communication consisting in moving a part of the body (typically a hand or the head) with an underlying meaning.
Semaphoric Gesture: A gesture requiring prior knowledge or learning based on a formalized dictionary used to trigger a pre-defined action.
3D User Interface: A human-computer interface involving 3D interaction, i.e., in which the user performs tasks directly in a 3D spatial context.
Manipulative Gesture: A gesture meant to control an entity with the hands by acting directly on it in a real or virtual environment.