Please use this identifier to cite or link to this item:
Title: GSI DEMO: Multiuser Gesture / Speech Interaction over Digital Tables by Wrapping Single User Applications
Authors: Tse, Edward
Greenberg, Saul
Shen, Chia
Keywords: Computer Science
Issue Date: 18-May-2006
Abstract: Most commercial software applications are designed for a single user using a keyboard/mouse over an upright monitor. Our interest is exploiting these systems so they work over a digital table. Mirroring what people do when working over traditional tables, we want multiple people to interact with the tabletop application and with each other via rich speech and hand gestures. In previous papers, we illustrated multi-user gesture and speech interaction on a digital table for geospatial applications Google Earth, Warcraft III and The Sims. In this paper, we describe our underlying architecture: GSI DEMO. First, GSI DEMO creates a run-time wrapper around existing single user applications: it accepts and translates speech and gestures from multiple people into a single stream of keyboard and mouse inputs recognized by the application. Second, it lets people use multimodal demonstration instead of programming to quickly map their own speech and gestures to these keyboard/mouse inputs. For example, continuous gestures are trained by saying Computer, when I do [one finger gesture], you do [mouse drag] . Similarly, discrete speech commands can be trained by saying Computer, when I say [layer bars], you do [keyboard and mouse macro] . The end result is that end users can rapidly transform single user commercial applications into a multi-user, multimodal digital tabletop system.
Appears in Collections:Greenberg, Saul

Files in This Item:
File Description SizeFormat 
2006-833-26.pdf563.21 kBAdobe PDFView/Open
2006-833-26.ps5.63 MBPostscriptView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.