CUSTOMIZABLE PHYSICAL INTERFACES FOR INTERACTING WITH CONVENTIONAL APPLICATIONS

Abstract

Most of today's complex software products rely solely on graphical controls (GUI widgets) for user interaction. However GUI widgets can be difficult to find and use. Physical controls are often simpler to manipulate and arrange sensibly about one's workspace. Thus, we wish to link a physical interface to existing commercial applications, e.g., an office productivity suite. To do so we must tap in to its functionality in ways that do not require access to its source code. We present our widget picker/taps package. It gives developers access to the functionality of an existing application via the semantics of its GUI widgets. This approach works well with many present-day commercial applications, unlike two other common approaches: hooking into application-specific APIs, and simulating raw user input. We present examples that illustrate how this package is used to link existing application widgets to physical controls. Our implementation prompts a number of issues relevant to makers of windowing systems and GUI toolkits.