A multi-surface system brings together a variety of different devices – such as a tabletop, tablet, mobile phone and wall display – into a single cohesive system. This integration allows users to take advantage of the unique capabilities of each device in ways that would not be possible using those devices separately. But creating usable interactions for moving content and control between all these devices has proven a difficult problem. Spatially augmented gestures, which are gestures which incorporate the spatial layout of the room as well as the people and devices in it, might provide a solution to this problem. Building such gestures into a multi-surface systems is difficult and tedious to develop. It represents too large an investment of time and effort for developers to bear. To decrease the cost of developing such systems, we have created an API – called MSE-API – that allows developers to quickly and efficiently add gestural interactions to multi-surface applications. In developing such an API we focused especially on making it usable for developers. Specifically we insured the API was learnable and discoverable for inexperienced developers but still an efficient tool for more experienced developers.
This thesis presents the requirements and structure of an API for developing multi-surface systems with spatially augmented gestures. The result of two case studies, in which the API was used to develop real world multi-surface applications, are also presented.