INDUCING PROGRAMS IN A DIRECT-MANIPULATION ENVIRONMENT

Abstract

End users who need to program within highly interactive direct- manipulation interfaces should be able to communicate their intentions through concrete demonstration rather than in terms of symbolic abstractions. This paper describes a system that learns procedures in interactive graphics taught to it "by example" by minimally trained users. It shows how techniques of machine learning and reactive interfaces can support one another-the former providing generalization heuristics to identify constraints implicit in user actions, the latter offering immediate feedback to help the user clarify hidden constraints and correct errors before they are planted into the procedure. The teacher's attention is focused on the learning system's perceptual and inferential shortcomings through a metaphorical apprentice called Metamouse, which generalizes action sequences on the fly and eagerly carries out any actions it can predict. The success of the induction process is assessed quantitatively by counting erroneous predictions made during example tasks.