On Modelling Events with an ‘Analogical’ Representation

Abstract

This paper is concerned with representing observed trajectories of moving objects. By modelling such spatio-temporal events we want to support certain tasks involving knowledge about typical object motion, such as trajectory prediction. We consider this problem in the context of a natural-language guided scene analysis task where simple questions like “Did a car turn off Schluterstreet?” are used to start top-down controlled image sequence analysis. For an effective control of the vision processes it is essential to provide knowledge about the spatio-temporal constraints implied by the verb ‘turn-off’ and other verbs for that matter. Previous work (NEUMANN and NOVAK 83, NEUMANN 84) has dealt with the bottom-up path, i.e. assigning meaning to observed object motions in terms of verbal descriptions. To this end event models have been defined which capture the spatio-temporal meaning of locomotion verbs by qualitative predicates which must be conjunctively true for — say — a turn-off event to take place. Propositional event models have been shown to provide an effective means for computing natural language scene descriptions (NOVAK and NEUMANN 86) but cannot be used effectively for the inverse: generating visualizations from verbal descriptions. For this we need an explicit representation of the spatio-temporal knowledge associated with verbs. As there is no unique trajectory which would serve for this purpose, we are faced with the problem of devising a representation for ‘typical’ trajectories, possibly involving uncertainty, incompleteness and fuzzyness.