Lecture Notes in Artificial Intelligence 1195 Subseries of Lecture Notes in Computer Science Edited by .J G. Carbonell and J. Siekmann Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen Robert Trappl Paolo Petta (Eds.) Creating Personalities for Synthetic Actors sdrawoT suomonotuA Personality Agents Springer Series Editors Jaime G. Carbonell, Carnegie Mellon University, Pittsburgh, PA, USA J6rg Siekmann, University of Saarland, Saarbrticken, Germany Volume Editors Robert Trappl Paolo Petta Austrian Research Institute forArtificial Intelligence S chottengasse 3, A- 1010 Vienna, Austria E-mail: robert @ ai.univie.ac.at paolo@ ai.univie.ac.at Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Creating personalities for synthetic actors : towards autonomous personality agents / Robert Trappl ; Paolo Petta (ed.). - Berlin ; Heidelberg ; New York ; Barcelona ; Budapest ; Hong Kong ; London ; Milan ; Paris ; Santa Clara ; Singapore ; Tokyo : Springer, 1997 (Lecture notes in computer science ; 5911 : Lecture notes in artificial intelligence) ISBN 3-540-62735-9 NE: Trappl, Robert Hrsg.; GT The cover illustration is based on Figure 7, page 901 (contribution by Barbara Hayes-Roth, Robert yon Gent, and Daniel Huber) CR Subject Classification (1991): 1.2, 1.3, J.5, H.5.1 ISBN 3-540-62735-9 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer -Verlag. Violations are liable for prosecution under the German Copyright Law. (cid:14)9 Springer-Verlag Berlin Heidelberg 1997 Printed in Germany Typesetting: Camera ready by author SPIN 10548995 06/3142 - 5 4 3 2 1 0 Printed on acid-free paper Preface Progress in computer animation has attained such a speed that computer- generated human faces and figures on ubiquitous screens wilt soon be indistin- guishable from those of real humans. The potential both for scripted films and real-time interaction with users is enormous. However, in order to cope with this potential, these faces and figures must be guided by autonomous "personality agents". But what is the current state of the art in this far less visible domain of research? To answer this question, we decided to organise a two-day workshop which took place at the premises of the Austrian Research Institute for Artificial Intel- ligence in Vienna. To this workshop we invited scientists known for their work in this area, ranging from computer animation specialists, computer scientists, and specialists in AI, to psychologists and philosophers. This book is the collection of the papers which were presented as first drafts at that occasion. The lively discussion was taped, transcribed, and sent to the participants. We are extremely grateful to them for the pains they took to con- sider the comments, to enhance their papers with new material, and to submit them on time. At least one author of each of the papers presented in this book was present at our workshop, with the sole exception of Barbara Hayes-Roth, who regrettably had already accepted other duties when the date of the event was set. We want to thank all our colleagues at the Austrian Research Institute for Artificial Intelligence, especially Dr. Bernhard Pfahringer, who was not only very instrumental in the scientific preparation and organization of the workshop but also proposed its name, and thus the title of this book, and to Gerda Helscher, MA, for preparing the transcripts of the tape recordings and for her help in the finalization of this volume. Both Prof. JSrg Siekmann, one of the LNAI series editors, and Mr. Alfred Hofmann of Springer-Verlag were ideal partners in our endeavor. We are grateful to Dr. Erhard Busek, then Federal Minister for Science and Research, for his support of an agent project of which the workshop was an integral part, and to Dr. Ren~ Fries from the same Ministry, who took great care to keep the project going. We hope we will get to be as grateful to Dr. Caspar Einem, now Federal Minister for Science and Transport, since we have applied to his Ministry for its financial support of a research project on the development of personality agents. It is our hope that this book will serve as a useful guide to the different approaches to create personalities for synthetic actors and to stimulate future research in this promising field. Vienna, Austria, January 1997 Robert Trappl and Paolo Petta Contents Why to Create Personalities for Synthetic Actors .................... 1 Paolo Petta and Robert Trappl Dressing Virtual Humans ............................................... 9 Nadia Magnenat Thalmann and Pascal Volino Autonomous Virtual Actors Based on Virtual Sensors ............. 25 Daniel Thalmann, Hansrudi Noser, and Zhiyong Huang Towards Personalities for Animated Agents with Reactive and Planning Behaviors ............................... 43 Norman I. Badler, Barry D. Reich, and Bonnie L. Webber IMPROV: A System for Real-Time Animation of Behavior-Based Interactive Synthetic Actors ..................... 58 Athomas Goldberg Multi-level Control for Animated Autonomous Agents: Do the Right Thing... Oh, Not That ................................. 74 Bruce Blumberg and Tinsley Galyean Tools for an Interactive Virtual Cinema ............................. 83 David Blair and Tom Meyer Acting in Character .................................................... 92 Barbara Hayes-Roth, Robert van Gent, and Daniel Huber Some Requirements and Approaches for Natural Language in a Believable Agent ........................ 113 A. Bryan Loyall Personality Parameters and Programs .............................. 120 Dave Moffat What Sort of Control System Is Able to Have a Personality? ..... 166 Aaron Sloman Personalities for Synthetic Actors: Current Issues and Some Perspectives ................... ~ ........... 209 Paolo Petta and Robert Trappl Personalities for Synthetic Actors: A Bibliography ................ 219 Paolo Petta Index ................................................................... 45 Why to Create Personalities for Synthetic Actors Paolo Petta, Robert Trappl Austrian Research Institute for Artificial Intelligence*, Vienna, and Department of Medical Cybernetics and Artificial Intelligence, University of Vienna {paolo, robert @ai.univie.ac. } at 1 Introduction The last decade has seen not only the impressive developments from the first brief appearance of a computer-generated character in a feature film ,2 over the first digital performer in a movie ,3 all the way to the first fully computer-generated theatrical motion picture4; synthetic actors have also conquered demanding real- time domains from the (co-)moderation of live television shows to the evaluation of complex real-world scenarios modded in detail in advanced simulators. Over this timespan the rendering of the external appearance has progressed from the basic geometrical and physical modeling and animation over the inclu- sion of dependencies on the simulated environment to event- and object-based behaviours. As synthetic actors were placed in simulated worlds of growing com- plexity, an obvious requirement that came about was to make them perform in these worlds: for convincing realistic behaviour they ought to perform as if they were gathering information as it became available to them and they should interact with objects -- e.g., tracking, avoiding, manipulating,...them -- in plausible ways. It soon became evident that the first approach of setting out from an omniscient position where all the data of the simulated world was made available to programs driving the actors was very inefficient in a number of re- spects. For instance, those pieces of information that could actually be accessed given the actor's current state in the world had to be laboriously identified and progressively singled out. Similarly, all sequences of actions making up basic behaviours of an agent had to be explicitly prescribed: take as an example an object "catching the eye" of the actor which would then turn around to inspect it more closely. Instead, taking the step forward and equipping actors with vir- tual sensors, turning their bodies into virtual effectors, and using these devices to guide the execution of behaviours proved to be a superior alternative, with * The Austrian Research Institute for Artificial Intelligence is supported by the Aus- trian Federal Ministry of Science, Transport, and the Arts. Young Sherlock Holmes, directed by Barry Levinson, visual effects by Industrial Light & Magic. 3 Casper, directed by Brad Silberling, visual effects by Industrial Light & Magic. 4 Toy Story, directed by John Lasseter, visual effects by Pixar. the additional benefit of introducing a well defined interface between the actor and its environment, allowing to deploy actors in different worlds .5 However, all of these external aspects still cover only part of what makes up an actor: as entities that "act a part or take part in an affair", synthetic actors also have to be endowed with internal capabilities, both affective and cognitive--i.e., in concordance with the following definition by OshersonlO Cognitive Science is the study of human intelligence in all of its forms, from perception and action to language and reasoning. The exercise of intelligence is called cognition. Under the rubric of cognition fall such di- verse human activities as recognizing a friend's voice over the telephone, reading a novel, jumping from stone to stone in a creek, explaining an idea to classmates, remembering the way home from work, and choosing a profession. Cognitive processes are essential to each of these activities; indeed, they are essential to everything we do. p.ix From the point of view of the different applications, the actual require- ments calling for these internal faculties can vary widely: feature film animators might just want to be able to instruct their puppets using high level commands (e.g., "walk leisurely across the room") or be assisted in the more delicate and ephemeral task of ensuring consistent patterns of behaviour of a virtual actor over a longer time span; in other settings, achieving e.g. various degrees of agent autonomy can play an essential role in providing effective assistance to users or in achieving credible and interesting performance in virtual environments. Espe- cially in this latter context, the inevitable limitations in terms of incompleteness of information about the world that can be perceived by the actor via its vir- tual sensors as well as restrictions in terms of available resources 6 become of particular relevance. Different approaches have been taken to tackle this prob- lem. Some favour runtime performance and acceptance by a broad public even though adhering to scientifically disproved preconceptions of folk psychology -- as in the Oz project directed by Joseph Bates at Carnegie Mellon University. Others place the emphasis on the soundness of the implementation of a cognitive theory resting on plausible first principles, as illustrated in the chapters by Dave Moffat and Aaron Sloman in this volume. Also, it must not be overlooked that as the exterior architecture of agents and their environments become more com- plex and structured, control of these worlds necessarily also becomes of a more indirect, higher-level nature, and consequently has to rely on the availability of interior mechanisms of corresponding functionality. All of these circumstances thus give rise to concerns regarding the characteris- tics of the high-level behaviour of the synthetic actors, both in terms of reliability 5 A related effort which gained a certain publicity was the work carried out by an undergraduate student at the MIT Media Lab who interfaced their ALIVE system to the game world of DOOM. See 12, 3 for a more detailed discussion of these topics. Computational resources and timing constraints, especially for characters '%hrown" into worlds where they have to act in real time. and predictability--ensuring that the performance remains within anticipated or prescribable limits--and interestingness--the manifestation of individual distin- guishing marks due to which the actors become recognizable individuals: in other words, the design of tailored personalities. The papers included in the present collection span the range from theoretical to applied and application-specific as- pect.s, laying a foundation for the creation of multifarious and more believable synthetic actors. From its very beginning the event at which these works were presented was characterized by an informal atmosphere which engendered an- imated exchanges of opinions. The results of these discussions are reflected in the contents of the revised versions of the workshop papers, tn the following we will briefly summarize some key points presented in the single contributions, deferring a principled discussion of the subject domain to the final chapter of this volume. 2 Contents 2.1 Virtual Humans An essential point of emphasis of the research supervised by Nadia Magnenat- Thalmann and Daniel Thalmann at the MIRALab in Geneva and the EPFL in Lausanne is the pursuit of the highest qualitative standards (i.e., photorealism) of the appearance of their creations: achievement of flexible real-time behaviour is subordinated to this primary goal. Consequently, special consideration is given e.g. to the simulation of different surfaces, such as clothes made of different fabrics and materiMs, or human hair and skin 7. Similarly, single courses of actions--such as the grasping of objects or the facial expressions--are modeled in accurate detail 12. These latter procedures also rely on the last focus of re- search to be mentioned here, namely the synthetic sensing of the environment. In particular this includes the virtual vision, tactile, and audition, which contribute decisively to a "natural" behaviour of the virtual actors. Most fO this research is strongly application-oriented, with telecooperation--following the maxim that the realistic representation of Avatars 7 has a crucial impact of the acceptance of the system--and the entertainment industries (e.g., the co-moderation of prime time shows) as important application areas. The research efforts presented by Bonnie Webber and Norman Badler 1 also direct their attention on realism: the work centered on Jack@ focuses on ergonomic (e.g., of complex controls systems such as a helicopter cockpit) and lo- gistic (e.g., in military domains) evaluations of scenarios. Jack comprises a whole family of variations on a basic theme, the biomechanical simulation of a human body in real time. The overall design of Jack follows the three-level-architecture which was popularized in robotic research: on the lowest level a sense-control- act loop provides the interface to the environment and ensures robust "reactive" 7 This commonly adopted designation for figures representing human users in cy- berspace was first introduced in the Habitat System .9 behaviour, making Jack capable of e.g., "stumbling" over obstacles s. At the highest level, the behaviour of all instances of Jack in one running simulation are centrolied by parallel transition net~orks, in which all possible courses of action for a given scene are defined in an abstract manner (roughly comparable to Roger Schank's scripts): to solve occurring partial problems, specialized prob- lem solving methods can be activated as required (e.g., heuristics and planning procedures to chose a hiding place in the context of a game of hide-and-seek). At New York University's Media Research Lab (NYU MgL) an alternative approach is taken for a similar subject 4. In the framework of a procedural animation technique, single movements of human figures are defined along with parameter settings for all possible transitions between these actions. As eval- uation criterion for this latter step the assessment of the "naturalness" of the resulting impression as given by human experts was drawn up. This exact mod- eling is subsequently softened by the superposition of stochastic noise functions, so that each repetition of a single movement is performed with slight variations; but also when at a standstill, the joints of the figures thereby feature a slight "unrest", which contributes to the realistic appearance. Given that in this way the single movements can be recalled in an efficient manner, it is possibie to control the whole animation at ~ higher level in real time. This functionality is provided by behavioural scripts, which define the "vocabulary" of each object (actors as well as any other entities comprised in the scene). Instead of equipping each actor e.g. with comprehensive knowledge about all possible actions, the respective relevant information is thereby stored in a distributed fashion (a tea cup "knows", that it can be sipped from, a ve- hicle knows that it can be mounted and driven, etc.). Among the opportunities arising from this approach, it is also easily possible to achieve the effect of a stepwise acquisition of skills in mastering recurrent tasks by instantiating and updating variables stored with the given actor. The considered application sce- narios for this technology include the interactive design of virtual choreographies and virtual sociM environments akin to graphicM Internet MUD (Multi-User Di- mensions) and their more recent, commercial variants. 2.2 Virtual Ethology Bruce Blumberg and Tinsley Galyean 3 present further developments of the Hamsterdam architecture which is being applied in the cont, ext of the ALIVE (ArtificiM Life Interactive Video Environment) project at the MIT Media Lab. ALIVE implements the metaphor of a "magic mirror" which allows users to immerse in a virtual environment without requiring them to wear any special equipment: this mirror is realized as a wide-area screen on which the image of the user as acquired by video cameras is displayed after being composited with As n~ interesting side note, the use of reinforcement learning Mgorithms was found to be of indispensable help in the adjustment of the numerous p~r~meters of this complex numerical system.