IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 347 The Amulet Environment: New Models for Effective User Interface Software Development Brad A. Myers, Member, IEEE, Richard G. McDaniel, Robert C. Miller, Alan S. Ferrency, Member, IEEE Computer Society, Andrew Faulring, Bruce D. Kyle, Member, IEEE Computer Society, Andrew Mickish, Alex Klimovitski, and Patrick Doane Abstract—The Amulet user interface development environment makes it easier for programmers to create highly-interactive, graphical user interface software for Unix, Windows and the Macintosh. Amulet uses new models for objects, constraints, animation, input, output, commands, and undo. The object system is a prototype-instance model in which there is no distinction between classes and instances or between methods and data. The constraint system allows any value of any object to be computed by arbitrary code and supports multiple constraint solvers. Animations can be attached to existing objects with a single line of code. Input from the user is handled by “interactor” objects which support reuse of behavior objects. The output model provides a declarative definition of the graphics and supports automatic refresh. Command objects encapsulate all of the information needed about operations, including support for various ways to undo them. A key feature of the Amulet design is that all graphical objects and behaviors of those objects are explicitly represented at run-time, so the system can provide a number of high-level built-in functions, including automatic display and editing of objects, and external analysis and control of interfaces. Amulet integrates these capabilities in a flexible and effective manner. Index Terms—Toolkits, user interface tools, user interface development environments, user interface management systems (UIMSs). —————————— F —————————— 1 INTRODUCTION CREATING user interface software has proven to be very easier, while supporting flexible experimentation with new difficult and expensive because it is often large, com- styles. Amulet includes many design and implementation plex, and challenging to implement, debug, and modify. innovations including new models for objects, constraints, One study found that an average of 48 percent of the appli- animation, input, output, commands, and undo. cations’ code is devoted to the user interface, and that about In addition to incorporating innovations into its own de- 50 percent of the implementation time is devoted to the sign, Amulet has an open architecture to enable user interface user interface portion [24]. Most of today’s toolkits and in- researchers and developers to easily investigate their own teractive tools are still quite hard to use and lack flexibility. innovations. For example, Amulet is the first system that For example, to create new kinds of “widgets” (sometimes supports multiple constraint solvers operating at the same called “controls”) such as a scroll-bar with two handles, or time so that researchers might be able to easily investigate to add support for gesture recognition, is quite difficult new kinds of constraint solvers. The undo model also sup- with today’s tools. ports new designs. The widgets are implemented in an open Amulet, a new user interface development environment fashion using the Amulet intrinsics so that researchers can for C++ that runs on X/11, Windows 95, Windows NT, and replace or modify the widgets. The goal is that researchers the Macintosh, facilitates user interface research and devel- will only have to implement the parts that they are interested opment. Amulet aims to make the design, prototyping, im- in, relying on the Amulet library for everything else. In addi- plementation and evaluation of user interfaces significantly tion, we aim for Amulet to be useful for students and general developers. Therefore, we have tried to make Amulet easy to ———————————————— learn and to have sufficient robustness, performance and • B.A. Myers is with the Human Computer Interaction Institute, School of documentation to attract a wide audience. Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. Amulet, which stands for Automatic Manufacture of Us- E-mail: [email protected]. • R.G. McDaniel, R.C. Miller, A. Faulring, B.D. Kyle, and P. Doane are able and Learnable Editors and Toolkits, is implemented in with the Computer Science Department, Carnegie Mellon University, 5000 C++. Amulet is based on our group’s substantial experience Forbes Ave., Pittsburgh, PA 15213. from creating the Garnet user interface development envi- • A.S. Ferrency is with Luce McQuillin Corp., 5001 Baum Blvd., Pitts- ronment [21], which was implemented in Common Lisp. burgh, PA 95217. • A. Mickish is with Vanderbilt University, Eskind Biomedical Library, Amulet brings to C++ the dynamic and rapid user interface Nashville, TN 37232. design and implementation capabilities that Garnet pro- • A. Klimovitski is at Dornacher Strasse 1, D-85622 Feldkirchen-Munich, vided in Lisp, while adding many new capabilities. Germany. Since Amulet provides a structure for implementing the Manuscript received Nov. 12, 1996; revised May 14, 1997. application-specific parts of a user interface, it can properly Recommended for acceptance by J.D. Dannon. be called an application framework [18]. It is clearly much more For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number 101264.0. 0098-5589/97/$10.00 © 1997 IEEE 348 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 than a “toolkit,” which generally refers to a collection of wid- • Students. By aiming for Amulet to be useful to stu- gets such as scroll-bars and buttons. A key reason that Amu- dents, we are continually striving to make Amulet let provides a higher level of support than other systems is easier to learn. The success of SUIT [26] and Micro- that all of the user interface objects are available at run-time soft’s Visual Basic show that it is possible to provide for inspection and manipulation through a standard proto- useful functionality in a way that is easy to learn, but col. This allows high-level, built-in utilities to be provided unlike those other systems, Amulet provides a natural which, in other toolkits, must be reimplemented for each ap- growth path to the complete fully-functional system. plication. For example, the graphical selection-handles widget Amulet has been used in at least ten courses at seven can get the list of graphical object, and move and resize the universities, which have provided feedback used in selected object, using a standard protocol, even if the objects refining the interface. are custom-created and application-specific. Other facilities • General Developers. We also want Amulet to be use- provided by Amulet include undo and operations like cut, ful for general user interface software construction. copy, paste, save and load. This paper provides an overview For this reason, Amulet runs on X/11, Windows 95, of all the parts of the Amulet system. Windows NT, and the Macintosh. We also provide a high level of robustness, documentation and per- formance. A complete reference manual including a 2 GOALS tutorial is available [19]. A few commercial products An important research objective of the Amulet project is to are even being built with Amulet. provide high-level support for the insides of application programs. Conventional toolkits like the Macintosh Tool- box and Motif provide a collection of widgets like menus, scroll-bars, buttons and text input fields. However, for graphical applications like drawing editors, CAD pro- grams, visual language editors, visualizations, and charting programs, most of the programming for the user interface deals with the contents of the graphic windows, which do not contain any widgets. Consider the user interface shown in Fig. 1. Other toolkits will provide the menu-bar at the top, and possibly the palette on the left, but they provide no help with the main area where the circuits are drawn. In- stead, programmers must code directly at the window manager level without much support. In contrast, Amulet provides high-level support for these kinds of graphical applications, including: • Automatically redrawing the graphics, • Constraints that automatically keep the wires at- tached to the circuit elements, • Widgets such as selection handles which make inter- active behaviors easy to implement, and, • Built-in editing commands, such as cut, copy, paste, to- top, to-bottom, save, load and undo, that can often be used directly from the library without modification. Fig. 1. A simple circuit design program created with Amulet. These facilities are made possible because Amulet’s novel models make the components and structure of the user in- terface and application visible and manipulatable by stan- 3 EXAMPLE: A CIRCUIT DESIGNER dard utilities. Suppose you wanted to build an application like the circuit We distribute Amulet for use by others because we feel design program of Fig. 1. This program should work in the this will help to demonstrate that the innovations in Amulet standard way: are sound and effective, and will hopefully facilitate tech- nology transfer. Amulet is in the public domain and can be • Clicking on the palette on a circuit element and then used for free. Version 1 of Amulet was released in July 1995, in the work window should create an object of that Version 2 was released in May 1996, and Version 3 was re- type. Gridding should be used to help lay out the ob- leased in March, 1997. To get Amulet, including the complete jects neatly. source code, visit http://www.cs.cmu.edu/~amulet or • The wires should be dragged from the source gate to send mail to [email protected]. the destination gate with the usual “rubber band” We want to make Amulet useful for: feedback. • Selection handles should appear on objects when they • Researchers. Over 30 research projects all over the are selected, and objects can be dragged (moved) in the world are already using Amulet, including a number standard way. Multiple objects should be selected in of thesis projects at all levels. the standard way, either by holding down the SHIFT MYERS ET AL.: THE AMULET ENVIRONMENT: NEW MODELS FOR EFFECTIVE USER INTERFACE SOFTWARE DEVELOPMENT 349 key while clicking on objects or dragging out a region. for example the slot named Am_LEFT1 might hold the value The wires must stay attached when elements are 10. Slots are similar to member variables or instance variables dragged. Resizing the gates should not be allowed. in other object systems but with some important differences: • All the standard editing operations should be sup- they can be dynamically created at run time, they are dy- ported, such as cut, copy, paste, clear, clear-all, select- namically typed (so the same slot can hold an integer at one all, to-top, and to-bottom. When a gate is deleted, any time and a string later), they can hold methods or data, and attached wires must be removed. inheritance is dynamically determined by whether an in- • All operations must be undoable. stance has a local value for the slot. • Circuit diagrams can be saved to a file and loaded A new object is created by making an instance of another back in. object, which is called the prototype. An instance starts off inheriting all of its slot values from the prototype, and the In addition, this program should provide some advanced slots can then be set with new values. Alternatively, an object features: can be copied, in which case it immediately gets a copy of all • The program should simulate the operation of the cir- values in the original. For example, the following creates an cuit by showing the values calculated by the gates. And_Gate as an instance of the built-in Am_Bitmap object, The input nodes can be toggled and the outputs and then sets the Am_IMAGE slot to the appropriate picture. should display the correct values. Animations should Am_Object And_Gate = Am_Bitmap.Create() be used to make the execution more understandable. .Set(Am_IMAGE, and_bitmap_image); • Instead of repeatedly going back to the palette, the user should instead be able to use gestures to create all Any object can serve as a prototype to create other objects. the kinds of objects using the right mouse button (on There is nothing special about the objects in the Amulet li- the Macintosh, using the Option keyboard key while brary. For example, the following creates an instance of the pressing on the one mouse button). For a gesture, the And_Gate and then puts it in a particular place: path of the mouse is important, not just its start and Am_Object new_gate = And_Gate.Create() end position. For example, the user can draw an “o” .Set(Am_LEFT, 10) to make an OR gate, an “A” to make an AND gate, a .Set(Am_TOP, 43); “>“ to make a NOT gate, a line to make wires, etc. We allow the Set operations to be chained together, but The dot gesture should perform undo, to make it easy the previous code could instead be written: to correct errors. • The application should run on Unix, Windows NT, Am_Object new_gate = And_Gate.Create(); new_gate.Set(Am_LEFT, 10); Windows 95, and the Macintosh and use widgets new_gate.Set(Am_TOP, 43); with an appropriate look-and-feel on each platform. To make objects appear on the screen, they are simply With most toolkits, such as Motif, Microsoft Foundation added as a part to a window which is added to the screen. Classes, MetroWerks PowerPlant, Visual Basic, Borland’s Because all the graphics on the screen are represented by Delphi, Java AWT, etc., the code for this application would objects in memory, changes to the screen are accomplished be tens of thousands of lines of code. However, using simply by setting the slots of objects with new values. For Amulet, this entire application requires about 850 lines of example, the new_gate could be made red by executing the C++ code, due to Amulet’s high-level features such as: statement: • Graphical objects with automatic refresh. • Interactors that handle standard behaviors. new_gate.Set(Am_LINE_STYLE, Am_RED); • Command Objects that handle editing operations. When slots are set, Amulet automatically redraws the • Constraints that maintain relationships among objects. object, as well as any other objects that overlap it, so the • Gesture Recognition as a built-in kind of interactor. screen is appropriately updated. • Animations, as a special form of constraint. The following is the complete “hello world” program in Amulet that displays a string, and redraws the string if the The rest of this paper will describe these and other fea- window becomes covered and then uncovered. This pro- tures of Amulet that make creating interactive applications gram would be about two pages long in Motif. easier. #include <amulet.h> 3.1 Graphical Objects void main (void) { An important goal is to make Amulet easy to learn and use Am_Initialize (); //initialize Amulet Am_Screen for developers, even though it has a large number of features. //add a window to the screen using all of the default values Therefore, we have concentrated on giving Amulet a uniform .Add_Part (Am_Window.Create () structure based on a few simple concepts. The main concept .Add_Part (Am_Text.Create () //create a text object and is that everything in Amulet is represented as an object which add it to the window has a set of slots. The objects use a prototype-instance object .Set (Am_TEXT, (cid:147)Hello World!(cid:148)))); //set the string system implemented on top of C++. In a prototype-instance Am_Main_Event_Loop (); //display the window and then handle all input events object system, there is no distinction between classes and Am_Cleanup (); //clean up Amulet instances: any object can be used as a prototype for other } objects. A slot has a name and can hold a value of any type, 1. All exported names in Amulet start with “Am_”. 350 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 3.2 Interactors standard Get method that accesses values of slots can tell To make objects respond to input, an instance of an interac- whether it is being invoked from inside of a formula, and if tor object is attached to the graphics. Different types of in- so, in addition to returning the value, it also sets up a de- teractors handle different types of behaviors. For example, pendency. Then, whenever a slot’s value changes, Amulet to make the new_gate object be movable, a move-grow knows which constraints depend on that value, and can interactor can be attached to it: cause the constraints to recalculate. For example, the left of the scrolling region in Fig. 1 stays to the right of the tool new_gate.Add_Part(Am_Move_Grow_Interactor.Create()); palette, even if the size of the tool palette changes, by using By default, the interactor starts when the left mouse button the following formula in its Am_LEFT slot. is hit over the object, but this is easily changed by setting a // define a formula called right_of_tool_panel_formula which returns an int slot of the interactor. The following will allow newgate to Am_Define_Formula(int, right_of_tool_panel_formula) { be selected with the right mouse button: // 5 pixels away from the right of the tool_panel return (int)tool_panel.Get(Am_LEFT) + new_gate.Add_Part(Am_Choice_Interactor.Create() (int)tool_panel.Get(Am_WIDTH) + 5; .Set(Am_START_WHEN, (cid:147)right_down(cid:148))); } ... When an instance is made of an object that has parts, scrolling_window.Set(Am_LEFT, right_of_tool_panel_formula); then Amulet makes instances of all the parts as well. Wid- gets, like buttons and scroll-bars, contain graphical and Am_Define_Formula is a macro which defines a for- interactor objects as parts, but the designer can make an mula object which returns the type of its first argument instance of the widget in the same way as instances of (here int) where the object is named with the second ar- primitive objects are created: gument (here right_of_tool_panel_formula). The Am_Object my_button = Am_Button.Create () macro then defines a procedure to be executed by the for- .Add_Part(Am_COMMAND, Am_Quit_Command.Create() mula, and the code following the macro is used as the pro- .Set (Am_LABEL, (cid:147)Goodbye, world!(cid:148))))); cedure’s body. 3.5 Gesture Recognition One of the built-in types of interactor objects supports ges- ture recognition. Using an interactive tool called Agate, the designer gives about 10 examples of each gesture desired in the interface and associates a string name with each ges- All of the widgets in the Amulet library are defined with ture. Then, in the program, the designer can associate each multiple look and feels, so a program can be written using name with an operation. Built-in commands provide a Amulet, and it can then be compiled on Unix, Microsoft standard interface between the “normal” direct manipula- Windows or the Macintosh without editing the source code, tion commands, for example to create and delete objects, and it will have the correct look and feel for the target plat- and the gestures. For example, part of the code to handle form. gestures in the circuit program is: Am_Object gesture_reader = 3.3 Command Objects Am_Gesture_Interactor.Create((cid:147)gesture_reader(cid:148)) When interactors or widgets are operated by the user, in- //gestures work when you hold down the right mouse button stead of calling “call-back procedures” as in other toolkits, .Set (Am_START_WHEN, (cid:147)any_right_down(cid:148)) they allocate an instance of a Command object, and execute //an object to show the gesture while in progress (interim feedback) .Set (Am_FEEDBACK_OBJECT, gesture_feedback) the method in its Am_DO_METHOD slot. The method can be //the gesture classifier read from a file that was created using the Agate tool any C++ code, which so far is pretty much like using a call- .Set (Am_CLASSIFIER, gc) back. However, command objects also have slots to support //first, a command to handle gestures that are not recognized Undoing, Redoing and Repeating, as well as enabling and .Add_Part(Am_COMMAND, disabling (greying out), and providing help. Many com- Am_Gesture_Unrecognized_Command.Create()) mand objects are available in the Amulet library and can //now, a list of commands, one for each of the gestures defined often be used without change. The Goodbye-World code by example .Set (Am_ITEMS, Am_Value_List () above uses the built-in Quit command, so my_button can //first, the gesture for the “And” gate be added to a window to form the complete “goodbye .Add (Am_Gesture_Create_Command.Create() world” program, which exits when the button is pressed. .Set (Am_LABEL, (cid:147)and(cid:148)) //string defined in Agate for this gesture 3.4 Constraints .Set (NEW_OBJECT_PROTO, and_proto) Another important feature of Amulet is the support for con- //object to create straints, which are relationships that are declared once and .Set (Am_CREATE_NEW_OBJECT_METHOD, then maintained by the system. Amulet supports multiple gesture_creator)) //next, the gesture for the “OR” gate kinds of constraints, but the main kind is formula constraints, .Add (Am_Gesture_Create_Command.Create() which act like spreadsheet formulas. When put into a slot of .Set (Am_LABEL, (cid:147)or(cid:148)) an object, formulas compute the value of the slot based on .Set (NEW_OBJECT_PROTO, or_proto) slots of the same object or other objects. Formula constraints .Set (Am_CREATE_NEW_OBJECT_METHOD, can contain arbitrary C++ code. This works because the gesture_creator)); MYERS ET AL.: THE AMULET ENVIRONMENT: NEW MODELS FOR EFFECTIVE USER INTERFACE SOFTWARE DEVELOPMENT 351 3.6 Animations “Gilt” Interface Builder Amulet has a flexible constraint system, which allows new Widgets kinds of constraint solvers to be created. We used this facility Command Objects to create an animation constraint solver. When an animation “Interactors” “Opal” constraint is attached to a slot and the slot changes value, the Input Handling Output Handling animation constraint removes the new value, resets the slot Constraint System with the old value, and smoothly sets the slot with values interpolated from the old value to the new value. For exam- “ORE” Object System ple, to move the little red numbers along with wires in the “Gem” circuit program, the following code is used: Graphics & Input Layer //first create the animator object Windows NT, Windows 95, X/11, or Macintosh Am_Object number_animator = Am_Interpolator.Create (); //now create the prototype number to be moved Am_Object animation_proto = Fig. 2. The overall structure of the Amulet system. Am_Text.Create((cid:147)animation_proto(cid:148)) .Set(Am_TEXT, (cid:147)0(cid:148)) model, as was done in Garnet [21], Amulet’s widgets are .Set(Am_FONT, small_font) implemented more efficiently using Gem-level drawing .Set(Am_LINE_STYLE, Am_Red) routines. .Set(Am_LEFT, Am_Animate_With (number_animator)) .Set(Am_TOP, Am_Animate_With (number_animator)); 4.2 Object System //when ready to start an animation, first create an instance of animation_proto, //and set the position to one end of the wire with no animations The Ore (Object Registering and Encoding) layer of Amulet Am_Object anim = animation_proto.Create(); implements a prototype-instance object system on top of anim.Set(Am_LEFT, (int)line.Get(Am_X1), C++. In a prototype-instance object system, there is no dis- Am_NO_ANIMATION); tinction between classes and instances: every object can be anim.Set(Am_TOP, (int)line.Get(Am_Y1)-10, Am_NO_ANIMATION); used as a prototype for other objects. Slots of the prototype //then set the left and top again, to the other end of the wire, but this time letting can be inherited by instances, so that changes to the proto- //the animations make the object move smoothly type’s slot will be seen by any instances which do not over- anim.Set(Am_LEFT, (int)line.Get(Am_X2)); ride it. We felt that we needed to create a new object system anim.Set(Am_TOP, (int)line.Get(Am_Y2)-10); due to a number of deficiencies we found with C++: • Primarily, we needed a system that was more dy- 4 DETAILS OF THE DESIGN namic. For the rapid prototyping required for user in- The Amulet toolkit is divided into a number of layers (see terface work, being able to dynamically change the Fig. 2). These layers include an abstract interface to the slots of objects, and the types of the contents of the window managers, novel models for objects, constraints, slots is important. input, output, and commands, a set of widgets, and an in- • We needed to be able to determine the complete in- teractive layout tool. The following sections describe the formation about an object at run time. This has par- overall design of each of these. tially been addressed by the new Run-Time-Type- Information (RTTI) interface for C++, but this was not 4.1 Gem: Abstract Interface to the Window Managers available when we started, and still is not universally Amulet provides a portable interface to various window supported. managers called Gem, which stands for the Graphics and • The ability to override a method in an instance, in- Events Manager. Gem uses ordinary C++ objects and stead of requiring a new sub-class as in C++, makes mechanisms to provide a simple graphics and input inter- the interface to Amulet more uniform. face used by the rest of Amulet. Any code written using All of these features are elaborated in the following Gem will port to different windowing systems (Windows sections. 95 or NT, Macintosh, or X/11) without change. Most other toolkits and frameworks only provide an interface at the 4.2.1Slots Gem level, or require that all graphics use the underlying Programming with a prototype-instance object system is a low-level window manager drawing interface. However, quite different style than conventional object-oriented lan- typical Amulet users never see the Gem interface, since the guages. Much of the code is devoted to defining the slots and higher-level parts of the Amulet framework provide access default values for prototype objects, and then creating in- to the same capabilities in an easier to use way. The circuit stances, possibly overriding some slots. For example, to cre- program of Fig. 1 did not require any programming at the ate an instance of the zero_one_proto object, which is the Gem layer. We export the Gem interface for advanced prototype for the zero-one buttons, the program just does: Amulet users. If the programmer wants to make something Am_Object new_obj = zero_one_proto.Create () very efficient, calling Gem directly may be appropriate. For .Set (Am_LEFT, new_obj_left) .Set (Am_TOP, new_obj_top); example, although widgets such as buttons and scroll-bars can be implemented using the high-level Opal output Slots can be set with any type of value: 352 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 obj.Set(Am_LEFT, 40); the instance, so later changes to the prototype do not affect obj.Set(Am_TEXT, (cid:147)Hello(cid:148)); the instance. A slot can be declared local, so the slot does obj.Set(OTHER_OBJ, and_gate1); not appear in the instance at all. This is useful for slots that The object system is dynamic in that slots in objects can be hold information that is particular to the object. In the defi- added and removed from objects at run time, and the types nition of the circuit_object_proto, the slots that will in slots can also change. Amulet performs run-time type hold the connected input and output wires are declared to checking if a type is declared for the slot. be local so that when copies or instances are made of an To allow the same Set and Get to work for all types in object, every instance of a circuit element will have unique C++, we provide accessor and setting methods for the stan- wires. Therefore, if the user duplicates a circuit element, dard built-in types, void (untyped) pointers, Amulet ob- Amulet ensures that the copy will start off with its input jects, and a special class called a “Wrapper.” Any new C++ and output slots empty. type that the programmer wants to store into objects and The inheritance mechanism is an important distinction have type-checked can use Wrappers. Amulet can also from other prototype-instance object systems, such as SELF handle memory management for Wrappers using reference [4], in which all the slots are always copied into instances so counting. changes to prototypes never affect instances. Although C++’s overloading and type-conversion capabilities Amulet’s model requires slightly more overhead, we think it make the interface very convenient. For example, the is useful for prototyping to be able to change properties of Am_Object class defines a number of Set routines: prototypes and see the effect on all instances immediately. Am_Object Set (Am_Slot_Key key, Am_Wrapper* value); 4.2.3Methods Am_Object Set (Am_Slot_Key key, void* value); An important feature of Amulet’s object system is that there Am_Object Set (Am_Slot_Key key, int value); is no distinction between methods and data: any instance can Am_Object Set (Am_Slot_Key key, float value); Am_Object Set (Am_Slot_Key key, char value); override an inherited method as easily as inherited data. In Am_Object Set (Am_Slot_Key key, const char* value); a conventional class-instance model such as SmallTalk or C++, instances can have different data, but only subclasses The compiler will choose the correct one based on which can have different methods. Thus, in cases where each in- type is actually used. Note that Set returns the original stance needs a unique method, conventional systems must object, allowing Sets to be cascaded, so the code above that use a mechanism other than the regular method invocation, sets the slots of obj could instead be: or create a new subclass and a single instance of that sub- obj.Set(Am_LEFT, 40).Set(Am_TEXT, (cid:147)Hello(cid:148)) class each time. For example, a button widget might use a .Set(OTHER_OBJ, and_gate1); regular C++ method for drawing but would have to use a different mechanism for the call-back procedure used when C++ does not allow overloaded functions to be chosen the user clicks on the button, since each instance of the based on the return type, but we were able to get around button needs a different call-back. In Amulet, the draw this by returning a special Am_Value type, which then has method and the callback use the same mechanism. In the type-conversion routines into the various primitive types. circuit code, we set a method into the tool panel so that This allows code like: whenever the user changes modes, the selection will be int i = circuit_object_proto.Get(Am_VALUE); cleared. This method is coded as follows: bool b = this_command.Get(Am_GROW_INACTIVE); //the next statement will work no matter what type is in the slot Am_Define_Method(Am_Object_Method, void, Am_Value v = tool_panel.Get(Am_IMPLEMENTATION_PARENT); clear_selection, if (v.type == Am_BOOL) ... (Am_Object /*cmd*/)){ my_selection.Set(Am_VALUE, NULL); In the last lines we use the special Am_Value type which } permits programmers to dynamically access and set the The method value can be put into a slot the same way as type and value. data is set into slots: 4.2.2Slot Inheritance tool_panel.Get_Part(Am_COMMAND) When an instance of an object is created, the slots that are .Set(Am_DO_METHOD, clear_selection); not specified inherit their values from the prototype ob- The Am_Define_Method macro works similarly to the ject’s. If a slot of the prototype is changed, then the value Am_Define_Formula macro. It creates a new method ob- also changes in all of the instances that do not override that ject of the type of the first argument (here property. For example, changing the Am_WIDGET_LOOK slot Am_Object_Method). This method object invokes a func- in the zero_one_proto will change the look in all in- tion which returns the type of the second argument (here stances, so it is easy to see what the circuit program will void), which has the name of the third argument (here look like when running on a Macintosh or Windows. How- clear_selection) and which takes as parameters the list ever, changing the Am_LEFT slot of the zero_one_proto in the fourth argument (which must be specified in an extra will not affect new_obj since it has a local value for Am- set of parentheses). The Am_Define_Method macro then LEFT. If the programmer does not want this behavior, then defines a function for the method to invoke, and the body Amulet allows the inheritance of each slot to be specified as of the function follows the macro. copy or local. When an instance is made for a slot with copy inheritance, the value is copied into a new slot created in MYERS ET AL.: THE AMULET ENVIRONMENT: NEW MODELS FOR EFFECTIVE USER INTERFACE SOFTWARE DEVELOPMENT 353 4.2.4Part-Owner Hierarchy Amulet’s constraint system (discussed below), any slot of The object system also implements a part-owner hierarchy. the item can depend on the list of values supplied. For ex- The owner is usually a “group” (aggregate) object or a ample, a list of strings might be supplied for a menu, a list window, and the parts are either other groups or primitives of objects for a palette, or a list of locations for a scatter plot. like rectangles, lines and text. However, we have found 4.2.5Other Features many other uses for the part-owner hierarchy that are in- Amulet’s object system also contains many other features dependent of graphical relationships. Supporting the part- that may be useful for programmers. Automatic memory owner hierarchy in the object system allows Amulet to pro- management using a reference counting scheme is used for vide “structural inheritance,” which means that when an objects and “wrappers” (used to “wrap” C++ types so they instance is made of a group which contains parts, the new can be put into Amulet objects with full type-checking). A instance will have instances of all the prototype’s parts, as flexible “demon” mechanism allows procedures to be at- shown in Fig. 3. This can be distinguished from “normal” tached to objects or slots for invocation when the slots slots, where just the value of the slot is copied, so if a nor- change. This mechanism enables Amulet to redraw objects mal slot contains a reference to an object obj1 which is not when their graphical properties change. Programmers can a part, then the instance will contain another pointer to the also create their own demons. Type checking of slots is same obj1 object, rather than having a new object created as supported by Amulet, so that programmers can declare that an instance of obj1. a slot can only hold a specific kind of value. A complete set The result is that programmers can create instances of of querying functions allows objects’ properties to be ex- any type of object without knowing whether it is a primi- amined at run-time. These are used by the debugging fa- tive or a group, and the system will make sure that the in- cilities described below, and they can also be useful for ap- stance has the same structure as the prototype. Changing plication programs. the xor_gate from a simple bitmap to a group containing a 4.2.6Performance bitmap and three lines as input and output ports only re- The main disadvantage of the prototype-instance model quired changing the prototype—none of the uses of the over the conventional class-instance model has been per- prototype needed to change. formance. When slots are accessed, the system must per- Some systems, such as FormsVBT [2] have hard-wired form a search through the object to see if the slot is there, some slots to inherit values from their prototypes and oth- and if not, it must search the prototypes up to the root. The ers to inherit from their owners. Because the constraint same search is needed for both method and data slots. Dy- mechanism is so easy to use and flexible in Amulet, it is namic type checking also adds some overhead. The forward sufficient to use constraints whenever slots should get their and backward pointers and space for the types add space values from their owners rather than from their prototypes. overhead. The SELF prototype-instance system [4] uses Amulet also provides a special form of group called an extensive compiler techniques to try to remove some of this Am_Map that computes its parts dynamically. The Am_Map search, but we have not found this necessary, and the per- object uses a constraint to build a list of parts, usually based formance of the Amulet system is quite good. There are no on a single prototype object, called the “item prototype.” noticeable delays for normal size programs on a variety of Typically, a list of strings, objects, commands, or something modern hardware platforms. For example, the circuit pro- else is provided, and the Am_Map creates an instance of the gram performs well on Unix, Macintosh and PC platforms. item prototype for each value in the list, setting a particular We optimized our previous Garnet prototype-instance slot of the instance with the corresponding value from the object system for speed by copying all values to all in- list. The programmer will define a constraint somewhere in stances, even if they were the same as the prototype’s. the item prototype that depends on the particular value However, this had a significant space penalty, so in Amulet copied from the list. Note that due to the flexibility of Am_Graphical_Object Am_Screen Am_Group Window_74 circuit_object_proto Prototype: Scrolling_Group_72 xor_proto xor_58 PICTURE PICTURE_59 Key: Is Instance Of OUTPUT_1_PORT OUTPUT_1_PORT_60 Is Part Of INPUT_1_PORT INPUT_1_PORT_61 INPUT_2_PORT INPUT_2_PORT_62 Fig. 3. The xor_proto object contains four parts: a bitmap called PICTURE, and output and input ports which are instances of lines. The xor- proto object is an instance of the circuit_object_proto object, which in turn is an instance of an Am_Group, which is an Am_Graphical_Object. When an instance is made of the xor_proto, Amulet automatically creates instances of each of the parts. If instances are not named, then Amulet makes up a name by appending a number. The instance, called xor_58, has been made a part of a scrolling group, which is part of a window, which is part of the screen, so it will be visible. 354 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 we only store the local slots, which in practice is only about Amulet. Constraint expressions can contain arbitrary C++ half of the slots. code, and the Get call checks to see whether it is inside a formula invocation or not. If so, in addition to returning the 4.2.7Discussion value of the slot, it also sets up a dependency link so that the There are many advantages of the prototype-instance model. constraint will be re-evaluated when the other slot’s value Having no distinction between classes and instances, or be- changes. tween methods and data, means that there are fewer con- Currently, Amulet does not use a preprocessor, so the cepts for the programmer to learn and a consistent mecha- syntax for specifying constraints is a little verbose. The macro nism can be used everywhere. Another advantage of the Am_Define_Formula creates a constraint object of the speci- prototype-instance object system is that it is very dynamic fied name using the code that follows. The constraint object and flexible. All of the properties of objects can be set and stores a pointer to the procedure to execute, the name of the queried at run time, and interactive tools can easily read and constraint for debugging and tracing, and the list of slots set these properties. In fact, most of today’s toolkits imple- used by and using this constraint. As an example, the fol- ment some form of “attribute-value pairs” to hold the prop- lowing constraint from the circuit program determines the erties of the widgets, but Amulet’s object system provides picture used for the zero-one buttons based on the value. significantly more flexibility and capabilities. Note that constraints can return any type. Here, the con- Amulet’s predecessor, Garnet, also used a prototype- straint is returning an image, which is Amulet’s machine- instance object system [20], as have a few other systems independent representation for a bitmap. such as SELF [4], Apple’s NewtonScript and General Am_Define_Image_Formula(zero_one_formula) { Magic’s MagicCap. Amulet’s design is more complete and int value = self.Get(Am_VALUE); flexible, and we fixed a number of problems we experi- if (value ==1) return one_image; else if (value == 0) return zero_image; enced with Garnet, including adding control over the in- else return question_image; heritance of slots, automatic management of a part-owner } hierarchy along with the prototype instance hierarchy, ... support for multiple constraint solvers, and a flexible de- obj.Set(Am_IMAGE, zero_one_formula); mon mechanism. Finally, it is worth pointing out that we The circuit program uses 24 custom constraints not are able to provide dynamic slot typing, a dynamic proto- counting all the constraints that are built into the objects type-instance system, and constraints in C++ without using themselves (for example, the built-in Am_Text object has a preprocessor or a scripting language. constraints in its width and height slots that compute its Garnet supported multiple inheritance, but we found it was dimensions based on the current string and font). Formulas not useful or necessary. In Amulet, we instead use the con- are set into slots using the standard Set. In the future we straint mechanism to copy values among objects, which pro- hope to add a preprocessor to support a conventional “dot” vides complete flexibility and control. Omitting multiple- notation for slot access (obj.slot) and to allow constraint inheritance has simplified much of Amulet’s implementation expressions to appear inside the Set instead of only as a leading to an easier to understand object creation procedure top-level procedure. and better efficiency when searching for slots. It also elimi- Slots are accessed the same way whether they contain nates the ambiguity and complexity for the programmer of constraints or constant values, and the code containing the resolving collisions of slot names from multiple prototypes. Get normally does not know how the value was calculated. Although designed to support the creation of graphical For example, the button widget does not care that the im- objects, many Amulet users have discovered that the proto- age was computed with a constraint. The object system is type-instance object system is useful for representing their tied into the graphics system using demons so that when- internal application data. The flexibility and dynamic nature ever the value of a slot changes, either because the pro- of the objects make them ideal when varied and changing grammer set it or due to constraints, the object will be re- data types are necessary. Amulet objects are somewhat like drawn automatically. “frames” used by artificial intelligence systems so AI appli- The Amulet constraint solver handles cycles in the con- cations may find the model useful. The constraint system is straints, so that a slot of object A can depend on a slot of also useful for maintaining data dependencies and consis- object B and vice versa. This is used in the circuit program tency in application-specific data structures. so that the zero-one buttons get their values from the input wires if present, and if not present then the value comes 4.3 Constraints from the toggle button. This is implemented, as shown in Amulet integrates constraint solving with the object system. Fig. 4, by having a constraint from the zero_one_proto to This means that instead of containing a constant value like a the button, and another constraint from the button to the number or a string, any slot of any object can contain an ex- zero_one’s value. In evaluating circular constraints, pression which computes the value. If the expression refer- Amulet simply goes around the cycle once and uses the old ences slots of other objects, then when those objects are value of any constraint that is already being evaluated. If changed, the expression is automatically re-evaluated. Thus, the programmer uses constraints that are consistent, the the constraints are primarily “one-way,” like those of Artkit values will be correct and this can be an effective way to set [9] and Rendezvous [7]. This kind of constraint resembles a up mutual dependencies. spreadsheet formula, so it is called a “formula constraint” in MYERS ET AL.: THE AMULET ENVIRONMENT: NEW MODELS FOR EFFECTIVE USER INTERFACE SOFTWARE DEVELOPMENT 355 Fig. 4. The VALUE slot of the zero_one objects have a constraint that depends on either the value of the input wire, if it exists, or else the value of the button if there is no wire (if the INPUT1 slot is NULL). The button in the PICTURE slot has a constraint that depends on the value for the zero_one object so it will have the right value when there is a wire. This creates a cycle of constraints. 4.3.1Indirect Constraints predefined constraints, which can be used for many of the The Amulet constraint system supports dynamic computa- basic relationships frequently found in user interfaces. It is tion of the objects to which a constraint refers, so a constraint worth reiterating that unlike other systems such as can not only compute the value to return, but also which ob- SubArctic [9] that only supply these predefined constraints, jects and slots to reference. This allows such constraints as Amulet allows any code in constraints. “the width is the maximum of all the components” which 4.3.2Multiple Constraints will be updated whenever components are added or re- Amulet also allows slots to contain multiple constraints at moved as well as when one of the components’ position the same time. We find this very useful for situations where changes. An example from the circuit program is that the an inherited formula is necessary for the correct operation value of a gate is computed by first accessing the input wire objects from the INPUT slots of the gate, and then indirectly of an object, but the programmer wants an additional for- mula so values can flow in multiple directions. For exam- accessing the value of the line objects. For example, the value ple, we set a constraint into the Am_VALUE slot of the button of the OR gate is calculated using the formula: widgets used in the zero-one objects to display the value of Am_Define_Formula(int, OR_value) { the input port, if any. Internally, however, the button wid- Am_Object in_line_1 = self.Get(INPUT_1); get uses a constraint in the Am_VALUE slot to make the slot Am_Object in_line_2 = self.Get(INPUT_2); change values when the user clicks on the widget. Both of if (in_line_1.Valid()&& in_line_2.Valid()) { int v1 = in_line_1.Get(Am_VALUE); these constraints can coexist in the Amulet constraint sys- int v2 = in_line_2.Get(Am_VALUE); tem. When there are multiple constraints in a slot, normally return v1 | v2; only one will become invalid at a time, and so that one will } be the one that is requested to recalculate the slot’s value. else return -1; //return for when have an illegal value When multiple constraints become invalid at the same time, } Amulet evaluates the constraint that first becomes invalid. Most other constraint systems cannot handle these kinds 4.3.3Side Effects of constraints. These “indirect constraints” [30] are also im- portant for supporting object inheritance. When an instance Our experience with Garnet suggested that people wanted is created of an object, Amulet also creates instances of any to put side effects into constraint expressions and use them like “demon procedures” or “active values.” Therefore, constraints in that object. These constraints refer to other Amulet’s constraints are eagerly evaluated and can contain objects indirectly using the structure of the groups. For ex- ample, in Fig. 3, the constraint for the top of the IN- arbitrary side effects, even creating and destroying objects. For example, a constraint is used in the Am_Map object to PUT_1_PORT is computed based on the center of the pic- create the instances of the item prototype based on the list ture, which is its sibling in the part-owner hierarchy: in the Am_ITEMS slot. This constraint creates objects which Am_Define_Formula(int, picture_center_y) { themselves will contain constraints which need to be evalu- Am_Object picture = self.Get_Sibling(PICTURE); ated. Another use is that even though formula constraints return (int)picture.Get(Am_TOP) must be put into a single slot, they can have the effect of +(int)picture.Get(Am_HEIGHT)/2; multiple outputs by simply setting the other slots as side } effects. For example, for efficiency, the constraint on the Note that even though many input and output port objects first end point of the wires is put into the Am_X1 slot but share this same constraint, they will each calculate different also sets the Am_Y1 slot: values because the pictures will be at different places. Am_Define_Formula(int, line_x1y1) { The indirect contraints are a form of “procedural ab- Am_Object source_obj = self.Get(INPUT_1); straction” since the constraints can be thought of as rela- int x1 = (int)source_obj.Get(Am_WIDTH) tionships that can be reused in multiple places, with differ- + (int)source_obj.Get(Am_LEFT); ent values for their parameters. In fact, there is a library of int y1 = (int)source_obj.Get(Am_HEIGHT)/2 356 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 23, NO. 6, JUNE 1997 + (int)source_obj.Get(Am_TOP); reuse. The programmer has independent control over the self.Set(Am_Y1, y1); //set Y1 by side effect for efficiency graphics to be animated, the start and end values of the return x1; animation, the path through value space, and the timing of } the animation. Animations can be attached to any object, Unlike previous systems such as Rendezvous [7], Amu- even existing widgets from the toolkit, and any type of let does not require the programmer to use a special value can be animated: scalars, coordinates, fonts, colors, mechanism for side effects: the regular Set and Create line-widths, vertex lists (for polygons), Booleans (for visi- calls are used. This works because we store any new con- bility), etc. straints that need to be evaluated in a queue. When a con- A number of built-in animation constraints are used for straint evaluation creates new constraints that need to be special effects. For example, Fig. 5 shows the effects of differ- evaluated, they are simply added to the end of the queue. ent constraints added to the Am_VISIBLE slot of a pop-up Amulet continues to evaluate constraints on the queue until menu. A library of useful animation constraints is provided the queue is empty, at which point Amulet redraws the in the toolkit, including support for exaggerated, cartoon- objects that have changed. style effects such as slow-in-and-slow-out, anticipation, and When using side effects in constraints, programmers followthrough [5]. The programmer can also create custom must be careful to avoid situations that will create an infi- animation types, if the built-in ones are not sufficient. nite loop. Constraints without side effects will always be evaluated exactly once each time the values change, since Amulet orders the constraint evaluation and checks for cy- cles of constraints, as discussed above. However, a pro- grammer could set up a set of constraints that invalidated each other through side effects. If the constraints are con- sistent, so that slots are set to the same values no matter which constraints are used, then the evaluation will termi- nate even if the constraints contain cycles of dependencies and side effects. However, if the constraints calculate and set different values, an infinite loop can result. 4.3.4Multiple Solvers An important research area in user interface software is (a) creating new kinds of constraint solvers (e.g., [6], [9], [29]). Therefore, Amulet contains an architecture that allows multiple solvers to coexist. Currently, in addition to the one- way solver described above, Amulet supports a multi- output, multiway solver called a “web” and an animation constraint solver. The web constraint can have an arbitrary number of in- put and output slots, and it can dynamically compute the dependencies like formula constraints. Webs also keep track of the order that dependencies change. We use this solver to keep the various slots of lines and polygons con- sistent. The line object has two sets of input slots. One set is (b) point based and has slots called X1, Y1, X2, and Y2. The Fig. 5. Animation constraints can be added to the Am_VISIBLE slot of other set is rectangle-based and has slots called LEFT, TOP, a pop-up menu to make it (a) fade in using halftoning; or (b) grow from WIDTH, and HEIGHT which are set when the line is moved the top. without changing its orientation. However, if the slots X1, TOP, and WIDTH were set, the normal one-way formula Often, animation constraints can be added to an existing mechanism would not necessarily evaluate the constraints application with only a single extra line of code, which in the correct order, but the web maintains the original or- makes it easy to explore many new uses for animations. For der of slot changes, so the final result will be correct. example, undoing operations can animate the objects back Animations. We have also created a novel animation con- to their original appearance, which might make it easier for straint solver for animating objects [23]. Adding animation users to see what has happened. The code to support the to interfaces is a very difficult task with today’s toolkits, animation of the small numbers travelling along the wires even though there are many situations in which it would be in the circuit program is only about 30 lines. Of course, useful and effective. An animation constraint detects animations can also be used to construct games (see Fig. 6) changes to the value of the slot, immediately restores the and dynamic visualizations. original value, and causes the slot to take on a series of val- ues interpolated between the original and new values. The advantage over previous approaches is that anima- tion constraints provide significantly better modularity and