Constructing Training Demonstrations Dan Fu, Randy Jensen, Alex Davis Roy M. Elam Jr. Stottler Henke Associates, Inc. Directorate, Training, Doctrine, and Combat Development, 951 Mariners Island Blvd, #360 Training Development Division, New Systems Training Branch San Mateo, CA 94404 Fort Knox, KY 40121 (650) 931-2700 (502) 624-5828 {fu,jensen,davis}@stottlerhenke.com [email protected] Abstract—Demonstrations have been favored by the Army such as Second Life, where layman participants can as a complement to more traditional training materials customize avatars and environments. Educational classes because they accelerate learning, stimulate interest, and can be held in the simulated world. Together, these two communicate better than text. Unfortunately, developments suggest that demonstrations can be demonstrations have received little attention in the research constructed in the virtual world. The benefits are many, literature and there is little consensus on what constitutes a such as reductions in development time, online good demonstration. We describe two parallel avenues of dissemination, and lessening production cost. research towards the rapid construction of effective demonstrations. The first avenue’s goals are to: clearly In this paper we describe ongoing efforts to (1) create an articulate the nature and purpose of demonstration; compare instructional framework in which demonstrations can be related areas of research to identify factors influencing measured, (2) explore the use of virtual environments by demonstration effectiveness; and define a set of component investigating technology platforms, and (3) combine results capabilities, guidelines and best practices for creating of the first two to construct effective virtual demonstrations. effective demonstrations. The results inform the second For the instructional framework, Rosen et al. [20] have been avenue’s investigation of how a demonstration authoring conducting research in organizational psychology. The toolset can be constructed from existing virtual training psychology literature has established that virtual or environments using 3-D multiplayer gaming technologies. constructive environments can accelerate the learning Together these avenues inform our effort to create process by illustrating correct behaviors, establishing a demonstrations for Army curricula. 1 2 shared mental model of team behavior, and supporting such advanced techniques as cross training [22]. Demonstrations TABLE OF CONTENTS have been considered only within a broader training context. Demonstrations by themselves have received little attention 1. INTRODUCTION.................................................................1 with little to no agreement on what makes a demonstration 2. THE NATURE OF DEMONSTRATION.................................1 effective. Our second effort is technology investigation, 3. PLATFORMS AND TECHNOLOGIES...................................4 which examines and evaluates approaches and platforms to 4. EXAMPLE USE CASE.........................................................4 be employed for demonstrations, such as film, video, CONCLUSION........................................................................7 computer-based training, videogames, and simulations [10]. ACKNOWLEDGEMENTS........................................................7 The third effort seeks to combine the earlier two. Here we REFERENCES........................................................................8 will discuss the construction of a team demonstration using BIOGRAPHY..........................................................................9 guidelines from the psychology literature along with a 3-D virtual environment technology platform. Altogether these 1. INTRODUCTION efforts make up a demonstration authoring system called RADX: Rapid Authoring of Demonstrations for eXperience. The use of demonstrations in Army training environments is pervasive. While most will agree that live demonstrations The rest of this paper is organized as follows. We describe are the most effective way to convey information to the theoretical basis for demonstrations. This includes an warfighters, this conjecture has never been proven. What’s analysis of the relevant literature with demonstration- needed is a framework that can taxonomize training oriented elements called out. We then characterize the demonstrations, and prescribe ways to measure the space of technology platforms with a focus on 3-D game usefulness of any given demonstration. Moreover, such a engines. With these two pieces of work in mind, we metric would suggest ways to improve existing examine team training applications for breaching operations. demonstrations, or how one might devise future demonstrations. As we consider these metrics, there are two important developments in education. First, traditional 2. THE NATURE OF DEMONSTRATION teaching methods in a classroom have seen changes in In this section we summarize the nature and purpose of recent years with the growing popularity of distance demonstration. See [20][21] for an expanded version. learning. Whole degrees can be earned online. The second Training is the systematic acquisition of the knowledge, important development is the use of virtual environments, skill, and attitude (KSA) competencies targeted for 1 acquisition. Generally training consists of five core 1978-1-4244-2622-5/09/$25.00 ©2009 IEEE. elements: the provision of information (e.g., classroom 2 IEEEAC paper #1338, Version 2, Updated January 16, 2009 1 Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 16 JAN 2009 2. REPORT TYPE 00-00-2009 to 00-00-2009 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Constructing Training Demonstrations 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Stottler Henke Associates Inc,951 Mariners Island Blvd #360,San REPORT NUMBER Mateo,CA,94404 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES Proceedings IEEE Aerospace Conference, Big Sky, MT, 7-14 Mar 2009 14. ABSTRACT see report 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 9 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 lectures), demonstration (e.g., live demonstrations, video (2) retention (wherein what is observed must be stored recorded examples of task performance), practice (e.g., symbolically in order to affect future behavior), simulation, guided on-the-job performance), feedback (e.g., results analysis), and remediation (i.e., the selection of (3) production (whereby the stored symbolic knowledge future training [23]). This section forwards a conceptual must be reconverted into overt actions), and definition of a demonstration and a review of the theoretical basis underlying the use of demonstrations for training. (4) motivation (whereby the perceived consequences of Guidelines for developing effective demonstrations are performing the observed behavior must be favorable summarized. enough to strengthen the likelihood of future performance). Although an exact and widely accepted definition of a This theory has received much empirical attention with the demonstration is currently lacking [26], demonstration- majority of research conducted under the general based training can be understood as a learner’s observation observational learning heading tending to involve lower of task performance, components of task performance (i.e., level motor tasks. Hence, the generalizability of the part-task performance) either in real time or through some empirical findings from these studies to types of complex form of recorded or computer generated medium, or tasks trained by organizations is suspect. Still, Bandura’s characteristics of the task environment that have been observational learning theory remains the most widely targeted for acquisition. Demonstrations are often an researched and applied. example of task performance; however, demonstrations are rightfully thought of as engineered experiences where Behavioral Modeling Training learners are prompted to actively process the informational content of the example, and to systematically and reliably Behavioral modeling training (BMT) is one of the most acquire targeted KSA’s and transfer them to the work extensively used training methods available to modern environment. In this vein, we propose a working definition organizations [25]. BMT is based on Bandura’s social of demonstration: “A demonstration is a strategically learning theory [13]. Utilizing the model provided by social crafted, dynamic example of partial or whole task learning theory, BMT includes processes such as modeling, performance or of characteristics of the task environment a retention process, behavioral rehearsal, feedback, and intended to increase the learner’s performance by methods of training transfer to encourage the greatest illustrating (with modeling, simulation, or any visualization transfer of training possible [8][16]. Specifically, during approach) the enactment of knowledge, skills, and attitudes BMT: (KSA’s) targeted for skill acquisition.” (1) trainees are given a list of well-defined skills and facts Demonstrations vary in terms of information, physical to be learned during training, characteristics, and the learner’s activities prior to, during and after the example of task performance. We distinguish (2) during training models and visual aids are used to between an example, which is the observational component illustrate effective behaviors and skills, of the demonstration, and the demonstration, which is the entirety of the example plus additional activities and (3) trainees are provided ample opportunities to practice information provided. In the following section we review newly learned skills, the theoretical literature pertinent to designing effective (4) trainees are provided feedback and social demonstrations. reinforcement by trainers and other trainees, and Theoretical Basis for Demonstration-based Training (5) trainers and the organization utilize many methods to Learning through observation has been one of the promote transfer of training [7]. fundamental means of acquiring knowledge and skills in both systematic and informal training. This section briefly Using all these methods, behavioral modeling training has reviews two of the research traditions in behavioral science proven to be an effective training tool in developing skills, that form the cornerstones of our understanding of resulting in high transfer of training. Additionally, BMT has demonstration-based training: observational learning and been tested and found effective in a number of scenarios behavior modeling training. including training technical and interpersonal skills. Observational Learning A Typology of Demonstrations Bandura [2] describes four observational learning processes: We have created a typology of demonstrations shown in Figure 1. It represents classes of features that can be (1) attention (whereby people must actively process what included within a demonstration. Any one demonstration they are observing in order to learn), may (and likely will) have features from more than one category. This framework organizes the space of 2 possibilities and provides a common language for First, passive demonstrations do not require any activity on discussing demonstrations. the part of the learner outside of the act of observing. These are by far the most frequently encountered demonstrations There are two types of knowledge: procedural and strategic. in day to day life and training programs. Passive Procedural knowledge is “how-to” knowledge; it involves demonstrations rely entirely on the content of the example knowledge about the sequences of actions involved in task and sometimes guiding information to focus the attention of performance. It is a rehearsed and static sequence of the learner, but do not incorporate any directions that behaviors performed to reach a task goal, such as require action (behavioral or cognitive) on the part of the performing a “stack” as part of a forced entry sequence. learner. Active demonstrations impose demands on the Strategic knowledge is “how-to-know-when-to-do-what” learners outside of passively observing an example of task knowledge [15] and is generally associated with problem performance. They require the learner to engage in activities solving. Strategic knowledge involves learning aspects of designed to increase the retention of knowledge and transfer the task that are not specific to one context, such as deciding of skill. Table 1 summarizes the six categories of when to initiate communication during an operation. demonstrations. Other than guided vs. unguided, the types are not mutually exclusive, and so a demonstration can be There are two high level categories concerning the types of both active-preparatory and active-retrospective for activities and information provided in the demonstration. example. Demonstration Procedural Strategic Knowledge Knowledge Focused Focused Passive Active Guided Unguided Preparatory Concurrent Retrospective Prospective Figure 1: Typology of Demonstrations for Simulation-based Training [24] Table 1: Description of demonstration types (adapted from [24]) Demonstration Description Example features Citations Type Passive-unguided Learners given no requirements or information N/A [1][3][4][19] outside of that present in the example of task performance or task environment characteristics Passive-guided Learners are given pre-demonstration Attentional advice, provision of learning points [7][14] information intended to increase learning Active-preparatory Learners engage in activities (designed to orient instruction on self-regulatory skills for [5][12] and focus the learner) before viewing the observation, goal setting, and perceived self- example for the observation experience to come efficacy before viewing the example Active-concurrent Learners engage in activities during observation note taking, perspective taking [18] of example Active-retrospective Learners engage in activities after viewing the symbolic mental rehearsal, learner-generated [6][13] demonstration designed to focus attention on learning points salient aspects of performance Active-prospective Learners engage in activities after observing the goal setting exercises, the generation of practice [17][25] example that focus the learner on how it can be scenarios by the learners applied to other contexts 3 3. PLATFORMS AND TECHNOLOGIES massively multiplayer. Using these two dimensions, we now highlight the most popular combinations. In this section we examine the types of game technologies and link them to demonstration. Fu, Jensen, and Hinkelman 2-D Single Player Games depict a point of view either from [10] categorize technologies along two dimensions: overhead or from the side. 2-D depictions could be most depiction and plurality (see Table 2). The most popular useful for “big picture” understanding, such as training depiction is 3-D, which makes the visualization as realistic coordination among teammates. 2-D game engines, as possible, as opposed to 2-D. Plurality refers to the compared to others, offer the lowest amount of fidelity. number of participants: single player, multiplayer, or Their use for demonstration-based training is limited. Table 2: Basic categories of game engines DEPICTION 2-D 3-D The player’s avatar operates in a 3-D environment. Avatar in a 2-D environment. The perspective Single First-person shooters and real-time strategy games Y is an overhead or side view. are common. T I Multiple players control an avatar in the 2-D L Typically less constructive than the 3-D single A Multiplayer world. Typically an extension of single player. R player games as there are human players. U Much less common. L A small handful exist as free games. Several P Massively Similar to 3-D multiplayer except players can commercial 3-D based systems have ancillary Multiplayer number in the thousands with persistent worlds. 2-D views as well. 3-D Single Player Games display the virtual environment by Guidelines for Developing Effective Demonstrations rendering it from parameters and descriptions of 3-D Much of the existing training research concerns tasks that objects. It assumes the player is the only person operating are more abstract and simple rather than the types of tasks in the environment, and that anything else independently the Army generally choose to train. Still, we were able to moving is controlled by artificial means. The engine may gather general principles in the form of seven preliminary support many “cameras” or viewpoints within the and empirically based guidelines as shown in Table 3 (see environment, such as first person, tethered, overhead, or a [21] for additional information). user-controllable point of view. It may support display of several cameras simultaneously on one screen. There are Table 3: Suggested guidelines three major genres: first-person shooter (FPS), real-time strategy (RTS), and role-playing game (RPG). Briefly, FPS depicts a first-person point of view. Emphasis is on real- Guideline time shooting ability. RTS depicts scenes from an 1. The KSA’s targeted for demonstration-based training overhead, angled perspective. The player will control must be perceivable by the learner. several units from above. RPG is similar to RTS, but there 2. Direct the learner’s attention to the cues relevant to is no real-time component. learning. 3. Use instructional narratives to make covert aspects of 3-D Multiplayer Games increase the number of human performance accessible to learners. players involved. One might think of multi-player as the 4. Utilize mixed models, as opposed to positive-only same as single player except that the control for avatars is models, to display both positive and negative behaviors supplanted by real human control. and outcomes. 5. Show the consequences of behaviors. 3-D Massively Multiplayer Online Games (MMOG) are similar to 3-D multiplayer except on a bigger scale. They 6. Instruct learners to create their own scenarios in which feature a huge virtual world, potentially as big as the earth, to rehearse behaviors. where one may explore and meet other avatars and objects. 7. Instruct learners to symbolically or mentally rehearse Unlike the multiplayer games whose participants assemble behaviors and skills before rehearsing them. temporarily and then disperse when the game round concludes, MMOG’s retain history in the virtual world: the world changes and so do the avatars in it. 4 a 2-D “bird’s eye view” of the vehicle movement. In 4. EXAMPLE USE CASE addition, only one Soldier at a time can “move” vehicles while the other Soldiers are passive observers. There is a In this section we describe an example use case currently lag time that occurs in the demonstration and the practice underway. It is composed of (1) existing curricula at the because a student must physically pick up and move his Army’s Armor School at Fort Knox, Kentucky; (2) a “micro armor” vehicles among check points. The lag time comparison of demonstration approaches; and (3) a use case and passiveness diminish the realism of this training and involving breaching operations. does not immerse the Soldier into the training. Armor School Training Technology Platform Opportunity The Armor School’s mission is to educate and train Soldiers Given the terrain board-based method for demonstrations, in the art and science of Mounted Maneuver Warfare which potential visualization and manipulation technologies are includes the control of tanks, armored infantry carriers, applicable here. Considering platforms, there could be artillery, and so forth. The school implements curricula that straightforward depictions of the terrain in both an overhead range from simple individual tasks to complex collective view using a 2-D map and a virtual 3-D environment. We tasks. There are many competencies that Soldiers are identify seven potential advantages: expected to gain while attending courses at the Armor School but the most important outcome is that Soldiers (1) Instructor-controlled viewpoint. The instructor can become agile and adaptable leaders. adopt camera viewpoints ranging from bird’s eye view, to behind-vehicle, to first person. Leadership courses conducted at the Armor School’s Noncommissioned Officers Academy uses a Small Group (2) Varying time. The instructor can control the passage Instruction method for content delivery. PowerPoint slides of time. It’s then possible to demonstrate movement in are developed and used to help guide the instruction. The real-time, faster than real-time, slow motion, or advantage of this type of instruction is it allows the reverse. instructor the opportunity to facilitate discussions (e.g., active-concurrent, active-prospective) that provide a basis (3) Both instructors and trainees can operate or move for active learning. The Soldiers are able to share their assets within the demonstration, either by simulated experiences with their peers and the instructor is able to “driving” of the vehicles, or simply shift objects in the bridge these experiences to a learning objective. Through 3-D world with ease. the sharing of experiences critical reflection (e.g., active- retrospective) occurs and provides a means for Soldiers to (4) The instructor and trainee are not necessarily co- become agile and adaptable. located; i.e., that demonstrations can be broadcast. This brings up the possibility of distance learning. In addition to the discussions that take place in the (5) Demonstrations can be recorded for later (unlimited) classroom the instructors present demonstrations on playback, using standard media (video capture) or competencies such as command and control of maneuver within the virtual world. elements and offensive and defensive operations. To accomplish the demonstration, the instructor uses a 3-D (6) Cross-training is possible for team instruction where model terrain board and micro armor (miniature vehicles trainees are able to view perspectives of other that can be easily moved) and walks the students through teammates. the process of accomplishing tasks associated with the competencies. The Soldiers are allowed to practice the (7) Terrain can be switched quickly. Rather than employ tasks and then assessed using the terrain board. a single terrain with varying vehicles, a completely different terrain can be used easily; e.g., switching There are both advantages and disadvantages associated from desert to urban terrain. with the Armor School’s terrain board-based demonstration technique. The biggest advantage is that Soldiers are able to Implementing a demonstration in a virtual environment has learn with their peers by vocally sharing their experiences in the potential to improve the instruction of the current model the field. In addition, the demonstration resources required and diminish the disadvantages outlined earlier. In this next to “move” vehicles are minimal in the scope of a live section we consider a particular portion of the Armor School training event where vast amounts of land, fuel and curriculum for construction of a demonstration. personnel are required. Soldiers are able to practice each task as many times as they need to become proficient in the Breaching Use Case task while receiving immediate feedback from their peers as well as their instructor. Breaching is the employment of a combination of tactics, techniques, and procedures to project combat power to the far side of an obstacle. Obstacles are any obstructions that The disadvantages of the demonstration and practice model stop, delay, divert, or restrict movement. They are usually described above are that the students are provided with only 5 covered by observation and enhanced by direct or indirect students to experience a complex maneuver such as a fires. breaching exercise. Through a role playing demonstration, each student would be able to be the leader of each of the Conducting a breaching exercise for an Armor platoon is breaching forces required to effectively conduct a breaching complex and requires a specific process to reach the desired exercise. In addition, the students while role playing a Tank outcome. In order to conduct a breaching exercise several Commander, would see the exercise from the Tank teams must maneuver in tandem and each element must be Commander’s hatch, hastening acquisition of strategic able to do their specific task without error. Breaching knowledge. This would allow the Tank Commander to gain operations usually entail the coordinated efforts of three task experiences and facilitate the agile adaptable leader concept. organized elements: The facilitator and students could provide constructive (retrospective) feedback to each other and learn from each (1) The support force- employed to suppress identified and other’s experiences. characterized enemy elements that are overwatching the obstacle. The support force uses direct and indirect Table 4 summarizes the three demonstration approaches— fires to accomplish its mission. live, terrain board-based, and virtual. For completeness, “live” is included though it is used sparingly by the Armor (2) The breach force- creates and proofs a lane through the School. The rows are the various factors discussed. obstacle, allowing the assault force to secure the far side of the obstacle. (3) The assault force- moves through the cleared obstacle to secure the far side of the obstacle allowing other elements to maneuver through the obstacle. Teaching this exercise in a live environment requires a vast amount of resources including vehicle, fuel, Observer Controller personnel, and land. It also requires personnel to pose as the opposition force in order to make the exercise realistic. In order to practice this exercise and allow each student an opportunity to act as a leader for each element would is very time consuming. The obstacle must be reconstructed for each iteration while students reset back to a starting point for each iteration. An After Action Review (AAR) can be conducted each time, but the exercise must be filmed and there must be a facility available to view the film and to conduct an AAR. This will add to the already extensive resources required for this type of learning activity. To reduce the resources required to conduct a live demonstration and exercise, training developers at the Armor School develop PowerPoint slides that describe the functions of each of the teams associated with a breach exercise. Students move to the 3-D terrain board and are talked through the exercise using the micro-armor as mentioned earlier. In this setting all of the students are not engaged as only one student at a time will conduct the exercise. Thus, for one student there is training while the rest are passively guided. The exercise requires little resources and students are permitted to offer feedback to each other (active concurrent) as the exercise is being conducted. The disadvantage of using a terrain board is it is not as engaging as a live event and the students see the task from only a bird’s eye view perspective. If the facilitator had a virtual demonstration tool that permitted the same engagement opportunity as a live exercise and eliminated the resources required of a live exercise, it would provide a more efficient way of allowing 6 Table 4: Comparison of Armor School demonstration approaches Terrain Board Live Demonstration Virtual Demonstration Demonstration Both 2-D bird’s eye and 3-D Viewing Perspective Real, situated, tactical Model, 2-D bird’s eye view tactical views, either instructor- or trainee-driven Small-scale model terrain Computer-based terrain Resources Live use of real assets construction construction Longest, involving physical Fast, requiring placement of Fast, virtual world Scene Arrangement placement micro armor assets placement of assets Time Efficiency Real-time Faster than real-time Faster than real-time Shared Experience Yes (team – no) Yes Potentially yes Yes, requires moving assets, Yes - with simulation Repeatable Yes, requires moving assets role players control Stimuli Real life Abstraction Life like + abstraction Feedback Delayed Immediate Immediate with playback Team Training Capable Yes - limited cross-training Yes - limited cross-training Yes No - All participants must No - All participants must Distribution Potentially yes be present be present CONCLUSION This paper forwards two threads of investigation: an instructional framework, and virtual technologies. These were considered in an Armor School use case where we contrasted the current terrain board-based technology with virtual technology. ACKNOWLEDGEMENTS This work was funded and sponsored by the US Army Research Institute under the SBIR program. Opinions expressed are those of the authors and do not necessarily represent an official position of the Department of the Army or the Army Research Institute. 7 REFERENCES [14] Jentsch, F., Bowers, C., & Salas, E. (2001). What determines whether observers recognize targeted [1] Austin, S., & Laurence, M. (1992). An empirical study of behaviors in modeling displays? Human Factors, 43(3), the SyberVision golf videotape. Perceptual and Motor 496-507. Skills, 74(3, pt 1), 875-881. [15] Kontogiannis, T., & Shepherd, A. (1999). Training [2] Bandura, A. (1986). Social foundations of thought and conditions and strategic aspects of skill transfer in a action: A social cognitive theory. Rockville, MD: simulated process control task. Human-Computer National Institutes of Mental Health. Interaction, 14, 355-393. [3] Berry, D. C. (1991). The role of action in implicit [16] Kraut, A. I. (1976). Developing managerial skills via learning. The Quarterly journal of experimental modeling techniques: Some positive research findings- A psychology, 43(4), 881-906. symposium. Personnel Psychology, 29, 325-328. [4] Blandin, Y. & Proteau, L. (2000). On the cognitive basis [17] Latham, G.P. & Saari, L.M. (1979). Importance of of observational learning: Development of mechanisms supportive relationships in goal-setting. Journal of for the detection and correction of errors. The Quarterly Applied Psychology, 64(2), 151-156. Journal of Experimental Psychology A, Volume 53, Number 3, 1 August 2000, 846-867. [18] Lozano, S.C., Hard, B.M., & Tversky, B. (2006) Perspective-taking Promotes Action Understanding and [5] Cumming, J., Clark, S.E., Ste-Marie, D.M., McCullagh, Learning. Journal of Experimental Psychology: Human P., & Hall, C. (2005). The functions of observational Perception and Performance, 32(6), 1405-1421. learning questionnaire (FOLQ). Psychology of sport and exercise, 6, 517-537. [19] Palmiter, S. & Elkerton, J. (1993). Animated demonstrations for learning procedural computer-based [6] Davis, F.D., & Yi, M.Y. (2004). Improving computer tasks. Human-Computer Interaction, 8(3), 193-216. skill training: behavior modeling, symbolic mental rehearsal, and the role of knowledge structures. Journal [20] Rosen, M.A., Salas, E., & Pavlas, D. (2009). of Applied Psychology, 89(3), 509-523. Demonstration-based Learning: Typology of Instructional Features, and Some Initial Propositions. [7] Decker, P. J. & Nathan, B. R. (1985). Behavior modeling Unpublished manuscript. training: Principles and applications. New York: Praeger. [21] Rosen, M. A., Salas, E., & Upshaw, C. L. (2007). [8] Doo, M. Y. (2005). The effects of presentation format for Understanding Demonstration-based Training: A behavior modeling of interpersonal skills in online Conceptual Framework, Some Principles and Guidelines. instruction. Journal of Educational Multimedia and Unpublished manuscript. Hypermedia, 14(3), 213-235. [22] Salas, E., & Cannon-Bowers, J.A. (2000). The anatomy [9] FM 7-1 (2003). Battle Focused Training. Washington, of team training. In S. Tobias and J.D. Fletcher (Eds.), DC: Department of the Army. Training & Retraining (pp. 312-335). New York: Macmillan Reference. [10] Fu, D., Jensen, R., & Hinkelman, E. (2008). “Evaluating Game Technologies for Training,” IEEE Aerospace [23] Salas, E., Priest, H. A., Wilson, K. A., & Burke, C. S. Conference. (2006). Scenario-based training: Improving military mission performance and adaptability. In C. A. C. A.B. [11] Fu, D., Jensen, R., Salas, E., Lampton, D., & Kusumoto, Adler, and T.W. Britt (Eds.), Military life: The L. (2008). Constructing Virtual Training Demonstrations. psychology of serving in peace and combat (Vol. 2: Proceedings of the Industry/Interservice, Training, Operational Stress, pp. 32-53). Westport, CT: Praeger Simulation & Education Conference (I/ITSEC 2008). Security International. [12] Hard, B. M., Lozano, S. C., & Tversky, B. (2006). [24] Salas, E., Rosen, M. A., Pavlas, D., Jensen, R., Fu, D., Hierarchical encoding: Translating perception into Ramachandran, S., Hinkelman, E. (2008). Understanding action. Journal of Experimental Psychology: General, Demonstration-based Training: A Definition, Conceptual 135, 588-608. Framework, and Some Initial Guidelines. Army Research Institute Technical Report, to appear. [13] Hogan, P. M., Hakel, M.D., & Decker, P.J. (1986). Effects of trainee-generated versus trainer-provided rule [25] Taylor, P. J., Russ-Eft, D.F., & Chan, D.W.L. (2005). A codes on generalization in behavior-modeling training. Meta-analytic Review of Behavior Modeling Training. Journal of Applied Psychology, 71(3), 469-473. Journal of Applied Psychology, 90(4), 692-709. 8 [26] Williams, A. M., Davids, K., & Williams, J.G. (1999). Visual perception and action in sport. London: Spon. BIOGRAPHY Dan Fu is a group manager at Stottler Henke Associates. He joined in 1998 and has worked on several artificial intelligence (AI) systems including AI authoring tools, wargaming toolsets, immersive training systems, and AI for simulations. He is the principal investigator for SimBionic, which enables users to graphically author entity behavior for a computer simulation or game. Dan holds a B.S. from Cornell University and a Ph.D. from the University of Chicago, both in computer science. Randy Jensen is a group manager at Stottler Henke Associates, Inc., working in training systems since 1993. He has developed numerous Intelligent Tutoring Systems for Stottler Henke, as well as authoring tools, simulation controls, after action review tools, and assessment logic routines. He is currently leading projects to develop automated after action review for Marine Corps combined arms training, a framework for ITS interoperability with distributed learning architectures for the Joint ADL Co-Lab, and an authoring tool for virtual training demonstrations for the Army. He holds a B.S. with honors in symbolic systems from Stanford University. Alex Davis is an AI Researcher at Stottler Henke. Mr. Davis has been a researcher, knowledge engineer, and software engineer at Stottler Henke over the last ten years. He has worked on a variety of cognitive agent architectures and training simulations, including several recent projects on the computational representation of cultural influences on human behavior, for training simulation applications. He graduated summa cum laude from the College of William and Mary in math and philosophy, and has an MS in computer science from SUNY Buffalo. Roy Elam is an Instructional Systems Specialist for the New Systems Training Branch Training Development Division at Ft. Knox Kentucky. He has worked in the U.S. Army training field as both a Soldier and a civilian and has served in the capacity of training developer and project lead in a variety of areas at the Armor School which included the design and development of both distance and resident instruction. He has developed training strategies for the Armor School’s Noncommissioned Officer Academy including the model for the implementation of DARWARS Ambush!, into the program of instruction. He is currently working on game effectiveness research with TCM Gaming and the Army Research Institute. He holds a Masters in Distance Education from the University of Maryland, University College and a M.S. in Human Resource Education from the University of Louisville. 9