AFRL-HE-WP-TR-2000-0009 UNITED STATES AIR FORCE RESEARCH LABORATORY THE COMBAT AUTOMATION REQUIREMENTS TESTBED (CART) TASK 1 FINAL REPORT: IMPLEMENTATION CONCEPTS AND AN EXAMPLE Bryan E. Brett Jeffrey A. Doyal David A. Malek SCIENCE APPLICATIONS INTERNATIONAL CORPORATION 4031 COLONEL GLENN HIGHWAY BEAVERCREEK OH 45431 Edward A. Martin David G. Hoagland HUMAN EFFECTIVENESS DIRECTORATE CREW SYSTEM INTERFACE DIVISION WRIGHT-PATTERSON AFB OH 45433-7022 JANUARY 2000 INTERIM REPORT FOR THE PERIOD OCTOBER 1998 TO SEPTEMBER 1999 Approved for public release; distribution is unlimited Human Effectiveness Directorate Crew System Interface Division 2255 H Street Wright-Patterson AFB, OH 45433-7022 DTIC QUALITY INSPECTED 2 20000509 131 NOTICES When US Government drawings, specifications, or other data are used for any purpose other than a definitely related Government procurement operation, the Government thereby incurs no responsibility nor any obligation whatsoever, and the fact that the Government may have formulated, furnished, or in any way supplied the said drawings, specifications, or other data, is not to be regarded by implication or otherwise, as in any manner licensing the holder or any other person or corporation, or conveying any rights or permission to manufacture, use, or sell any patented invention that may in any way be related thereto. Please do not request copies of this report from the Air Force Research Laboratory. Additional copies may be purchased from: National Technical Information Service 5285 Port Royal Road Springfield, Virginia 22161 Federal Government agencies and their contractors registered with the Defense Technical Information Center should direct requests for copies of this report to: Defense Technical Information Center 8725 John J. Kingman Road, Suite 0944 Ft. Belvoir, Virginia 22060-6218 TECHNICAL REVIEW AND APPROVAL AFRL-HE-WP-TR-2000-0009 This report has been reviewed by the Office of Public Affairs (PA) and is releasable to the National Technical Information Service (NTIS). At NTIS, it will be available to the general public. This technical report has been reviewed and is approved for publication. FOR THE COMMANDER T / MARIS M. VIKMANIS Chief, Crew System Interface Division. Air Force Research Laboratory Form Approved REPORT DOCUMENTATION PAGE OMBNo. 0704-0188 Public reporting burden for this collection of information is estimated ,o average . hour per response, inc uding the „me ^^T^^T^^^S^^ gathering and maintaining the data needed, and completing and reviewing the collection of ,nforn,at,on Send comments regar^ *^" o^erlt 0^^ Reports 2?^« on Collection of information, including suggestions for reducing this burden, to Washington Headquarters Services Directorate ^^^J^T^^^^l *" Davis Highway Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DL 2UWJ. 1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED January 2000 Interim, October 1998 - September 1999 5. FUNDING NUMBERS 4. TITLE AND SUBTITLE The Combat Automation Requirements Testbed (Cart) Task 1 Final Report: C: F41624-98-C-6012 Implementation Concepts and an Example PE: 6323IF PR: 2830 TA: 29 6- AUTHOR(S) WU: 10 * Bryan E. Brett, * Jeffrey A. Doyal, * David A. Malek ** Edward A. Martin, ** David G. Hoagland 8. PERFORMING ORGANIZATION 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) REPORT NUMBER * Science Applications International Corporation (SAIC) 4031 Colonel Glenn Highway Beavercreek OH 45431 10. SPONSORING/MONITORING 9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) AGENCY REPORT NUMBER ** Air Force Research Laboratory, Human Effectiveness Directorate Crew System Interface Division AFRL-HE-WP-TR-2000-0009 Air Force Materiel Command Wright-Patterson AFB OH 45433-7022 11. SUPPLEMENTARY NOTES 12b. DISTRIBUTION CODE 12a. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution is unlimited 13. ABSTRACT (Maximum 200 words) Military weapon systems are built to satisfy requirements levied by the warfighter. These weapon systems are all manned in some sense, yet tools for quantifying the effectiveness with which a crewstation must support operator performance are lacking The Combat Automation Requirements Testbed (CART) program will provide human performance modeling methods and tools for generating objective, performance-based crew system requirements. CART will integrate readily with techniques currently used within the acquisition process to generate system requirements. CART will enable acquisition analysts to consider the crew system at the same time other key system components are being evaluated during the system definition activities that occur early in the acquisition process. This report begins with a discussion of why performance-based crew system requirements are needed, followed by a description of how modeling and simulation are currently used m the acquisition process to generate system requirements. Next, it describes the modeling requirements for developing performance-based crew system requirements and CART's human performance modeling architecture. The discussion then turns to a notional example of how CART would be applied to generate a set of creW system requirements. Finally, the last section discusses the costs associated with applying CART's capabilities and the benefits to be derived therefrom. 1 5. NUMBER OF PAGES 1It4.. SDUUDBJJCE<C_TI T I ECRIMMVISO _, , Human Performance Model, Constructive Simulation Analysis of Alternatives, Use-Centered _D_ Design, Human-Centered Design, Human-System Interface, Human Behavior Representation 16. PRICE CODE 17. SECURITY CLASSIFICATION 1 8. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF ABSTRACT OF REPORT OF THIS PAGE OF ABSTRACT UNL UNCLASSIFIED UNCLASSIFIED UNCLASSIFIED Standard Form 298 (Rev. 2-89) (EG) Prescribed by ANSI Std. 239-18 This page left blank intentionally. PREFACE This effort, "Task 1, Crew Systems Requirements Establishment," was conducted under contract number F41624-98-C-6012 with the Crew Systems Development Branch, Crew System Interface Division, Human Effectiveness Directorate of the Air Force Research Laboratory (AFRL/HECI), Wright-Patterson Air Force Base, Ohio 45433-7022, for the period October 1998 to September 1999. Science Applications International Corporation (SAIC), 4031 Col Glenn Highway, Beavercreek, Ohio 45431-7753 was the contractor. Mr. David Hoagland (AFRL/HECI) was the Program Manager. This effort supported Work Unit 28302910, "Combat Automation Requirements Testbed (CART)." in This page left blank intentionally. IV TABLE OF CONTENTS 1.0 INTRODUCTION ] 2.0 THE NEED FOR OBJECTIVE, PERFORMANCE-BASED CREW SYSTEM REQUIREMENTS 2 3.0 CURRENT STATE-OF-THE-ART FOR GENERATING SYSTEM REQUIREMENTS IN THE ACQUISITION PROCESS 5 3.1 Overview of the Acquisition Process 5 3.2 Trade Studies ^ 3.3 Generating Subsystem Requirements with Modeling and Simulation 8 3.4 The Lack of Operator Models and Its Consequences 10 4 0 HUMAN PERFORMANCE MODELING IN CART 12 12 4.1 Overview 4.2 Human as an Information Processor 13 4.3 CART Human Performance Modeling Architecture 17 4.3.1 Human Performance Modeling Capabilities for Generating Performance-Based Crew System Requirements 4.3.2 Integration of CART Human Performance Models with Different Constructive Testbeds 19 4.3.3 An Easy to Use Tool that Can Be Applied by Personnel Who Support Constructive Testbeds..l9 4 4 Using Task Networks to Implement the Human Information Processor Model 20 4.4.1 Goal Functions zu 4.4.2 Tasks 21 4.4.3 Task Networks 23 4.4.4 Variables and Macros ■ "* 4.4.5 Implementing the Human Information Processor 25 5.0 CART IMPLEMENTATION 28 5.1 Factors and Considerations that Drive CART Implementation 28 5.2 Overview of CART Implementation Activities 30 5.2.1 Mission Decomposition 31 5.2.2 Human Performance Model Development and Integration 36 5.2.3 Constructive Simulation and Data Analysis 40 5.2.4 CART-Based Crew System Performance Requirements 41 6.0 CART COSTS AND BENEFITS 43 6.1 CART Costs 43 6.1.1 Human Performance Model Development 43 6.1.2 HLA Integration 44 6.1.3 Constructive Simulation Augmentation 44 6.1.4 Test Execution and Analysis 44 6.2 CART Benefits 45 6.2.1 CART Benefits for System Acquisition 45 6.2.2 CART'S Impact on the Use of Human-in-the-Loop Simulation 46 7.0 SUMMARY AND CONCLUSIONS 48 8.0 BIBLIOGRAPHY • 49 9.0 GLOSSARY OF ACRONYMS 51 APPENDIX A 53 LIST OF FIGURES Figure 1. Phases and Milestones in the Acquisition Cycle 6 Figure 2. Tradeoff Analyses Throughout the Acquisition Lifecycle.: 7 Figure 3. Levels of Effectiveness Modeling 8 Figure 4. How Subsystem Requirements Are Derived 10 Figure 5. Basic Human Information Processor (HIP) Model 14 Figure 6. CART Human Performance Modeling Architecture 17 Figure 7. Implementation of the HIP Using Task Network Modeling 26 Figure 8. Factors and Considerations that Drive CART Implementation 29 Figure 9. The CART Implementation Process 31 Figure 10. Excerpt from a Means-Ends Hierarchy 32 Figure 11. Goal State Definitions and Interactions 33 Figure 12. Information-Decision-Action Analysis 34 Figure 13. Example of MOE Extension and Definition of Data Collection Requirements 35 Figure 14. Human Performance Model Development - 37 Figure 15. Integrating HPM with the HLA Federation 39 Figure 16. Data Analysis • •••• 41 Figure A-l. A JDAM Weapon Mounted on an Eight Station Rotary Launcher 57 Figure A-2. A Graphical Depiction of the Blockage Problem 58 Figure A-4. Initial Human Interface for JDAM 61 Figure A-5. The JDAM Re-designation Study 63 Figure A-6. Results of the JDAM Re-designation Study 64 Figure A-7. How CART Could Have Ensured a Successful JDAM Interface 66 VI 1.0 INTRODUCTION The Combat Automation Requirements Testbed (CART) program provides human performance modeling methods and tools for generating objective, performance-based crew system requirements. This constructive simulation capability will integrate readily with techniques currently used within the acquisition process to generate system requirements. CART will enable acquisition analysts to consider the crew system at the same time other key system components are being evaluated during the system definition and refinement activities that occur early in the ac quisition process. Ultimately, CART will provide for more effective crew system designs that reliably achieve mission performance goals, take less time to develop, and have fewer flaws that require subsequent redesign cost and effort. This report describes the rationale for why CART is needed and how it will be implemented. It begins with a discussion of why performance-based crew system requirements are needed, followed by a description of how modeling and simulation are currently used in the acquisition process to generate system requirements. Next, it describes the modeling requirements for developing performance-based crew system requirements and CART's human performance modeling architecture. The discussion then turns to a notional example of how CART would be applied to generate a set of crew system requirements. Finally, the last section discusses the costs associated with applying CART's capabilities and the benefits to be derived therefrom. 2.0 THE NEED FOR OBJECTIVE, PERFORMANCE-BASED CREW SYSTEM REQUIREMENTS Despite significant levels of automation applied in modern weapons and command and control systems, human operators still play significant roles in those systems. Consequently, the performance of an operator can have significant impacts on the performance of the entire system and, indeed, on the outcome of a mission. Among the factors that influence operator performance are the tasks assigned to the operator and the design of the crew system that enables performance of those tasks. Ideally, the crew system design process should produce a crew station that results in a level of operator performance needed to successfully perform a mission. In practice, this is often not the case, and crew systems are fielded with problems that must be corrected. Doyal, Irvin, and Ramer (1995) reported on an evaluation of cursor system gain functions implemented in the original design of the B-2 bomber. The impetus for the study was a problem related to the aircrew's ability to control the cursor used for navigation and target updates. While it represents a small component of the overall crew system, the cursor is a very important element in the employment of B-2 precision guided weapons. As originally implemented, aircrews were unable to accurately and quickly control cursor placement on desired targets and aim points in a radar image. This resulted in errors and delays in target designation that could have an impact on degree of target destruction. Not all crew system design problems occur as a function of developing completely new systems. Often, capabilities are added to an existing system to enhance or extend mission performance. These enhancements can have operator performance implications. In such instances, the crew system must be modified to fully achieve the mission performance gains offered by the enhancement. Appendix A presents an unpublished study conducted by two of the authors. The study documents how crew system design is typically conducted within the acquisition process (especially, system modifications) and how design errors can occur. The study evaluated a weapon assignment interface that was developed for an attack aircraft to support employment of a new weapon for the aircraft. An important requirement for the interface was the ability to reassign weapons when target sets were changed during the course of a mission. Unique characteristics of the weapon as it was installed on the aircraft made reassignment of weapon a cognitively complex task. The interface originally provided to support the reassignment task