ARI Research Note 90-135 Proposed Method for Military Intelligence Job Ability Assessment 0,, 00 N Frederick A. Muckler, Sally Seven, and Allen Akman Akman Associates, Inc. for Contracting Officer's Representative Beverly G. Knapp ARI Field Unit at Fort Huachuca, Arizona DTIC Julie A. Hopson, Chief Systems Research Laboratory F-L ECTT E Robin L. Keesee, Director NOV021990 l D September 1990 United States Army Research Institute for the Behavioral and Social Sciences Approved for public release; distribution Is unlimited U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES A Field Operating Agency Under the Jurisdiction of the Deputy Chief of Staff for Personnel EDGAR M. JOHNSON JON W. BLADES Technical Director COL, IN Commanding Research accomplished under contract for the Department of the Army . - Ackman Associates, Inc. Technical review by -iu !o n/ David D. Burnstein -: tLbility Codes ,. ii and/or Special NOTICES DISTRIBUTION: This report has been cleared for release to the Defense Technical Information Center (DTIC) to comply with regulatory requirements. It has been given no primary distribution other than to DTIC and will be available only through DTIC or the National Technical Information Service (NTIS). FINAL DISPOSITION: This report may be destroyed when it is no longer needed. Please do not return it to the U.S. Army Research Institute for the Behavioral and Social Sciences. NOTE: The views, opinions, and findings in this report are those of the author(s) and should not be construed as an official Department of the Army position, policy, or decision, unless so designated by other authorized documents. UNCLASSIFIED SECURITY CLASSIFICATION OF THIS PAGE Form Approved REPORT DOCUMENTATION PAGE OMB No. 0704-0188 la. REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGS Unclassified 2a. SECURITY CLASSIFICATION AUTHORITY 3. DISTRIBUTION/AVAILABILITY OF REPORT Approved for public release; 2b. DECLASSIFICATION /DOWNGRADING SCHEDULE distribution is unlimited. 4. PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMBER(S) ARI Research Note 90-135 6a. NAME OF PERFORMING ORGANIZATION 6b OFFICE SYMBOL 7a. NAME OF MONITORING ORGANIZATION Akman Associates, Inc. (If applicable) U.S. Army Research Institute -- Field Unit at Fort Huachuca 6c. ADDRESS (City, State, and ZIP C d'e) 7b. ADDRESS (City, State, and ZIP Code) 8555 Sixteenth Street, Suite 400 Fort Huachuca, AZ 85613-7000 Silver Spring, MD 20910 8a. NAME OF FUNDING/SPONSORING 8b. OFFICE SYMBOL 9. PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER ORGANIZATION U.S. Army Research (If applicable) Institute for the Behavioral DAHC35-89-D-0028, D.O. 0001 and Social Sciences PERI-S 8c. ADDRESS(City, State, and ZIP Code) 10. SOURCE OF FUNDING NUMBERS PROGRAM PROJECT i TASK WORK UNIT 5001 Eisenhower Avenue ELEMENT NO. NO. NO. ACCESSION NO. Alexandria, VA 22333-5600 62785A 790 1306 C04 11. TITLE (Include Security Classification) Proposed Method for Military Intelligence Job Ability Assessment 12. PERSONAL AUTHOR(S) Muckler, Frederick A.; Seven, Sally; and Akman, Allan 13a. TYPE OF REPORT 13b. TIME COVERED 14. DATE OF REPORT (Year, Month, Day) 15. PAGE COUNT Final FROM 89/11 TO 90/01 1990, September 138 16. SUPPLEMENTARY NOTATION Contracting Officerts Representative, Beverly G. Knapp 17. COSATI CODES 18. SUBJECT TERMS (Continue on reverse if necessary and identify by block number) FIELD GROUP SUB-GROUP Military intelligence Ability requiremcnts, IEW systems Abilities and skills 19, ABSTRACT (Continue on reverse if necessary and identify by block number) > In 1987, the U.S. Army Intelligence Center and School (USAICS) requested that the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) investigate a number of military occupational specialty (MOS) issues related to the development and introduction of new intelligence/electronic warfare (IEW) collection and processing systems. One problem is the need to develop an in-depth taxonomy that can identify and assess the abilities and skills associated with military intelligence (MI) MOS and new IEW systems, in short a "Job Abilities Assessment System (JAAS)." Based on the Manual of Ability Requirements Scales (MARS), the JAAS method described here can evaluate 52 different abilities and skills appropriate to MI jobs and tasks. The method was applied by 13 raters to two MI MOS: 96H (Aerial Intelligence Specialist) and 97E (Interrogator) and demonstrated high inter-rater reliability and coherent profile patterns. Further, the method was used to assess abilities and skills requirements for the Unmanned Aerial Vehicle (UAV). , . 20. DISTRIBUTION/AVAILABILITY OF ABSTRACT 21. ABSTRACT SECURITY CLASSIFICATION IEUNCLASSIFIED/UNLIMITED 0 SAME AS RPT. 0 DTIC USERS Unclassified 22a. NAME OF RESPONSIBLE INDIVIDUAL 22b TELEPHONE (Include Area Code) 22c OFFICE SYMBOL Beverl (. Knapp (602) 538-4704 PERI-SA DD Form 1473, JUN 86 Previous editionsa re obsolete. SECURITY CLASSIFICATION OF THIS PAGE UNCLASSIFIED PROPOSED METHOD FOR MILITARY INTELLIGENCE JOB ABILITY ASSESSMENT EXECUTIVE SUMMARY Requirement: In 1987, the U.S. Army Intelligence Center and School (USAICS) requested that the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) investigate a number of military occupational specialty (MOS) issues related to the development and introduction of new intelligence/electronic war- fare (IEW) collection and processing systems. There has been a need to develop an in-depth taxonomy that can be used to iden- tify and determine the suitability of the skills and abilities associated with military intelligence (MI) MOS and IEW systems. Procedure: One method for identifying skills and abilities associated with MOS and IEW systems is based on the Manual of Ability Re- quirements Scales (MARS) developed by E. A. Fleishman and his associates. In the 1984 version of MARS, 52 different abilities were defined. To identify abilities requirements in new systems and ability capabilities in MOS, a flow diagram technique was expanded from 40 to the complete set of 52 abilities. To iden- tify how much of an ability is needed, an existing MARS scalar measurement method was adopted and extended. These two instru- ments, designated as the "Job Abilities Assessment System" or JAAS, were applied by 13 raters to MOS 96H (Aerial Intelligence Specialist) and MOS 97E (Interrogator). A JAAS analysis was also made of the abilities demands for operations and launch and re- covery of the Unmanned Aerial Vehicle (UAV). Findings: The test applications of JAAS to two MI MOS and one IEW system indicated that JAAS can be used in the MI context as it has been in other military job and task settings. Refinements in both the identification and the scaling parts of JAAS were specified. iii Utilization of Findings: The revised JAAS technique can potentially be used for determining the suitability of MI MOS to meet the ability and skill requirements associated with new IEW systems. JAAS can produce ability profiles of MOS, identify ability and skill requirements associated with new IEW systems, and permit an assessment in the same dimensions of the match between MOS and new systems. JAAS data are also useful for job redesign, devel- opment of training requirements, and establishment of new selec- tion criteria. iv PROPOSED METHOD FOR MILITARY INTELLIGENCE JOB ABILITY ASSESSMENT CONTENTS Page INTRODUCTION .............. ....................... 1 Background .......... .... ....................... 1 Objective .......... ....................... 1 JAAS METHOD FOR ASSESSMENT OF MOS CAPABILITIES AND IEW SYSTEM REQUIREMENT ...... ................ 4 Generating New Systems MOS Requirements ......... 4 Identifying Ability and Skill Requirements in IEW Systems ............ .................... 4 A Flow Diagram Method ....... ... ................. 8 Clustering the Abilities .......... ............... 10 Scaling the Abilities ...... ................. 11 Using JAAS Results ........ ................... 12 TEST APPLICATIONS ......... ..................... 15 Applying JAAS to MI MOSs ...... ................ 15 Applying JAAS to UAV ....... ................. 28 Comparing UAV and MOS 96H ...... ............... 32 CONCLUSION .......... ........................ 37 REFERENCES ........... ........................ 39 APPENDIX A. JOB ABILITIES ASSESSMENT SYSTEM (JAAS), PART 1 ........ .................... A-I B. JOB ABILITIES ASSESSMENT SYSTEM (JAAS), PART 2 ........ .................... B-I C. JOB DESCRIPTIONS, MOS 97E AND MOS 96H . . . . C-I LIST OF TABLES Table 1. Comparison variables for new IEW systems at the job level ........ ................ 5 2. MARS abilities and skills dimensions .... ...... 7 3. JAAS analysis--MOS 96H abilities and skills. . . 17 v CONTENTS (Continued) Page Table 4. JAAS analysis--MOS 96H abilities and skills--ordered by JAAS Part 1 "Yes" responses and Part 2 means .. ........... .19 5. JAAS analysis--MOS 97E abilities and skills. . . 20 6. JAAS analysis--MOS 97E abilities and skills--ordered by JAAS Part 1 "Yes" responses and Part 2 means .......... ... 22 7. Comparison of skills and abilities-- MOS 96H and 97E ...... ................. .. 24 LIST OF FIGURES Figure 1. An illustrative example of using the Job Abilities Assessment System (JAAS) to derive required job or task abilities .............. 9 2. An example of the Job Abilities Assessment System (Part 2) ..... ................ .13 3. Abilities and skills profile comparisons. . .. 23 4. Skills and abilities: Number of "yes" responses ....... ................... . 26 5. Comparison--UAV OPS vs. UAV L&R by clusters .. .......... ........ 29 6. Comparison--UAV OPS vs. UAV L&R by clusters and abilities .. ........... .30 7. Comparison--MOS 96H vs. UAV OPS by clusters ...... .................. . 33 8. Comparison--MOS 96H vs. UAV OPS by clusters and abilities .. ........... .35 vi PROPOSED METHOD FOR MILITARY INTELLIGENCE JOB ABILITY ASSESSMENT Introduction In earlier work, an initial model of a military intelligence (MI) military occupational specialty (MOS) taxononty was developed (Muckler, Seven, and Akman, 1989). The purpose of developing that taxonomy was to formulate an analytical framework which potentially could be used to assess the capabilities of soldiers with specific MI MOSs as well as to determine the requirements for skills and abilities associated with intelligence/electronic warfare (IEW) systems. This research note describes a technique which may be used to convert and integrate IEW system demands into the MI MOS taxonomy. The work is part of a research effort focusing on the development of methodologies and techniques which can be used to determine the suitability of MI MOSs to meet future IEW system demands. Backcround In 1987, the U.S. Army Intelligence Center and School (USAICS) requested that ARI investigate a number of MOS issues arising from the development and introduction of new IEW collection and processing systems. An important component of this research is the development of a taxonomy of the parameters critical for evaluating each MI MOS and career management field (CMF). In previous work, an initial MI MOS taxonomy was developed which included descriptors at three major levels: job-level variables, MOS-level variables, and CMF-level variables. These levels assume a sequential process that starts with micro-level evaluation of the impact of job changes on soldier's tasks and leads to macro-level evaluation where the impact on MOS aggregates and CMF can be measured. Of the many dimensions included in the initial taxonomy, those concerning abilities and skills map most directly to the issues associated with the psychological descriptors which are critical elements in establishing the suitability of existing MOSs to meet future IEW system demands. These dimensions, of the many included in the initial taxonomy, also pose more difficulty with respect to their application and measurement in manpower, personnel, and training (MPT) analysis. Obiective The objective of the current research is to refine a method to determine the abilities and skills represented by MOSs and potentially required for the operation and maintenance of IEW systems. This method for identifying and scaling abilities and skills, which is based on work by E.A. Fleishman, is here called the Job Abilities Assessment System (JAAS). JAAS in its present form is a research product requiring testing, evaluation, and refinement before operational use can be recommended. In the present instance, the focus is on its use in determining the ability and skill requirements associated with IEW systems. Its development and application, at this stage, is part of an evolutionary process. The present formulation of JAAS was used experimentally to assess its effectiveness. Test applications were made to specific IEW systems as well as to MI MOSs of the 96 CMF. At each stage of its application, the results were evaluated to determine the reliability of its use in identifying and scaling abilities and skills. Initial applications were performed by project team personnel representing a variety of different backgrounds including research psychology, MPT analysis, and Army personnel planning. The results of these limited applications are presented in a subsequent chapter, and the results were used to refine JAAS. The modified JAAS will then be made available to subject matter experts (SME) representing MOSs in the 96 CMF and IEW system development. The objective of these test applications, in addition to refining and validating JAAS, is to create catalogues of the ability and skill attributes associated with the 96 CMF MOSs, including: 96B Intelligence Analyst 96D Imagery Analyst 96H Aerial Intelligence Specialist 96R Ground Surveillance Systems Operator 97B Counterintelligence Agent 97E Interrogator 97G Counter-Signals Intelligence Specialist. In addition, the ability and skill requirements associated with four emerging IEW systems will also be catalogued. The following IEW system requirements will be identified: Joint Surveillance/Target Acquisition Radar (JSTARS) Unmanned Aerial Vehicle (UAV) Imagery Processing and Dissemination System (IPDS) Commander's Tactical Terminal (CTT). The results of the JAAS application will set the stage for additional research aimed at developing methods for determining whether the MOS and IEW abilities profiles are compatible and, if 2 not, indicating possible approaches for aligning capabilities and requirements for more effective man-machine systems. In sum, under this current effort, JAAS will be the focus of test applications, analysis, and refinement in order to determine whether its decision flow diagrams and scalar techniques can be used to assess the suitability of MI MOS capabilities in meeting IEW system demands. 3