AD-A260 816 ii I 1 " ,' i la ;i Effect of Aptitude on the Performance of Army Communications Operators John D. Winkler, Judith C. Fernandez, DTI J. Michael Polich uC E L.ECT FE- FEB 2 3 1993 E 93-02874 ov c jb-l 1R1l1l 1111 111111 111 9 2 12 183 R A , D - .... AR OY.......... TE= Ri 'TIhe, re'4,cAw dcý,rilt'd ill hI1r ýri J'l wvv's spo!(cid:127)(-,.d by lik.: United a.te. Armiy, (oiatact No. MI)A(X)03.)I ((9 h, Library of( Cowigress Cataloging In Publication Data Winkler, John D.. 1953- Effect of aptitude on the perfornance of army communications -)perators / John D. Winkler, Judith C. Fernandez,. J. Michael Polich. p. cm "Prepared for tie United States Army." "R-4143-A." Includes bibliographical references. ISBN 0-8330-1212-6 1. United States. Anny--Occupational specialties. 2. United States. Army-Electronic technicians-Abdlity teating. 3. United States. Army-Electronic technicians-Rating of. 4. United States. Army-Communication systems. I. Fernandez, Judith C., 1948- UI. Policb, J. Michael. 1I1. Title. UB337.W56 1992 623'.73--dc2O 91-42075 C1P) RAND is a nonprofit institution that seeks to improve public policy through research and analysis. Publications of RAND do not necessarily reflect the opinions or policies of the sponsors of RAND research. Published 1992 by RAND 1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138 R-41 43-A Effect of Aptitude on the Performance of Army Communications Operators John D. Winkler, Judith C. Fernandez, J. Michael Polich Accesion For NTIS CRA&I Prepared for the DTIC TAB U,tannounced El United States Army Justification ....................... By ...................... Dist, ibution I Availability Codes "-~ A'Avail and /or Dist Special DTIC QUALMI'T N8SPEC"¶D 3 RAND Approved for public release; distribution unlimited PREFACE This report documents results of RAND Arroyo Center research on the linkage between the aptitude of enlisted personnel and their abil- ity to operate and troubleshoot tactical communications systems. The purpose of the research was to improve the ability of the Army to set appropriate performance standards and to estimate the effects of per- sonnel quality levels on Army operational performance. This study was one of several research efforts on soldier performance conducted by RAND and the U.S. Army Research Institute. The results should be of interest to manpower analysts in the Army, the Office of the Secretary of Defense, and the other services, as well as to policy ana- lysts interested in the relationship between the aptitude of military enlisted personnel and their performance on combat-related tasks. THE ARROYO CENTER The Arroyo Center is the U.S. Army's federally funded research and development center (FFRDC) for studies and analysis operated by RAND. The Arroyo Center provides the Army with objective, inde- pendent analytic research on major policy and management concerns, emphasizing mid- and long-term problems. Its research is carried out in four programs: Strategy and Doctrine, Force Development and Technology, Military Logistics, and Manpower and Training. Army Regulation 5-21 contains basic policy for the conduct of the Arroyo Center. The Army provides continuing guidance and over- sight through the Arroyo Center Policy Committee (ACPC), which is co-chaired by the Vice Chief of Staff and by the Assistant Secretary for Research, Development, and Acquisition. Arroyo Center work is performed under contract MDA903-91-C-0006. The Arroyo Center is housed in RAND's Army Research Division. RAND is a private, nonprofit institution that conducts analytic re- search on a wide range of public policy matters affecting the nation's security and welfare. iii iv Lynn E. Davis is Vice President for the Army Research Division and Director of the Arroyo Center. Those interested in further informa- tion about the Arroyo Center should contact her office directly: Lynn E. Davis RAND 1700 Main Street P.O. Box 2138 Santa Monica CA 90407-2138 SUMMARY BACKGROUND AND OBJECTIVE The modern Army's battlefield operations depend to a great extent on the rapid availability of communications. During combat, comnmuni- cation among dispersed units is made possible by communications op- erators in mobile Army signal centers, which typically represent "nodes" in a network handling multichannel signals. This report de- scribes RAND research to assess the performance of signal operators and to link that performance to personnel aptitude, as measured by scores on the Armed Forces Qualification Test (AFQT), the Defense Department's test of general aptitude.' This study was one of several research efforts sponsored by the U.S. Army to develop quantitative analyses based on objective measure- ment of soldier and unit performance. The primary purposes of these research efforts were to improve the Army's ability to set appropriate performance standards and to develop quantitative estimates of the link between personnel aptitude and Army operational performance. This research examined duty tasks performed by military occupa- tional specialty (MOS) 31M, Multichannel Communications Equip- ment Operator, whose members operate communications systems providing division- and corps-level command and control. A key feature of this research was its examination of wartime-related tasks using the facilities of a high-fidelity tactical communications simula- tor to provide an objective and systematic test of performance. The research examined two principal functions: Communications system operation: the ability to establish an op- erating communications network, a task that requires interaction and teamwork among individuals at different communications fa- cilities. 'AFQT scores, along with scores on other composite scales of occupational aptitude, are derived from the Armed Services Vocational Aptitude Battery (ASVAB), the written test used to screen military applicants. The AFQT measures general mental aptitude as percentile scores from 1-99 normed on the U.S. youth population. AFQT categories are in turn defined by AFQT scores as follows: category I, percentiles 93-99; category II, 65-92; category IIIA, 50-64; category TuB, 31-49; category IV, 10-30. Category V persons, percentiles 1-9, are excluded by law from military service. v vi Communications system troubleshooting: the ability to isolate faults in multichannel communications systems, also a task requir- ing teamwork. For both functions, we examined graduates of Advanced Individual Training (AIT) at the Army Signal Center, Fort Gordon, Georgia. For communications system operation, we also report results for soldiers from active-duty Signal Corps units. The report covers two major sets of analyses. The first set examined the performance of Signal Corps operators in establishing and operating a functional communications system. We examined the effect of different levels of AFQT scores on the likelihood that a three-person group will successfully operate the system. We also examined the performance of individual operators on related tasks (those involved in preparing equipment for operation), showing the extent to which proficiency depends on individual AFQT score. The second set of analyses examined the performance of groups of operators in isolating faults in malfunctioning communica- tions systems. Taken together, these results imply that AFQT score has a direct, consistent effect on the ability of communications per- sonnel to provide effective battlefield communications to Army units. COMMUNICATIONS SYSTEM OPERATION Group Performance We used the Reactive Electronic Equipment Simulator (REES)-a high-fidelity, computer-controlled simulation facility-to obtain mea- sures of a group's ability to install and operate a realistically config- ured communications network, such as that connecting a division command post to division artillery. Groups of three operators (two at terminals and one in a relay position) were assigned to communica- tions nodes in a controlled experiment to provide a realistic wartime mix of aptitude levels and to allow us to analyze performance for varying levels of group aptitude and experience. We examined the performance of 240 such three-person groups, which were formed from a set of 720 new graduates of the Signal Center's AIT course for MOS 31M, Multichannel Communications Equipment Operator. We also examined 84 three-person groups (252 31Ms) from active-duty signal battalions. The tests were performed between September 1988 and April 1989. We wanted to assess how group outcome (i.e., success or failure in establishing a functioning system) was affected by various soldier characteristics, such as aptitude, experience, and demographic and educational background. To make this assessment, we used regres- vii sion models that allowed us to predict the effects of varying levels of aggregate group aptitude (measured as the average AFQT score of the group's members) while controlling for other differences. The results of our analyses demonstrate that the average AFQT score of a three-person group of operators is an important determinant of group success in system operation when measures of demographic background, education, and military experience are controlled. The groups with lower average AFQT scores were significantly less likely than those with higher average AFQT scores to establish a function- ing communications system involving two terminals, one relay, and two 12-channel systems. Some of our results fcr members of active- duty signal battalions can be used to illustrate these differences. The model predicts that for randomly selected groups of three soldiers in which the average AFQT score is at the midpoint of category IIIA, 63 percent will successfully operate the system within the allotted time. However, if group aptitude is reduced so that the average AFQT score falls to the midpoint of category IIIB (and all other fac- tors are held constant), the prediction is that only about 47 percent of the groups will be successful. The model thus suggests that for groups of soldiers performing the specified task, the effect of lowering the average AFQT score from the midpoint of category IIIA to the midpoint of category IIIB is to reduce the probability of successful system operation by 16 percentage points.2 The results of our analyses further indicate that the aptitudes of all group members contribute to the probability that the group will oper- ate the system successfully. For example, we found out how the probability of successful system operation relates to the number of group members (out of three) whose scores fall within categories I through IIIA.3 Our models suggest that as the number of group members in categories I through IIIA increases, the group is more likely to operate the system successfully. Each additional "high-scor- ing" member improves the probability that the group will succeed by about 8 percentage points. 2For groups of three AIT graduates, the model predicts that the probability of suc- cessful system operation for groups whose average AFQT score falls at the midpoint of category IlIA will be 15 percentage points higher than that for groups whose average AFQT score falls at the midpoint of category IIIB. 31ndividuals with scores in this range are commonly referred to as high-aptitude personnel. viii Individual Operator Performance For measures of the individual proficiency of terminal operators, we drew on data collected in earlier RAND studies that permitted us to examine the relationship between AFQT score and individual profi- ciency. The initial objective of these earlier studies was to examine the effectiveness of alternative training strategies at the Signal Center, but they also provided the basis for subsequent analyses to examine the effects of AFQT score on performance. Our analyses showed that for a variety of tasks and equipment, per- sonnel with higher AFQT scores are significantly more likely than their lower-scoring counterparts to install their assemblages cor- rectly. The first of these analyses involved 340 AIT students in MOS 31M who were tested in the REES facility. The results show that higher-scoring operators are more likely to accomplish the initial steps of the system operation task (preset and cabling of the AN/TRC- 145 terminal). Other analyses examined 336 trainees in MOS 31Q (Tactical Satellite/Microwave Systems Operator), whose performance was measured in hands-on tests administered by objective assessors who were unaware of the soldiers' AFQT scores. Again, the results show that the higher the AFQT score, the more likely the 31Q opera- tor will be to successfully prepare the equipment for operation (i.e., perform alignments and adjustments of tropospheric scatter radios) according to Army technical standards. Across these various tasks, the models predict performance differences of approximately 5 to 8 percentage points by AFQT category. Thus, for instance, if individual AFQT scores fall from the midpoint of category IIIA (close to current levels) to the midpoint of category IIIB (through a reduction in acces- sion standards, for example), we would expect the ability of operators to successfully perform the three preset, alignment, and adjustment tasks to decrease by 5 to 8 percentage points per task. COMMUNICATIONS SYSTEM TROUBLESHOOTING The final set of analyses examined the ability of groups to perform a second major function: isolate faults that interfere with the proper operation of a communications system. For this test, we used the REES facility to introduce malfunctions in an operating system com- posed of two terminals and two relays. The faults were selected so that teamwork would be a significant factor in identifying the sources of the problems and determining the appropriate corrective actions. As in our test of system operation, groups of three AIT graduates were formed and assigned to nodes to provide a range of group aptitude. Each group of AIT graduates received six malfunctions to ix isolate.4 Altogether, the test involved 187 three-person groups, rep- resenting 561 individuals. The analyses used regression models to predict group success at iso- lating faults given various levels of group aptitude and controlling for differences in the experience and the demographic and educational background of the group members.5 The results show that as the av- erage AFQT score of the group increases, so does group success at iso- lating faults to the correct assemblage and component. For example, the model predicts that 60 percent of randomly composed groups of operators whose average AFQT score falls at the midpoint of category IIIA will find two or more bugs (the median value). If the average group AFQT score falls to the midpoint of category IIIB, however, the same outcome is expected only 43 percent of the time. Thus, the de- crease in performance predicted for a decline in average AFQT score from the midpoint of category IIIA to the midpoint of category IIIB is 17 percentage points. CONCLUSIONS Our results provide considerable evidence that AFQT score has a direct effect on the ability of signal operators to provide usable battle- field communications to the Army. The effects of AFQT score on per- formance are remarkable in several respects. They manifest them- selves across a range of tasks and situations. We found the effects of AFQT score to be statistically significant for group performance on two different tasks-system operation and system troubleshooting- and for individual proficiency on tasks involving very different kinds of equipment and functions for two different operator MOSs (31M and 31Q). Moreover, for system operation, the AFQT score effects were found to be similar for both AIT graduates and unit members, even though the latter group has more experience. Perhaps more important, we found evidence that AFQT score has a sizable effect on group performance. In general, we observed that 4Each group received three troubleshooting trials of 10 minutes each with two faults inserted per trial. The assignment of faults was counterbalanced: "bugs" appeared in two different nodes per trial; over the course of the test, each examinee received an equal number of bugs. The faults consisted of two malfunctions inserted in a radio transmitter, three in a radio receiver, and one in a multiplexer. The symptoms of the faults ranged from red alarm lights to incorrect meter readings to audible cues (alarms or buzzers) that failed to sound. In all chree trials, a fault could be diagnosed most quickly if team members cooperated with one another. 5Specifically, we used ordered polytomous logistic regression to predict the proba- bility of finding some minimum number of bugs, e.g., one or more, two or more.