• 818 il ~AD-A269 DMDC TECHNICAL REPORT 93007 MILITARY APTITUDE TESTING: THE PAST FIFTY YEARS Milton H. Maier DTIC ELECTE IJ ~SEP 2 7. 1993 ,D ~B 93-22242 JUNE 1993 Approved for public release; distribution is unlimited. Personnel Testing Division f DEFENSE MANPOWER DATA CENTER This report was prepared for the Directorate for Accession Policy, Office of the Assistant Secretary of Defense (Personnel and Readiness). The views, opinions, and findings contained in this report are those of the author and should not be construed as an official Department of Defense position, policy, or decision, unless so designated by other official documentation. DEFENSE MANPOWER DATA CENTER 99 Pacific Street, Suite 155-A 0 Monterey, CA 93940-2453 Telephone: (408) 655-0400 Telefax: (408) 656-2087 MILITARY APTITUDE TESTING: THE PAST FIFTY YEARS Milton H. Maier JUNE 1993 Personnel Testing Division DEFENSE MANPOWER DATA CENTER ACRONYMS USED IN THIS REPORT ACB Army Classification Battery AFES Armed Forces Examining Stations AFHRL Air Force Human Resources Laboratory (now Armstrong Laboratory) AFQT Armed Forces Qualification Test AFVTG Armed Forces Vocational Testing Group AFWST Armed Forces Women's Selection Test AGCT Army General Classification Test AL Armstrong Laboratory AQB Army Qualification Battery AQE Airman Qualifying Examination ART Army Research Institute for the Behavioral and Social Sciences ASP Adaptability Screening Profile ASVAB Armed Services Vocational Aptitude Battery AVF All Volunteer Force BUPERS Bureau of Per-sncl (Navy) CAST Computer Adaptive Screening Test CAT Computer-adaptive testing CEP Career Exploration Program CNA Center for Naval Analyses DAC Defense Advisory Committee on Military Personnel Testing DFK Deliberate Failure Key DMDC Defense Manpower Data Center ECAT Enhanced Computer-Administered Testing ECFA Examen Calificacion de Feurzas Armadas EST Enlistment Screening Test ETP Enlistment Testing Program GAO General Accounting Office GATB General Aptitude Test Battery GED General Education Development Program IOT&E Initial Operational Test and Evaluation JPM Job Performance Measurement Project MAP Military Applicant Profile MAPWG Manpower Accession Policy Working Group MARDAC Manpower Research and Data Analysis Center MEPCOM Military Entrance Processing Command MEPS Military Entrance Processing Stations METS Mobile Examining Team Sites MTP Military Testing Program NAS National Academy of Sciences NGCT Navy General Classification Test NORC National Opinion Research Center NPRDC Navy Personnel Research and Development Center OASD-FM&P Office of the Assistant Secretary of Defense (Force Management and Personnel) OASD-FM&P-AP Directorate for Accession Policy in OASD- FM&P PTD Personnel Testing Division of Defense Manpower Data Center (The Testing Center) STP Student Testing Program USAREC US Army Recruiting Command TABLE OF CONTENTS EXECUTIVE SUMMARY ....................................... i CHAPTER 1 OVERVIEW OF MILITARY SELECTION AND CLASSIFICATION TESTING ....................... I Matching Abilities of Recruit- to the Needs of the Services ............... 2 Developing Military Selection and Classification Tests .................. 4 Interpreting the Validity Coefficient ............................... 6 Content of the Current ASVAB .................................. 7 Defining Aptitude Composites ........ ....................... 10 Test Fairness ........................................... 13 Score Scales and Qualifying Standards . ........................... 14 The ASVAB Score Scales ..................................... 15 Selection Standards . ....................................... 16 Classification Standards .................................... 17 Maintaining the Integrity of Aptitude Test Scores ..................... 19 Stability of the Military Aptitude Test Score Scales ................... 22 Structure of Military Selection and Classification Testing ................ 24 CHAPTER 2 MANAGING THE MILITARY SELECTION AND CLASSIFICATION TESTING PROGRAM ............... 27 MANAGING THE DAY-TO-DAY OPERATIONS OF THE MTP ......... 27 Current Management Structure ................................ 27 Previous Management Structures .. ........................... .30 Personnel Testing Center as Executive Agent for the ASVAB ............ 32 MANAGING MAJOR CHANGES TO THE MTP .................... 35 Inflation of AFQT Scores . .................................. 35 Addition and Deletion of Tool Knowledge Items ..................... 35 Supplementary Testing ...................................... 36 The ASVAB for High School Students ........................... 36 The Watershed Time for Aptitude Testing in the 1970s ................ 36 The All-Volunteer Force .................................... 37 The ASVAB Miscalibration ................................... 38 The 1980 ASVAB Score Scale . ............................... 38 The ASVAB Content Changes . ............................... 38 MANAGING TEST-VALIDATION EFFORTS ..................... 41 The Job Performance Measurement Project (JPM) .................... 41 r Joint-Service Evaluation of Test Fairness .......................... 41 Joint-Service Validation of Enhanced Computer-Administered Testing (ECAT) ............... 42 Computer-Adaptive Testing ................................... 43 o Development of the Applicant Screening Profile (ASP) ................ 44 Joint-Service Validation of the ASVAB ........................... 46 Distribution/ Availability e(cid:127)o&e A.t vail aSeoedr blot Specils] CHAPTER 3 THE DEPARTMENT OF DEFENSE STUDENT TESTING PROGRAM ........................ 49 The ASVAB Career Exploration Program (CEP) ..................... 49 Supporting Materials for Earlier Versions of the STP .................. 51 Content of the ASVAB and Scores Reported in the STP ................ 51 Professional Reviews of the ASVAB and the STP .................... 54 Marketing the STP . ....................................... 56 Administration of the STP .................................... 57 Validity of the ASVAB for the STP .............................. 59 Issues About Reporting ASVAB Scores in the STP ................... 61 Relationship Between the STP and the Joint-Service Program ............. 64 CHAPTER 4 NORMING AND SCALING MILITARY SELECTION AND CLASSIFICATION TESTS ......................... 67 Development of the World War II Score Scale ...................... 67 Calibrating the AFQT and Classification Tests in the 1950s and 1960s . ................................ 69 Calibrating the ASVAB 5/6/7 in the 1970s ......................... 70 Impact of the ASVAB Miscalibration ............................. 71 Reasons Why the ASVAB Miscalibration Occurred ................... 73 The Unfolding of the ASVAB Miscalibration. ....................... 75 Aftermath of the ASVAB Miscalibration ........................... 77 Development of the 1980 Youth Population Scale .................... 78 Equating the ASVAB in the 1980s ............................... 79 Validating the ASVAB: the Job Performance Measurement Project .......... 81 APPENDXES A 21st Annual Report: Qualitative Distribution ................. 83 B Lineage of the AFQT and the ASVAB ...................... 89 C Development or the ASVAB 5/6/7 ......................... 95 REFERENCES ................................................. 97 TABLES I Content of the Current ASVAB ............................. 8 2 Current Composite Definitions ............................. 9 3 AFQT Score Distributions .............................. 23 4 Predictive Validity of Interest Measures in the ASVAB 6/7 .......... 39 5 Tests in Forms 1 and 2 of the ASVAB and Composite Scores ................................ 52 6 Tests in Form 5 of the ASVAB and Composite Scores ............. 53 7 Tests in Form 14 of the ASVAB and Composite Scores ............ 55 8 Recruiter Contact Options for the STP ....................... 57 9 Correlation of the ASVAB and GATB Tests ................... 59 10 Validity of the ASVAB for Predicting Performance in Civilian Occupations ............................... 61 I1 Definitions of the Mechanical Composite by the Services and in the STP . ........................... 65 12 ASVAB 5/6/7 AFQT Percentile Scores ...................... 72 EXECUTIVE SUMMARY The purpose of the military selection and classification testing program is to improve the quality of personnel decisions in all the Military Services. In the process of accessioning military recruits, personnel decisions are made at three different times: "* The first is selection in or out of the Service, depending on whether or not a person meets the minimum qualification standards. "* The second is classification, in which the occupational specialties for which a person meets the qualifying standards are determined. "* The third stage is assignment to a specific occupational specialty for which a person is qualified. Assignment to a specific occupational specialty is based in part on qualification standards and in part on the needs of the Service. Qualification standards are more complex than just mental standards; they also include medical and moral standards. Mental standards include educational level as well as aptitude test scores. The military testing program (MTP) began with the aptitude tests used during World War II, the Army General Classification Test (AGCT) and the Navy Basic Test Battery which included the Navy General Classification Test (NGCT). The AGCT was taken by over nine million recruits who entered the Army, Air Force, and Marine Corps during World War II, and the NGCT was taken by over three million sailors. During World War II, the Army, Navy, and Air Force each set up research facilities to develop military aptitude tests. These research centers remain in existence, and their functions have expanded to include other personnel areas in addition to aptitude testing; research on training, recruiting, human factors, and team performance currently are other areas studied. From the time the peacetime draft was initiated in 1948, until the end of the Vietnam War in 1973, military accessions were obtained through a combination of the draft and voluntary enlistment. Beginning in 1973 with the All Volunteer Force (AVF), the accessioning process was restructured to obtain a sufficient number of voluntary enlistees, and the MTP likewise was radically restructured to facilitate recruiting. CHAPTER 1 OVERVIEW OF MILITARY SELECTION AND CLASSIFICATION TESTING Classification tests are designed to measure aptitudes related to performance in different occupational areas, notably maintenance and repair occupations, clerical and administrative occupations, and other occupations, such as medical, combat arms, intelligence, and operators of equipment. Aptitude composites, made up of scores received from combinations of tests in the classification batteries, are used to help determine qualification of recruits for assignments to military occupational specialties. Procedures for developing these military aptitude tests typically include the following steps: "* Identify the skills and knowledge that underlie performance in an occupational area. "* Develop experimental tests that may predict performance in the area. "* Administer the new tests to people assigned to military occupational specialties in that area. "* Follow the examinees to evaluate how well the test scores predict subsequent performance in the occupational area; usually final grades in training courses for the occupational specialties in the area are used as the criterion measure of performance. "• Evaluate the predictive validity of the new tests. "* Retain tests that improve the prediction of performance in one or more occupational areas. "* Prepare a test battery for administration to new or potential recruits for use in making personnel decisions. Procedures for systematically assigning recruits to occupations did not become widespread until the 1950s, when improved technology enabled more timely matching of the aptitudes of recruits and the needs of the Services. An elaborate communication system was established to project the personnel vacancies in field units in the various specialties, and from these projected vacancies to determine the number of people to be trained in each specialty. Accessioning goals were set to help ensure an adequate supply of qualified people to fill the training vacancies. In the 1960s, computer programs were developed to assign batches of recruits to occupational specialties in such a way as to minimize transportation costs from the sites of basic training to the sites of occupational training courses, and simultaneously maximize the mean expected performance of all recruits for all occupational specialties. The assignments were optimal in these respects. Variations of these assignment procedures have been used by all Services since then for matching people and occupational specialties. Currently, each Service derives a set of aptitude composites to help set classification standards and make classification decisions. In the 1980s, the Air Force and Marine Corps each had four aptitude composites, the Army had nine, and the Navy had twelve. The number of aptitude composites each Service uses has been based to some extent on empirical data and to a large extent on traditional practices. The Services continue to compute their own sets of aptitude composites from the ASVAB, each tending to use different combinations of tests in the various composites. Joint-service Testing With the inception of the peacetime draft in 1948, the need for a joint-service selection test to test potential inductees became apparent. The Armed Forces Qualification Test (AFQT), modeled after t.e AGCT, was introduced on January 1, 1950, and taken by millions of registrants for the draft and applicants for enlistment from 1950 until 1973. Beginning in 1973, when use of the separate AFQT was made optional by the Office of the Assistant Secretary of Defense for Force Management and Personnel (OASD-FM&P), the Services obtained an AFQT score from their classification batteries. The Army used a version of the Army ii Classification Battery introduced in 1973. The Navy used its Basic Test Battery, introduced some years earlier. The Air Force and Marine Corps used a version of the Armed Services Vocational Aptitude Battery (ASVAB) that was parallel to the one used in the Student Testing Program (STP). Thus, in the years 1973, 1974, and 1975, the examining stations had to administer three separate classification batteries, each of which required over three hours of testing time and separate testing facilities. In addition to the strain of the transition from the draft to the AVF environment, the examining stations had to cope with the burden of administering three separate test batteries. On January 1, 1976, the Services all stopped using their own classification batteries and started using the joint-service ASVAB, which was introduced to facilitate the accessioning of recruits by (a) permitting applicants to shop among the Services without taking several batteries, and (b) reducing the testing burden on examining stations. Since 1976, all applicants for all Services take the same battery of tests, and the separate Service batteries are no longer used. The AFQT score is derived from the ASVAB and is used to help set selection standards, help determine eligibility for special treatment (i.e., enlistment bonuses), facilitate manpower management, and report the quality of accessions to the Congress. Qualification Standards Qualification standards are set so that qualified people have a sufficiently high probability of being satisfactory performers, while unqualified people are likely to be unsatisfactory performers, consistent with the needs of the Services for an adequate number of recruits. Test scores are indicators of performance, and score scales are used to express the scores in terms that imply levels of expected or predicted performance. During World War II, the aptitude tests had limited use to help make selection decisions; that is, few people were excluded from serving solely because of low test scores. The widespread use of aptitude tests to set military selection standards began after World War II; since the Korean War, the bottom ten percent of the mobilization population have been excluded from serving. CHAPTER 2 MANAGING THE MILITARY SELECTION AND CLASSIFICATION TESTING PROGRAM The first AFQT, conceived in 1948 to meet the need5 of the draft, was developed by a joint- service committee that was supplemented with outside statistical and testing experts. The Army served as the executive agent for developing and administering the AFQT. Beginning in 1973, testing with the Service classification batteries was moved from recruit centers to the examining stations located throughout the country. The first ASVAB Working Group, composed of technical and policy representatives from each Service and initially chaired by a representative from the OASD-FM&P, started meeting in 1974 to develop the first joint-service ASVAB for use in testing applicants for enlistment. The ASVAB Working Group later was expanded to include representatives from the Military Entrance Processing Command (MEPCOM). iii Responsibility for research on measurement issues related to the accessioning process has been carried out by various executive agents. The Air Force served as the executive agent for development of the ASVAB from 1972 until 1989, and the Navy has served as the executive agent for related research efforts, such as developing computer-based testing. In 1989, management of the MTP took a new direction with formation of the Personnel Testing Division of the Defense Manpower Data Center. This Testing Center currently serves as the executive agent for the ASVAB, and as other research efforts reach the stage of operational use, they will become the responsibility of the Testing Center. The Testing Center works closely with the Manpower Accession Policy Working Group (MAPWG), the title assumed by the ASVAB Working Group in the mid 1980s. Issues that affect the MTP are discussed and resolved if possible at the working-group level. The MAPWG and the Testing Center keep the Defense Advisory Committee for Military Personnel Testing (DAC), composed of testing experts from academia and industry, informed of plans and results, and this Committee provides input as it sees fit. The Manpower Accession Policy Steering Committee is composed of flag officers responsible for the Services' military manpower policies, plus the Commander of MEPCOM; it is chaired by the Director for Accession Policy in OASD-FM&P (OASD-FM&P-AP). The Steering Committee reviews and approves larger MTP issues, such as introducing new forms of the ASVAB and providing troop support for research efforts. Final authority for the MTP resides in the ASD-FM&P. In the 1980s, research efforts were begun to change (a) the testing medium of the ASVAB from the paper-and-pencil mode to computer-based testing, and (b) the battery content by substituting and/or adding new kinds of tests. These efforts have received the careful attention of the entire military manpower management community. The principals are ,ware of the need to maintain the technical rigor of the MTP, and they help provide the resources needed to maintain its quality. CHAPTER 3 THE DEPARTMENT OF DEFENSE STUDENT TESTING PROGRAM Military aptitude tests are made available to students in high schools and post-secondary institutions through the Department of Defense Student Testing Program (STP). The purposes of the STP are (a) to provide recruiting leads, (b) to help determine mental qualifications for enlistment, and (c) to provide aptitude test scores and other information useful in vocational counseling and career exploration. The STP had its origins in 1958 when the Air Force offered to test students with a version of the Airman Classification Battery. Other Services quickly followed suit, and soon schools were visited by recruiters from each of the Services, all offering to test students with a military classification battery. In 1966, the OASD-FM&P directed the Services to develop a single test battery that could be used for vocational counseling of students and that would provide information useful in the accessioning process, such as accurate estimates of AFQT scores. With the Army serving as executive agent, the Services worked together to develop the first ASVAB, and it was introduced in 1968 for testing students. The first editions of the ASVAB had modest materials for use by counselors and schools on how iv