THE ARTS This PDF document was made available CHILD POLICY from www.rand.org as a public service of CIVIL JUSTICE the RAND Corporation. EDUCATION ENERGY AND ENVIRONMENT Jump down to document6 HEALTH AND HEALTH CARE INTERNATIONAL AFFAIRS The RAND Corporation is a nonprofit NATIONAL SECURITY research organization providing POPULATION AND AGING objective analysis and effective PUBLIC SAFETY solutions that address the challenges SCIENCE AND TECHNOLOGY SUBSTANCE ABUSE facing the public and private sectors TERRORISM AND around the world. HOMELAND SECURITY TRANSPORTATION AND INFRASTRUCTURE Support RAND WORKFORCE AND WORKPLACE Purchase this document Browse Books & Publications Make a charitable contribution For More Information Visit RAND at www.rand.org Explore RAND Project AIR FORCE View document details Limited Electronic Distribution Rights This document and trademark(s) contained herein are protected by law as indicated in a notice appearing later in this work. This electronic representation of RAND intellectual property is provided for non-commercial use only. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. Permission is required from RAND to reproduce, or reuse in another form, any of our research documents for commercial use. For information on reprint and linking permissions, please see RAND Permissions. This product is part of the RAND Corporation monograph series. RAND monographs present major research findings that address the challenges facing the public and private sectors. All RAND mono- graphs undergo rigorous peer review to ensure high standards for research quality and objectivity. The Weighted Airman Promotion System Standardizing Test Scores Michael Schiefer, Albert A. Robbert, John S. Crown, Thomas Manacapilli, Carolyn Wong Prepared for the United States Air Force Approved for public release; distribution unlimited PROJECT AIR FORCE The research described in this report was sponsored by the United States Air Force under Contract FA7014-06-C-0001. Further information may be obtained from the Strategic Planning Division, Directorate of Plans, Hq USAF. Library of Congress Cataloging-in-Publication Data Is Available The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. R ® is a registered trademark. © Copyright 2008 RAND Corporation All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from RAND. Published 2008 by the RAND Corporation 1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138 1200 South Hayes Street, Arlington, VA 22202-5050 4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213-2665 RAND URL: http://www.rand.org To order RAND documents or to obtain additional information, contact Distribution Services: Telephone: (310) 451-7002; Fax: (310) 451-6915; Email: [email protected] Preface This study, conducted in the Manpower, Personnel, and Training Pro- gram of RAND Project AIR FORCE (PAF), is a follow-on to Air Force Enlisted Force Management: System Interactions and Synchronization Strategies (Schiefer et al., 2007). This monograph explores a potential modification to the enlisted promotion system, one of the primary sys- tems that affect the enlisted force. Brig Gen Glenn Spears sponsored this work in fiscal year 2006 as the Director of Force Management Policy, Deputy Chief of Staff for Personnel, Headquarters U.S. Air Force. The research was accom- plished as part of a project entitled Enlisted Force Management. This monograph should be of interest to those responsible for Air Force enlisted testing and promotion policies, to those who develop enlisted promotion tests, to strength managers, to the Chief’s Group at the Air Force Personnel Center, and to enlisted career field managers. We appreciate that the findings in this monograph will not gener- ate change without the support of key leaders within the Air Force. The document was prepared with that audience in mind. RAND Project Air Force RAND Project AIR FORCE, a division of the RAND Corporation, is the U.S. Air Force’s federally funded research and development center for studies and analyses. PAF provides the Air Force with independent analyses of policy alternatives affecting the development, employment, combat readiness, and support of current and future aerospace forces. iii iv The Weighted Airman Promotion System: Standardizing Test Scores Research is conducted in four programs: Aerospace Force Develop- ment; Manpower, Personnel, and Training; Resource Management; and Strategy and Doctrine. Additional information about PAF is available on our Web site: http://www.rand.org/paf/ Contents Preface ............................................................................. iii Figures ............................................................................. ix Tables .............................................................................xiii Summary ..........................................................................xv Acknowledgments .............................................................. xix Abbreviations .................................................................... xxi CHAPTER ONE Introduction ....................................................................... 1 The Air Force Cannot Be Achieving Its Primary Enlisted Promotion Objective ....................................................................... 2 The Motivation for Enlisted Promotion Equity ................................ 4 The Air Force Has Not Achieved All TOPCAP and Secondary Promotion Objectives ......................................................... 7 Organization of the Monograph ................................................10 CHAPTER TWO The Weighted Airman Promotion System: Motivation, Mechanics, Reality, and Theory .........................................................13 A Fundamental Relationship ....................................................17 Reality ..............................................................................18 Theoretical Impacts of Differences in Variation .............................. 24 Measuring Variation ............................................................. 26 E5 WAPS Component Impacts .............................................. 26 E6 WAPS Component Impacts ...............................................31 E7 WAPS Component Impacts ...............................................33 v vi The Weighted Airman Promotion System: Standardizing Test Scores E8 WAPS Component Impacts .............................................. 36 E9 WAPS Component Impacts ...............................................37 Chapter Summary ................................................................39 CHAPTER THREE Standardizing Test Scores ......................................................41 What Is Test Standardization? .................................................. 42 Why Standardize? ................................................................ 42 Approaches to Standardizing PFE/SKT Scores ............................... 43 Standardization Mechanics ..................................................... 44 An Alternative Approach to Standardization ................................. 46 Disclaimer ..........................................................................47 CHAPTER FOUR Testing Impact and Selection Timing .......................................49 Selections to E2–E4 ..............................................................49 Selections to E5 ................................................................... 50 A Univariate Perspective of Selections to E5 .................................52 A Multivariate Perspective of Selections to E5 ............................. 54 Selections to E6 .................................................................. 56 A Univariate Perspective of Selections to E6 ................................57 A Multivariate Perspective of Selections to E6 ..............................58 Selections to E7 ................................................................... 60 A Univariate Perspective of Selections to E7 ................................61 A Multivariate Perspective of Selections to E7 ..............................63 Selections to E8 .................................................................. 64 E8 Selection Factors ............................................................65 Selections to E9 ....................................................................67 Chapter Summary ............................................................... 68 CHAPTER FIVE Effects ..............................................................................71 Inconsistent and Random Selection Standards ................................71 Senior NCO (E7–E9) Manning ................................................72 Unequal Opportunities to Make E8 and E9 ...................................73 Contents vii Disproportionate Selectivity for E9 Nominative and Commander- Involvement Positions ........................................................74 Standardization Strategies ........................................................76 Transition Issues ...................................................................79 Standardization Costs ........................................................... 80 CHAPTER SIX Conclusions and Recommendation ..........................................83 Conclusions ........................................................................83 Recommendation .................................................................83 APPENDIXES A. The Impact of WAPS Factors by Grade and AFSC .................. 87 B. AFSC Titles ................................................................. 97 C. WAPS Changes over Time .............................................. 101 D. Periodic WAPS Revalidation ........................................... 105 E. Four Approaches to Measuring the Impacts of WAPS Factors ... 107 F. Multivariate Models to Predict Selection Rates to E5 ............. 113 G. Multivariate Models to Predict Selection Rates to E6 ............. 125 H. Multivariate Models to Predict Selection Rates to E7 ............. 131 I. ACT, SAT, and ASVAB Approaches to Standardization ........... 137 References ....................................................................... 143