CORPORATION Air Force Management of the Defense Acquisition Workforce Development Fund Opportunities for Improvement John A. Ausink, Lisa M. Harrington, Laura Werber, William A. Williams, John E. Boon, Jr., Michael H. Powell For more information on this publication, visit www.rand.org/t/RR1486 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-0-8330-9420-9 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2016 RAND Corporation R ® is a registered trademark. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface The Defense Acquisition Workforce Development Fund (DAWDF) was established in the fiscal year 2008 National Defense Authorization Act to assist in the recruitment, training, and retention of U.S. Department of Defense (DoD) acquisition personnel so that they can properly perform their mission, provide appropriate oversight of contractor performance, and ensure that DoD receives the best value for the expenditure of public resources. The DAWDF charter requires the Director of Acquisition Career Management in each of the services to provide DoD with measurable objectives and to track the performance of programs receiving resources from the fund. To satisfy this requirement, the Air Force’s Office of Acquisition Career Management, SAF/AQH, asked the RAND Corporation to help develop a framework for using DAWDF to effectively invest in recruiting, retaining, and developing the civilian acquisition workforce. This report discusses process, data collection, and metrics improvements that will help the Air Force’s DAWDF program manager better initiate and track the performance of DAWDF initiatives. It also describes a general framework for prioritizing DAWDF proposals that could be useful for all DoD DAWDF program managers. Finally, it presents insights on the use of recruiting, training, and retention initiatives outside DoD, based on interviews with 21 commercial organizations that have been listed among Fortune magazine’s “100 Best Companies to Work For.” This research was conducted within the Manpower, Personnel, and Training Program of RAND Project AIR FORCE. It should be of interest to DAWDF program managers in other services, the Joint Staff, and DoD agencies. RAND Project AIR FORCE RAND Project AIR FORCE (PAF), a division of the RAND Corporation, is the U.S. Air Force’s federally funded research and development center for studies and analyses. PAF provides the Air Force with independent analyses of policy alternatives affecting the development, employment, combat readiness, and support of current and future air, space, and cyber forces. Research is conducted in four programs: Force Modernization and Employment; Manpower, Personnel, and Training; Resource Management; and Strategy and Doctrine. The research reported here was prepared under contract FA7014-06-C-0001. Additional information about PAF is available on our website: www.rand.org/paf iii iv Table of Contents Preface ............................................................................................................................................ iii Figures and Tables ........................................................................................................................ vii   Summary ........................................................................................................................................ ix   Acknowledgments ......................................................................................................................... xv   Abbreviations .............................................................................................................................. xvii   1. Introduction ................................................................................................................................. 1   Research Approach ................................................................................................................................... 2   Outline of This Report .............................................................................................................................. 3   2. Overview of the Defense Acquisition Workforce Development Fund ....................................... 5   DAWDF Funding ..................................................................................................................................... 5   DAWDF Management .............................................................................................................................. 7   DAWDF Initiative Categories ................................................................................................................ 13   3. DAWDF Management Phases .................................................................................................. 15   Phase One: Developing and Communicating a Strategy ........................................................................ 16   Phase Two: Developing the Call for Requirements and Collecting Requests ........................................ 21   Phase Three: Approving Initiatives and Monitoring Execution ............................................................. 27   4. Evidence-Based Justification and Metrics ................................................................................ 33   Data Sources and Limitations ................................................................................................................. 33   Metrics .................................................................................................................................................... 36   Tuition Assistance: A Data and Metrics Example .................................................................................. 36   Conclusions ............................................................................................................................................. 39   5. Lessons from Industry ............................................................................................................... 41   Introduction ............................................................................................................................................. 41   Individual-Level Strategies: Financial .................................................................................................... 43   Individual-Level Strategies: Nonfinancial .............................................................................................. 47   Corporate-Level Incentives ..................................................................................................................... 51   Strategy Portfolio Management .............................................................................................................. 56   Lessons for the Air Force ........................................................................................................................ 58   6. A Proposed DAWDF Decision Framework ............................................................................. 61   Evaluation Framework ............................................................................................................................ 61   Conclusion .............................................................................................................................................. 65   7. Conclusions and Recommendations ......................................................................................... 67   Appendix A. Air Force DAWDF Initiatives for FY 2015 and Line Item Descriptions ............... 69   Appendix B. Detailed Comments Related to Lessons from Industry ........................................... 73   Appendix C. Industry Practices Interview Protocol ..................................................................... 79   Selected Bibliography ................................................................................................................... 83     v vi Figures and Tables Figures 2.1. DAWDF Funding Sources in FY 2015 .................................................................................... 6 2.2. Chains of Command for DAWDF Management ................................................................... 10   2.3. Notional Air Force Acquisition Organizational Structure ..................................................... 11   3.1. DAWDF Process Timeline .................................................................................................... 16   5.1. Industry Breakdown of Fortune 100 Interview Sample ......................................................... 42     Tables S.1. Evaluation Framework for DAWDF Initiatives ................................................................... xiii 2.1. Mandated and Minimum DAWDF Funding Amounts Through 2018 .................................... 6   2.2. DoD DAWDF Management .................................................................................................... 8   2.3. Air Force DAWDF Spending FY 2015–2017 ($ M) ............................................................. 14   4.1. Data Requirements for DAWDF Program Line Items .......................................................... 35   4.2. Merged AFAEMS and CAE Data on Tuition Assistance Recipients .................................... 38   5.1. Summary of Individual-Level Financial Strategies Cited by   RAND’s Sample of Fortune’s “100 Best Companies to Work For” .................................... 43 5.2. Summary of Individual-Level Nonfinancial Strategies Cited by   RAND’s Sample of Fortune’s “100 Best Companies to Work For” .................................... 47 5.3. Summary of Corporate-Level Strategies Cited by   RAND’s Sample of Fortune’s “100 Best Companies to Work For” .................................... 52 6.1. Evaluation Framework for DAWDF Initiatives .................................................................... 62   6.2. Recruit and Hire: First Stage of Framework Analysis ........................................................... 63   6.3. Hiring Metrics ........................................................................................................................ 64   6.4. Train and Develop: First Stage of Framework Analysis ....................................................... 65   6.5. Training and Development Metrics ....................................................................................... 65   B.1. Examples of Employee Referral Bonus Programs in   RAND’s Sample of Fortune’s “100 Best Companies to Work For” .................................... 74 B.2. Examples of Fortune 100 Merit Award Programs ................................................................ 75   B.3. Examples of Sabbatical Programs ......................................................................................... 76   B.4. Employee Perception Data Examples ................................................................................... 77     vii viii Summary The Defense Acquisition Workforce Development Fund (DAWDF) was established in 2008 to provide funds for the recruitment, training, and retention of acquisition personnel. The fund, which is financed by a combination of direct appropriations and funds provided by military departments and defense agencies, is meant to pay for initiatives in three major categories: recruit and hire new acquisition personnel, train and develop members of the existing workforce, and retain and recognize highly skilled personnel. Since the fund’s establishment, the U.S. Air Force has contributed more than $600 million to DAWDF and received more than $451 million for various initiatives. The Air Force’s Director of Acquisition Career Management (DACM), who is in the Office of the Assistant Secretary of the Air Force for Acquisition, Air Force Acquisition Career Management [SAF/AQH]), is responsible for managing the Air Force’s share of the money. In recent years, resources available to the Air Force through DAWDF have been sufficient to pay for all proposals received by the DACM, so there has been little need to develop a formal process to prioritize them. Recognizing that this will not always be the case, the DACM asked the RAND Corporation to help explore ways to ensure that the funds are used effectively. To do this, we examined legislation, regulations, and other documents related to the fund; interviewed acquisition workforce subject-matter experts and users of DAWDF money in headquarters organizations, major commands (MAJCOM), and centers; analyzed acquisition workforce databases; and interviewed managers in 21 companies that have been recognized by Fortune magazine as being among the “100 Best Companies to Work For.” Our findings and recommendations can be summarized in four categories: DAWDF management processes, evidence-based justification and metrics, lessons from industry, and a proposed evaluation framework. DAWDF Management Processes For convenience, we divide the annual DAWDF approval process into three phases: • Phase One: developing and communicating a strategy • Phase Two: developing the call for requirements and collecting requests • Phase Three: approving initiatives and monitoring execution. We examined the activities undertaken by the Air Force in each phase, looked at how similar functions were handled in the U.S. Army and the U.S. Navy, and identified a number of ways in which Air Force processes could be improved. In Phase One, we found that potential Air Force users of DAWDF funds were unaware of strategic guidance related to the acquisition workforce and were therefore unable to link potential ix DAWDF initiatives to strategic objectives. When we initially reviewed the process for gathering DAWDF initiatives from acquisition organizations and functions, we observed that MAJCOM and center functional managers in the Air Force had little knowledge of the DAWDF process and were not reviewing annual submissions being made through personnel channels. This clearly hindered attempts to address acquisition-workforce development issues from a strategic perspective. SAF/AQH needs to provide better guidance about DoD and Air Force–level strategic priorities related to acquisition and the acquisition workforce so that personnel and functional managers can better shape their DAWDF proposals to Air Force needs. Recent calls for requirements (Phase Two) have done a good job of providing basic information about how to apply for DAWDF funding, but the Air Force needs to do a better job of ensuring that proposed initiatives clearly justify the need for the proposal and that valid metrics will be used to judge the performance of the initiative. Additionally, the DACM needs to encourage communication between the personnel and functional communities at all levels to ensure that good ideas are shared and innovative suggestions are not stifled. The DACM should consider hosting a DAWDF conference that brings together functional and personnel representatives who work at the headquarters, MAJCOM, and center levels to help ensure that proposed initiatives address important needs. The Air Force currently has no formal documentation of its DAWDF processes. It would be especially useful to formalize the DAWDF initiative approval process of Phase Three. This will address any concerns about process transparency, may improve interaction with potential fund users, and will help SAF/AQH respond to queries or audits that may seek justification for funding decisions. Evidence-Based Justification and Metrics The DAWDF charter requires DACMs to develop measurable objectives and to track performance metrics for approved initiatives. Doing so has been a difficult task for all organizations that use DAWDF money, not just the Air Force. Despite the difficulties, however, evidence-based justification and monitoring need to be improved. One difficulty is the determination of appropriate data to use for assessing performance. Enterprise-wide databases, such as the Defense Civilian Personnel Data System (DCPDS), may be useful for assessing the impact of Air Force–wide hiring initiatives, for example, but may not be useful for DAWDF initiatives that target small, installation-level populations. In addition, there may have been insufficient time for the impact of some DAWDF programs to show up in enterprise-wide data. Thus, different data sources are needed to track the effectiveness of different types of DAWDF initiatives. In some cases, available databases need to be merged to better justify proposed initiatives and facilitate the development and tracking of metrics associated with the performance of DAWDF initiatives. x