Addressing Risk ManageTmUeTnOt iRn INADLI Acquisition Programs ADDRESSING RISK MANAGEMENT IN NON-DEVELOPMENTAL ITEMS ACQUISITION PROGRAMS Michael R. Steves Non-developmental Items (NDI) acquisition programs are enjoying popular support as faster, cheaper alternatives to new start research and development programs. Unfortunately, DoD policy on risk management in NDI programs is lacking. In the new, less-restrictive DoD Directive 5000.1 Acquisition Policy and DoD Regulation 5000.2-R, the program manager (PM) is expected to tailor risk management practices to the needs of the program. Tailoring DoD risk management policy to support NDI program management leaves the PM too much guesswork. An NDI PM’s risk management program cannot reasonably benefit from DoD risk management guidance, procedures, and tools because he or she is focused on new development program risks and risk management practices. Missing is any explicit consideration of unique NDI risks and risk management requirements. NDI PMs need more structured policy and instruction regarding NDI risk management for the streamlined, accelerated NDI acquisition environment. A lesson that we have learned is that we need a published risk management plan as the source of NDI risk management program decisions and actions. This article makes specific recommendations for including NDI risk management in DoD policy. T oday’s era of defense downsizing when commercial or nondevelopmental has the military services scrambling solutions exist. Risky acquisition strate- to protect research, development, gies and mistakes in new research and de- and acquisition (RD&A) funds and to pre- velopment acquisitions have become serve their acquisition programs. Quality harder to defend. A new program’s risks of life issues, training and readiness, and will demand close management to prevent peacekeeping missions continue to draw unanticipated or poorly planned-for events funds away from RD&A programs and from exposing the program to funding cuts modernization. The armed services and or cancellation. their acquisition program managers are To the rescue, or so many expect, comes challenged to justify lengthy and costly the Non-developmental Item (NDI). NDI new research and development programs system acquisitions give PMs and the gov- 39 Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 1997 2. REPORT TYPE 00-00-1997 to 00-00-1997 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Addressing Risk Management in Non-Developmental Items Acquisition 5b. GRANT NUMBER Programs 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION U.S. Army Test & Evaluation Command,Aberdeen Proving REPORT NUMBER Ground,MD,21005 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES Acquisition Review Quarterly, Winter 1997 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 16 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 Acquisition Review Quarterly—Winter 1997 ernment rapid, lower cost access to ad- RISK MANAGEMENT AND vanced technology. NDI acquisition strat- NON-DEVELOPMENTAL ITEM OVERVIEW egies offer the promise of risk mitigation and a lower risk means of meeting the DoD’s risk management concepts are armed services’ urgent mission needs and consistent with those in industry. DoD operational requirements. NDI program guides on risk management are similar to management is not, however, without risk. the Program Management Institute’s An NDI program manager must tailor the (PMI) body of knowledge on risk man- program’s risk management to the unique agement. The PMI approach reflects the risks and uncertainties in NDI system ac- general body of knowledge on risk and quisitions.1 NDI risk management de- risk management that may be found in mands more than tailoring. Problems arise business and industry today. with NDI risk management because NDI risks and necessary risk management pro- RISK DEFINITION cesses are not well understood or de- Webster’s 3d International Dictionary scribed in Department of Defense (DoD) defines risk as “the possibility of loss, in- policy. NDI risk management demands jury, disadvantage, or destruction; some- policy, framework, and tools to assist the one or something that creates or suggests PM. An NDI acquisition’s purpose is to a hazard or adverse chance; the product simplify and accelerate the acquisition of the amount that may be lost and that process to meet the program’s, and ulti- probability of losing it (p. 1961). DoD mately, our soldier’s needs. Successful Directive (DoDD) 5000.1 and DoD Regu- NDI acquisition programs would benefit lation (DoDR) 5000.2-R do not define risk from a more clearly mapped risk manage- to coincide with discussions of risk man- ment policy structure, rather than broad agement or risk reduction. Risk is often expectations of tailoring, to fulfill their characterized by type: cost, schedule, tech- promise of simple, rapid, and reduced-risk nical performance, supportability, and pro- acquisitions. The simple, fast, cheaper grammatic risk (DSMC, 1989). route to NDI risk management success These five types of risk interact to af- requires good directions and signposts fect a program’s overall performance. advising PMs of risks and precautionary measures. 1 Tailoring is a practice that means modifying the acquisition process to reduce the time required to meet user (soldier) needs. Tailoring is expected to be done according to common sense, effective business practice, laws, regulations, and a program’s anticipated fielding date (DoD Directive 5000.1, 1996). The concept of tailoring risk management is considered valuable by DoD as part of the program manager’s authority. MAJ Micheal R. Steves is Test and Evaluation Coordinator for the PATRIOT Missile System, Forward Area Air Defense Systems, and Task Force XXI Tactical Operations Centers at Head- quarters, U.S. Army Test and Evaluation Command, Aberdeen Proving Ground, Maryland. 40 Addressing Risk Management in NDI Acquisition Programs Their basis is in the five-phase, four-mile- Upon identification of a program’s stone, full-scale development acquisition risks, a four-part process begins. The four cycle found in the now-rescinded DoD defense risk management process compo- Instruction 5000.2. nents are planning, assessment, analysis, and handling. Once these components are RISK MANAGEMENT implemented, the risk mitigation effort is The DoDD 5000.1 (1996, p.4) defines underway. Risk mitigation is the combi- risk management as an approach that “en- nation of risk handling and risk controls. compasses risk identification, mitigation, DoDD 5000.1 notes that technology dem- and continuous tracking, and control pro- onstrations, prototyping, and test and cedures that feed back through the pro- evaluation are the available techniques to gram assessment process to decision au- assess and handle risk. The PM should thorities.” This definition differs from the document the risk management program, business definition in that it stresses a sys- process, and techniques. Publication of a tematic approach rather than “art and sci- risk management plan (RMP) is one ence.” Risk management identifies and means of accomplishing this. evaluates the program areas vulnerable to high levels of uncertainty. Its purpose is NON-DEVELOPMENTAL ITEMS to provide a means of comparing risk An NDI is a previously developed fed- management performance to a standard eral, state, local, or foreign government and of tracking risk-related information. item with little or no development effort By doing so, DoDR 5000.2-R adds that required (5000.2-R, Part 3, p.5). NDI ac- risk management becomes an essential el- quisition strategies look to governments ement of a program’s acquisition strategy or commercial vendors as sources for a (1996, Part 3, p. 3). developed and available item. DoDD 5000.1 encourages the armed services to THE DEFENSE RISK MANAGEMENT PROCESS use the most cost-effective materiel solu- A PM’s risk management process tion over a system’s life cycle. The first should set objectives and continually as- choice among materiel alternatives after sess the program for obstacles that hinder buying commercial off-the-shelf items is accomplishment of those objectives. It al- to procure or modify previously developed lows the PM to formulate alternative courses U.S. military or Allied systems or equip- of action and to make rational decisions on ment (p. 6). monitoring and controlling the outcomes or The armed services in the past have de- consequences of program events. fined NDIs more specifically.2 The Army 2 Confusion sometimes results from the attempt to distinguish an NDI from a commercial off-the-shelf (COTS) item. A COTS item does not make up a portion of NDIs. COTS items are commercial hardware or software items not yet modified by the government, items that are in the commercial inventory or production, that have proven their performance in a similar environment, that have an existing support structure, have an internal support structure, have an internal configuration that flows with commercial changes, and generally are integrated with other hardware and software items to become part of a system or subsystem capability. An NDI therefore might be described as a noncommercial off-the-shelf item. 41 Acquisition Review Quarterly—Winter 1997 version distinguishes NDI in three catego- 2. Preparing minimum but not objec- ries (Quindlen, 1989): tive (“gold-plated”) user require- ments to support lower program 1. Category A (basic NDI). No modi- costs and rapid system fielding. fications are required. 3. Providing DoD with greater access 2. Category B (NDI adaptation). This to state-of-the-art technology to category is sometimes referred to as keep pace with changing threats, “Ruggedized NDI.” emerging technologies, and innova- tive combat systems. 3. Category C (NDI integration). This category is sometimes referred to as The danger to NDI programs and their “Militarized NDI.” effective risk management is in forcing NDI solutions on PMs and their custom- ADVANTAGES AND DISADVANTAGES OF NDIS ers (the user community), when those NDIs are considered a means of miti- NDIs neither meet requirements nor tran- gating program cost and schedule risks. sition effectively make the transition from Risk mitigation can be implemented in a current government or commercial appli- shortened or streamlined Program Defi- cations to battlefield environments. nition and Risk Reduction Phase and En- Broadly stated DoD risk management gineering and Manufacturing Develop- policy and guidelines then contribute to ment (EMD) Phase of the acquisition pro- disadvantages as well as advantages of cess. The advantages of NDIs, according to buying NDIs. the Government Accounting Office (GAO), The disadvantages of NDIs and NDI include shorter acquisition time, reduced acquisition strategies stem from the new cost, proven technical performance or ma- start research and development program ture technologies, and simplified contract- paradigms in the acquisition community. ing procedures involved in procuring estab- A developmental item PM traditionally lished products. Yet NDI acquisitions do not faces cost, schedule, performance, sup- eliminate risk they may actually introduce portability, and programmatic risks; new, significant risks. These risks will be NDI program risks are often unique. In examined shortly. reviewing the literature, I have found that An effective NDI acquisition program the following areas make up the most must fulfill the service’s needs by field- pressing potential risks facing NDI pro- ing mature technology that contributes to gram objectives: a) requirements, b) NDI combat readiness while satisfying the acquisition management environment, c) user’s expectations. The three features that performance specifications and nongov- attract substantial DoD and Congressional ernment standards (S&S), d) test and support for NDI or commercial alterna- evaluation (T&E), and e) integrated logis- tives but challenge the PM are: tics support (ILS). 1. Meeting user requirements based on The NDI PM’s principal disadvantage available government or commer- is a lack of both DoD policy attention paid cial market solutions. to the above NDI program risks and the 42 Addressing Risk Management in NDI Acquisition Programs procedures and tools to manage them. This the NDI-specific risks listed earlier. The void in NDI-specific risk management consequences of NDI-specific risk events guidance leaves the PM without current in these areas can have tremendous affect references for structuring a risk manage- on cost and schedule in the long run. ment program or publishing an RMP. NDI PMs currently conduct risk iden- Meanwhile, the pressure to use NDI solu- tification, planning, analysis, assessment, tions at the expense of a service’s original handling, and documentation according to needs and requirements may yield fielded the 5000.1 and 5000.2-R. In doing so, the items that aren’t compatible or interoper- PM applies developmental acquisition risk able with the intended operational envi- management policy and procedures focus- ronment or battlefield systems. ing on traditional developmental program risks to nondevelopmental systems. The point is that many NDI acquisition PMs THE DIVERGENT PATHS must “think on the move” without explicit directions or road maps. Without DoD or The crux of the NDI risk management service-specific NDI risk management issue is that the paths of risk management policy, instructions, templates, or unoffi- policy and NDI policy do not cross. DoD cial emphasis in Defense Systems Man- policy emphasizes both risk management agement College (DSMC) risk manage- and NDI but in differing directions. Risk ment publications or instruction, PMs will management policy in the DoD 5000 se- continue to rely on gut feelings, develop- ries keys on developmental items and ac- mental lessons learned, and incomplete quisition strategies. This is demonstrated references. NDI in the traditional cost, schedule, and per- risk manage- “The crux of the NDI formance risk orientation of current policy. ment planning risk management NDI policy focuses on risk mitigation can be second- issue is that the gained through use of established govern- guessed since paths of risk man- ment sources of supply. Cost and sched- no official refer- agement policy and ule benefits aside, no description or in- ences can be NDI policy do not struction for NDI program risk manage- cited to justify cross.” ment appears in the DoD 5000 series. The risk manage- 5000.1 and 5000.2-R policy voids exist ment decisions. Lack of published DoD within the expectation that the program RMP formats increases this risk. manager will “tailor” (overlap, combine, There are reasons for this policy void. or omit) risk management requirements to The risks in NDI acquisition are not as an NDI acquisition strategy. widely described or documented as the NDI acquisitions streamline acquisition benefits. The absence of documentation policy and particularly, risk management and description is due to lack of familiar- policy, and result in tradeoffs between pro- ity by PMs, users, and DoD overall with gram objectives. While NDI programs the NDI acquisition risks in competitive, may mitigate risks in cost and schedule, commercial (defense- and nondefense-re- technical performance, supportability, and lated) industries. These industry sources programmatic risks may be heightened by provide DoD with a growing number of 43 Acquisition Review Quarterly—Winter 1997 NDI systems not initially developed to phase of the acquisition process. Risk meet a specified armed service requirement. management encompasses identifi- The defense acquisition community’s un- cation, mitigation and continuous familiarity with NDI risks and risk man- tracking, and control procedures that agement means a lack of understanding feed back through the program as- with commercial risk management prac- sessment process to decision authori- tices and standards. Without formal risk ties. management policy and guideline initia- tives by DoD acquisition participants, de- This guidance lacks reference to a de- finitive policy and instruction has insuffi- finitive risk management planning struc- cient support or momentum to be officially ture. Currently, NDI PMs are expected to implemented. tailor the DoDD 5000.1 risk management guidance to their NDI systems acquisi- tions. The following could be added for MERGING THE TWO PATHS more structural content to both NDI and developmental risk management planning: The challenge of DoD is to bridge the gap between developmental acquisition Each milestone decision point will risk management policy and practices and include a review of the updated risk those that apply specifically to NDI risks. management plan (RMP) and mea- Effective application of NDI-based risk sures taken to identify, assess, ana- management policy and practice can save lyze, handle, and document program NDI and possibly COTS PMs valuable risks. Continuity will be maintained time, money, and manpower resources between the RMP and discussions of while assisting them in effectively meet- program risks in the Integrated Pro- ing program goals and objectives. gram Summary, Acquisition Plan, Systems Engineering Master Plan, PROPOSED DOD POLICY REVISIONS Test and Evaluation Master Plan, and DoDD 5000.1 Risk Management Integrated Logistics Support Plan. Policy. DoDD 5000.1 contains five areas which could be revised to make its risk The RMP could be modeled after management guidance suitable for NDI DSMC Risk Management or DSMC Sys- PMs. The first area is the risk management tems Engineering Management Guides’ subsection of Part 1 (p. 4). It presently RMP formats and tailored to the require- states: ments of the individual program. It would require the “teaming” of the user and the [Program mangers] and other acqui- PM to integrate a risk management focus sition managers shall continually that supports operational need and require- assess program risks. Risks must be ments development. It could then be moni- well understood, and risk manage- tored through the Integrated Product Team ment approaches developed, before Process. decision authorities can authorize a The second area is the Event-Oriented program to proceed into the next Management subsection of Part 2 (p. 6), 44 Addressing Risk Management in NDI Acquisition Programs which describes “a rigorous, event-ori- these risks. Trade-offs shall be con- ented management process that empha- sidered primarily when current sys- sizes effective acquisition planning, im- tems exist to sustain the force until proved and continuous communications the NDI system is fielded. with users, and prudent risk management by both the government and industry.” The third area of interest is a combined This statement overlooks risk areas, es- look at the Test and Evaluation and Mod- pecially those in NDI programs: require- eling and Simu- ments, NDI acquisition management, lation subsec- “The risks in NDI acquisitions are T&E, S&S, and ILS. tions in Part 2 testing too little, too The DoD 4245.7-M, published in 1985, (p. 6). The Risk much, and testing to described the areas of risk that jeopardize Assessment and the wrong or modi- achievement of successful cost, schedule, Management fied requirements and performance objectives. It did not subsection re- using contractor test describe NDI risks acting as cost, sched- fers to test and and evaluation ule, or performance drivers. The Hierar- evaluation. In- inputs.” chy of Materiel Alternatives subsection tegrated testing immediately following Event-Oriented and evaluation, including modeling and Management currently doesn’t refer to simulation, is used to identify areas of them either. Event-oriented managers are technical risk, to “assess attainment of empowered with the authority to make technical performance parameters... to re- trade offs between performance and duce the time, resources, and risks of the schedule for the sake of cost objectives. acquisition process” (p. 6). The Risk As- The authority to trade off performance or sessment and Management subsection it- schedule to control program cost risks can self reads: “To assess and manage risk, imply the acceptance of force readiness [program managers] and other acquisition and doctrinal capability gaps. If no opera- managers shall use a variety of techniques, tionally effective replacement systems including technology demonstrations, exist, then such trade offs can have sig- prototyping, and test and evaluation. nificant impact on force training and com- NDI acquisition strategies often include bat effectiveness. accelerated and streamlined (depending on The following could be added to reflect the quantity and quality of contractor test NDI risks: programs and data) testing and evaluation processes or simply contractor test and NDI programs shall examine and evaluation data reviews. The risks in NDI track the effects of requirements, acquisitions are testing too little, too much, NDI acquisition management, Test and testing to the wrong or modified and Evaluation, Specifications and requirements using contractor test and Standards, and Integrated Logistics evaluation inputs. The following could be Support risks on cost, schedule, and included to address these concerns: performance objectives. Measures of performance for risk handling shall NDI market analysis of proposed de- be proposed in the RMP concerning signs, NDI test and evaluation pro- 45 Acquisition Review Quarterly—Winter 1997 grams, and NDI performance speci- sponsibility in NDI programs should rest fications shall carefully consider in large part with the contractor, given the trade offs in test and evaluation for supposedly mature system being offered. the sake of short-term cost and The following could be added: “Contrac- schedule objectives. Such trade offs tor RMPs shall be specified in solicitations introduce risks to NDI life-cycle as a deliverable. These shall be consistent costs and support. NDI contractors’ with DoD program managers’ RMPs. Risk test and evaluation data shall be sharing shall be emphasized in contracts screened and validated indepen- and monitored by the DoD–contractor dently by the operational test and team through integrated product teams evaluation community. (IPTs). The fifth area combines its focus on the The fourth area considers contractor re- Tailoring, IPT, and Management Control sponsibilities again from the Risk Assess- subsections in Part 2 (p. 8). The guidance ment and Management subsection. Previ- makes no mention of NDI risks even as ously, solicitation documents required materiel alternatives are repeatedly called contractors to identify risks and specify for prior to beginning a new major defense plans to assess and eliminate risks or re- acquisition program. Risk plans, assess- duce them to acceptable levels. Now, ments, and controls, which are subject to DoDD 5000.1 directs that (p. 4) “To en- tailoring, should be comprehensive in na- sure an equitable and sensible allocation ture and include the full spectrum of risks of risk between government and industry, expected for a NDI program. Tailoring, [program mangers] and other acquisition IPTs, and management control could ad- managers shall develop a contracting ap- dress the five NDI-specific risk areas by proach appropriate to the type of system directing that an RMP be prepared and re- being acquired.” viewed at each milestone. The shortfall in this guidance on con- The DoDD 5000.1 risk management tractor risk management is the lack of par- policy revisions are an important first step. allelism between the NDI PM’s RMP and Policy must clearly state the risk manage- requirements ment requirements and guidelines as they and those of the specifically affect a NDI PM’s risk man- “The NDI risk man- contractor. The agement program. These changes will pro- agement efforts NDI risk man- vide impetus for revisions to DoDR should emphasize teaming and risk- agement efforts 5000.2-R, which will be examined next. sharing to relieve should empha- the burden of risk size teaming DoDR 5000.2-R Risk Management managment from and risk-sharing Policy and Procedures. The NDI pro- resting primarily on to relieve the gram managers need more definitive and the government.” burden of risk explicit policy and procedures in DoDR management 5000.2-R pertaining to their risk manage- from resting primarily on the government. ment programs. The 5000.2-R, even with This is particularly true with the acceler- its increased commercial and nondevelop- ated nature of NDI acquisitions. Risk re- mental item emphasis, neglects NDI risk 46 Addressing Risk Management in NDI Acquisition Programs management. With increasing competition Additionally, with NDI and streamlining for shrinking DoD RD&A budgets and re- initiatives more common today, more de- sources, DoD and Congressional risk tol- tail is required than currently described or erance will decline, pressure will inten- provided in those sets of templates in the sify to manage programs according to supportability and programmatic sources sound business practices, and “safer” of risk as well as the NDI-specific Re- acquisitions will attract more support. In quirements, S&S, T&E, ILS, and NDI these circumstances, explicit NDI risk program management environment risk management instruction will become criti- areas. cal to NDI program management. In the 5000.2-R, milestone 0 concen- Two parts of DoDR 5000.2-R could trates on materiel alternatives to a new better serve NDI PMs with more explicit major defense acquisiton program. The discussion of NDI risks and risk manage- milestone I, II, and III reviews, however, ment implementation procedures. The first repeatedly focus on acquisition program is Part 3. The regulation encourages ac- baselines (APB), acquisition strategies, quisition program stakeholders to exam- and exit criteria. The APB content, espe- ine the full range of mature or developing cially pertain- materiel alternatives before initiating a ing to cost, is “The targeted risks are technical, manu- new program. Emphasis is placed on cost, subject to re- facturing, and sup- schedule, performance, interoperability, finement during port risks that must trade offs, and risk management. The use each program be controlled before of IPTs and tailoring are introduced as phase. A refined the next milestone additional major program considerations APB encour- decision point.” for each milestone decision point. A ages risk reduc- program’s decision authority, documenta- tion efforts. Risk reduction efforts are tion, and milestone reviews will be sub- based on careful risk assessments, which ject to the program size, complexity, and are a basis for overall cost parameters and risk, as well as the flexibility intended realistic cost estimates. But no format is through IPTs and tailoring. provided in the regulation. No tie-in is As part of the original DoD Instruction made to NDI APBs or NDI risk assess- 5000.2, risk assessment formats were pro- ments and unique risk reduction require- vided in the now-rescinded DoD 5000.2- ments. Program acquisition strategies each M. The format description did more to have risk management as an essential ele- encourage tailoring than to depict a spe- ment. Again, while NDI programs will cific risk assessment format; it provided have their own acquisition strategies and little to support outlining an NDI risk as- risk management programs, no format or sessment, much less an RMP. The current tools are offered. The exit criteria or 5000.2-R provides no formats. PMs are “gates” that the system must pass to meet given wider latitude to prepare their own program goals and enter a new acquisi- risk assessments and risk management tion phase are selected to track progress plans and programs. Any current format in important technical, schedule, or man- would be helpful if it were to refer to the agement risk areas. No explicit mention the old DoD 4245.7-M risk areas. is made of NDI, Requirements, S&S, 47