Teaching Risk Analysis in an Aircraft Gas Turbine Engine Design Capstone Course Aaron R. Byerley1, Cory A. Cooper2, and Devin O. O’Dowd3 Department of Aeronautics, USAF Academy, CO 80840 I. Introduction This paper describes the introduction of risk analysis in an undergraduate aircraft engine design capstone course that is taught in concert with a companion course in airframe design. The two preliminary designs, one for the engine and the other for the airframe, must be integrated as subsystems within a system to satisfy the performance requirements of a given mission as outlined in a single “request for proposals”. In recent years, systems engineering majors have been added to the design teams to work alongside the aeronautical engineering majors to analyze and report on costs, schedule, and technical risk factors in addition to the operational performance factors that have previously been the sole focus of the course. The teaming of systems engineering majors and aeronautical engineering majors has been driven by a heightened emphasis on system affordability and risk reduction. To tie all of the performance, cost, development time, and technical risk issues together, the paper presents an “analysis of alternatives” example that considers three different engine cycle alternatives. The design tools presented in this paper will provide a strong foundational understanding of how to systematically weigh and evaluate the important tradeoffs between aircraft turbofan engine performance and risk factors. Equipping students with the insight and ability to perform these multidisciplinary trade studies during the preliminary engine design process is this paper’s most important contribution. II. Background Since 2002, when the Secretary of the Air Force encouraged the USAF Academy to initiate systems engineering (SE), there has been greater emphasis placed on developing “systems minded” officers who are capable of managing large, complex systems and the integration of the many subsystems that comprise the larger system. As described by Rolling et al (Rolling, 2012), the SE cadets have been integrated into the traditional senior engineering capstone design courses where they are responsible for such systems engineering tasks as tracking performance, system / subsystem interfacing, cost, schedule, and technical risk. Previous publications by the same authors have dealt with estimating dry engine weight, engine development costs, engine production costs, and scheduling (Byerley A. R., 2013) as well as the linkage between turbine inlet temperature, blade cooling effectiveness, and maximum blade material temperature (Byerley A. R., 2015). This paper will provide a clearer explanation of the generic risk analysis SE majors have studied and how this is linked to the specific issues they must face in aircraft gas turbine engine design. Aeronautical and Systems Engineering majors are assigned to engineering projects as their senior year design capstone experience. Aeronautical Engineering majors participate in either a full year of aircraft design, or one semester of aircraft design followed by one semester of aircraft engine design. Systems Engineering majors are assigned to over 30 different capstone projects spanning 7 different engineering domains (Cooper 2016). A subset of this group are assigned to the capstone engine design course and have had two propulsion courses (AE 241 Intro Thermo and Compressible Flow and AE 361 Introduction to Air-breathing Propulsion / Cycle Analysis) as well as SE courses that introduce risk analysis. The Aeronautical Engineering majors in this course have had an additional course, AE 466, which focuses on gas turbine engine components. 1 Professor of Aeronautics, Department of Aeronautics, USAF Academy, CO, 80840, AIAA Associate Fellow 2 Assistant Professor / Director of Systems Engineering Program, HW USAFA/DFSE, USAF Academy, CO, 80840 3 Associate Professor, Department of Aeronautics, USAF Academy, CO, 80840 1 American Institute of Aeronautics and Astronautics III. Introducing Risk Analysis to Systems Engineering Majors The USAF Academy’s SE curriculum introduces the concept of risk management in several courses. The concept of risk is initially introduced by relating it to the students’ daily lives and operations. The Air Force presents this as Operational Risk Management (ORM) and focuses largely on safety risks and how they relate to personal and professional choices (SAF/IEE, 2012) (DoD, 2012). This approach to understanding risks from a personal standpoint serves well as a touchpoint to understanding how to identify, assess, handle and track risks in other areas of system development (Cooper, 2015). As SE majors, the students then are introduced to more in-depth understanding of risk management through system lifecycle and project management courses. As described by the International Council on Systems Engineering (INCOSE) (INCOSE, 2011), the main steps of this process include identification, analysis, treatment and monitoring of the risks. While these main activity steps are presented in slightly different ways throughout the curricukum and broader SE discipline (Blanchard, 2011) (Haimes, 1999) (Kerzner, 2006), the students can observe the overall risk management process that aids design decision making. Risk identification, as the first main step of the risk management process, requires the designer to both think broadly (i.e. what are all of the system-level interactions and dependencies?), and deeply (i.e. how do specific system elements work and what are their failure modes?). To aid students in the former area, additional risk categories are taught. The certifying standard for SE is based on INCOSE’s SE Handbook, and it defines the following types of risks seen in Figure 1. Figure 1. Typical relationship(s) among the risk categories (INCOSE, 2011). The SE students take courses in project management, engineering economics, and domain-specific engineering disciplines that prepare them to identify and assess risks in the Technical, Cost, Programmatic, and Schedule Risk categories. Where it may be expected for typical engineering students to be able to identify some system risks and assess the likelihood and severity of those risks, SE are trained and then expected to perform full risk management activities. For the purposes of this course, the careful consideration of risk will be limited to Technical Risk and the possible impact of technical problems on Cost Risk and Schedule Risk. While Cost Risk, Schedule Risk, and Programmatic Risk can also work in the opposite direction and have an impact on Technical Risk, the in-depth treatment of cost, schedule, and programmatic risk are beyond the scope of this paper. This paper will focus on technical risk as it is traded off against increased engineering performance. Following risk identification, the process activity of risk analysis requires the use of a framework to appropriately compare risks. The concept of a hazard risk index (HRI) value is used to categorize risks in a relative manner to 2 American Institute of Aeronautics and Astronautics enable effective design decision making. A sample HRI used to introduce the concept to students is shown in Figure 2. Figure 2. Nominal Hazard Risk Index. The design team in conjunction defines it own scales and definitions based upon engineering judgment. Identified risks are analyzed for their likelihood and consequences so that they can be placed in the HRI matrix for relative comparison. Each system design effort or program is really meant to tailor their HRI matrix axes and definitions to meet their specific needs (i.e. the consequence scale for a mini-unmanned aerial vehicle design project will be much lower than such a scale for the development of the F-35 Joint Strike Fighter program office). This analysis is sometime qualitative in nature during early stages of design, but can be highly quantitative in nature as more details of the design are established, and as expert assessment of the design is incorporated in the risk process (Eggstaff, 2014). Tools such a Failure Modes, Effects, and Criticality Analysis (FMECA), Faulty Tree Analysis (FTA), and others can also serve to characterize the risks (Blanchard, 2011) (Otto, 2001). The last steps of risk management, treatment and tracking, are especially important for subsequent design activities as they will determine what resources a project manager will need to expend. Ths paper will focus largely on the first two steps of the risk management process and show a unique way to embed technical risks for engine design into the performance tradeoffs that are necessary. In traditional aircraft and engine design processes, this explicit step of incorporating risk is largely absent. For instance, texts that explore the fundamentals of aircraft design, go through the steps of conceptual, preliminary, and detailed design, make the assumption that technology will support the required performance specifications (Brandt, 2004) (Raymer, 1989). There is no specific step to treat a selected design performance parameter with a risk management mindset (i.e. what is the risk that the desired afterburner stage pressure ratio will not achieve the technical performance parameter that our engine design needs by the time it is needed?). A risk mindset, when it comes to making tradeoffs in performance parameters allows the designer to “weight” a desirable, but possibly improbable, parameter value in such a way that a designer does not chase impossibilities. This also allows the designer to approach design in a top-down, bottom-up approach that is necessary in the design of complex systems. It acknowledges that a designer must design to top-level, driving requirements, while at the same time, being conscious of existing and imminent technology needed to achieve the design itself (Blanchard, 2011) (Aslaksen, 2009). IV. Introducing Technical Risk in the Context of Gas Turbine Engine Design and Development 3 American Institute of Aeronautics and Astronautics The systems engineering majors chosen to join the aircraft engine design teams need to understand more about the technical risks associated with gas turbine engines and how the incentives for increasing engine performance, while reducing the engine development costs, and shortening the engine development schedule must be counterbalanced by adopting the risk mindset introduced in generic terms earlier in the systems engineering curriculum. To provide a good sense of these issues as they relate to gas turbine engines, these students were required to read “The Air Force and the Great Engine War”2. The book describes the challenges associated with procuring and maintaining the F100 high performance engine. Early USAF performance requirements called for a thrust to engine weight ratio of 10-to-1 which was more than double that for previous engines. The push for low weight during design and development exposed technical challenges associated with fan blade flutter, fan rotor-to- stator rubbing which ignited titatium fires, and turbine rotor failures. Drewes2 argued that the “F100 entered production before it had been completely developed and tested”. Upon fielding the F100 in the F-15, the risk of the design requirements not matching the actual experience in operational use became apparent. There was a six-fold increase in full-throttle transients (or thermal cycles) which greatly reduced the structural life of the engine which led to turbine failures. This was made worse by the pressure to shorten the overall schedule from design to fielding. According to Drewes, the F100 “never had a chance to be operated moderately at first and then gradually eased into full production and operational use. It never had a chance to mature”. The book does an excellent job in highlighting the importance of a risk mindset and the tradeoffs between system performance, development cost, technical risk, and scheduling pressure. The students were required to write a paper which summarized each of the issues above. V. Risk Analysis in Preliminary Engine Design Since the focus of this capstone course is preliminary engine design, the treatment of technical risk is limited to the risk associated with engine component figures of merit described by “Level of Technology”. Mattingly et al (Mattingly, 2002) presented engine and component performance parameters in terms of Level of Technology 1 through 4 as shown in the Table 1 below. These levels correspond roughly to 20 year increments starting in 1945. The Level of Technology 5 is an estimate based upon extrapolating the trends into the future and on insight gained from informal discussions with industry representatives about the progress made possible by using advanced computational fluid dynamics, a tool that was not in widespread use as a design aid in earlier time periods. The risk associated with attempting to predict component figures of merit 10 years into the future is strongly highlighted in class. 4 American Institute of Aeronautics and Astronautics Level of Technology in Time Periods 1 2 3 4 5* 1945-19651965-19851985-20052005-20252025-2045 Diffuser (subsonic engine in nacelles) 0.90 0.95 0.98 0.995 0.997 Diffuser (subsonic engine inside airframe) π 0.88 0.93 0.96 0.97 0.99 d max Diffuser (supersonic engine inside airframe) 0.85 0.90 0.94 0.96 0.98 Compressor e 0.80 0.84 0.88 0.90 0.93 c Fan e 0.78 0.82 0.86 0.89 0.92 f Burner π 0.90 0.92 0.94 0.96 0.98 b η 0.88 0.94 0.99 0.995 0.997 b Turbine (cooled) e 0.80 0.83 0.87 0.89 0.92 t Afterburner π 0.90 0.92 0.94 0.95 0.960 ab η 0.85 0.91 0.96 0.97 0.980 ab Nozzle (fixed-area conv) 0.95 0.97 0.98 0.995 0.997 Nozzle (variable-area conv) π 0.93 0.96 0.97 0.985 0.990 n Nozzle (variable C-D) 0.90 0.93 0.95 0.98 0.985 Max Tt4 (deg R) 2000 2500 3200 3600 4000 Max Tt7 (deg R) 2500 3000 3600 4000 4100 Table 1. Component performance parameters for Levels of Technology 1-4 from Mattingly, 2002 (Table 4.4, p. 107.) Note: Level of Technology 5 is estimated. As an example of technology improvement over time, the polytropic efficiencies for the fan and high pressure compressor are shown in the figure below. The steady increase in polytropic efficiencies began to level off between Levels of Technology 3 and 4 but it is estimated that the upward trend will return with the earlier rate of change from 4 to 5 because of the advances in CFD which are used to make design decisions. The issue of technical risk in this capstone course which is focused on preliminary design is limited to the risk of failing to achieve the predicted values of component figures of merit. Figure 3. 5 American Institute of Aeronautics and Astronautics These component figures of merit are important for conducting parametric cycle analysis (PCA) because they have a direct impact on the estimates of the key engine performance parameters which are uninstalled specific fuel consumption (S) and uninstalled specific thrust (ST). The trade study described below determines the performance improvements from going a riskier Level of Technology 5 from the contempory Level of Technology 4. The design point for this example is a supercruise mission leg at Mach 1.5 and altitude of 36,000 feet. The figure below shows the graphical user interface for the software tool AEDsys which accompanies the course textbook. . Figure 4. Sample AEDsys graphical user interface with Level of Technology 4 inputs. The results of PCA can be presented as “carpet plots” which show lines of constant bypass ratio and overall pressure ratio where fan pressure ratio, turbine inlet temperature, and % turbine cooling flow are held constant as depicted in Figures 5 and 6 below. The solution space is bracketed by lines of maxiumum acceptable value of specific fuel consumption (S ) and minimum acceptable specific thrust (ST ). The combinations of cycle max min parameters that exceed S are not workable because the aircraft will run out of fuel. Those combinations that do max not meet ST will not produce enough thrust to sustain the mission leg. Two different design scenarios were min considered. Design Scenario A is intended to maximimize specific thrust while staying below S . This would be max useful for minimizing the mass flowrate of air necessary to produce the required thrust. A smaller mass flowrate for a given Mach number and altitude would require a smaller engine inlet area. Design Scenario B is intended to minimize S while still producing the required specific thrust. This would be useful when reengining an aircraft with a known inlet area with the goal of reducing fuel consumption. Figure 5 below is based upon Level of Technology 4 6 American Institute of Aeronautics and Astronautics while Figure 6 is based upon Level of Technology 5. Both are based upon the supercruise design condition of Mach 1.5 and altitude 36,000 feet. Figure 5. Parametric cycle analysis carpet plot for supercruising (M=1.5, h=36,000 ft) with Level of Technology 4 inputs to include Tt4=3600 R with 18% coolant flow bled from the high pressure compressor and injected evenly into the turbine nozzle guide vane and first stage rotor. Cycle design choice for Scenario A – alpha (bypass ratio) = 0.64, OPR = 35, Pi = 4.5; for Scenario B – alpha (bypass ratio) = 0.72, OPR = 35, Pi = 4.5. f f 7 American Institute of Aeronautics and Astronautics Figure 6. Parametric cycle analysis carpet plot with Level of Technology 5 inputs to include Tt4=4000 R with 20% coolant flow bled from the high pressure compressor and injected evenly into the turbine nozzle guide vane and first stage rotor. Cycle design choice for Scenario A – alpha (bypass ratio) = 0.89, OPR = 35, Pi = 5.0. Design Scenario f B – alpha (bypass ratio) = 1.22, OPR = 35, Pi = 5.0. f The first observation to make about the two carpet plots is that the higher level of technology increases the number of bypass ratio and overall pressure ratio combinations that are in the solution space which is expected. To quantify the specific performance improvements associated with the Level of Technology 5, a comparison for each of the two design scenarios is included in the table below. A 9.4% increase in ST is possible for a fixed value of S while a 3.4% decrease in S is possible for a fixed value of ST. Design Scenario A LOT 4 Specific Thrust LOT 5 Specific Thrust % increase (increase specific thrust while just staying below 58.4 lbf/(lbm/sec) 63.9 lbf/(lbm/sec) 9.4% max allowable specific fuel consumption) Design Scenario B LOT 4 Specific Fuel LOT 5 Specific Fuel % decrease (decrease specific fuel Consumption Consumption consumption while staying 3.2% above minimum allowable 1.015 (lbm/hr)/lbf .9822 (lbm/hr)/lbf specific thrust) In previous years, the design teams would always recommend the higher performance option but recently, the treatment of technical risk has been made a much more important element in the final design brief. The systems engineering majors have added this discussion to the discussion on predicted dry engine weight, development and production costs, and development schedule that were introduced in Byerley et al5. 8 American Institute of Aeronautics and Astronautics VI. Summary This paper described how risk analysis is taught in a gas turbine engine capstone design course. The generic treatment of risk analysis is extended to address an assortment of technical risk issues as they appear in trade studies where the state of the art levels of technology may be pushed in order to improve gas turbine engine performance. If these risks are taken, they should be taken in the context of full disclosure and with the confidence that possible improvements in system performance outweigh the technical risks. References Rolling, August J., Byerley, Aaron R., Wisniewski, Charles F., 2012. Integrating Systems Engineering Into the USAF Academy Capstone Gas Turbine Engine Course. Feb 2012, ASME Journal of Engineering for Gas Turbines and Power, Vol. 134, pp. 021601-1. Byerley, Aaron R., Rolling, August J., Van Treuren, Kenneth W., 2013. Estimating Gas Turbine Engine Weight, Costs, and Development Time During the Preliminary Aircraft Design Process. 2013. ASME GT2013- 95778. Byerley, Aaron R., Rolling, August J., 2015. Exploring the Impact of Elevated Turbine Blade Cooling Effectiveness and Turbine Material Temperatures on Gas Turbine Engine Performance and Cost. 2015. ASME GT2015- 44061. Cooper, Cory A., Joseph M. Fulton, Jeremy J. Homan, “Initial application of a multi-spectrum characterization framework for interdisciplinary capstone design experiences,” in Proceedings of the 14th Annual Conference on Systems Engineering Research (CSER), Huntsville AL, March 22-24, 2016. International Council on Systems Engineering (INCOSE). 2011. Systems Enginering Handbook, v3.2.2. 2011. Office of the Secretary of the US Air Force (SAF/IEE). Air Force Policy Directive (AFPD) 90-8, Environmental, Safety & Occupational Health Management and Risk Management, Feb 2012. Department of Defense (DoD), Standard Practice for System Safety, MIL-STD-882E, 11 May 2012 Cory A. Cooper, Michael Anderson, Chad Bruce, Sarah Galyon Dorman, Daniel Jensen, Kevin Otto, Kristin Wood., “Designettes in Capstone: Initial Design Experiences to Enhance Students’ Implementation of Design Methodology,” in Proceedings of the 122nd American Society for Engineering Education (ASEE) Annual Conference and Exposition, Seattle, WA, June 14-17, 2015. Blanchard, Benjamin S., Fabrycky, Wolter J. 2011. Systems Engineering and Analysis, 5th ed. : Prentice Hall, 2011. Haimes, Yacov Y. 1999. Risk Management. [book auth.] Andrew P., Rouse, William B. Sage. Handbook of Systems Engineering and Management. : John Wiley and Sons, Inc., 1999, pp. 137-173. Kerzner, Harold. 2006. Project Management: A Systems Approach to Planning, Scheduling, and Controlling, 9th ed. : John Wiley and Sons, Inc., 2006. Eggstaff, Justin W., Mazzuchi, Thomas A., Sarkani, Shahram. 2014. The Development of Progress Plans Using a Performance-Based Expert Judgment Model to Assess Technical Performance and Risk. 4, : Wiley Periodicals, 2014, Systems Engineering, Vol. 17, pp. 375-391. Otto, Kevin, Wood, Kristin. 2001. Product Design: Techniques in Reverse Engineering and New Product Development. Prentice Hall. Brandt, Steven A., Stiles, Randall J., Bertin, John J., Whitford, Ray. 2004. Introduction to Aerodynamics: a Design Perspective, 2nd ed.: AIAA Education Series, 2004. Raymer, Daniel P. 1989. Aircraft Design: A Conceptual Approach.: AIAA Educational Series, 1989. Aslaksen, E. W. 2009. Designing Complex Systems: Foundations of Design in the Functional Domain. Boca Raton, FL : CRC Press, 2009. Mattingly, Jack D., Heiser, William H., Pratt, David T. 2002. Aircraft Engine Design, 2nd ed.: AIAA Education Series, 2002. Drews, Robert W. 2006. The Air Force and the Great Engine War. : University Press of the Pacific, 2006. 9 American Institute of Aeronautics and Astronautics