SE&I AND MANAGEMENT N9 3- 4_. SYSTEMS ENGINEERING & INTEGRATION AND MANAGEMENT FOR MANNED SPACE FLIGHT PROGRAMS /_"_, /// by Owen Morris /_ /_( The development of systems engineering and In these projects, essentially all of the tech- program management in NASA manned nical responsibility was delegated to one of space programs has grown in a largely un- the Centers, which were primarily expert in coordinated manner over the last 30 years. the technical area being explored (i.e., aero- However, the systems and practices that dynamics, stability, control and structures) have been developed form a proven pattern but did not have experts in the development for successfully integrating large, technical- of hardware. Accordingly, NACA entered ly complex programs executed in several geo- into agreements with the Air Force or Navy graphical locations. This development has to manage the actual development of the not been recorded in a comprehensive man- aircraft. The NACA Centers focused their ner, and much of the reasoning behind the direction on the technical requirements and decisions made is not obvious. performance characteristics to be demon- For the purposes of this discussion, sys- strated by the aircraft. The contractor's tems engineering is defined as the inter- responsibility was similar to that for the disciplinary engineering that is necessary to development of any aircraft, and the contrac- achieve efficient definition and integration tor usually furnished test pilots for early of program elements in a manner that meets demonstration flights. the system-level requirements. Integration With the formation of NASA and the is defined as the activity necessary to de- start of major manned space programs, it velop and document the systems' technical was necessary for NASA to develop the capa- characteristics, including interface control bility to manage complex development requirements, resource reporting and analy- activities. Very little SE&I capability exist- sis, system verification requirements and ed within the functional organizations of the plans, and integration of the system NASA Centers. As a result, SE&I expertise elements into the program operational was developed within each of the program scenario. offices. In particular, the Gemini program This paper discusses the history of SE&I office was set up with autonomous capability management of the overall program archi- to manage SE&I and direct the development tecture, organizational structure and the contractor. relationship of SE&I to other program orga- With the advent of the Apollo program, nizational elements. A brief discussion of the SE&I was again managed from the project method of executing the SE&I process, a offices at the development centers. The summary of some of the major lessons project offices used specialized technical learned, and identification of things that capability from the Center functional orga- have proven successful are included. nizations and prime contractors and initiat- ed the practice of hiring support contractors HISTORY to assist in implementing SE&I. After the Apollo I fire, a review committee was estab- NASA, then the National Advisory Commit- lished to determine the cause of the fire and tee for Aeronautics (NACA), participation in recommend modifications to the program. the management of major aerospace pro- One of the recommendations made was that grams began shortly after World War II with NASA acquire a technical integration and the advent of the X series research aircraft. engineering support contractor to assist in 87 READINGS IN SYSTEMS ENGINEERING accomplishing SE&I activity. The Washing- integration. JSC, then called the Manned ton program office selected Boeing as the Space Center, managed both development contractor and managed the contract for this and flight operational aspects of the Mercury activity; however, a large portion of the work and Gemini programs with the checkout and force was located at the Centers. The con- preflight testing being performed by support tractor's responsibilities included moni- elements at Cape Canaveral. toring the development and operational Apollo became organizationally more activities at the Centers, forming integrated complex (Figure 1). The spacecraft develop- assessments of the activity, and making ment was managed by JSC, the launch vehi- recommendations to the program director for cle development by Marshall Space Flight improvements. As the program matured, the Center (MSFC), the prelaunch activities by contrac t.focus was changed, and the contrac- Kennedy Space Center (KSC)--by then an tor provided a significant number of person- independent NASA Center--and the flight nel to directly support the Centers in SE&I operations by JSC, In all of these programs, and systems development activities. the responsibility for the development of the With the initiation of the Space Shuttle flight hardware was delegated to the program and the adoption of the Lead Center Centers, and the interfaces between projects concept, it was decided to manage the Level were intentionally kept as simple as possi- II integration activity, including SE&I, by ble. The Washington office, under direction providing a small management core within of the program director, was responsible for the program office and using many of the overall direction of the program including Centers' functional organizations to provide budgetary allocations, congressional rela- technical support in a matrix fashion. At the tions, and management of development Johnson Space Center (JSC), the lead person issues between the project offices at the from the functional organization was gener- different Centers. The actual integration ally a branch head or an assistant division activity (SE&I) was coordinated by a series chief. JSC had a relatively large staff to of panels and working groups in which draw from to provide the specific technical individuals from the Washington program expertise and the level of effort needed to office served as either chairperson or accomplish a given task. members, with the program director over- The Space Station Freedom program was seeing the activity. In the early programs started using the Space Shuttle program as a (Mercury and Gemini), this activity was the model. As the Lead Center, JSC managed in- responsibility of a single Center, and the tegration. Later, the Level II function was Washington office was coordinated in an moved near Washington, D.C., under the informal manner, but by the end of the deputy program director, and an indepen- Apollo program, the management of the pan- dent contractor was brought in to assist the el and working group activity was relatively integration process. The Space Station Free- formal. In all of these programs the Center dom management organization will be dis- directors took an active part and personally cussed in more detail in the next section. felt responsible for the technical excellence of the work performed by their Centers. This PROGRAM MANAGEMENT intercenter involvement was accomplished ORGANIZATIONAL STRUCTURE primarily through the management council and major program reviews where Center A single NASA Center largelymanaged ear- directors personally participated in major ly NASA manned space flight programs, decisions. which allowed for a relatively simple organi- In part of the Apollo program, the zational structure to accomplish program Washington office retained the responsibil- 88 SE&I AND MANAGEMENT Level I Apollo Program Director Gen. S. Philips Level II Ai_rO11oSpacecraft Saturn V Launch Flight ogram Office Office Operations Operations G. Low H. Rudolph R. Petrone C. Kraft ...........t..................t...................................................... lLM sistIa esIi siv Instrument Unit Project Project Project Project Project Project Figure 1 Apollo Program Management Organization performing the SE&I activity with the actu- interfaces of much greater complexity; and al work being led by Bellcom, a division of the employment of one of the major hard- Bell Laboratories. Ultimately, this approach ware development contractors as the inte- was abandoned, at least partly because much gration support contractor. The complex of the Center director's responsibility was interfaces made SE&I activity voluminous lost, and an adversarial relationship be- and involved and required the commitment tween the program director and the Center of a larger percentage of the program re- organizations developed. The execution of sources to this activity. the SE&I was returned to the Centers with The Space Station Freedom program was management and coordination of intercenter structured so that the interface activity activities achieved through the use of work- between the work packages was even more ing groups, panels and management re- complex than that of the Shuttle program. views. Initially, the Lead Center approach to SE&I At the outset of the Space Shuttle pro- activity was adopted, but the implementa- gram (Figure 2), the management of SE&I tion was not effective. As a result of recom- was markedly changed. Some of the more im- mendations made by study groups and the portant changes were adoption of the Lead committee reviewing the Challenger acci- Center management concept in which one of dent, it was decided to transfer the responsi- the participating Centers was delegated the bility for program integration activity, management of program level integration including SE&I, to the deputy program including SE&I activities; the adoption of a director in Reston, Virginia, and to bring on configuration with functional and physical a contractor to provide program integration 89 READINGS IN SYSTEMS ENGINEERING Level I Space Shuttle Program Director M. Malkin Level II Space Shuttle Program Manager R.F. Thompson 1 1 [ Resources and Systems Management Operations Schedules Integration Integration Integration Integration O. Morris R. Machell D. Cheatham R. Young I MSFC Space Shuttle Projects Office Integration J. Lovingood t R. Lindstrom Leve1lII I L .t I Orbiter SSME External Tank SRB Project Project Project Project A. Cohen J.R.Thompson J. Odom G. Hardy Figure 2 Space ShuttleProgram Management Organization support (Figure 3). Contractors having sig- contractors negotiate the definition and nificant hardware development contracts execution of much of the detailed integration were excluded from the contract competition. process directly between themselves. This The first approach was to provide detailed proved ineffective, however, because there management of SE&I activity by the Reston was no clear lead responsibility and no clear civil service personnel with the integration way to resolve differences. As a result, contractor providing support in executing because of the complexity of program in- the activity. Additionally, it was thought tegration and the lack of in-depth backup ca- that much of the technical integration could pability, this management approach has not be accomplished by having the work package been completely effective. 90 SE&I AND MANAGEMENT Level I Space Station Freedom I I Program Director Level II Space Station Freedom / Deputy Program Director ] I Deputy Program Manager Deputy Program Manager for for Integration (JSC) Utilizations &Operations 1 I I Element System Integration Integration Manager Manager (MSFC) (JSC) I l l I I Systems Management International Program Program Engineering & Integration Programs Control Assessments Analysis Group Group Group Office Group ..................................... _ ............................................... Level III [ 1 lm l 1 I Work Work Work Work Package 1 Package 2 Package 3 Package 4 KSC Langley (MSFC) (JSC) (GSFC) (LeRC) Figure 3 Space Station Freedom Program Management Organization (1990) Recently, it was decided to give the inte- had the advantage of drawing from the in- gration support contractor direct responsibil- depth technical capability residing at the ity for the integration of the program but Centers. Simultaneously, the integrating without authority to directly manage the contractor's work force at the Centers was work packages or their contractors. In an increased in both responsibilities and num- attempt to obtain more in-depth capability, bers. the program director and deputy program director decided to execute the systems in- GROWING PROGRAM COMPLEXITY tegration portion of the SE&I activity at two of the Centers with the deputy director for One of the major factors determining the integration physically located at one of the efficiency of the integration of a program is Centers. Since these functions were still re- the methodology used to delegate the engi- tained organizationally within the program neering and development responsibilities to office, they were under the control of the dep- the project offices at the Centers. It has been uty program director and, at the same time, found that less complex organizational 91 READINGS IN SYSTEMS ENGINEERING structures and simple interfaces are ex- originates in the spacecraft batteries. The tremely important to allow efficient manage- main point is that a single person can fully ment of SE&I activities. Each of NASA's understand this interface and can cope with manned space programs has been organiza- all the effects of a change on either side of the tionally more complex than its predecessor interface. If there had been 10 times as many and has had more complex interfaces. In both wires, it probably would have taken a hun- the Mercury and Gemini programs, the dred (or a thousand?) times as many people flight elements were divided into two parts, to handle the interface." However, the oper- spacecraft and launch vehicle, and the phys- ational complexity of the Apollo vehicle ical and functional interfaces between the demanded a more extensive integration two were quite simple. The induced environ- activity between the Centers and for the first mental interfaces were somewhat more com- time posed the problem of accomplishing plex but readily amenable to experimental detailed technical coordination between and analytical determination. Centers. The Apollo program involved a major in- One of the basic tenets of the Space crease in program complexity. The space- Shuttle was to have an integrated vehicle craft was divided into two project offices and that would recover the most expensive ele- the launch vehicle was divided into four ments of the system for reuse. This led to a project offices. By assigning the four launch design concept that placed a great majority vehicle projects to the same Center (MSFC), of the electronics and major components of the integration between launch vehicle the main propulsion systems in the orbiter. stages could be accomplished at the Center This design concept led to very large level. Similarly, both spacecraft projects increases in interface complexity between were assigned to one center (JSC) for the the program elements and, more important- same reason. The physical and functional in- ly, between the Centers. For instance, the terfaces between the spacecraft and launch number of electrical wires running between vehicle, and hence between Centers, was rel- the external tank and the orbiter was more atively simple. In a 1971 paper titled "What than an order of magnitude greater than Made Apollo a Success," George Low stated: between the spacecraft and launch vehicle of "Another important design rule, which we Apollo, and for the first time, major fluid have not discussed as often as we should, systems ran across the interfaces. This reads: minimize functional interfaces be- represented a formidable increase in the ef- tween complex pieces of hardware. Examples fort required to successfully accomplish the in Apollo include the interfaces between the SE&I activity. As previously noted, a new spacecraft and launch vehicle and between program management structure (Figure 1) the command module and the lunar module. was adopted to accommodate the increase. Only some 100 wires link the Saturn launch The accomplishment of program-level SE&I vehicle and the Apollo spacecraft, and most was given to a "Lead Center." The program of these have to do with the emergency detec- director at Headquarters was still respon- tion system. The reason that this number sible for program budgetary control, Con- could not be even smaller is twofold: redun- gressional relations and a technical staff dant circuits are employed, and the electrical sufficient to assure that the program tech- power always comes from the module or nical activity was being properly implement- stage where a function is to be performed. ed. At JSC, which was the Lead Center for For example, the closing of relays in the the Shuttle program, a Level II program launch vehicle could, in an automatic abort office was established totally separate from mode, fire the spacecraft escape motor. But the Level III orbiter project office located at the electrical power to do this, by design, the same Center. = 92 SE&I AND MANAGEMENT The development of the flight hardware projects and to serve as the primary interface was delegated to four project offices with the to the Level II systems integration office. orbiter office located at JSC, as mentioned The flight hardware developmental dele- above, and the other three--the Space Shut- gation of the Space Station Freedom tle main engine office, the external tank program was formulated in an even more office, and the solid rocket booster office-- complex manner (Figure 4). End-to-end located at MSFC. In addition to the hard- developmental responsibility for each of the ware development project offices, a pre- major functional systems was delegated to launch processing office was formed at KSC. one of four project offices called work pack- All of the project offices reported to the Level age offices in the Space Station Freedom II program manager for all programmatic program. Responsibility for assembling and direction except budget allocation, which delivering the flight hardware was broken was retained by the program director at down by launch elements, again assigned to Headquarters. one of the work package offices. Each of these The SE&I activity was delegated to the launch elements incorporates components of systems integration office located within the most of the distributed systems, neces- JSC Level II office. The orbiter contractor, sitating the transfer of an extremely large Rockwell International, was selected to be number of hardware and software items the integration support contractor, but to between work packages prior to their deliv- increase objectivity, the integration activity ery to the Government. This resulted in was made a separate exhibit to the contract another major increase in the complexity of and technical direction was delegated to the the program-level SE&I process and directly Level II systems integration office. The contributed to the difficulty of implementing MSFC Space Shuttle project office appointed a satisfactory SE&I process in the Space an integration manager to manage the Station Freedom program. integration of the Marshall Space Shuttle Figure 4 Space Station Integration Job 93 READINGS IN SYSTEMS ENGINEERING SE&I SCENARIO and reviews is central to accomplishing hori- zontal integration among the project offices As a program develops from concept to oper- and is discussed in more detail later. ational status, the characteristics of the In preparation for the preliminary design SE&I activity vary greatly. Early in the pro- review (PDR), SE&I defines the minimum gram, conceptual SE&I is intimately in- content required in the PDR data packages volved in defining systems that will meet the and is responsible for preparing system-level overall program objectives and in evaluating documents supporting the Integrated the relative merits of each. This is usually System PDR. During the PDR process, SE&I accomplished in NASA manned programs by representatives participate in the project- the civil service organizations, often in con- level reviews with particular emphasis on cert with Phase A/B contracts with industry. the compliance of the project to the system- After the general systems specification has level requirements. During the Integrated been developed and a detailed evaluation of System PDR, emphasis is placed on assuring systems concepts has been completed, SE&I that the preliminary designs proposed by the provides a lead in the preparation of the pro- projects are compatible across the interfaces curement specifications for the Phase C and and that the integrated system is capable of D activities and is usually directly involved meeting the operational requirements of the in the source selection process. After award program. The SE&I organization is inti- of the Phase C and D contracts and final mately involved with the evaluation and dis- selection of the design approach chosen for position of review item discrepancies (RIDs) implementation, SE&I is responsible for pre- that are submitted during the review. paring system-level technical specifications, As a result of the PDR process, changes to which define the performance requirements the requirements and modifications to the to be satisfied by each of the major program preliminary design of the elements are incor- elements. SE&I then develops the system porated. A new characterization cycle is then characterization process to be used (dis- initiated to evaluate the compatibility be- cussed in detail later) and starts an initial tween the modified requirements and pro- analysis cycle. The results of this cycle are posed system capabilities. At this time, the extremely important in verifying the valid- drafts of the interface control documents are ity of the system technical specifications and expanded and quantitative detail is added to providing a technical basis for conducting assure that the documents are mature the Program Requirements Review (PRR). enough to become baseline requirements in After completion of the PRR and updating of the program. This maturation process inevi- the technical specifications, SE&I starts the tably results in the identification of physical definition of the interface control document and functional disconnects among the ele- tree and the initial document drafts. An- ments and in a significant number of other system characterization cycle is start- changes to the baseline. ed, based on the updated specifications and In a similar manner, the verification the hardware and software concepts chosen plans of the elements and the integrated to assess the adequacy of the proposed pre- system are refined and baselined. The liminary design approach. responsibility for executing the test and ana- By this time in the program, the ad hoc lysis required by the integrated system ver- organizational structure should be well in ification plan are delegated to appropriate place and functioning routinely. The commu- organizations that prepare detailed plans for nication and management overview provided accomplishing the assigned portions of the by this structure of working groups, panels verification. 94 SE&I AND MANAGEMENT Detailed mission operational scenarios The SE&I organization's participation and timelines are prepared by the operations throughout the program development cycle organizations, and the operations and SE&I supports operational planning and real-time organizations jointly conduct an analysis of operations. SE&I is the repository of corpo- the system capabilities to support the sce- rate knowledge of the details of system narios. Concurrently, the acceptance test capability, which is vital to the effective and and prelaunch operations requirements and efficient operation of the system. plans are prepared and delegated for execu- tion. RELATIONSHIP OF SE&I TO OTHER In preparation for the critical design PROGRAM FUNCTIONS review (CDR), another system characteriza- tion cycle is performed, based upon the To effectively accomplish the SE&I task, the detailed design of the elements. This cycle SE&I management organization must main- typically uses mature models to synthesize tain good communications and obtain the the hardware and software systems and also support of other program office organiza- incorporates the results of tests performed to tions. Some of the more important interac- that time. SE&I participates in the conduct tions are discussed below. of the CDR in a manner similar to that of the PDR. After completion of the CDR, the Configuration Management. The in- system requirements and design changes re- teraction between SE&I and configuration sulting from the CDR are incorporated into management is particularly strong. As the the documentation, and another complete or developers and keepers of the systems speci- partial system characterization cycle vali- fications, SE&I has an interface with the dates the decisions made during CDR. configuration management function that is After CDR, the primary activity of the extremely active throughout the life of the SE&I organization is to analyze test results program. The SE&I office recommends the and conduct analysis to verify the capability baselining of the technical requirements as of the system that is being manufactured. they become sufficiently mature and then Particular emphasis is given to verifying the serves as the office of primary responsibility interface characteristics of the elements as for defining and evaluating most of the pro- defined by the interface control documents. posed changes to this baseline. The SE&I of- This activity directly supports the prepara- rice, after proper coordination throughout tion for the design certification review the integration function, also recommends (DCR), and provides interface information the processing of noncontroversial changes necessary to allow acceptance of the system outside of the formal control board meetings, hardware and software by the Government. where appropriate. This significantly re- The DCR is conducted similarly to the duces the board's workload and conserves the PDR and CDR but addresses the as-built time of the key managers who are members hardware and software. Successful comple- of the change control board. As significant is- tion of the DCR certifies the acceptability of sues are referred to the board, SE&I presents the as-built elements and the ability to be an analysis of the issues involved and makes integrated into an overall system that will appropriate recommendations for action. satisfy the initial program operational re- quirements. Final operational certification Program Control. SE&I supports the of the system is obtained by a combination of program control function in the development the DCR process and analysis of information of program schedules and budgets. The key obtained during early flight operation of the to making this support effective is the use of system. the SE&I logic networks and estimates of the READINGS IN SYSTEMS ENGINEERING manpower required to accomplish the activi- SR&QA office is responsible for setting the ties. Because of SE&I's interdisciplinary requirements for SR&QA activities and for nature, SE&I can assist in planning activi- evaluating the outcomesmwhile other orga- ties in many areas of the program. nizations are delegated the responsibility for Early in the program, SE&I helps define executing the work--then SR&QA must de- the content and schedule milestones of each fine and obtain baseline approval of task re- project to permit coherent development of quirements, monitor execution of the task by project-level schedules and cost estimates. SE&I, and evaluate the results to assure sat- SE&I also provides program control with the isfactory achievement. engineering master schedules (EMS) and The former mode of operation was exem- associated budget estimates for incorpora- plified during the early Apollo program, in tion in the overall schedule and budget which the SR&QA activities were largely ac- system. SE&I also works with program complished within the SR&QA office using control in planning major program reviews; basic engineering information obtained from provides technical leadership in conducting SE&I and other program organizational the reviews; and frequently chairs the offices. Later in the Apollo program, the screening groups and pre-boards. second mode of execution was adopted; the engineering offices, primarily SE&I, actual- Operations. In all of the NASA manned ly performed the work and made a first-level space programs to date, the SE&I function analysis before formally transmitting the has been managed in an organization differ- results to SR&QA for authentication. This ent from the operations definition and plan- latter method was considered more effective ning function. Although this is undoubtedly primarily because problems and discrepan- the best choice in the later phases of the pro- cies were often discovered by the originating gram, it may result in a less thorough incor- engineering office and corrected even before poration of operational requirements in the the task was completed. systems specifications and other SE&I pro- ducts early in the program. It may be desir- SE&I EXECUTION able to combine the management of SE&I and operations in the same office early in the Techniques developed in past NASA manned program and then separating them later, programs have proven effective and have perhaps at the completion of the preliminary become an integral part of implementing design review. The stated reason for separat- SE&I activities. The following paragraphs ing the functions in the past has been that describe, in no particular order, some of the they serve as a check and balance on each most important techniques in planning and other; however, the separation also discon- implementing new programs. nects the detailed interfaces between the two functions. Importance of SE&I Early in a Pro- gram. In the early stages of complex SR&QA. The interactions between SE&I programs, comprehensive SE&I support and the system reliability and quality assur- helps determine the architecture to be used ance (SR&QA) functions depend on how to delegate project responsibility. This is responsibility for executing the program is accomplished by dividing the program into delegated. If a large part of the SR&QA the next lower level of management, the pro- activity is accomplished within the SR&QA ject offices. The primary outputs are compre- organization, SE&I is used as a reservoir of hensive and clear program requirement information or to perform specific tasks as specifications, identification of major pro- requested by SR&QA. However, if the grammatic interfaces, development of the ad 96