@"dB SPACE SHUTTLE MAIN ENGINE PROPELLANT PATH LEAK DETECTION USING SEQUENTIAL IMAGE PROCESSING L. Montgomery Smith*, J. A. Malone**, B. W. Bornart, and R. A. Crawfordtt - University of Tennessee Calspan Center for Space Transportation and Applied Research University of Tennessee Space Institute Tullahoma, Tennessee 37388-8897 ABSTRACT Initial research using theoretical rsdiation transport models established that the occurmnce ofa leak m the powerhead of the SSME is accompanied by a sudden but sustained change in intensity in a given region of an image. During this work period, temporal pmcesing of video images on a he-by-frame basis has been used to detect leaks within a gim field of view, The kak detection algorithm developed in this study consists of a digital highpass filter cascaded with a moving average filter. The absolute value of the output image is then averaged over the full h e to produce a singk time-vsrying mean value estimate that is indicative of the intensity and extent of a leak. Four video sequences h m a n actual SSME test firing were analyzed using this technique. A hydrogen gas kak was detected approximately four seconds before existing sensors initiated shutdown during the test. The resulting plots of the fulI frame mean value versus time verify the effectiveness of the system. A detailed specification of the hardware and software required to impkment the d,prithm m in tiuze was pperfomed. The proposed system consists of a personal-computer-based system with ommen5ally av&k add-in cards capable of digitizing and pmcesing standard RS170 format video signals at 30 frames per second I. INTRODUCTION lines and internal components due to thermal shock, mechanical stress, erosion, and material The rapid detection of propellant leaks fatigue often result in failure modes with from the Space Shuttle Main Engine (SSME) sufficiently long time constants to allow during test firing is crucial to the prevention of detection and safe shut down. Recent advances catastrophic failures. Ruptures of high-pressure in imaging and image processing technology provide the hardware necessary for visual and infrared observation of these phenomena and the computing capability required for processing the *UTSI/CLA/CSX'AR, Affistant Prof-r of ECE signals and detecting the occurrence of a leak "ERC/CSTAR, Fbearch Engineer within the f~ldof view. Thus, a system capable tUTSI/CSTAR, h i a t eP rofessor of ECE of detecting leaks from images acquired ttUTSI/CLA/CSTAR, Professor of AE/ME sequentially during test fring in real time is of This work was sponsored jointly by the Center for Space value to the development of the SSME and is Transportation and Applied Research and the National with current technology. This study Aeronautics and Space Administration under Grant NAGS- investigated this approach and established its 140, through the Center for Laser Applications (CLA) at feasibility and applicability to the program. the University of Tennessee Space Institute (UTSI). Previous work in this area by Shoha.de The leak detection algorithm has been and Crawford [I], 121 concentrated on implemented on a general purpose digital image establishing the feasibility of observing hot or processor and successfully applied to actual test- cold leaks using infrared imaging. The theoretical stand fning data Four video sequences of models developed were used to predict radiation a SSME test-stand fuing were analyzed with this transport in absorbing, emitting and scattering technique to evaluate its performance under media. These models predicted the intensity of actual fuing conditions. A hydrogen leak was both background and plume radiation mxhing a detected in three of the four camera views sensor location, and they were used for designing approximately 4 sec before existing sensors validation experiments. The feasibility of infrared initiated an early shutdown. detection of leak plumes was demonstrated on subscale simulated plumes to determine In order to implement the algorithm n sensitivity, signal-to-noise ratio, and general real-time with a standard 30 frames/sec video suitability. Both hot and cold leaks were readily signal, a system with memory for approximately detected as measurable intensity changes by the 16 previous frames and a processing throughput sensor. of 37 million operations/sec is required. A personalcomputer-based system meeting these However, to detect the occurrence of a specifications has been designed with leak, the temporal aspects of the process must commercially-available add-in cards utilized where be considered. The previous analysis showed possible. Verification of timing requirements has that the occunence of a leak should result in a been carried out with a software simulator to sudden change in intensity in a given region of an establish proper system operation. image. Furthermore, the change should be sustained for a typical persistent leak. The time The following section describes the leak variation of the intensity at a point within the detection algorithm and discusses some of the area of the leak should therefore be similar to considerations for implementing it on digital that of a step function, although other smaller image processing equipment. Section III intensity variations are also present due to discusses the processing of the four video normal operating conditions. The problem sequences from a SSME test firing. becomes that of detecting a step function in the Specifications for a real-time data acquisition and presence of additive noise. processing system implementing the algorithm are presented in Section IV. A summary and The leak detection system presented here conclusions are given in Section V. was designed to quickly and automatically detect a steplie change in intensity in a sequence of 11. THE LEAK DETECTION images [3], [4]. Temporal processing is carried out at each point in full-frame digitized video Time-varying intensity levels in video data. The system consists of a causal, recursive images are an inherently discrete signal with a high-pass filter that removes slowly-varying sampling rate of 30 Hz for standard television background intensities cascaded with a moving format. Therefore, techniques of digital signal average filter that accumulates transmitted processing are directly applicable to the analysis. sustained changes. The absolute value over the A detailed derivation of the step detection full output frame is averaged to produce a time- algorithm used in this study was given in [4] varying mean value indicative of the level and along with a numerical analysis of its spatial extent of any leak. performance. For completeness, a brief description of it is presented here. The processing algorithm developed for this application can be broken down into four - n 1 z-i component blocks. The first three blocks involve processes that are applied in parallel at each These two filtering operations can be combined point (pixel location) in the image, while the into a single cascade connection with an effective fourth incorporates the spatial information over transfer function given by the entire field *of view. In concept, the four steps can be described as follows: 1. A highpass filter is applied to remove the const ant or slowly-varying background intensity while passing sudden changes. which corresponds to the dierence equation 2. A moving average filter accumulates any sustained changes transmitted through the highpass filter. Equation (4) is the difference equation 3. The absolute value of the output from the defining the temporal filtering that is carried out moving average is taken to allow for at each point in the image. Processing k; detection of positive or negative intensity completed by taking the absolute value of the changes. resulting signals at all pixel locations and averaging them over the full field of view. 4. The pixel values of the output image are summed or averaged over the field of view to While equation (4) conceptually describes produce a single time-varying quantity the filtering process at each pixel location, the indicative of the extent and intensity of a actual implementation consists of the recursive leak. computation given by For speed of response and simplicity of computation, the highpass filter has been chosen + as a single-pole recursive filter with a z-domain Multiplication by the gain factor (1 J3)/2N is transfer function given by performed on the sum of the absolute values of the output pixels in conjunction with normalization to obtain the mean value. This substantially reduces computational requirements. The cutoff frequency of the filter is determined by choice of the pole value P. It has been shown [4] that the signal-to- noise ratio of the output signal can be maximized The moving average fdter has a transfer for a given value of J3 by choosing the number of function terms in the moving average according to somewhat distorted.) Figure l(a) (camera position 1) shows the typical vapor clouds that are present during an engine test. This condition sets the memory requirements of the system in terms of previous input images Thii image data was transferred to an that must be stored. optical disc recorder/player and processed frame- by-frame with the leak detection algorithm. 111. APPLICATION TO TEST STAND Because of the large amount of noise in this DATA data, it was found that values for and N of 0.91 and 13, respectively, were required for The system has been applied to visible processing. (Use of higher values was limited by wavelength image data acquired during an actual the number of frame buffers available in the SSME test firing in which a premature shutdown image processor.) Processing began just prior occurred. Following the ignition sequence, the to ignition following a pre-test synchronization engine entered mainstage mode and operated tlash and concluded with the pt-test mode. For normally for roughly 3.5 sec. At that time, each sequence, a plot of the output mean value hydrogen gas began to leak from the low (average intensity of the output image) versus pressure fuel turbopump (LPFTP). Thii leaked frame number was computed and is shown n gas ignited intermittently, causing small flames P i . 2(a-d). about the powerhead for approximately 4 sec until a large flash fire was detected by an The peaks in the plots of Figs. 2(a-d) external sensor and the shutdown sequence was correspond to the events occurring during the initiated. engine test. The peaks detected during ignition were caused by lights beii turned on and by the Four image sequences of this test plume forming at the bottom of the nozzle. transcribed from film onto magnetic videotape Ignition also caused the nozzle to vibrate which were supplied by the National Aeronautics and shook loose frost that had accumulated on Space Administration (NASA). These sequences various cold engine components. These events correspond to four different views of the engine caused the first large peaks in the output mean powerhead taken during the same test firing plots. As the engine entered mainstage mode, from data acquisition cameras which were the mean value output reduced to that caused denoted numbers 1, 6, 7, and 8 by NASA. by the noise. Figure 1 shows the fields of view for each of these four camera positions. These images were The leak near the LPFTP resulted in a acquired prior to ignition and therefore are large peak in the output mean value at clearer than those processed during the f i . approximately 6.1 sec in Fig. 2(d) (camera The most prominent features in these position 8), and smaller, but still noticeable photographs are the low pressure fuel peaks at the same time in Figs. 2(a) and 2(b) turbopump (LPFTP) discharge duct (Fig. (camera positions 1 and 6). While the leak was l(a,d)) and the low pressure oxidizer turbopump detected in three of the four sequences, it was (LPOTP) discharge duct (Fig. l(b,c)). The most obvious from camera position 8. The leak LPETP from which the leak occurred is barely caused a flow that purged the vapor clouds visible in the upper right hand corner of Fig. l(d) normally surrounding the powerhead and cleared (camera position 8). (Because of the wide-angle the field of view. Flames are visible from 6.1 to lenses used on the cameras, features are 10.1 sec in camera position 6 data and resulted in substantial output mean values during that operation is carried out in a dedicated frame time as shown in Fig. 2@). grabber and data transferred to other processing hardware, the continuous transfer The flash fire at 10.1 see is evident in a1 rate of the internal data bus is nominally 7.4 the plots of Fig. 2. It was detected by the Mbytes/sec. monitoring systems presently in use and the shutdown sequence was initiated. The engine Examination of equation (5) shows that - vibration and water spray associated with at each pixel, 3 arithmetic operations 1 - shutdown and post-test procedures also result in multiply and 2 adds are required for the large peaks in the output mean value in all four filtering process. Also, the absolute value plots of Fig. 2. constitutes one operation/pixel, and the sum over the field of view requires one As the plots in Fig. 2 show, if this operation/pixel. For a 480 x 512 image pixel system had been impIemented for this engine array and a 30 Hz framii rate, the necessary test, the leak/no leak decision would have effective processing speed is thus roughly 37 probably been positive at approximately 6.1 sec. million operations/sec. Thus, the engine would have begun shutdown 4.1 sec before the systems presently implemented Memory requirements for the processors indicated. This analysis of these four data sets are governed by the number of terms in the thus demonstrates the potential value of this moving average N, which is determined by the technique for monitoring SSME test fuings. criterion given in equation (6). Practical experience with SSME test stand data has shown that up to 15 previous input frames in addition IV. SYSTEM SPECIFICATIONS to the present can be required for proper performance. One additional frame is needed for The conceptual structure of the system the previous output frame. While input frames required to implement the leak detection can be stored in 1 byte/pixel integer format, the algorithm is shown in Fig. 3 along with a output frame must be in 4 bytelpixel flmting- summary of the nominal system specifications. point format. The total memory required is An AID converter (or frame grabber) diitizg thus 5 Mbyte of acas memory. the input video voltage signal. This data is then transferred via an internal data bus to one or This system design is intended to more high-speed digital signal processors where compute the sum of the absolute values of the the bulk of the numerical computations 6 carried outputs defined by (5), normalize by the out. The resulting output sequence of mean appropriate scale factor ((1 + $)/(2N x 480 x values is then transferred to the host computer 512)), and transfer those values to the host for display and archival storage. computer via its internal bus at the framing rate of 30 Hz. This produces the final mean value at The proposed system is intended for diirete time intervals. This value will be written standard RS-170 format input video signals with to disk for archival storage. In addition, the a framing rate of 30 Hz and 480 lineslframe. value can be compared with a threshold value Adequate resolution is achieved by digitizing each during processing to determine whether the "red line into 512 samples. The dynamic range of the line" condition has been exceeded indicating a intensity values can be covered with 8 bits (1 leak has occurred. Another feature is that a byte) per pixel. If this A/D conversion manual override from the keyboard must be provided to prevent false alarms from occurring continuously processed. Sufficient memory is during the ignition sequence and other planned available in this system for storing the image anomalous events. currently being processed and 15 past images. Therefore, a value as large as N = 15 can be Figure 4 is a block diagram of the used in (5). recommended system. An overlay frame grabber (OFG) from Imaging Technology, Inc. was selected for the video digitizing function V. CONCLUSIONS because of its ability to continuously output all frames of digitized video data over a standard A method for detecting abrupt, steplike - synchronous digital data bus the VISIONbus. changes in a time sequence of images has been The VISIONbus data is split into four separate developed, implemented on general-purpose streams, each of whiih is routed to a separate equipment, and tested. The detection algorithm Texas Instruments TMS320C40 32-bit floating- functions by filtering the input data with a point digital signal processor using a custom- cascade connection of a highpass filter and a built interface board which sends either every moving average. The absolute value of the fourth pixel or every fourth row of pixels to the resulting image is then averaged over the field of same processor. This interface is the only view to determine a mean value estimate. This custom-built component in the system and can time-varying value can then be compared to a be fabricated on a single PC-plug-in card with pre-set threshold to determine a "mi-line" four synchronous first-in first-out memories condition for shutdown. (FIFOs) and five state machines. The four C40 processors are located on two Spirit4 AT/ISA This method has been applied to actual dual-C40 PC-plug-in boards from Sonitech video data acquired during a SSME test firing. International, Inc. The analysis of the test-stand data indicates the applicability of this technique to actual SSME A C40 assembly language program was test firings and its ability to identify anomalous written to verify that the C40 processors can events. perform the processing of (5) in the time available. This program carries out the A system has been identifd and specified calculations of (S), takes the absolute value of for real-time on-line implementation of this the pixels computed from (5), and sums these method for SSME leak detection. This system absolute values. The program was run on a uses an IBM compatible personal computer as a C40 simulator and was optimized to minimize host platform and an Imaging Technology the processing time required per pixel. With Overlay Frame Grabber with the VISIONbus for appropriate local and global bus memory, and 40 data acquisition and transfer. Processing is MHz C40 processors, the program requires 9 carried out by four TMS320C40 processors instruction cycles of 50 nsec each to process one located on two Sonitech Spirit-40 add-in cards pixel giving a total processing time of 450 and requires that a custom interface be nsec/pixel. Although pixels become available on fabricated to transfer data from the frame the VISIONbus at an average rate of one every grabber to the processors. The processing I36 nsec, each of the four C40 processors only algorithm has been programmed and proper receives a new pixel every 544 nsec. Thus, the timing to achieve the required effective 450 nsec/pixel processing time is sufficient to throughput rate has been verified. guarantee that incoming video data can be Future work in this effort includa examining the effectiveness of the algorithm in infrared imaging applications and other test stand data sequences. Work is also planned to extend this technique to color image processing to detect sudden changes in plume color that otherwise may not result in detectable intensity variations. F d y , application to actual launch- pad data is also a possibility. ACKNOVVLEDGEMENTS The authors would like to express their sincere appreciation to Prof. T. Dwayne McCay for several useful discussions regarding the [I] A. A. Shohadaee, "Leak detection feasibility investigation using infrared radiation transfer n absorbing, emitting and scattering media," Doctoral dissertation, Dept. of Mech. Engr., Univ. of Tennessee, Knoxville, TN, 1990. [2] A. A. Shohadaee and R. A. Crawford, "SSME leak detection feasibility investigation by utilization of infrared sensor technology," Center for Advanced Space Propulsion Second Annual Technical Symposium Proceedings, Tuliahoma, TN, Nov. 1990. [3] J. A. Malone, "A system for leak detection using sequential image pmessing," Master's thesis, Dept. of Elec. Engr., Univ. of Tennessee, Knoxville, TN, 1991. [4] J. A. Malone and L. M. Smith, "A system for sequential step detection with application to video image processing," IEEE Trans. on Industrial Electronics, vol. IE39, pp. 277- 284, Aug. 1992. Figure 1. Filds of view for each of the four cameras: (a) pition 1, (b) position 6, (c) position 7. (d) position 8. Camera Position 1 Time (sec) (a) Camera Position 6 14 - - - 12 -- I 1 I I I I I - 10 r - f - 8 - - c 6 - a 4 - - 0 2 4 6 8 10 12 14 16 Time (sec) (b) Camera-Position 7 Time (sec) (4 Camera Position 8 14 - -- - - 5 - I I I I I 1 1 - - 12 - I 10 - 3 8 - P - c 6 - a 4 - 0 2 4 6 8 10 12 14 16 Time (sec) (4 Figure 2. Mean value of output images versus time for test-stand video data. Note the detection of the leak near 6.1 sec. (Some initial data from camera position 7 (c) was missing, so output was set to zero in that range.) GL-3119 A Video Signal 1 I Video ND Input: 30 frameslseconds Frame Grabber I I Format: 480 x 51 2 x I byte pixelslframe Data BUS Transfer Rate: 7.4 Mbyteslsecond Processor(s) Memory: 16 previous frame buffers L I I Host Bus Algorithm: 5 operationslpixels Processing Rate: 37 MOpslsecond Host Computer CG-2497 Figure 3. Basic system configuration and nominal specifications. 4 - Spirit 40 4 + ...........A...T..I.I.S...A.. ......... Dual TMS320C40 - + 4 t Processor Board Vision Vision Bus Video Overlay In Frame Bus * to TMS320C40 Grabber Interface - Spirit 40 ATIISA ................................ TMS320C40 Processor Board PC Bus CG-2581 Figure 4. Block diagram of recommended system using TMS320C40 processon. GL-3120