AIAA-2002-0532 Dot Projection Photogrammetric Technique for Shape Measurements of Aerospace Test Articles T. W. Jones and R. S. Pappa NASA Langley Research Center Hampton, VA 40th AIAA Applied Aerodynamics Conference 14-17 January 2002 / Reno, NV For permission to copy or republish, contact the American Institute of Aeronautics and Astronautics 1801 Alexander Bell Drive, Suite 500, Virginia 20191-4344 i P i AIAA-2002-0532 Dot Projection Photogrammetric Technique for Shape Measurements of Aerospace Test Articles Thomas W. Jones " and Richard S. Pappa NASA Langley Research Center Hampton, VA 23681-2199 Abstract Videogrammetry, while capable of providing spatial Results from initial laboratory investigations with the positions of targets in near real-time at higher dot projection photogrammelric technique are accuracy than PMI, has traditionally relicd on a presented for three wind-tunnel test articles with a limited number of either painted or relro-reflcctive range of surface scattering and reflection properties. targets attached to the surface of a model. This These test articles are a semispan model and a micro limited number of targets may provide incomplete air vehicle with a latex wing that arc both diffusely surface characterization and, in addition, requires reflecting, and a highly polished specularly reflecting time for target application. model used for high Reynolds number testing. Results using both white light and laser illumination Dot projection techniques attempt to fill the gap arc presented. Some of the advantages and between the two aforementioned systems, providing a limitations of the dol projection technique arc completely non-contacting spatially complete surface discussed. Although a desirable final outcome of this measurement technique suitablc for near real-time research effort is the characterization of dynamic applications. The method of obtaining object space behavior, only static laboratory results are presented coordinatcs described here is based on the projection in this preliminary effort. of a dense dot pattern (targets). Images arc collected and processed from two instrumentation grade video Introduction cameras to yield the three-dimensional shape of the Several image-based, non-contacting video object based on the spatial coordinates of the measurement techniques, including Proiection Moir6 projected dots. While sevcral systems employing Interferometry (PMI) and Videogrammetry have been retro-reflective and painted targets and projected used for shape characterization in wind tunnel Moir6 fringes have been widely used in wind tunnels applications at NASA and elsewhere I. These at NASA, dot projection techniques have not yet been techniques are capable of providing quantitative used at all. measurements of model deformation and dynamics, but each has some limitations or constraints. The The projection of structured light onto a surface has often extensive processing requirement for classic been extensively used for the surface reconstruction PMI restricts its use in real-time applications. of static objects for more than a decade in various research and industrial applications. More recently efforts to develop non-contacting optical techniques have expanded to include not only model deformation measurements during wind tunnel tests, but also ground based testing of a new generation of Ultra-Light Inflatable Space Structures (UIS) 2'3'4. Concurrently, the aeronautics community is focusing technology development toward a new generation of biologically inspired flight systems (BIFS) and micro air vehicles (MAV) -_. One common aspect these *Research Engineer technology development programs share is their -tSenior Research Engineer, Member A1AA focus on membrane construction for critical Copyright ©2001 bythe American Institute of Aeronautics components. An example of each is shown in Figure and Astronautics, Inc. No copyright is asserted in the I. The material density requirements for each of these United States under Title 17, U.S. Code. The U.S. programs will restrict the use of attached retro- Government has aroyalty-free license to exercise all rights under the copyright claimed herein for Governmental purposes. All other rights are reserved by the copyright owner 1 American Institute of Aeronautics and Astronautics reflective targets and therefore there is a need characterization, validation, and control of these for a comparably accurate system with near membrane structures. real-time capacity to support dynamic structural but the rapid development of CCD technology led to the successful migration from film to video technology. Photogrammetry combined with image processing has been found to yield accurate and near real-time data, but at a limited number of points on a given model. At the other end of the technology spectrum is the PMI technique, with generally lower Figure1: UISandMAVVehicle Examples accuracy than the photogrammetric techniques, but with more complete spatial coverage. The PMI technique projects a grid of equally spaced The potential for commercialization of a dot- parallel lines onto the test article. The basic projection photogrammetric system has also principle of the method is that any height been recognized by industry. Both Geodetic difference from a reference plane causes a shift Systems,Inc. and ShapeQuest, Inc. have in the projection line of light in the image plane commercially available products based on the of a recording camera. 8 While the technique concept of dot projection. 6'14 provides full-field measurement it is not well suited for real-time applications. There are As technology advancements continue at a rapid several factors that can affect measurement pace in high-resolution CCD video cameras, accuracy for the PMI technique; among them storage capacity, and central processing are projected grid pitch, field of view, optical speeds, the development of numerous digital modulation transfer function, and illumination. image processing software packages has automated many of the tasks that previously The dot projection technique proposed here will required a detailed understanding of the seek to find a "middle of the road" in the optical principles of photogrammetry and image measurement technology spectrum, between processing, Thus a range of high-accuracy the superior accuracy associated with the use of applications are feasible, even for the novice. photogrammetric principles and retro-reflective For our purposes we seek to provide a targets (as they have been developed incurrent continuous spatial model of the test article videogrammetric systems) and the spatially without the requirement for attached targets and continuous measurements provided with the lengthy post-processing requirements. The goal PMI technique. The basic goal of dot projection of this initial investigation was to evaluate the is to apply the same photogrammetric principles advantages and constraints associated with the used in the aforementioned videogrammetry use of dot projection as a method of gathering techniques, but without the use of attached three-dimensional shape data. targets. The dot projection technique as presented here is based upon two video Technology Spectrum cameras, but photogrammetric techniques with The basic data for photogrammetric more than two cameras would be required to measurements are images. In general, an capture extreme 3-D motion where targets may image isthe resultof a perspective projection of become occluded occasionally. 9 a three-dimensional (3-D) object to two dimensions (2-D). The photogrammetric Experimental Discussion solution provides a quantitative relationship Toevaluate the accuracy of white light projected between 3-D position, and/or deformation and targets a laboratory expe'rimen(was designed_to the 2-D image plane data recorded by one or directly compare the three-dimensional more cameras. Over the last two decades a ( photogrammetric accuracy of projected targets variety of photogrammetric techniques have against the adopted standard provided by been used in a number of wind tunnel tests to retroreflective targets. Figure 2 shows the provide wing twist and model deformation laboratory configuration. The hardware measurements7 Originally, thetechniques were configuration included two Hatachi CCD developed for use with still-frame film cameras, (640x480) progressive scan video cameras with 2 American Institute of Aeronautics and Astronautics Sony 25-mm focal length lenses, and a Newport been completed, any camera movement will MP-1000 slide projector. The test article for this result in erroneous data. Similarly, any vibration experiment was a mock-up of the 16 percent or motion of the projector once the initial scale F18-E/F planform Smart Wing developed reference is established will prevent accurate by Northrop Grumman and tested at the NASA ............ in image sequences. Langley Transonic Dynamics Tunnel in March 1999. l° Projection Hardware A white-light projector used for the dot projection technique does not have to be calibrated, as is the case with the PMI technique. The projected pattern is imaged simultaneously by two CCD ,i i!i ii ¸¸ cameras. As with most two-camera techniques Ill!tin _" _ _:_._'_L_iC__:_!C.O_ • an angle of around 90 degrees between the Projector _ _ ii optical axes of the two cameras is desirable to maintain precision in the approximate direction that the cameras are pointed. Video Camera :::..... A series of slides with different dot sizes were Figure 2: Hardware Configuration produced, from which the best projection pattern was selected depending on the object surface Calibration properties and the desired resolution. The slides Perhaps the most critical aspect of any used stainless steel screens that were photo- photogrammetric measurement technique is the chemically machined using a photochemical camera calibration. The camera calibration etching process. _2The material is commercially determines the camera orientation parameters available in sheet-form. Because they are that relate the 2D image plane and 3D object photochemically etched, the metal is not space. The process is developed with a set of mechanically perforated; resulting is greater collinearity equations. _ three-tier calibration material integrity, and a cleaner, burr-free target plate, shown in Fig. 3, provided a three- perforation. Using the metal screen as a dimensional known target field. The coordinates projection medium produces maximum contrast of the target field, which were independently in the pattern as well as high thermal stability measured with a 3-D coordinate measuring compared to a film slide. The slides were cut to machine, provide an input for the camera fit a standard slide projector. For the results calibration process. presented here a Newport MP-1000 Moir@ projector fitted with a 50 mm Nikon lens was used in place of a standard slide projector. Experimental Configuration To generate out-of-plane motion of the wing in this experiment, it was attached to a Velmex 12 inch linear translation table. The table provided measurement accuracy of 0.001 inch as verified X with electronic dial gauges accurate to 0.0001 inch. The projector axis was aligned with the ¥ translation axis of the table to ensure the wing 11111 m remained inthe field of projection throughout the range of motion. A perspective of the wing under Figure 3: 3-Step Calibration Target Plate dot projection is shown in Figure 4. The calibration process establishes a coordinate system in which all measured points are referenced. We aligned the axis of motion defined by a translation table with the Z-axis as established by the calibration plate. For this configuration, the Z-axis represents motion toward the cameras. Once the calibration has 3 American Institute of Aeronautics and Astronautics Point to PoLarDifference: Measured Travel w/Retro Target _0980 1 2 3 4 5 fi 7 8 9 l0 Tr'atshfioma[]_¢*_m_at(Inches) Figure 5: Retro Target Incremental Change Figure 6 shows the scatter in the Z component between the two targeting methods over the Figure4:Test Articlewith DotProjection (SimulatedImage) measured range. Retros&Projected Four retro-reflective targets were positioned at 15 two spanwise locations along the wing. This °°°t comparative analysis used 38 of the projected targets. The tracked retro targets are identified O in green (most extreme targets) and the o Retros ] projected targets in white (middle set of targets) i i i o Projected Jl in Figure 4. The wing translated over a ten-inch 2 4 6 8 10 range with image sequences acquired at one AZure, inch 0.5 inch increments. The axis of motion was along the Z-axis as defined by the camera calibration D process. ,Jil Results ? As expected, the retro-reflective targets tracked .050L 2 &4__ G__ 8 10 the motion of the wing with high accuracy and AZs_a_,Inck very small scatter over the measured range. °'°_t Using the retro data to check for possible error " o ¢ k o in the alignment of the photogrammetric axis (Z) @ o 8 o # o R J;; and the axis of translation identified a slight )i °2t 0 0 C. _ misalignment of 0.42 degrees, ttwas determined 0 this small misalignment would not significantly 0 4 6 8 10 affect the data over the measured range. Figure AZstaae, inch 5 highlights the delta change for the retro targets over the measured range and identifies a mean Figure 6: Comparison Retro vs. Projected of 0.9997 inches with a standard deviation of 0.0048 inches. The standard deviation for both A closer inspection of the resuits indicates a the X and Y-axis was approximately 0.001. The substantial variation in the XY plane for the better precision in the XY plane was due to the projected targets while the attached retro targets small convergence angle between the two show only small variations inthat plane over the cameras that led to reduced precision in the Z- same range of motion. Figure 7 highlights the axis. recorded motion of the projected targets over the measured range. With the motion of the model along the Z-axis the projected targets appear to move closer together. As would be expected the motion of the projected pattern appears to converge toward a point of symmetry for the projection. This convergence is seen to bea point inthe upper left quadrant of the plot. 4 American Institute of Aeronautics and Astronautics Target Variation Over Measured Range Residuals from linear least squares fi! (projected only) 4.:o0.2,,t [ [] 'P,uj+u+Id 6 [] ,'r°.ie°ted | t t % [] Retros I [] i! m H B e _ e B it 4 13 [] 2 [3 o 4O 5 inch + AZs_aoe, ._ o.1] [3 o+of, -2 [] O m H § = u .4 D ..6 o e 1o i I i ' ' +_.-' ' 0 ' _,Zsta_e,inch 2 4 6 8 10 12 14 16 48 X, inch Figure8:LinearLeastSquaresFit (Proj) :; RProejtercotesd 1 Plots of the computed residuals for the projected targets for both the X- and Y-axes are shown in Figure 8. One of the targets has a significantly larger residual variation over the range. Manual inspection of the image sequences identified the i,-. u 0 _c target in question as an edge target with an incorrect background calculation. This -2 discrepancy isdiscussed inthe next section. Discussion - Target Recognition -8 A factor in achieving maximum accuracy with either retro-reflective or projected targets is the 0 2 4 6 8 40 12 14 16 48 image plane diameter of the target. During test X, inch planning, the user should consider the Figure 7: Retro/Projected Motion inXYPlane necessary field of view and the resolution ofthe cameras. For best centroiding accuracy, the To interpret the true surface motion as defined diameter of targets inthe camera image plane by the projected targets, a fixed reference point should be approximately 5-7 pixels, in the case would need to be included in the data set. The oflargerstructures suchasproposed solar sails, repeating dot pattern of projected targets can be this requirement may dictate the need for targets exported to a CAD environment to generate several inches in diameter. A noted benefit of splines along various regions of interest. An using projected targets is the ability to quickly obvious shortcoming of projected targets is the change projection slides to adjust from larger to inability to provide time histories of a specific smaller targets when the research focus point of interest onthe test article. changes to an individual component of a larger model. Target recognition can be achieved in several ways. Several of the systems currently inuse at NASA Langley require initial manual identification of each target for each camera. It is obvious that such an effort for a system using hundreds of closely spaced projected dots would be excessively labor intensive and prone to human error. It is therefore desirable that the identification of targets be automated. A useful algorithm for matching the projected dots between two images is based on the epipolar 5 American Institute of Aeronautics and Astronautics geometry. Epipolar geometry can help establish correspondence between dots for each camera view. As shown in Figure 9 for object point P, the projection centers of the two images, O1 and 02, form a plane (the epipolar plane) that intersects the two image planes. As a result, two epipolar lines (one for each image) are created. Corresponding points fall on these lines. Once the camera positions and orientations are determined from calibration, the equations of the epipolar lines are known. 13Since there are many points that may fall on this line the approximate distance from the camera to the test article is used as an additional constraint for the algorithm. Z 1 I i--'pP<ir,ri,l_- t p2 1 ,i i::f"_'_<': _,. I | : _ | if.-"- ----J Figure 10: Target Recognition Errors Ol 0"_ Discussion - Target Tracking Y There are three widely adopted techniques used fortarget tracking. They are centroid, edge, and /-# P correlation tracking. Centroid tracking is a brightness based segmentation technique that computes the centroid for the segmented area. Edge tracking seeks todefine the target withan edge segmentation scheme that isfollowed by a Figure 9: Epipolar Geometric Model14 centroid calculation. The correlation technique identifies a template to track from frame to Several factors may contribute to mistaken frame. For our experiment a centroid tracking correspondence of targets. Object features technique was applied. such as slots, edges or holes will often cause a projected dot to appear as two distinct targets There are a number of methods that have been from one camera perspective, but appear as a developed to find the subpixel location of a normal ellipse from another. Figure 10 shows target. 15 Among the various techniques the same region of interest on the model but available are Binary centroid, Gaussian from opposite camera positions. Note that the Distribution Fitting, Ellipse Least Squares, edge between the wing and the flap creates Perimeter Average and Squared gray scale several erroneous targets that must be detected centroid. and discarded by the software during image processing. This type of error isalso more likely With projected dots several issues can affect the when the separation angle between cameras is accuracy of the subpixel algorithm. An accurate large, however reducing this angle negatively determination of the target background is affects sensitivity in the Z-axis. Automating the necessary. With dot projection, the combination process of false target discrimination for large of a high-density dot pattern and a model that target fields is difficult. covers a wide depth of field can negatively influence the definition of background. In many applications the search window for the targets edge is defined as some factor times the largest target diameter found on the object. For our 6 American Institute of Aeronautics and Astronautics experiment the depth of field of the projected not critical, the projector was positioned 10 feet pattern created targets that ranged from 21 to 45 from the test article to ensure sufficient intensity pixels over the full span of the model. When using a high-density projection pattern, adjacent targets only a few pixels away may negatively When considering the use of white-light dot influence target centroiding when a general projection over the more common retro-reflective search window based on the diameter of the targets, consideration must be given to the largest target is used to define the background. reflective nature and surface finish of the test With our experiment the 38 projected targets article. Typically wind tunnel models have were selected over a region that minimized their surface finishes ranging from highly polished size variation and thereby avoided this issue of steel to painted diffuse white panels. As background definition. demonstrated in Figure 12, the contrast in target and background is very low for a highly reflective As simulated in this paper using a linear stage to model surface due to the substantial reflection of move the entire wing in the out-of-plane light away from the cameras. In cases where direction, the technique could support a wide the specular reflection direction is toward one of range of dynamic motion. A reconstructed the cameras, the light source may create bright rendering of the model surface created using all spots that prevent target recognition in the of the projected targets from a single image set immediate area. of the experiment is shown in Figure 11. With the use of external control targets (i.e. stationary Deformation measurements at the National background targets) a rigid frame of reference Transonic Facility (NTF), where model surfaces could be established to reference sequential are always highly polished, routinely require the renderings of the model. targets to be painted on the model surface. 16 The polished paint diffuse targets are typically 0.0005 inch thick with a surface roughness of less than 10 pinch. Ordinary lights (non-laser) can provide sufficient illumination. While this technique has been demonstrated to be effective for deformation measurements at the facility, the procedure for painting the targets is detailed and time consuming. Similarly, repair time for damaged targets can be substantial. Figure 11: Surface Rendering of Wing Ambient lighting can also create reflections on a painted target model. Since a laboratory The step from capturing static surface contour investigation of the use of projected targets on measurements to dynamic motion may require highly reflective model surfaces yielded poor additional hardware. One possibility is a results due to the low contrast and surface configuration using a pulsed light source as the reflections of the source projector, further projector. The pulsed light output would freeze developments will be necessary before the dot the motion to prevent blurring of the projected projection technique is suitable for high dots. To ensure the light pulse occurs during Reynolds number facilities such as the NTF. the camera open shutter period an external synchronized trigger will be required. Model Surface and Lighting To take advantage of the full dynamic range available with a standard 8-bit digital video camera the settings should be optimized with the combination of shutter speed and aperture to remove extraneous light. Due to the variations in ambient lighting in many wind tunnel facilities a higher power projector may be necessary to ensure adequate contrast. Since the distance between the retro targets and the light source is 7 American Institute of Aeronautics and Astronautics laser beam and optical axis of a camera. The A.G. Davis index head has an accuracy of better than 1 arc second. For this experiment a 5mW HeNe laser was selected. The MAV model wing section used was made of latex and provided a consistent contrasting background for target comparison. The model was placed 48 inches from the laser and camera station. The camera optical axis and the projected beam axis were aligned by sight. The setup is shown in Figure 13. A stationary Kodak DCS 460 6 megapixel (3060x2048) camera acquired the image sequences. The camera was stationary during the image sequences. The camera focal length and aperture settings remained constant for all images. The camera electronic shutter was Figure 12: Highly Polished Metal Surface varied to obtain approximately the same relative Laser Projected Targets exposure for each trial. All targets were approximately 0.25 inches insize. There are advantages and disadvantages to using a laser as a replacement source for Camera calibration yielded an estimated projecting the dot pattern. A suitable laser will provide a concentrated diffraction limited beam sensitivity of 0.0055 inches/pixel. The calibration scheme employed for the Kodak DCS that is generally visible above the ambient lighting in most environments. This high professional series camera was based on a photogrammetry technique known as "bundle intensity beam may be able to provide comparable signal-to-noise ratios commonly adjustment." The scheme utilizes multiple camera stations, multiple convergent views, and found with the use of retro targets. In addition, a different camera angles to determine accurately laser could also be chosen that would operate in the near infrared. A pulsed laser diode emitting the interior orientation parameters. Using_:the -Commercial Photorn-0deie_ software, from EOS in the near infrared similar to that used in the aforementioned PMI systems, coupled with Systems Inc., the process requires the user to manually identify a series of control pointson suitable wavelength filtering, may supply each of the Calibration images. The bundle sufficient contrast to allow lights-on operations in most research facilities. solution then Computes the interior orientation and distortion parameters of the camera. Among other advantages noted is the narrow ............J..J........................... spectral bandwidth of the laser, which provides the user the unique ability to remove most of the extraneous light with an interference filter on the i¸ camera. Diode lasers are another option that compare favorably to available white light projectors. A diode laser offers a small (few inches) and self-contained package that operates with low power requirements. When coupled with a diffractive pattern dot generator the combination is routinely used in laser machining, laser marking, alignment systems and computer vision. However, note that safety considerations will be a factor when using a laser as the projection light source. Figure13. LaserTarget ExperimentConfiguration As part of the evaluation of the centroiding Visual observations indicated that laser speckle accuracy using retroreflective, laser, or white might adversely impact centroid determination. light projected targets, a micro air vehicle (MAV) Due to the concentration of light for the laser the test article was mounted on a precision rotary target saturated (255) on an 8-bit grayscale index head approximately perpendicular to the 8 American Institute of Aeronautics and Astronautics