Compressive Underwater Video Camera Frank M. Caimi Harbor Branch Oceanographic Institution, Department of Electrical and Computer Engineering Ft. Pierce, FL 34946 phone: (561) 465-2400 x256 fax: (561)-465-2446 email:[email protected] Gerhard X.Ritter and Mark S. Schmalz University of Florida, Department of Computer and Information Sciences Gainesville, FL 32611 Award #: N0001496-1-5019 http://www.hboi.edu/engineering/englaser.html http://www.cise.ufl.edu/~mssz/ LONG-TERM GOAL The primary long term goal of our research team is the development of image compression transforms with high compression ratios (consistently exceeding 200:1), which can facilitate the transmission of moderate resolution imagery (e.g., S-VHS format) across low-bandwidth channels such as acoustic uplinks from autonomous underwater vehicles (AUVs). Secondary goals include the development of image quality measures (IQMs) that can support rigorous comparison between compression transforms, as well as the implementation of best-performance transforms on fast digital signal processors (DSPs) or reconfigurable architectures suitable for AUVs. A tertiary goal, which we plan to explore in depth in future research, is the integration of image compression with automated target recognition (ATR). Two cases are of interest to this study. First, we have shown that one can achieve increased processing efficiency by applying specially configured ATR operations to compressed imagery [Sch97c,d]. Second, we have developed techniques for prefiltering multi-target imagery to downselect candidate targets prior to compression [Sch96]. The first technique uses compression to increase computational efficiency of ATR processes, while the second technique employs ATR to increase the effective compression ratio. OBJECTIVES In FY98, our primary goal was to establish that the compression transform developed under the scope of this study, called EBLAST (Enhanced Blurring, Local Averaging, Sharpening, and Thresholding), performed better than published compression transforms such as EPIC, SPIHT, VPIC, and wavelet packet compression. The second goal was to implement EBLAST, or its predecessor BLAST, on DSP-based hardware, to determine feasibility for underwater image compression via laboratory or field tests. A third objective emphasized the enhancement of our existing suite of IQMs to include measures of linear feature distortion, as well as statistical techniques for quantifying the blurring effect of a Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 1998 2. REPORT TYPE 00-00-1998 to 00-00-1998 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Compressive Underwater Video Camera 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Harbor Branch Oceanographic Institution,Department of Electrical and REPORT NUMBER Computer Engineering,Ft. Pierce,FL,34946 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES See also ADM002252. 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 6 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 compression transform. This was expressed in terms of cutoff frequency, which was computed from the modulation transfer function of a decompressed image and its corresponding source image. APPROACH Image quality measures that describe feature distortion were derived from the Hough transform (sinusoidal formulation). From the Hough transform peaks that correspond to vertical and horizontal lines, we derived a measure of blocking effect, a common distortion in block-oriented compression transforms. Cutoff frequency was determined by (1) collecting in a set D the intersection points of the first quintile of the MTF's range with the MTF itself, then (2) computing the centroid of spatial frequencies in D. Salient technique was published in [Sch96] and [Cai98b]. The IQMs derived in this study were used to evaluate EBLAST against several published compression transformations, whose authors claimed compression ratios exceeding 100:1 and low mean-squared error (MSE) in the decompressed image. EBLAST's predecessor transform, called BLAST [Sch96], was implemented on an Analog Devices SHARC 3000 dual processor DSP. We investigated the effect of input subsampling on MSE and compression ratio, in order to reduce the computational work requirement of EBLAST. HBOI plans to implement EBLAST on the Analog Devices SHARC 3000 Quad processor. Preparatory to formulating an Advanced Technology Summary to direct possible future work, we analyzed the implementation of EBLAST, VPIC, JPEG, and EPIC on various parallel and reconfigurable embedded DSPs. Results were published in [Cai98a] and [Sch98a]. WORK COMPLETED HBOI and UF have completed the following tasks under the proposed statement of work: 1) image analysis, compression algorithm and implementation requirements development, 2) selection of several best-performance compression algorithms from the literature, 3) development of an improved high- compression algorithm, 4)comparison of the developed algorithm with prior art (following testing on database imagery), and 5) algorithm DSP implementation/testing. We have also held advisory meetings on the implementation of BLAST and EBLAST in DSP hardware, and have authored/co-authored nine conference papers as a result of this research. Final project documentation and preparation of three journal submissions is in progress. RESULTS The image quality measures derived from the Hough transform (sinusoidal formulation) can be computed from MxN-pixel source imagery in O(MN) work. The measure of blocking effect derived from the Hough transform peaks that correspond to vertical and horizontal lines has been employed in compression transform comparison and optimization, as has the IQM related to cutoff frequency. Following computation of the MTF of an MxN-pixel decompressed image, the cutoff frequency computation requires O(max(M,N)) multiplications, additions, and comparison operations. When tested via the aforementioned IQMs (including MSE) EBLAST performed better than EPIC, VPIC, SPIHT, and JPEG, with performance comparable to wavelet packet compression, but at much lower computational cost (measured as the space-time bandwidth product). EBLAST was able to achieve compression ratios exceeding 200:1 on a database of 32 realistic underwater images, with MSE less than 10 percent. Additionally, EBLAST requires very little parameter adjustment to adapt to realistically non-ergodic inputs, versus the considerable modifications required by EPIC, SPIHT, and wavelet packets. Furthermore, unlike these hierarchical or pyramid-structured transforms, EBLAST is block-structured and can thus be computed on small DSPs or SIMD-parallel processors with small local memories [Jac97]. Because EBLAST employs multiplication, addition, and table lookup, it can be computed at eight-bits precision, while preserving compression ratio and MSE [Sch98b]. To the best of our knowledge at this time, this key advantage of reduced-precision computation is not shared by EPIC, SPIHT, JPEG, or wavelet-based compression algorithms published to date, at the levels of precision at which EBLAST can be computed. Source image. EPIC, CR = 98:1. BLAST, CR = 98:1. EBLAST, CR = 240:1. Example of image reconstruction after compression using underwater coral image containing test target. Current Hardware implementation results compression at frame rates of one per second using the BLAST algorithm. With optimized coding and input subsampling techniques, we expect to achieve real-time compression at frame rates of one frame per second. Analysis of test data indicates that source image encoding blocks (typically 8x8 pixels to 10x10 pixels in size) can be subsampled at ratios of 1:4 without degradation of compression ratio or MSE. This means that BLAST or EBLAST can be speeded up by a factor of four without increasing computational resources. IMPACT/APPLICATION The development of EBLAST, with its low MSE and high compression ratio, represents a significant advance in image compression technology. The development of feature-based image quality measures is a significant advance in image analysis technology, which we have directly applied to image compression transform evaluation and optimization. Additionally, the discovery that EBLAST can be computed at reduced precision (eight-bit addition and multiplication operations, as opposed to the customary 16-bit single-precision arithmetic operations) opens new horizons for image compression using embedded processors. For example, it may be possible to use table lookup only, instead of costly arithmetic logic units (ALUs), which directly implies a memory-based architecture. This could result in reduced power consumption and smaller form factor, which are key advantages for implementation on autonomous vehicles that have highly constrained power supply and stringent space requirements. A further application of the EBLAST transformation is expected in videotelephony and Internet video. We are currently exploring possible collaboration with commercial firms to develop and market EBLAST algorithms or compression chips, as described below. Military applications of EBLAST are also attractive, for example, compression of airborne surveillance imagery for transmission along low- bandwidth secure communication channels. The integration of image compression and target recognition algorithms shows great promise for efficient ATR on autonomous vehicles, due to a significantly decreased work requirement. In the case of ATR applied to compressed imagery, speedups are often on the order of the compression ratio. When ATR algorithms are used to detect and downselect candidate targets prior to image compression, increases in the effective compression ratio can range as high as 100:1, depending on the fraction of image area occupied by the targets of interest. TRANSITIONS HBOI is developing an AUV test bed to be used with Florida Atlantic University’s Ocean Explorer II AUV. The high compression ratios achieved using this algorithmic approach should allow transmission of video and sonar data at very near real-time acquisition rates as underwater acoustic modems achieve increased dat thruput. In collaboration with Lockheed-Martin, Inc., UF investigators are assessing the utility of the EBLAST transform for Internet video and videotelephony applications. This effort could result in one or more small, low-power image compression chips. Additionally, UF investigators are collaborating with Dr. Jeff Bloch at Los Alamos National Laboratories on adaptation of target prefiltering and compression technology developed under the scope of this project for plume detection in large databases of multispectral and hyperspectral imagery. Currently, this effort is in a preliminary (planning) stage. RELATED PROJECTS 1) Under ONR funding, HBOI, UF, and Frontier Technologies, Inc. are researching low-power configurable architectures that could be readily implemented on autonomous underwater vehicles. The architectures would support ATR and image compression, with compression transforms analyzed or developed under this study being featured as candidate algorithms. 2) UF is being funded by DARPA to develop scheduling and control software for mapping ATR and image processing/compression algorithms to reconfigurable processors, such as field-programmable gate arrays (FPGAs) or embedded SIMD meshes [Bur96,Jac97, Rit96,Swe98]. The image compression algorithms and image quality measures analyzed or developed in this study are featured in UF's DARPA-funded research. REFERENCES [Bur96] Bursky, D. "Programmable arrays mix FPGA and ASIC blocks", Electronic Design (14 Oct 1996), p.69. [Cai98a] Caimi, F.M., M.S. Schmalz, and G.X. Ritter. "Parallel computation of image compression transformations", in Proceedings SPIE 3456 (1998). [Cai98b] Caimi, F.M., M.S. Schmalz, and G.X. Ritter. "Image quality measures for performance assessment of compression transforms", in Proceedings SPIE 3456 (1998). [Jac97] Jackson, R.D., P.C. Coffield, and J.N. Wilson. "New SIMD computer vision architecture with image algebra programming environment", Proceedings of the 1997 IEEE Aerospace Applications Conference (Snowmass Village, Colorado, USA) vol. 1, pp. 169-185. [Rit96] Ritter, G.X. and J.N. Wilson. Handbook of Computer Vision Algorithms in Image Algebra, Boca Raton, FL: CRC Press (1996). [Sch96] Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Data compression techniques for underwater imagery", in Proceedings IEEE OCEANS '96 Conference, Ft. Lauderdale, FL (1996). [Sch97c] Schmalz, M.S. "Detection of small targets in compressed imagery. 1. Performance of edge detectors and morphological operations over VQ- and VPIC-compressed imagery", in Proceedings SPIE 3068 (1997). [Sch97d] Schmalz, M.S. "Detection of small targets in compressed imagery. 2. Performance of component labelling and target classification algorithms over VQ- and VPIC-compressed imagery", in Proceedings SPIE 3068 (1997). [Sch98a] Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Mathematical methods for mapping image compression transforms to adaptive computing systems", in Proceedings of the IEEE Oceans `98 Conference, Nice, France (1998). [Sch98b] Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Limits on computational precision of image compression transformations", in Proceedings SPIE 3456 (1998). [Swe98] Sweat, M. and J.N. Wilson. "An overview of AIM: Supporting computer vision on heterogeneous high-performance computing systems", to appear in Proceedings SPIE 3452 (1998). PUBLICATIONS Caimi, F.M., D.M. Kocak, M.S. Schmalz, G.X. Ritter, “Comparison and Development of Compression Algorithms for AUV Telemetry: Recent Advancements,” MTS Ocean Community Conf. ‘98, Baltimore, MD, November 1998. Kocak, D.M., F.M. Caimi, “DSP Hardware Implementation of Transform-Based Compression Algorithm for AUV Telemetry,” IEEE Oceans ’98, Nice, France, September 1998. Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Mathematical methods for mapping image compression transforms to adaptive computing systems", in Proceedings of the IEEE Oceans `98 Conference, Nice, France (1998). Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Limits on computational precision of image compression transformations", in Proceedings SPIE 3456 (1998). Caimi, F.M., M.S. Schmalz, and G.X. Ritter. "Parallel computation of image compression transformations", in Proceedings SPIE 3456 (1998). Caimi, F.M., M.S. Schmalz, and G.X. Ritter. "Image quality measures for performance assessment of compression transforms", in Proceedings SPIE 3456 (1998). Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Performance analysis of compression algorithms for noisy multispectral underwater imagery of small targets", in Proceedings SPIE 3079 (1997). Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Performance evaluation of data compression transforms for underwater imaging and object recognition", in Proceedings IEEE OCEANS`97 Conference, Halifax, Nova Scotia (1997). Schmalz, M.S. "Detection of small targets in compressed imagery. 1. Performance of edge detectors and morphological operations over VQ- and VPIC-compressed imagery", in Proceedings SPIE 3068 (1997). Schmalz, M.S. "Detection of small targets in compressed imagery. 2. Performance of component labelling and target classification algorithms over VQ- and VPIC-compressed imagery", in Proceedings SPIE 3068 (1997). Schmalz, M.S., G.X. Ritter, and F.M. Caimi. "Data compression techniques for underwater imagery", in Proceedings IEEE OCEANS'96 Conference, Ft. Lauderdale, FL (1996).