Entropy and Information Optics Connecting Information and Time, Second Edition OPTICAL SCIENCE AND ENGINEERING Founding Editor Brian J. Thompson University of Rochester Rochester, New York RECENTLY PUBLISHED Entropy and Information Optics: Connecting Information and Time, Second Edition, Francis T. S. Yu Laser Beam Shaping Applications, Second Edition, Fred M. Dickey and Todd E. Lizotte Fundamentals and Basic Optical Instruments, edited by Daniel Malacara Hernández Tunable Laser Applications, Third Edition, edited by F. J. Duarte Laser Safety: Tools and Training, Second Edition, edited by Ken Barat Optical Materials and Applications, edited by Moriaki Wakaki Lightwave Engineering, Yasuo Kokubun Handbook of Optical and Laser Scanning, Second Edition, Gerald F. Marshall and Glenn E. Stutz Computational Methods for Electromagnetic and Optical Systems, Second Edition, John M. Jarem and Partha P. Banerjee Optical Methods of Measurement: Wholefield Techniques, Second Edition, Rajpal S. Sirohi Optoelectronics: Infrared-Visible-Ultraviolet Devices and Applications, Second Edition, edited by Dave Birtalan and William Nunley Photoacoustic Imaging and Spectroscopy, edited by Lihong V. Wang Polarimetric Radar Imaging: From Basics to Applications, Jong-Sen Lee and Eric Pottier Near-Earth Laser Communications, edited by Hamid Hemmati Slow Light: Science and Applications, edited by Jacob B. Khurgin and Rodney S. Tucker Dynamic Laser Speckle and Applications, edited by Hector J. Rabal and Roberto A. Braga Jr. Biochemical Applications of Nonlinear Optical Spectroscopy, edited by Vladislav Yakovlev Optical and Photonic MEMS Devices: Design, Fabrication and Control, edited by Ai-Qun Liu The Nature of Light: What Is a Photon?, edited by Chandrasekhar Roychoudhuri, A. F. Kracklauer, and Katherine Creath Introduction to Nonimaging Optics, Julio Chaves Introduction to Organic Electronic and Optoelectronic Materials and Devices, edited by Sam-Shajing Sun and Larry R. Dalton Fiber Optic Sensors, Second Edition, edited by Shizhuo Yin, Paul B. Ruffin, and Francis T. S. Yu Terahertz Spectroscopy: Principles and Applications, edited by Susan L. Dexheimer Photonic Signal Processing: Techniques and Applications, Le Nguyen Binh Please visit our website www.crcpress.com for a full list of titles Entropy and Information Optics Connecting Information and Time, Second Edition Francis T. S. Yu CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2017 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper International Standard Book Number-13: 978-1-138-55549-5 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and informa- tion, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright .com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Names: Yu, Francis T. S., 1932- author. Title: Entropy and information optics : connecting information and time / Francis T.S. Yu. Description: Second edition. | Boca Raton : Taylor & Francis, CRC Press,2017. | Includes bibliographical references and index. Identifiers: LCCN 2017032627| ISBN 9781138555495 (hardback : alk. paper) | ISBN 9781315149561 (ebook) Subjects: LCSH: Optical communications. | Entropy (Information theory) | Optics. | Optical data processing. Classification: LCC TK5103.59 .Y8 2017 | DDC 621.382/7--dc23 LC record available at https://lccn.loc.gov/2017032627 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Contents From the Series Editor .................................................................................................................................................................ix Preface..........................................................................................................................................................................................xi Author ........................................................................................................................................................................................xiii Chapter 1 Introduction to Information Transmission ..............................................................................................................1 1.1 Information Measure ....................................................................................................................................2 1.2 Entropy Information .....................................................................................................................................4 1.3 Communication Channels ............................................................................................................................7 1.4 Memoryless Discrete Channels ....................................................................................................................8 1.5 Continuous Channels with Additive Noise .................................................................................................10 1.6 Summary and Remarks ..............................................................................................................................15 References .............................................................................................................................................................16 Chapter 2 Diffraction and Signal Analysis ............................................................................................................................17 2.1 Introduction to Diffraction .........................................................................................................................17 2.2 Fresnel–Kirchhoff Theory ..........................................................................................................................18 2.3 Linear Systems and Fourier Analysis .........................................................................................................19 2.4 Finite Bandwidth Analysis .........................................................................................................................21 2.5 Degrees of Freedom of a Signal .................................................................................................................23 2.6 Gabor’s Information Cell ............................................................................................................................25 2.7 Signal Detection .........................................................................................................................................26 2.8 Statistical Signal Detection ........................................................................................................................27 2.9 Signal Recovering .......................................................................................................................................29 2.10 Signal Ambiguity........................................................................................................................................30 2.11 Wigner Signal Representation ....................................................................................................................31 2.12 Fourier Transform Properties of Lenses .....................................................................................................33 References .............................................................................................................................................................34 Chapter 3 Optical Spatial Channel and Encoding Principles ................................................................................................35 3.1 Optical Spatial Communication Channel ...................................................................................................35 3.2 Optical Message in Spatial Coding ............................................................................................................37 3.3 Spatial Channel with Resolution Cells of Different Sizes ..........................................................................39 3.4 Matching a Code with a Spatial Channel ...................................................................................................41 References .............................................................................................................................................................43 Chapter 4 Entropy and Information .......................................................................................................................................45 4.1 Fundamental Laws of Thermodynamics ....................................................................................................45 4.2 Physical Entropy and Information ..............................................................................................................46 4.3 Trading Entropy with Information .............................................................................................................47 4.4 Typical Examples .......................................................................................................................................48 4.5 Remarks ......................................................................................................................................................50 References .............................................................................................................................................................50 Chapter 5 Demon Exorcist and Cost of Entropy ....................................................................................................................53 5.1 Perpetual Motion Machine .........................................................................................................................53 5.2 Maxwell’s Demon .......................................................................................................................................54 5.3 Information and Demon Exorcist ...............................................................................................................54 5.4 Demon Exorcist, a Revisit ..........................................................................................................................57 5.5 Szilard’s Demon..........................................................................................................................................58 v vi Contents 5.6 Diffraction-Limited Demon .......................................................................................................................60 5.7 Minimum Cost of Entropy .........................................................................................................................60 5.8 Gabor’s Perpetuum Mobile of the Second Kind ........................................................................................62 References .............................................................................................................................................................63 Chapter 6 Observation and Information ................................................................................................................................65 6.1 Observation with Radiation ........................................................................................................................65 6.2 Simultaneous Observations ........................................................................................................................68 6.3 Observation and Information .....................................................................................................................69 6.4 Accuracy and Reliability in Observations ..................................................................................................71 6.5 Observation by Interference and by Microscope .......................................................................................74 6.6 Uncertainty and Observation ......................................................................................................................78 6.7 Remarks ......................................................................................................................................................79 References .............................................................................................................................................................79 Chapter 7 Image Restoration and Information ......................................................................................................................81 7.1 Image Restoration .......................................................................................................................................81 7.2 Uncertainty and Image Restoration ............................................................................................................84 7.3 Resolving Power and Information ..............................................................................................................85 7.4 Coherent and Digital Image Enhancement.................................................................................................87 7.5 Information Leakage Through a Passive Channel .....................................................................................88 7.6 Restoration of Blurred Images ....................................................................................................................90 References .............................................................................................................................................................93 Chapter 8 Quantum Effect on Information Transmission .....................................................................................................95 8.1 Problem Formulation and Entropy Consideration ......................................................................................95 8.2 Capacity of a Photon Channel ....................................................................................................................96 8.3 Informational Theoristic Approach ............................................................................................................98 8.4 Narrowband Photon Channel ...................................................................................................................100 8.5 Optimum Signal Power Distribution: A Special Case .............................................................................103 References ...........................................................................................................................................................104 Chapter 9 Coherence Theory of Optics ...............................................................................................................................105 9.1 Aspects of Coherence ...............................................................................................................................105 9.2 Spatial and Temporal Coherence ..............................................................................................................106 9.2.1 Spatial Coherence ........................................................................................................................106 9.2.2 Temporal Coherence ...................................................................................................................107 9.3 Coherent and Incoherent Processing ........................................................................................................108 9.4 Exploitation of Coherence ........................................................................................................................109 9.4.1 Exploitation of Spatial Coherence ...............................................................................................109 9.4.2 Exploitation of Temporal Coherence ............................................................................................110 9.4.3 Complex Amplitude Processing ...................................................................................................110 9.4.4 Incoherent Superposition ..............................................................................................................112 9.5 Remarks .....................................................................................................................................................112 References ............................................................................................................................................................112 Chapter 10 Wavelet Transforms with Optics ..........................................................................................................................113 10.1 Aspects of Wavelet Transform ...................................................................................................................113 10.2 Fourier Domain Processing (FDP) ............................................................................................................113 10.3 Wavelet Transform .....................................................................................................................................116 10.4 Optical Implementations ...........................................................................................................................118 10.5 Simulations ................................................................................................................................................118 Contents vii 10.6 Remarks .....................................................................................................................................................119 References ............................................................................................................................................................119 Chapter 11 Pattern Recognition with Optics .........................................................................................................................121 11.1 Optical Correlators ...................................................................................................................................121 11.2 Optical Disk–Based Correlator ................................................................................................................123 11.3 Photorefractive-Based Correlator .............................................................................................................123 11.4 Optical Neural Networks ..........................................................................................................................124 11.5 Composite Filters ......................................................................................................................................126 11.6 Remarks ....................................................................................................................................................127 References ...........................................................................................................................................................128 Chapter 12 Computing with Optics .......................................................................................................................................129 12.1 Logic-Based Computing ...........................................................................................................................129 12.2 Optical Interconnects and Shuffling .........................................................................................................129 12.3 Matrix–Vector Multiplication ....................................................................................................................131 12.4 Systolic Processor .....................................................................................................................................132 12.5 Matrix–Matrix Processing .......................................................................................................................133 12.6 Expert System and Artificial Intelligence ................................................................................................135 12.7 Remarks ....................................................................................................................................................138 References ...........................................................................................................................................................138 Chapter 13 Communication with Fiber Optics ......................................................................................................................139 13.1 Aspects of Fiber-Optic Communication ...................................................................................................139 13.2 Optical Fiber Structures ............................................................................................................................141 13.3 Fiber-Optic Transmission .........................................................................................................................142 13.4 Types of Optical Fibers ............................................................................................................................145 13.5 Fiber-Optic Communications ...................................................................................................................146 13.6 Remarks ....................................................................................................................................................148 References ...........................................................................................................................................................149 Chapter 14 Synthetic Radar Imaging with Optics .................................................................................................................151 14.1 Synthetic Aperture Radar .........................................................................................................................151 14.2 Radar Data Format Synthesis ...................................................................................................................151 14.3 Radar Imaging with Optics ......................................................................................................................153 14.4 Remarks ....................................................................................................................................................154 References ...........................................................................................................................................................154 Chapter 15 Wideband Signal Analysis with Optics ...............................................................................................................155 15.1 Time Signal to Two-Dimensional Spatial Conversion .............................................................................155 15.2 Spectrum Analysis with Optics ................................................................................................................155 15.3 Processing with Area Modulation ............................................................................................................157 15.4 Broadband Area Modulation Processing ..................................................................................................158 15.5 Remarks ....................................................................................................................................................159 References ...........................................................................................................................................................159 Chapter 16 Information in Myth of Science and Creation......................................................................................................161 16.1 Space and Information ..............................................................................................................................161 16.2 Time and Information ................................................................................................................................161 16.3 Entropy and Information ..........................................................................................................................163 16.4 Substance and Information .......................................................................................................................163 16.5 Uncertainty and Information ....................................................................................................................163 viii Contents 16.6 Certainty and Information ........................................................................................................................164 16.7 Relativity and Information .......................................................................................................................165 16.8 Creation and Information .........................................................................................................................167 16.9 Price Tag and Information ........................................................................................................................167 16.10 Remarks ....................................................................................................................................................168 References ...........................................................................................................................................................168 Chapter 17 Time: The Enigma of Space .................................................................................................................................171 17.1 Time and Energy .......................................................................................................................................171 17.2 Einstein’s Energy Equation .......................................................................................................................171 17.3 Energy Conversion and Reservoir .............................................................................................................171 17.4 Trading Mass and Energy ..........................................................................................................................171 17.5 Physical Substance and Space ..................................................................................................................172 17.6 Absolute and Physical Subspaces .............................................................................................................172 17.7 Time and Physical Space ...........................................................................................................................173 17.8 Electromagnetics and Laws of Physics ......................................................................................................173 17.9 Trading Time for Space .............................................................................................................................174 17.10 Relativistic Time and Temporal Space ......................................................................................................175 17.11 Time and Physical Space ...........................................................................................................................175 17.12 We Are Not Alone .....................................................................................................................................176 17.13 Remarks .....................................................................................................................................................176 References ............................................................................................................................................................176 Chapter 18 Time–Space Quantum Entanglement .................................................................................................................177 18.1 Science and Mathematics .........................................................................................................................177 18.2 Time and Temporal Space ........................................................................................................................177 18.3 Quantum Entanglement .............................................................................................................................178 18.4 Relativistic Quantum Entanglement ..........................................................................................................178 18.5 The Art of Quantum Limited-Subspace ...................................................................................................179 18.6 Remarks .....................................................................................................................................................181 References ............................................................................................................................................................181 Appendix A: Linear Difference Equation with Constant Coefficients ..............................................................................183 Appendix B: Solution of the A Priori Probabilities of Eqs. (5.37) and (5.38) ....................................................................185 Appendix C: Probability Energy Distribution .....................................................................................................................187 Appendix D: Einstein’s Energy Equation .............................................................................................................................189 Appendix E: Gravitation and EM Field Interaction ............................................................................................................191 Index .........................................................................................................................................................................................193 From the Series Editor Fundamentally optical beams and optical systems transmit defect, relative structural content, and correlation quantity and analyze information. The information can be analog or were concepts introduced by E. H. Linfoot in 1964. Formal digital. It can be three-dimensional, two-dimensional, or one- definitions of entropy and information content were to follow, dimensional. It can be in the traditional form of an image or and the field continues to expand, driven by the explosion of information that is coded and/or compressed. The light beam high-speed, high-data-rate, and high-capacity communica- carrying the information can be incoherent, coherent, or even tion systems. partially coherent. This volume discusses the fundamentals and the applica- In the early days of this important field, the concepts of tions of entropy and information optics by means of a sampling communication theory had a major impact on our understand- of topics in this field, including image restoration, wavelet ing and our descriptions of optical systems. The initial impe- transforms, pattern recognition, computing, and fiber-optic tus was to deal with images and image quality. Concepts of communication. impulse responses and transfer functions caused considerable rethinking about the design and evaluation of optical systems. Brian J. Thompson Resolution criteria were only the beginning; fidelity, fidelity ix