Naval Research tata Vol XLV CVI CW. S A random walk is a basic concept in probability theory. Brownian motion can be viewed as a random walk where the average jnuomnpe osifz et heism wedlolm idneafitneesd.. IFnr atchtisa l firgaurned otmh e wcaolnknse cdtoe dn otl ihnea vsee gamne natvser as ghoew j utmhpe tsriazej,e ctbourty joufm pa sf raocft aalll rlaenngdtohm scwaallke.s Tenhtee r,d otasn Sbltfoohee rlnlayego stwnsih unesgmg teehmtrrehe, ii nsct traBat l.jy sepWhcseetoi somuotrulf,yld as staphihanaoocdvnwe e- J ot.tf aih Kmetlve uae rfflbrtoraeuccarlit entaniydlnt o psPmpreh itoy ppswoeoiafr cl tkaftil luoo rhnwnaRa iselwn vigtbi theeop ewoiin at ns RL teleustey stnneeogrodfts l h,td thsoerV aomrilno.sau denm5edd8b l,oe t mort 1 ut1rhwo0beaf0lu -kl11.e1/0 n130cW 0,ep3h0.o 0 we0e(An rp c.b csa yo ar Tvd heiF.i lnsoHgT ca hityet oy ett oto hfr eiiyKoenr loy d wl amPisshow ygasadoisscesr avoolsce vuilR caoecstpveceseiadslde ifw nuw gbli,Aytl 4hyM3 a., ta htper8p a0ljp6eia-c _e t -dh 8w1hi0c h( 19r9e1f)l.e ctsT het hef raecnthala nrcaendd ommi xwianlgk daupep rtooa tcuhr bsulheonwceed Wtihteh fOlaNtRte nsedu pp(roerlta tiDvre. tHoa yloatmi niasr e xfplloo+rvi)n gv eloncuimteyr ipcroaflillye atchre ossism utlhaet ipoinp eo f Sthurlbeuslienngte rf luows ipnags tT rau ec yBlaisnidce r fours itnhge Mthaec fractal space-time random walk approach. The above figure was calculated by M. NAVAL RESEARCH REVIE aRvae eWe "E uIs ° C4 aVooXeL .V a Research Articles : . 2 4. CW Applications of Cha6s x: niinicating with Chaos and Fractals A “Laomas L. Carroll Michael F. Shlesinger ~ Louis M. Pecora 12 23 Controlling Chaos Nonlinear Resonance: Mark L. Spano > Noise-Assisted Information William L. Ditto Pag, / Processing in Physical and Neurophysiological Systems Adi Bulsara John K. Douglass Frank Moss 38 49 Dialogue on Noisy Chaos Attractors of Iterated Robert Cawley Systems and Their Application Guan-Hsong Hsu To Image Compression E. W. Jacobs R. D. Boss CHIEF OF NAVAL RESEARCH 2 2 RADM Marc Pelaez, USN DEPUTY CHIEF OF NAVALRESEARCH Profiles in Science TECHNICAL DIRECTOR Dr. Fred Saalfeld About the Cover CHIEF WRITER/EDITOR The trajectories (coded by color) of several hundred initial conditions are followed for a two-dimensional Hamiltonian William J. Lescure system. The underlying equations are called a Zaslavsky map and were chosen here to exhibit 6-fold symmetry. Some SCIENTIFIC EDITORS trajectories stay localized with periodic orbits while others exhibit a complex global dynamics traveling through the Dr. Robert J. Nowak 6-fold symmetric stochastic web. The dynamics are related to the behavior of charged particles in electromagnetic Dr. J. Dale Bultman fields. Computer calculations by M. Shlesinger, using True Basic for the Mac. Naval Research Reviews publishes articles about research conducted by the laboratories and contractors of the Office MANAGING EDITOR of Naval Research and describes important naval experimental activities. Manuscripts submitted for publication, Norma Gerbozy correspondence conceming prospective articles, and changes of address, should be directed to Code OPARI, Office of Naval Research, Arlington, VA 22217-5000. Requests for subscriptions should be directed to the Superintendent of ART DIRECTION Documents, U.S. Government Printing Office, Washington, DC 20403. Naval Research Reviews is published from Typography and Design appropriated funds by authority of the Office of Naval Research in accordance with Navy Publications and Printing Desktop Printer, Arnold, MD : Regulations. NAVSO P-35. Threei1993 1 Applications of Chaos and Fractals Dr. Michael F. Shlesinger, Director continuous, but non-differentiable function; Levy’s theory of ONR Physics Division the algebra of random variables with infinite moments; and Guest Editor Richardson’s discovery that the length of a country’s border e-mail: [email protected] depended, in a scaling manner, with the resolution of measure- ment. It is only in recent times that the deep power of fractal geometry is appreciated and its intricacies can be revealed Introduction through the use of computers. Fractals have led to a paradigm shift which has changed our view of nature. It is now realized There have been two great trends in 20th century physics; that many shapes in nature from turbulent dissipation fields to investigating the very small (atomic, nuclear, particle physics) mountain profiles are better served by fractal geometry than and investigating the very large (astronomy and cosmology). by standard Euclidean geometry. Fractal shapes also abound It is only in the last decade that a third trend has been recog- in the phase space of nonlinear dynamical trajectories. The nized; investigating the very complex. Complexity is all historical use of the term strange attractor obscures the domi- around us, but it is only recently that discoveries and methods nant feature that the trajectories are fractal, and highlights the have arisen which gives us hope to treat it as a science. The typical reaction of a field to encountering its first fractal. two words which are the most recognizable from this new Before the chaos revolution the main conclusion one drew discipline are chaos and fractals. about nonlinear systems is that they were difficult to analyze. Chaos arises as a common behavior in nonlinear dynam- The computer revolution has eased this perception. Usually, ical systems. Nonlinear dynamics arose with Newton’s equa- one just separated stable from unstable regions and chose to tions of motion and the scale-invariant nonlinear 1/r stay well within the stable bounds. Another early lesson was gravitational potential. While the earth-moon system was that the study of one nonlinear equation would not shed light solvable (integrable) the 3-body problem was not. We now on the properties of another. A major development in nonlinear know that chaos can appear in the 3-body problem. Newton dynamics is discovering how wrong the above statement is. worried about the 3-body problem because he wondered how In the early 1970’s nonlinear dynamics tumed to seeking orbital earth-moon stability could be maintained in the face of stable particle-like solutions (called solitons) to integrable additional bodies, such as comets, since there is no friction to nonlinear equations with an infinite number of conservation damp out perturbations. This question, in fact, was not an- laws. Solitons are the antithesis of chaos. The mathematics swered until the 1950’s (both stability and instability can behind solitons was quite complex, and kept the field only occur) with the work of Kolmogorov. He showed that in phase open to experts. In 1963, Lorenz published an example of a space that both islands of stability and a chaotic sea coexist. nonlinear system of three ordinary differential equations Which of these regions you explore depends on the energy of whose solutions was extremely sensitive to initial conditions. the system and your initial conditions. A strong Russian His equations were related to a meteorological problem of the school, steeped in tradition, advanced our knowledge of such convection of heated air. The trajectory of his solutions traced Hamiltonian systems. The Zaslavsky map on the cover is one out a highly intricate figure with what would now be called a such example. fractal structure. Such structures were named strange attrac- Fractals are beautiful hierarchical geometric figures tors by Ruelle and Takens in 1971. They were studying the which are by now familiar to many around the world. The field mathematics of strange attractors which arise after a two was mainly developed by Mandelbrot over the past 40 years. frequency (quasi periodic) response becomes unstable with a He was able to use the scaling properties of fractals to define change in a parameter which adds a third frequency to the mix. a continuous dimension, called the fractal dimension. Fractals All of these advances still left complexity as a small technical can be self-similar with a single dimension; multifractal with area of science. a distribution of dimensions; or self-affine with independent The field of nonlinear dynamics exploded in the late scaling in different directions. Perhaps, the most famous frac- 1970’s with the experimental verification of Feigenbaum’s tal is the Mandelbrot Set which now even appears as a popular period doubling universality in Libchaber and Maurer’s liquid screen saver on computers. We honor Professor Mandelbrot helium convection experiments. Not only did arcane theoret- with a brief biographical note and a sketch of his likeness, in ical notions about mappings of the unit interval into itself this issue. reveal themselves as fundamental in a real 3D fluid with untold The set of pre-Mandelbrot fractals is nonzero, but while degrees of freedom, but also the renormalization group found the early examples of fractals were interesting, the results were an extension into the realm of dynamical systems. The trickle isolated, and held up as pathologies and not as the seeds of a of research in dynamical systems soon became a flood cover- new field of scientific inquiry. Examples included Weierstrass’ ing diverse fields with a common language and phenomena y Naval Research Reviews based on the twin notions of instability and fractal dimension. The field of nonlinear dynamics is now represented by over half a dozen specialized journals, a large number of texts, a best selling historical account by James Gleick called “Chaos”, and it is even well covered in high school science fairs. The basis of the success of nonlinear dynamics and fractals lies in being able to understand very complex behav- iors from very simple laws. This fortunate turn of events allows a student to learn a fair amount rapidly without great effort. This simplicity can be misleading with the field being too easy for beginners and too hard for experts. Research is now proceeding from temporal chaos to spatio-temporal pat- terns and chaos, self-organization, and adaptive systems. The focus on applications is the theme of this issue of the Naval Research Reviews. Each article is based on research at a Navy laboratory. This is not by accident, but reflects the lead the Navy has taken in turning the state-of-the-art in chaos and fractals into applications. This is not just the judgment of ONR, but was recognized, as well, by Science in its May 10, 1991 issue where they stated that for applications of nonlinear dynamics that “its no coincidence that most of the pioneering work is being done in Navy laboratories. Since 1983 the Office of Naval Research has been the only government agency with a funding program specifically for chaos studies.” Our selection looks at synchronization of chaotic circuits with applications from communication to robotics; controlling chaos with ap- plications from pacemakers to actuators; exploring the inter- action of nonlinear dynamics with noise for novel detectors based on noise-induced resonance processes; employing non- linear dynamics for signal processing in phase space; and novel techniques for employing fractals for image compres- sion. This is just a sampling of the research underway in Navy laboratories. I leave it now to the authors of the articles to give you a glimpse into the frontiers they are inventing and explor- ing in the exciting world of chaos and fractals. 3 Three/1993 Communicating with Chaos Thomas L. Carroll and Louis M. Pecora Naval Research Laboratory The study of chaos, a complex form of motion, is rela- Communications depend on both deterministic and sto- tively new. It is only now that possible uses for chaos are chastic mechanics. Deterministic mechanical laws govern the beginning to emerge. We believe that many new technologies behavior of such common devices as oscillators or filters. useful to the Navy will eventually come from this field. One Stochastic mechanics is used in the study of noise and in possible application that we have been studying is the use of random number generators for encryption or spread spectrum chaos as a broad band signal for communications. communications. The type of motion known as chaos has some Until recently, the study of classical mechanics proceeded of the properties of both of these types of motion. Chaos is a in two different directions. On the one hand were deterministic highly irregular , nonperiodic form of deterministic motion mechanical systems, where one could write down all the laws which is extremely sensitive to initial conditions. In principle, of motion. Deterministic systems were believed to proceed in one could calculate the future motion of a chaotic system, but a small error in specifying the initial conditions will grow an orderly fashion, like clockwork; it was from these studies exponentially with time. Since all measurement involves some that the nineteenth century concept of God as a clock maker error, it is practically impossible to predict the future motion arose. Small perturbations in the motion were possible, but it of a chaotic system. The unique properties of chaos have not was believed that they would be damped out over time and yet been exploited for any useful systems; most applied re- have little effect. search has concentrated on suppressing chaos. We have been On the other hand, statistical mechanics developed as the studying how to apply chaos to problems of interest to the study of stochastic, or random, systems. No laws of motion Navy, such as communications. After a brief introduction to existed, so these systems were not predictable. If one studied chaos, we will describe one possible application of chaos. a large number of identical stochastic systems, one could find that their motion did fit certain statistical distributions. The study of statistical mechanics therefore involved the study of Introduction to Chaos long time averages of the motion of a system, of the average motion of many identical systems. The description of chaos came about as a result of ad- As early as the time of Newton, it was suspected that not vances in the field of nonlinear dynamics'. A dynamical sys- all motion fit into these two neat classes. Newton had specu- tem is a system that changes over time in a way that may in lated that the deterministic motion of a planet in orbit might principle be described by some set of rules, such as differential become highly irregular if it was perturbed by a third body, equations or recursion relations. A pendulum is an example of such as a comet. A foundation for the study of complex a dynamical system, as is an electrical oscillator. If one knows nonlinear dynamical systems was not laid until the late nine- the rules that govern a dynamical system, one may predict the teenth century by the mathematician Poincare. This work was future behavior of the system knowing only its starting point. not built on until the 1960’s, when a more serious study of For this reason, the type of dynamical systems that we describe nonlinear dynamics and chaos began. here are known as deterministic. 4 Naval Research Reviews The behavior of a dynamical system is typically described Figure 1. by plotting its trajectory in a phase space, where each direction corresponds to one dynamical variable. The systems we are Block diagram of synchronizing chaotic systems. The re- sponse system is a duplicate of part of the drive system. The concerned with are dissipative, so their motion in phase space response system is driven by the signal or signals that come will eventually settle down to a finite region known as an from the missing part of the system. attractor. A system with periodic motion will follow a closed path in phase space. A system whose motion possesses incom- mensurate periods will move on the surface of a torus. Complex motion is possible if one or more points in the phase space are unstable, so that motion in regions of phase space near these points diverges. Since all real systems are ms finite, this motion can not diverge forever. It may be that the system variables reach some maximum possible value, such as a power supply voltage, and stay there. In other systems, some mechanism for folding the motion back in towards the unstable region may exist. In this type of attractor, trajectories repeatedly are stretched apart near the instability and folded back in. This repeated stretching and folding in phase space produces the complex motion known as chaos. The stretching drive response and folding also cause the motion to depend sensitively on the initial conditions, making prediction a practical impossibility. The rate of this stretching is described by the Lyapunov originally saw from the full chaotic system’. Figure 1 is a exponents. One may measure what happens to small perturba- tions to the trajectory in phase space. The average change ina schematic of this arrangement. We call the full chaotic system perturbation is measured over the entire attractor. The natural the drive system, and we call the driven subsystem the re- logs of the components of the average change vector are the sponse system. The output of the response system depended Lyapunov exponents. A positive exponent means that pertur- on its Lyapunov exponents. If all the Lyapunov exponents of bations are increasing along some direction, a negative expo- the driven response system were less than zero, then the nent means that perturbations are decreasing, while a zero response system will follow the drive signal, so that a partic- exponent means that perturbations are staying the same size. ular signal in the response system is synchronized with the Achaotic attractor has at least one Lyapunov exponent greater corresponding chaotic signal in the drive system. Because the than zero, while the largest exponent for a periodic attractor is response system is stable, this method is not sensitive to small zero. amounts of noise or mismatch in the drive and response systems. The range of initial conditions over which this syn- chronization will occur varies from system to system, and Chaotic Synchronization depends mainly on the presence of other basins of attraction in the system. Since chaos looks like noise, most applied work on chaos We may demonstrate this synchronization with a numer- has been directed toward eliminating it. We saw these noise- ical experiment. We use the Lorenz equations, which are well like properties as useful. One might think of a chaotic system known in the field of nonlinear dynamics: as a noise source that, because of its deterministic nature, may #6 (y-x) be easily characterized. It occurred to us that if we could produce a chaotic signal at or@ location and reproduce an WY= xn +1x-y identical chaotic signal at a reiote location, then we could encode information onto the original chaotic signal and de- a= code it using the reproduced signal. Because of their great sensitivity to initial conditions, isolated chaotic systems not only will not synchronize with each other, their outputs will When o = 10.0, r = 60.0 and b = 8/3, these equations not even be correlated with each other. It is necessary to send produce chaos. We used a numerical integration routine on a information from one chaotic system to the other in order to workstation to integrate these equations. We chose a subsys- synchronize them. tem consisting of the x and z equations and drove them with Our approach was to view a chaotic system as a group of the y variable from the full chaotic system. Figure 2 shows interconnected subsystems. We could then reproduce one of how the z variable in the response system converged to the z the subsystems and drive it with whatever chaotic signals it variable in the drive system within a few cycles of the drive. Three/1993. 5 Cascading Synchronized information using the chaotic signal as a carrier. We wanted to show that this chaotic synchronization Chaotic Systems worked in real systems, so we built the simple chaotic circuit shown in fig. 4°. This circuit becomes chaotic through a period We may actually choose other response systems for the doubling cascade as shown in fig 5 (a)-(d), which shows a Lorenz equations above. The y and z equations together also series of attractors seen as resistor R12 is decreased. Figure 6 form a stable subsystem that may be used as a response system shows the power spectrum of the voltage measured at the x; when driven by the x variable. Since there are two possible point in the circuit. This power spectrum is broad band with a response systems, we may also cascade these response systems few prominent frequency peaks. More complex chaotic sys- as shown in Fig. 3. One may think of the cascaded response tems can have power spectra that resemble colored noise. systems as a black box where a chaotic signal from the drive Designing even a simple chaotic circuit such as this was signal goes in and a chaotic signal comes out. If the response a challenge. There are no general laws that determine when a systems are synchronized to the drive system, the output given system will be chaotic. We could merely use our intu- chaotic signal matches the input signal. If we change a param- ition and experiment to find chaotic circuits. We also had to eter in the drive system, then the output chaotic signal does not find a chaotic circuit that could be divided into at least two match the input chaotic signal. In this way we may send response subsystems. There is no general method for Figure 2. Time series from the computer simulation of the Lorenz equations showing how the response system converges to the drive system. 100 + 4444 4 4+ +4 + 4+ tft ttt tt tt tt tt 90 j drive 5 [ response 1 E ] 80 1 r -+>— L 4 r - [ h.4 70 - : e(t) [ 60 + 7 rm - [ 1 50 + - [ + 40 + 4 [ b 41 [ + 30 + c 4 | | } } ; oo -+-4 LAE SELES A A A A a a 0 0.5 1 1.5 2 2.5 3 6 Naval Research Reviews designing a circuit so that is has stable subsystems. We were _ Figure 3. able to come up with a few guidelines that work in simple . cases, but again much intuition was necessary. Block diagram of cascading the synchronized chaotic Lorenz The hardest problem to solve was matching the response CEMETONE. subsystems to the full circuit. Matching the circuits required finding nonlinear elements whose behavior matched to within 1 or 2 % over a wide range. Typical semiconductor devices by x x ' , themselves never come close to these figures. Some manufac- 4 ‘ turers produce chips that perform analog multiplication, but y ’ y r these still were not reproducible enough. We finally solved the problem of producing reproducible nonlinearities by using an z z' z" arrangement from analog computer technology known as a diode function generator. In this type of circuit, diodes are used to switch different linear characteristics on or off to produce a ES GQuesanenl piecewise linear approximation to any function. The boxes in y F fig. 4 labeled g1 and g2 contain diode function generators. This circuit could be divided into two subsystems, one containing the x4 variable and the other containing the x:, x2, Applying Chaotic x3 variables. Figure 7 shows schematically how the circuits ® ® were cascaded, with the full system driving the x4 subsystem, Synchronization which then drove the x:, x2 , x3 subsystem. With this arrange- We have tried several methods for sending information ment, the x; signal used as an input and the x; output matchto —_— with our chaotic carrier signal. The simplest method involves within about 2%. using a single non-cascaded response system. In this arrange- Figure 4. Schematic of a circuit used to study the cascading of synchronized chaotic systems. Voltages were sampled at the points labeled X1, X2, X3, X4. R3 R7 oo a ‘ R53 aN nei es Bes wv wv 7 4 + R6 T R173 3 CF S y D x1 R18 ri9/2 778 G a0 B :R 15 )a R12 R16 4 WAY a 2(x att Z x4 92(x) R13} R14% $ R20 Three/1993 7 EL ————————— Figure 5. (a)-(d) show a series of attractors seen in the circuit as resistor R12 is decreased. We 8 Naval Research Reviews