PRINCIPLES OF APPLIED MATHEMATICS James P. Keener PRINCIPLES OF APPLIED MATHEMATICS Transformation and Approximation James P. Keener Department of Mathematics University of Utah ADDISO N-W ESLEY PUBLISH ING CO M PANY The Advanced Book Program Redwood City, California • Menlo Park, California Reading, Massachusetts • New York • Amsterdam Don Mills, Ontario • Sydney • Bonn • Madrid • Singapore Tokyo • San Juan • Wokingham, United Kingdom Publisher: Allan M. W ylde Production A dm inistrator: Karen L. Garrison Editorial Coordinator: Pearline Randall Electronic Production Consultant: Mona Zeftel Prom otions Manager: Celina Gonzales Library of Congress Cataloging-in-Publication D ata Keener, James P. Principles of applied m athem atics : transform ation and approxim ation / James P. Keener, p. cm. Includes index. ISBN 0-201-15674-1 1. Transform ations (M athem atics) 2. Asym ptotic expansions. I. Title. QA601.K4 1987 515.7’23-dcl9 87-18630 CIP Copyright © 1988 by Addison-Wesley Publishing Company. This book was typeset in Textures using a Macintosh SE com puter. Camera-ready ou tput from an Apple LaserW riter Plus Printer. BCDEFGHIJ-IIA-89 0-201-15674-1 To my three best friends, Kristine, Samantha, and Justin , who have adjusted extremely well to having a mathem atician as husband and father. PREFACE Applied mathematics should read like good mystery, with an intriguing begin­ ning, a clever but systematic middle, and a satisfying resolution at the end. Often, however, the resolution of one mystery opens up a whole new problem, and the process starts all over. For the applied mathematical scientist, there is the goal to explain or predict the behavior of some physical situation. One begins by constructing a mathematical model which captures the essential fea­ tures of the problem without masking its content with overwhelming detail. Then comes the analysis of the model where every possible tool is tried, and some new tools developed, in order to understand the behavior of the model as thoroughly as possible. Finally, one must interpret and compare these re­ sults with real world facts. Sometimes this comparison is quite satisfactory, but most often one discovers that important features of the problem are not adequately accounted for, and the process begins again. Although every problem has its own distinctive features, through the years it has become apparent that there are a group of tools that are essential to the analysis of problems in many disciplines. This book is about those tools. But more than being just a grab bag of tools and techniques, the purpose of this book is to show that there is a systematic, even esthetic, explanation of how these classical tools work and fit together as a unit. Much of applied mathematical analysis can be summarized by the obser­ vation that we continually attempt to reduce our problems to ones that we already know how to solve. This philosophy is illustrated by an anecdote (apocryphal, I hope) of a mathematician and an engineer who, as chance would have it, took a gourmet cooking class together. As this course started with the basics, the first lesson was on how to boil a pot of water. The instructor presented the students with two pots of water, one on the counter, one on a cold burner, while another burner was already quite hot, but empty. In order to test their cooking aptitudes, the instructor first asked the engineer to demonstrate to the rest of the class how to boil the pot of water that was already sitting on the stove. He naturally moved the pot carefully v vi PREFACE from the cold burner to the hot burner, to the appreciation of his classmates. To be sure the class understood the process, the pot was moved back to its original spot on the cold burner at which time the mathematician was asked to demonstrate how to heat the water in the pot sitting on the counter. He promptly and confidently exchanged the position of the two pots, placing the pot from the counter onto the cold burner and the pot from the burner onto the counter. He then stepped back from the stove, expecting an appreciative response from his mates. Baffled by his actions, the instructor asked him to explain what he had done, and he replied naturally, that he had simply reduced his problem to one that everyone already knew how to solve. We shall also make it our goal to reduce problems to those which we already know how to solve. We can illustrate this underlying idea with simple examples. We know that to solve the algebraic equation 3x = 2, we multiply both sides of the equation with the inverse of the “operator” 3, namely 1/3, to obtain x = 2/3. The same is true if we wish to solve the matrix equation Ax — f where Namely, if we know A x, the inverse of the matrix operator A, we multiply both sides of the equation by A~l to obtain x = A~l f . For this problem, If we make it our goal to invert many kinds of linear operators, including matrix, integral, and differential operators, we will certainly be able to solve many types of problems. However, there is an approach to calculating the inverse operator that also gives us geometrical insight. We try to transform the original problem into a simpler problem which we already know how to solve. For example, if we rewrite the equation Ax — f as T ~ lAT (T~ lx) — T~l f and choose we find that T ' 1AT = With the change of variables y = T~ lx, g = T-1/ , the new problem looks like two of the easy algebraic equations we already know how to solve, namely 4t/i = 3/2, — 2?/2 = 1/2. This process of separating the coupled equations into uncoupled equations works only if T is carefully chosen, and exactly how vii this is done is still a mystery. Suffice it to say, the original problem has been transformed, by a carefully chosen change of coordinate system, into a problem we already know how to solve. This process of changing coordinate systems is very useful in many other problems. For example, suppose we wish to solve the boundary value problem u" — 2u — f ( x ) with u(0) = u(tt) = 0. As we do not yet know how to write down an inverse operator, we look for an alternative approach. The single most important step is deciding how to represent the solution u(x). For example, we might try to represent u and / as polynomials or infinite power series in x, but we quickly learn that this guess does not simplify the solution process much. Instead, the “natural” choice is to represent u and / as trigonometric series, oo oo u(x) = uk sm kx , f (x) = sm ^X* k=l k=1 Using this representation (i.e., coordinate system) we find that the original differential equation reduces to the infinite number of separated algebraic equations {k2 + 2) Uk = —fk- Since these equations are separated (the kth equation depends only on the unknown Uk), we can solve them just as before, even though there are an infinite number of equations. We have managed to simplify this problem by transforming into the correctly chosen coordinate system. For many of the problems we encounter in the sciences, there is a natural way to represent the solution that transforms the problem into a substantially easier one. All of the well-known special functions, including Legendre poly­ nomials, Bessel functions, Fourier series, Fourier integrals, etc., have as their common motivation that they are natural for certain problems, and perfectly ridiculous in others. It is important to know when to choose one transform over another. Not all problems can be solved exactly, and it is a mistake to always look for exact solutions. The second basic technique of applied mathematics is to reduce hard problems to easier problems by ignoring small terms. For example, to find the roots of the polynomial x2 + x + .0001 = 0, we notice that the equation is very close to the equation x2 -f x = 0 which has roots x = — 1, and x = 0, and we suspect that the roots of the original polynomial are not too much different from these. Finding how changes in parameters affect the solution is the goal of perturbation theory and asymptotic analysis, and in this example we have a regular perturbation problem, since the solution is a regular (i.e., analytic) function of the “parameter” .0001 . Not all reductions lead to such obvious conclusions. For example, the polynomial .0001x2 + x + 1 = 0 is close to the polynomial £ + 1 = 0, but the first has two roots while the second has only one. Where did the second root go? We know, of course, that there is a very large root that “goes to infinity” as “.0001 goes to zero,” and this example shows that our naive idea viii PREFACE of setting all small parameters to zero must be done with care. As we see, not all problems with small parameters are regular, but some have a singular behavior. We need to know how to distinguish between regular and singular approximations, and what to do in each case. This book is written for beginning graduate students in applied mathe­ matics, science, and engineering, and is appropriate as a one-year course in applied mathematical techniques (although I have never been able to cover all of this material in one year). We assume that the students have studied at an introductory undergraduate level material on linear algebra, ordinary and partial differential equations, and complex variables. The emphasis of the book is a working, systematic understanding of classical techniques in a mod­ ern context. Along the way, students are exposed to models from a variety of disciplines. It is hoped that this course will prepare students for further study of modern techniques and in-depth modeling in their own specific discipline. One book cannot do justice to all of applied mathematics, and in an ef­ fort to keep the amount of material at a manageable size, many important topics were not included. In fact, each of the twelve chapters could easily be expanded into an entire book. The topics included here have been selected, not only for their scientific importance, but also because they allow a logical flow to the development of the ideas and techniques of transform theory and asymptotic analysis. The theme of transform theory is introduced for matrices in Chapter one, and revisited for integral equations in Chapter three, for ordi­ nary differential equations in Chapters four and seven, for partial differential equations in Chapter eight, and for certain nonlinear evolution equations in Chapter nine. Once we know how to solve a wide variety of problems via transform theory, it becomes appropriate to see what harder problems we can reduce to those we know how to solve. Thus, in Chapters ten, eleven, and twelve we give a survey of the three basic areas of asymptotic analysis, namely asymptotic analysis of integrals, regular perturbation theory and singular perturbation theory. Here is a summary of the text, chapter by chapter. In Chapter one, we re­ view the basics of spectral theory for matrices with the goal of understanding not just the mechanics of how to solve matrix equations, but more impor­ tantly, the geometry of the solution process, and the crucial role played by eigenvalues and eigenvectors in finding useful changes of coordinate systems. This usefulness extends to pseudo-inverse operators as well as operators in Hilbert space, and so is a particularly important piece of background infor­ mation. In Chapter two, we extend many of the notions of finite dimensional vector spaces to function spaces. The main goal is to show how to represent objects in a function space. Thus, Hilbert spaces and representation of functions in a Hilbert space are studied. In this context we meet classical sets of functions such as Fourier series and Legendre polynomials, as well as less well-known ix sets such as the Walsh functions, Sine functions, and finite element bases. In Chapter three, we explore the strong analogy between integral equa­ tions and matrix equations, and examine again the consequences of spectral theory. This chapter is more abstract than others as it is an introduction to functional analysis and compact operator theory, given under the guise of Fredholm integral equations. The added generality is important as a frame- work for things to come. In Chapter four, we develop the tools necessary to use r-ectral decompo­ sitions to solve differential equations. In particular, distributions and Green’s functions are used as the means by which the theory of compact operators can be applied to differential operators. With these tools in place, the com­ pleteness of eigenfunctions of Sturm-Liouville operators follows directly. Chapter five is devoted to showing how many classical differential equa­ tions can be derived from a variational principle, and how the eigenvalues of a differential operator vary as the operator is changed. Chapter six is a pivotal chapter, since all chapters following it require a substantial understanding of analytic function theory, and the chapters pre­ ceding it require no such knowledge. In particular, knowing how to integrate and differentiate analytic functions at a reasonably sophisticated level is in­ dispensable to the remaining text. Section 6.3 (applications to Fluid Flow) is included because it is a classically important and very lovely subject, but it plays no role in the remaining development of the book, and could be skipped if time constraints demand. Chapter seven continues the development of transform theory and we show that eigenvalues and eigenfunctions are not always sufficient to build a transform, and that operators having continuous spectrum require a gen­ eralized construction. It is in this context that Fourier, Mellin, Hankel, and Z transforms, as well as scattering theory for the Schrodinger operator are studied. In Chapter eight we show how to solve linear partial differential and differ­ ence equations, with special emphasis on (as you guessed) transform theory. In this chapter we are able to make specific use of all the techniques introduced so far and solve some problems with interesting applications. Although much of transform theory is rather old, it is by no means dead. In Chapter nine we show how transform theory has recently been used to solve certain nonlinear evolution equations. We illustrate the inverse scattering transform on the Korteweg-deVries equation and the Toda lattice. In Chapter ten we show how asymptotic methods can be used to approx­ imate the horrendous integral expressions that so often result from transform techniques. In Chapter eleven we show how perturbation theory and especially the study of nonlinear eigenvalue problems uses knowledge of the spectrum of a linear operator in a fundamental way. The nonlinear problems in this chapter all have the feature that their solutions are close to the solutions of a nearby linear problem. X PREFACE Singular perturbation problems fail to have this property, but have so­ lutions that differ markedly from the naive simplified problem. In Chap­ ter twelve, we give a survey of the three basic singular perturbation problems (slowly varying oscillations, initial value problems with vastly different time scales, and boundary value problems with boundary layers). This book has the lofty goal of reaching students of mathematics, sci­ ence, and engineering. To keep the attention of mathematicians, one must be systematic, and include theorems and proofs, but not too many explicit cal­ culations. To interest an engineer or scientist, one must give specific examples of how to solve meaningful problems but not too many proofs or too much abstraction. In other words, there is always someone who is unhappy. In an effort to minimize the total displeasure and not scare off too many members of either camp, there are some proofs and some computations. Early in the text, there are disproportionately more proofs than computations be­ cause as the foundations are being laid, the proofs often give important in­ sights. Later, after the bases are established, they are invoked in the prob­ lem solving process, but proofs are deemphasized. (For example, there are no proofs in Chapters eleven and twelve.) Experience has shown that this approach is, if not optimal, at least palatable to both sides. This is intended to be an applied mathematics book and yet there is very little mention of numerical methods. How can this be? The answer is simple and, I hope, satisfactory. This book is primarily about the principles that one uses to solve problems and since these principles often have consequence in numerical algorithms, mention of numerical routines is made when appro­ priate. However, FORTRAN codes or other specific implementations can be found in many other good resources and so (except for a code for fast Walsh transforms) are omitted. On the other hand, many of the calculations in this text should be done routinely using symbolic manipulation languages such as REDUCE or MACSYMA. Since these languages are not generally familiar to many, included here are a number of short programs written in REDUCE in order to encourage readers to learn how to use these remarkable tools. (Some of the problems at the end of several chapters are intended to be so tedious that they force the reader to learn one of these languages.) This text would not have been possible were it not for the hard work and encouragement of many other people. There were numerous students who struggled through this material without the benefit of written notes while the course was evolving. More recently, Prof. Calvin Wilcox, Prof. Frank Hoppen- steadt, Gary deYoung, Fred Phelps, and Paul Arner have been very influential in the shaping of this presentation. Finally, the patience of Annetta Cochran and Shannon Ferguson while typing from my illegible scrawling was exem­ plary. In spite of all the best efforts of everyone involved, I am sure there are still typographical errors in the text. It is disturbing that I can read and reread a section of text and still not catch all the erors. My hope is that those that remain are both few and obvius and will not lead to undue confusion.