A Brief Course in Linear Algebra Leonard Evens Department of Mathematics Northwestern University Evanston, Illinois 1997 (cid:176)c Leonard Evens 1994, 1997 CONTENTS Chapter I. Linear Algebra, Basic Notions 1 1.1 Introduction 1 1.2 Matrix Algebra 4 1.3 Formal Rules 12 1.4 Linear Systems of Algebraic Equations 15 1.5 Singularity, Pivots, and Invertible Matrices 24 1.6 Gauss-Jordan Reduction in the General Case 36 1.7 Homogeneous Systems and Vector Subspaces 46 1.8 Linear Independence, Bases, and Dimension 51 1.9 Calculations in Rn 62 1.10 Review Problems 67 Chapter II. Determinants and Eigenvalues 71 2.1 Introduction 71 2.2 Definition of the Determinant 74 2.3 Some Important Properties of Determinants 82 2.4 Eigenvalues and Eigenvectors 89 2.5 Diagonalization 100 2.6 The Exponential of a Matrix 105 2.7 Review 108 Chapter III. Applications 111 3.1 Real Symmetric Matrices 111 3.2 Repeated Eigenvalues, The Gram–Schmidt Process 113 3.3 Change of Coordinates 118 3.4 Classification of Conics and Quadrics 125 3.5 Conics and the Method of Lagrange Multipliers 133 3.6 Normal Modes 139 3.7 Review 147 Chapter IV. Index 149 v vi CONTENTS PREFACE This text has been specially created for the course Mathematics B17 at North- western University. The subject matter is introductory linear algebra, which is covered in about six weeks in that course. Since time is limited, emphasis is on important basic concepts. Linearalgebraissomewhatmoretheoreticalthansomeofthesubjectsyoustud- ied previously in your calculus courses. What you have to learn is a collection of basic concepts and algorithms. Some of these concepts are a bit subtle, and in- steadofmemorizingformulasyouneedtolearnmoderatelycomplexproceduresfor solving problems. In developing the subject matter, we have tried to keep things concrete by concentrating on illustrative examples. Such examples exhibit the im- portant features of the theory. To describe a concept in complete generality will often require an extensive discussion and listing of many special cases and caveats. However, if you have a good understanding of the basic examples, you will usually beabletofigureoutwhattodoifyouencountersomethingsimilarbutnotexactly the same as the example. You won’t find as many exercises as you are used to in a calculus text. Most of theexercisestakesomewhatmoretimethanisusual,sotrytogleanasmuchasyou can from each rather than relying on repetition to drive a point home. There are fairly complete answers at the end of the book. However, don’t just try to get the rightanswers. Itismoreimportanttounderstandthemethodsandconcepts. Also, don’t concentrate so much on how to solve particular problems that you lose sight oftheideastheseproblemsaremeanttoillustrate. Therearealsomore‘theoretical’ questionsthanyoumaybeusedto. Suchproblemsareintendedtogetyoutocome to grips with important concepts in cases where just doing some more examples might not suffice. You need not write out formal proofs as long as you can give convincingexplanations. Theemphasisshouldbeonunderstandingratherthanon mathematical rigor. Unfortunately, there isn’t time in the syllabus to develop many of the beautiful andimportantapplicationsoflinearalgebra. Afewsuchapplicationsarementioned in the exercises, and two important applications are included at the end. However, linearalgebraisoneofthemostessentialmathematicaltoolsinscience,engineering, statistics,economics,etc.,soweaskyoutobearwithusifthegoinggetsabittough. Forcompleteness,wehaveincludedsomeproofsofcrucialtheorems,buttheyare not supposed to be a fundamental part of the course. Given that time is limited, it is not unreasonable to postpone the proofs for a more advanced course in linear algebra. Noonehaseverwrittenaperfectbook. Apublisheroncetoldmethatpeoplestill find typographical errors in the oft reprinted works of Charles Dickens. If you find iii iv PREFACE something that doesn’t seemto make any sense, in either the text or the problems, please mention it to your instructor. More important, if you find some discussion particularly murky, please let me or your instructor know. The exposition will be revised with such contributions in mind, and you may help generations of calculus students yet to come. IwouldliketothankProfessorDanielS.Kahnwhohashelpedenormouslywith the preparation of this text but who doesn’t want to be held responsible, as an author, for my misdeeds. I would also like to thank my teaching assistants for valuable comments. In particular, I incorporated a suggestion by Jason Douma which I hope clarifies the concept of eigenvector. This text was typeset using AMS-TEX. Leonard Evens, December, 1994 CHAPTER I LINEAR ALGEBRA, BASIC NOTIONS 1. Introduction In your previous calculus courses, you studied differentiation and integration for functionsofmorethanonevariable. Usually,youcouldn’tdomuchifthenumberof variables was greater than two or three. However, in many important applications, the number of relevant variables can be quite large. In such situations, even very basic algebra can get quite complicated. Linear algebra is a tool invented in the nineteenth century and further extended in the twentieth century to enable people to handle such algebra in a systematic and understandable manner. Westartoffwithacoupleofsimpleexampleswhereitisclearthatwemayhave to deal with a lot of variables. Example 1. Professor Marie Curie has ten students in a chemistry class and gives five exams which are weighted differently in order to obtain a total score for the course. The data as presented in her grade book is as follows. student/exam 1 2 3 4 5 1 78 70 74 82 74 2 81 75 72 85 80 3 92 90 94 88 94 4 53 72 65 72 59 5 81 79 79 82 78 6 21 92 90 88 95 7 83 84 76 79 84 8 62 65 67 73 65 9 70 72 76 82 73 10 69 75 70 78 79 The numbers across the top label the exams and the numbers in the left hand column number the students. There are a variety of statistics the teacher might want to calculate from this data. First, she might want to know the average score for each test. For a given test, label the scores x1,x2,x3,x4,x5,x6,x7,x8,x9,x10 so that xi is the score for the ith student. Then the average score is x1+x2+x2+x4+x5+x6+x7+x8+x9+x10 = 1 (cid:88)10 xi. 10 10 i=1 For example, for the second test the average is 1 (70+75+90+72+79+92+84+65+72+75)=77.4. 10 1 2 I. LINEAR ALGEBRA, BASIC NOTIONS Suppose she decides to weight the five scores as follows: the first, third, and fifth scores are weighted equally at 20 percent or 0.2, the second score is weighted 10 percentor0.1,andthefourthscoreisweighted30percentor0.3. Thenifthescores foratypicalstudentaredenotedy1,y2,y3,y4,y5,thetotalweightedscorewouldbe 0.2y1+0.1y2+0.2y3+0.3y4+0.2y5. If we denote the weightings a1 =a3 =a5 =0.2,a2 =0.1,a4 =0.3, then this could also be written (cid:88)5 a1y1+a2y2+a3y3+a4y4+a5y5 = aiyi. i=1 For example, for the third student, the total score would be 0.2·92+0.1·90+0.2·94+0.3·88+0.2·94=91.4. As you see, in both cases we have a number of variables and we are forming what is called a linear function of those variables, that is, an expression in which each variable appears simply to the first power (with no complicated functions). When we only have two or three variables, the algebra for dealing with such functions is quitesimple,butasthenumberofvariablesgrows,thealgebrabecomesmuchmore complex. Such data sets and calculations should be familiar to anyone who has played with a spreadsheet. Example 2. Instudyingcomplicatedelectricalcircuits, oneusesacollectionof rules called Kirchhoff’s laws. One of these rules says that the currents converging at a node in the circuit add up algebraically to zero. (Currents can be positive or negative.) Other rules put other restrictions on the currents. For example, in the circuit below 10 x 1 x 2 10 x 5 15 Numerical resistances in ohms 20 x x 4 3 5 50 volts Kirchhoff’s laws yield the following equations for the currents x1,x2,x3,x4,x5 in the different branches of the circuit. 10x1+10x2 =50 20x3+5x4 =50 x1−x2−x5 =0 −x3+x4−x5 =0 10x1+5x4+15x5 =50 1. INTRODUCTION 3 Don’t worry if you don’t know anything about electricity. The point is that the circuit is governed by a system of linear equations. In order to understand the circuit, we must have methods to solve such systems. In your high school algebra course,youlearnedhowtosolvetwoequationsintwounknownsandperhapsthree equationsinthreeunknowns. Inthiscourseweshallstudyhowtosolveanynumber ofequationsinanynumberofunknowns. Linearalgebrawasinventedinlargepart todiscussthesolutionsofsuchsystemsinanorganizedmanner. Theaboveexample yielded a fairly small system, but electrical engineers must often deal with very largecircuitsinvolvingmany, manycurrents. Similarly, manyotherapplicationsin other fields require the solution of systems of very many equations in very many unknowns. Nowadays, one uses electronic computers to solve such systems. Consider for example the system of 5 equations in 5 unknowns 2x1+3x2−5x3+6x4−x5 =10 3x1−3x2+6x3+x4−x5 =2 x1+x2−4x3+2x4+x5 =5 4x1−3x2+x3+6x4+x5 =4 2x1+3x2−5x3+6x4−x5 =3 Howmightyoupresentthedataneededtosolvethissystemtoacomputer? Clearly, the computer won’t care about the names of the unknowns since it doesn’t need such aids to do what we tell it to do. It would need to be given the table of coefficients 2 3 −5 6 −1 3 −3 6 1 −1 1 1 −4 2 1 4 −3 1 6 1 2 3 −5 6 −1 and the quantities on the right—also called the ‘givens’ 10 2 5 4 3 Each such table is an example of a matrix, and in the next section, we shall discuss the algebra of such matrices. Exercises for Section 1. 1. A professor taught a class with three students who took two exams each. The results were student/test 1 2 1 100 95 2 60 75 3 100 95 4 I. LINEAR ALGEBRA, BASIC NOTIONS (a) What were the average scores on each test? (b) Are there weightings a1,a2 which result in either of the following weighted scores? student score student score 1 98 1 98 or 2 66 2 66 3 98 3 97 2. Solve each of the following linear systems by any method you know. (a) 2x+3y =3 x+3y =1 (b) x+y =3 y+z =4 x+y+z =5 (c) x+y =3 y+z =4 x+2y+z =5 (d) x+y+z =1 z =1 2. Matrix Algebra In the previous section, we saw examples of rectangular arrays or matrices such as the table of grades   78 70 74 82 74 81 75 72 85 80   92 90 94 88 94   53 72 65 72 59   81 79 79 82 78   21 92 90 88 95   83 84 76 79 84   62 65 67 73 65   70 72 76 82 73 69 75 70 78 79