• comprehensively introduces infinite cardinalities and infinite- dimensional vector spaces; • thoroughly develops Permutation Theory to completely prove the properties of determinants; • presents large non-trivial matrices, especially symmetric matrices, that have multi-dimensional eigenspaces; • rigorously constructs Complex Euclidean Spaces and inner products, with complete proofs of Schur’s Lemma, the Spectral Theorems for normal matrices, and the simultaneous diagonalization of commuting normal matrices; • proves and applies the Fundamental Theorem of Linear Algebra, and its twin, the Singular Value Decomposition, an essential tool in modern computation; • presents application topics from Physics, Chemistry, Differential Equations, Geometry, Computer Graphics, Group Theory, Recursive Sequences, and Number Theory; • includes topics not usually seen in an introductory book, such as the exponential of a matrix, the intersection of two subspaces, the pre-image of a subspace, cosets, quotient spaces, and the Isomorphism Theorems of Emmy Noether, providing enough material for two full semesters; • features more than 500 additional Exercises since the 2nd Edition, including basic computations, assisted computations, The Third Edition of A Portrait of Linear Algebra builds on the true or false questions, mini-projects, and of course proofs, strengths of the previous editions, providing the student a unified, with multi-step proofs broken down with hints for the student; elegant, modern, and comprehensive introduction: • written in a student-friendly style, with precisely stated definitions and theorems, making this book readable for self- • emphasizes the reading, understanding, and writing of proofs, study. and gives students advice on how to master these skills; • presents a thorough introduction to basic logic, set theory, The author received his Ph.D. in Mathematics from the California axioms, theorems, and methods of proof; Institute of Technology in 1993, and since then has been a professor • develops the properties of vector and matrix operations as at Pasadena City College. natural extensions of the field axioms for real numbers; • gives an early introduction of the core concepts of spanning, linear independence, subspaces (including the fundamental matrix spaces and orthogonal complements), basis, dimension, kernel, and range; • explores linear transformations and their properties by using their correspondence with matrices, fully investigating injective, surjective, and bijective transformations; • focuses on the derivative as the prime example of a linear transformation on function spaces, establishing the strong connection between the fields of Linear Algebra and Differential Equations;       KH COLORS: cyan magenta yellow black final proof: 6-24-16 jjf BOOK: 8.5x11 SPINE: 1.74 for Perfect Binding A Portrait of LinAelagre bra ThiErddi tion JudTeh addSeoucsr ates PasaCdeintCayo llege KendHaulnlt publishing c ompany Jude Thaddeus Socrates and A Portrait of Linear Algebra are on Facebook. Please visit us! To order the print or e-book version of this book, go to: https://he.kendallhunt.com/product/portrait-linear-algebra Cover art: Linear Transformation by Jude Thaddeus Socrates, 2016 www.kendallhunt.com Send all inquiries to: 4050 Westmark Drive Dubuque, IA 52004-1840 Copyright © 2016 by Kendall Hunt Publishing Company ISBN 978-1-4652-9053-3 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the copyright owner. Printed in the United States of America Table of Contents ChapterZero.TheLanguageofMathematics: Sets,Axioms,Theorems&Proofs 1 Chapter1.TheCanvasofLinearAlgebra: EuclideanSpacesandSubspaces 25 1.1TheMainSubject:EuclideanSpaces 26 1.2TheSpanofaSetofVectors 41 1.3TheDotProductandOrthogonality 54 1.4SystemsofLinearEquations 67 1.5LinearSystemsandLinearIndependence 83 1.6IndependentSetsversusSpanningSets 99 1.7SubspacesofEuclideanSpaces;BasisandDimension 115 1.8TheFundamentalMatrixSpaces 125 1.9OrthogonalComplements 142 ASummaryofChapter1 155 Chapter2.AddingMovementandColors: LinearTransformations onEuclideanSpaces 157 2.1MappingSpaces:IntroductiontoLinearTransformations 158 2.2Rotations,ProjectionsandReflections 170 2.3OperationsonLinearTransformationsandMatrices 186 2.4PropertiesofOperationsonLinearTransformationsandMatrices 199 2.5Kernel,Range,One-to-OneandOntoTransformations 213 2.6InvertibleOperatorsandMatrices 228 2.7FindingtheInverseofaMatrix 238 2.8ConditionsforInvertibility 248 2.9Diagonal,Triangular,andSymmetricMatrices 256 ASummaryofChapter2 267 Chapter3.From TheRealtoTheAbstract: GeneralVectorSpaces 269 3.1AxiomsforaVectorSpace 270 3.2LinearityPropertiesforFiniteSetsofVectors 284 3.3LinearityPropertiesforInfiniteSetsofVectors 295 3.4Subspaces,BasisandDimension 310 3.5LinearTransformationsonGeneralVectorSpaces 329 iii 3.6CoordinateVectorsandMatricesforLinearTransformations 341 3.7One-to-OneandOntoLinearTransformations; CompositionsofLinearTransformations 358 3.8IsomorphismsandtheirApplications 376 ASummaryofChapter3 391 Chapter4.PeelingTheOnion: TheSubspaceStructureofVectorSpaces 393 4.1TheJoinandIntersectionofTwoSubspaces 394 4.2RestrictingLinearTransformationsandtheRoleoftheRowspace 403 4.3TheImageandPreimageofSubspaces 412 4.4CosetsandQuotientSpaces 422 4.5TheThreeIsomorphismTheorems 431 ASummaryofChapter4 445 Chapter5.From SquaretoScalar: PermutationTheoryandDeterminants 447 5.1PermutationsandTheDeterminantConcept 448 5.2AGeneralDeterminantFormula 461 5.3ComputationalToolsandPropertiesofDeterminants 477 5.4TheAdjugateMatrixandCramer’sRule 488 5.5TheWronskian 497 ASummaryofChapter5 501 Chapter6.PaintingtheLines: Eigentheory,Diagonalization andSimilarity 503 6.1TheEigentheoryofSquareMatrices 504 6.2ComputationalTechniquesforEigentheory 514 6.3DiagonalizationofSquareMatrices 526 6.4TheExponentialofaMatrix 540 6.5ChangeofBasisandLinearTransformationsonEuclideanSpaces 544 6.6ChangeofBasisforAbstractSpacesandDeterminantsforOperators 555 6.7SimilarityandTheEigentheoryofOperators 563 ASummaryofChapter6 575 Chapter7.GeometryintheAbstract: InnerProductSpaces 577 7.1AxiomsforanInnerProductSpace 578 7.2GeometricConstructionsinInnerProductSpaces 589 iv 7.3OrthonormalSetsandTheGram-SchmidtAlgorithm 599 7.4OrthogonalComplementsandDecompositions 613 7.5OrthonormalBasesandProjectionOperators 625 7.6OrthogonalMatrices 635 7.7OrthogonalDiagonalizationofSymmetricMatrices 646 7.8TheMethodofLeastSquares 653 7.9TheQR-Decomposition 662 ASummaryofChapter7 669 Chapter8.ImagineThat: ComplexSpacesandTheSpectralTheorems 671 8.1TheFieldofComplexNumbers 672 8.2ComplexVectorSpaces 685 8.3ComplexInnerProducts 694 8.4ComplexLinearTransformationsandTheAdjoint 702 8.5NormalMatrices 712 8.6Schur’sLemmaandTheSpectralTheorems 725 8.7SimultaneousDiagonalization 735 ASummaryofChapter8 751 Chapter9.TheBigPicture:TheFundamentalTheorem of LinearAlgebraandApplications 753 9.1BalancingChemicalEquations 754 9.2BasicCircuitAnalysis 760 9.3RecurrenceRelations 770 9.4IntroductiontoQuadraticForms 778 9.5RotationsofConics 788 9.6PositiveDefiniteQuadraticFormsandMatrices 796 9.7TheFundamentalTheoremofLinearAlgebra 807 9.8TheSingularValueDecomposition 817 9.9ApplicationsoftheSVD 827 AppendixA:TheRealNumberSystem 837 AppendixB:LogicalSymbolsandTruthTables 856 GlossaryofSymbols 861 SubjectIndex 866 TheAnswerKeytotheExercisesisavailableasafreedownloadat: https://he.kendallhunt.com/product/portrait-linear-algebra v Preface to the 3rd Edition In the three years since the 2nd Edition of A Portrait of Linear Algebra came out, I have had the privilege of teaching Linear Algebra every semester, and even during most of the summers. All the new ideas, improvements, exercises, and other changes that have been incorporated in the 3rd edition would not have been possible without the lengthy discussions and interactions that I have had with so many wonderful students in these classes, and the colleagues who adopted this book for their own LinearAlgebraclass. So let me begin by thanking Daniel Gallup, John Sepikas, Lyman Chaffee, Christopher Strinden, Patricia Michel, Asher Shamam, Richard Abdelkerim, Mark Pavitch, David Matthews, and Guoqiang Song, mycolleagues at Pasadena CityCollege whohave taught out ofmybook, for sharing their ideas andexperienceswithme,theirencouragement,andsuggestionsforimprovingthistext. IamcertainthatifIbegintonameallthestudentswhohavegivenmeconstructivecriticismsaboutthe book, I will miss more than just a handful. There have been hundreds of students who have gone through this book, and I learned so much from my conversations with many of them. Often, a casual remarkor a simple questionwould prompt me torewrite anexplanationor come up withaninteresting new exercise. Many of these students have continued on to finish their undergraduate careers at four-year institutions, and have begun graduate studies in mathematics or engineering. Some of them have kept in touch with me over the years, and the sweetest words they have said to me is how easily they handled upper-division Linear Algebra classes, thanks to the solid education they received from mybook. I give them mydeepest gratitude, not just for their thoughts, but also for giving me the best careerintheworld. It is hard tobelieve that tenyears ago, the idea ofthis bookdid not evenexist. None ofthis would have beenpossiblewithoutthehelpofsomanypeople. Thank you to Christine Bochniak, Beverly Kraus, and Taylor Knuckey of Kendall Hunt for their valuableassistanceinbringingthe3rdeditiontofruition. Manythanks to mylong-suffering husband, mybest friend and biggest supporter, Juan Sanchez-Diaz, for patientlyaccepting allthe nights and weekends that were consumed bythis book. And thankyou to Johannes, for your unconditionallove and for making me get up from the computer sowe cangofor a walkorplaywiththeball.Iwouldhavegonebonkersifitweren’tforyoutwo. TothemembersoftheSocratesandSanchez familiesallovertheplanet,maramingsalamat,ymuchas gracias,forallyourloveandsupport. Thanks to all my colleagues at PCC, my friends on Facebook, and my barkada, for being my unflaggingcheeringsquadandartisticcritics. Thankstomytennisandgymbuddiesforkeepingmemotivatedandphysicallyhealthy. Thank you to mylate parents, Dr. Jose Socrates and Dr. Nenita Socrates, for teaching me and all their childrentheloveforlearning. Andfinally,mythankstoourLord,forshoweringmylifewithsomanyblessings. JudeThaddeusSocrates ProfessorofMathematics PasadenaCityCollege,California June,2016 vii What Makes This Book Different? APortraitofLinearAlgebratakesauniqueapproachindevelopingandintroducingthecoreconcepts of this subject. It begins with a thorough introduction of the field properties for real numbers and uses them to guide the student through simple proof exercises. From here, we introduce the Euclidean spaces and see that manyof the field properties for the real numbers naturallyextend to the properties of vector arithmetic. The core concepts of linear combinations, spans of sets of vectors, linear independence, subspaces, basis and dimension, are introduced in the first chapter and constantly referenced and reinforced throughout the book. This early introduction enables the student to retain theseconceptsbetterandtoapplythemtodeeperideas. The Four Fundamental Matrix Spaces are encountered at the end of the first Chapter, and transitions naturally into the second Chapter, where we study linear transformations and their standard matrices. The kernel and range of these transformations tells us if our transformations are one-to-one or onto. Whentheyare both, we learnhow tofind the inverse transformation. We alsosee that some geometric operationsofvectorsin2 or3 areexamplesoflinearoperators. Once these core concepts are firmly established, they can be naturally extended to create abstract vector spaces, the most important examples of which are function spaces, polynomial spaces, and matrix spaces. Linear transformations on finite dimensional vector spaces can again be coded using matrices by finding coordinates for our vectors with respect to a basis. Everything we encountered in thefirsttwochapterscannowbenaturallygeneralized. One of the unique features of this book is the use of projections and reflections in 3, with respect to either a line or a plane, in order to motivate some concepts or constructions. We use them to explore the core concepts of the standard matrix of a linear transformation, the matrix of a transformation with respect to a non-standard basis, and the change of basis matrix. In the case of reflection operators, we see them as motivation for the inverse of a matrix, and as an example of an orthogonal matrix. Projectionmatrices,ontheotherhand,aregoodexamplesofidempotentmatrices. The second half of the book goes into the study of determinants, eigentheory, inner product spaces, complex vector spaces, the SpectralTheorems, and the materialnecessarytounderstand and prove the FundamentalTheoremofLinearAlgebra,anditstwin,theSingularValueDecomposition.Wealsosee severalapplicationsofLinearAlgebrainscience,engineering,andotherareasofmathematics. Throughout the book, we emphasize clear and precise definitions and proofs of Theorems, constantly encouragingthestudenttoreadandunderstandproofs,andtopracticewritingtheirownproofs. How this Book is Organized Chapter Zero provides an introduction to sets and set operations, logic, the field axioms for real numbers, and common proof techniques, emphasizing theorems that can be derived from the field axioms. This brief introductory chapter will prepare the student to learn how to read, understand and writebasicproofs. WebaseourdevelopmentofthemainconceptsofLinearAlgebraonthefollowingdefinition: Linear Algebra is the study of vector spaces, their structure, and the linear transformations thatmaponevectorspacetoanother. viii Chapter 1 rigorouslyexamines the archetype vector spaces: Euclidean spaces, their geometry, and the core ideas ofspanning, linear independence, subspaces, basis, dimensionand orthogonalcomplements. WewillseetheGauss-JordanAlgorithm,thecentraltoolofLinearAlgebra,anduseittosolvesystems of linear equations and investigate the span of a set of vectors. We will also construct the four fundamental matrix spaces: rowspace, columnspace and nullspace for a matrix and its transpose, and findabasisforeachspace. Chapter 2 introduces linear transformations on Euclidean spaces as encoded bymatrices. We will see how each linear transformation determines special subspaces, namely the kernel and the range of the transformation, and use these spaces to investigate the one-to-one and onto properties. We will define basicmatrixoperations,includingamethodtofinditsinversewhenthisexists. Chapter 3 generalizes the concepts from Chapters 1 and 2 in order to construct abstract vector spaces and linear transformations from one vector space to another. We focus most of our examples on function spaces (in particular, polynomial spaces), and linear transformations connecting them, especiallythose involving derivatives and evaluations. We willsee that inthe finite-dimensionalcase, a linear transformationcanbe encoded bya matrix as well. Byfocusing onfunctionspaces preserved by the derivative operator, the strong relationship between Linear Algebra and Differential Equations is firmlyestablished. Chapter 4 investigates the subspace structure of vector spaces, and we will see techniques to fully describe the join and intersection of two subspaces, the image or preimage of a subspace, and the restriction of a linear transformation to a subspace. We will create cosets and quotient spaces, and see one of the fundamentaltriptychs of modern mathematics: the Isomorphism Theorems of EmyNoether asappliedtovectorspaces. Chapter 5 explores the determinant function, its properties, especially its relationship to invertibility, andefficientalgorithmstocomputeit.WewillseeCramer’srule,atechniquetosolveinvertiblesquare systemsofequations,albeitnotaverypracticalone. Chapter 6 introduces the eigentheoryof operators both on Euclidean spaces as well as abstract vector spaces. We will see when it is possible to encode operators into the simplest possible form, that is, to diagonalizethem.Wewillstudytheconceptofsimilarityanditsconsequences. Chapter 7 generalizes geometry on a vector space by imposing an inner product on it. This allows us to introduce the concepts of norm and orthogonality in abstract spaces. We will explore orthonormal bases, the Gram-Schmidt Algorithm, orthogonalmatrices, the orthogonaldiagonalization of symmetric matrices,themethodofleastsquares,andtheQR-decomposition. Chapter8appliestheconstructionsthusfartovectorspacesoverarbitraryfields,especiallythefieldof complex numbers. The maingoalofthis chapter is toprove the SpectralTheorem ofNormalMatrices. One specific case of this Theorem tells us that symmetric matrices can indeed be diagonalized by orthogonal matrices. We also see that commuting diagonalizable matrices can be simultaneously diagonalizedbythesameinvertiblematrix,andpresentanalgorithmtodoso. Chapter 9 explores some applications of Linear Algebra in science and engineering. We develop the theories of quadratic forms and positive semi-definite matrices. These enable us to prove The Fundamental Theorem of Linear Algebra, an elegant theorem that ties together the four fundamental matrix spaces and the concepts of eigenspaces and orthogonality. Closely connected to this is the SingularValueDecomposition,whichhasapplicationsindataprocessing. This book is intended to serve as a text for a standard 15-week semester course in introductoryLinear Algebra. However, enough material is included in this text for two full semesters. This book is my visionofwhattoday’sstudentinscienceandengineeringshouldknowaboutthiselegantfield. ix