FUNDAMENTALS OF MATRIX ANALYSIS WITH APPLICATIONS FUNDAMENTALS OF MATRIX ANALYSIS WITH APPLICATIONS EDWARDBARRYSAFF DepartmentofMathematics CenterforConstructiveApproximation VanderbiltUniversity Nashville,TN,USA ARTHURDAVIDSNIDER DepartmentofElectricalEngineering UniversityofSouthFlorida Tampa,FL,USA Copyright©2016byJohnWiley&Sons,Inc.Allrightsreserved PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey PublishedsimultaneouslyinCanada Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyany means,electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptaspermittedunder Section107or108ofthe1976UnitedStatesCopyrightAct,withouteitherthepriorwrittenpermissionofthe Publisher,orauthorizationthroughpaymentoftheappropriateper-copyfeetotheCopyrightClearanceCenter, Inc.,222RosewoodDrive,Danvers,MA01923,(978)750-8400,fax(978)750-4470,oronthewebat www.copyright.com.RequeststothePublisherforpermissionshouldbeaddressedtothePermissions Department,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,(201)748-6011,fax(201) 748-6008,oronlineathttp://www.wiley.com/go/permissions. LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompletenessof thecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessfora particularpurpose.Nowarrantymaybecreatedorextendedbysalesrepresentativesorwrittensalesmaterials. Theadviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.Youshouldconsultwitha professionalwhereappropriate.Neitherthepublishernorauthorshallbeliableforanylossofprofitoranyother commercialdamages,includingbutnotlimitedtospecial,incidental,consequential,orotherdamages. Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontactourCustomer CareDepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat(317)572-3993orfax (317)572-4002. Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe availableinelectronicformats.FormoreinformationaboutWileyproducts,visitourwebsiteatwww.wiley.com. LibraryofCongressCataloging-in-PublicationData: Saff,E.B.,1944– Fundamentalsofmatrixanalysiswithapplications/EdwardBarrySaff,CenterforConstructiveApproximation, VanderbiltUniversity,Nashville,Tennessee,ArthurDavidSnider,DepartmentofElectricalEngineering, UniversityofSouthFlorida,Tampa,Florida. pages cm Includesbibliographicalreferencesandindex. ISBN978-1-118-95365-5(cloth) 1. Matrices. 2. Algebras,Linear. 3. Orthogonalizationmethods. 4. Eigenvalues. I. Snider,ArthurDavid,1940– II. Title. QA188.S1942015 (cid:2) 512.9434–dc23 2015016670 PrintedintheUnitedStatesofAmerica 10 9 8 7 6 5 4 3 2 1 1 2015 ToourbrothersHarveyJ.Saff,DonaldJ.Saff,andArthurHerndonSnider.Theyhave set a high bar for us, inspired us to achieve, and extended helping hands when we neededthemtoreachforthosegreaterheights. EdwardBarrySaff ArthurDavidSnider CONTENTS PREFACE ix PARTI INTRODUCTION:THREEEXAMPLES 1 1 SystemsofLinearAlgebraicEquations 5 1.1 LinearAlgebraicEquations, 5 1.2 Matrix Representation of Linear Systems and the Gauss-Jordan Algorithm, 17 1.3 TheCompleteGaussEliminationAlgorithm, 27 1.4 EchelonFormandRank, 38 1.5 ComputationalConsiderations, 46 1.6 Summary, 55 2 MatrixAlgebra 58 2.1 MatrixMultiplication, 58 2.2 SomePhysicalApplicationsofMatrixOperators, 69 2.3 TheInverseandtheTranspose, 76 2.4 Determinants, 86 2.5 ThreeImportantDeterminantRules, 100 2.6 Summary, 111 GroupProjectsforPartI A. LUFactorization, 116 B. Two-PointBoundaryValueProblem, 118 C. ElectrostaticVoltage, 119 CONTENTS vii D. Kirchhoff’sLaws, 120 E. GlobalPositioningSystems, 122 F. Fixed-PointMethods, 123 PARTII INTRODUCTION:THESTRUCTUREOFGENERAL SOLUTIONSTOLINEARALGEBRAICEQUATIONS 129 3 VectorSpaces 133 3.1 GeneralSpaces,Subspaces,andSpans, 133 3.2 LinearDependence, 142 3.3 Bases,Dimension,andRank, 151 3.4 Summary, 164 4 Orthogonality 165 4.1 OrthogonalVectorsandtheGram–SchmidtAlgorithm, 165 4.2 OrthogonalMatrices, 174 4.3 LeastSquares, 180 4.4 FunctionSpaces, 190 4.5 Summary, 197 GroupProjectsforPartII A. RotationsandReflections, 201 B. HouseholderReflectors, 201 C. InfiniteDimensionalMatrices, 202 PARTIII INTRODUCTION:REFLECTONTHIS 205 5 EigenvectorsandEigenvalues 209 5.1 EigenvectorBasics, 209 5.2 CalculatingEigenvaluesandEigenvectors, 217 5.3 SymmetricandHermitianMatrices, 225 5.4 Summary, 232 6 Similarity 233 6.1 SimilarityTransformationsandDiagonalizability, 233 6.2 PrincipleAxesandNormalModes, 244 6.3 SchurDecompositionandItsImplications, 257 6.4 TheSingularValueDecomposition, 264 6.5 ThePowerMethodandtheQRAlgorithm, 282 6.6 Summary, 290 viii CONTENTS 7 LinearSystemsofDifferentialEquations 293 7.1 First-OrderLinearSystems, 293 7.2 TheMatrixExponentialFunction, 306 7.3 TheJordanNormalForm, 316 7.4 MatrixExponentiationviaGeneralizedEigenvectors, 333 7.5 Summary, 339 GroupProjectsforPartIII A. PositiveDefiniteMatrices, 342 B. HessenbergForm, 343 C. DiscreteFourierTransform, 344 D. ConstructionoftheSVD, 346 E. TotalLeastSquares, 348 F. FibonacciNumbers, 350 ANSWERSTOODDNUMBEREDEXERCISES 351 INDEX 393