Linear Algebra Linear Algebra Step by Step Kuldeep Singh SeniorLecturerinMathematics UniversityofHertfordshire 3 3 GreatClarendonStreet,Oxford,OX26DP, UnitedKingdom OxfordUniversityPressisadepartmentoftheUniversityofOxford. ItfurtherstheUniversity’sobjectiveofexcellenceinresearch,scholarship, andeducationbypublishingworldwide.Oxfordisaregisteredtrademarkof OxfordUniversityPressintheUKandincertainothercountries (cid:2)c KuldeepSingh2014 Themoralrightsoftheauthorhavebeenasserted FirstEditionpublishedin2014 Impression:1 Allrightsreserved.Nopartofthispublicationmaybereproduced,storedin aretrievalsystem,ortransmitted,inanyformorbyanymeans,withoutthe priorpermissioninwritingofOxfordUniversityPress,orasexpresslypermitted bylaw,bylicenceorundertermsagreedwiththeappropriatereprographics rightsorganization.Enquiriesconcerningreproductionoutsidethescopeofthe aboveshouldbesenttotheRightsDepartment,OxfordUniversityPress,atthe addressabove Youmustnotcirculatethisworkinanyotherform andyoumustimposethissameconditiononanyacquirer PublishedintheUnitedStatesofAmericabyOxfordUniversityPress 198MadisonAvenue,NewYork,NY10016,UnitedStatesofAmerica BritishLibraryCataloguinginPublicationData Dataavailable LibraryofCongressControlNumber:2013947001 ISBN978–0–19–965444–4 PrintedintheUKby Bell&BainLtd,Glasgow LinkstothirdpartywebsitesareprovidedbyOxfordingoodfaithand forinformationonly.Oxforddisclaimsanyresponsibilityforthematerials containedinanythirdpartywebsitereferencedinthiswork. Preface My interest in mathematics began at school. I am originally of Sikh descent, and as a young child oftenfoundEnglishdifficulttocomprehend,butIdiscoveredanaffinitywithmathematics,auniversal languagethatIcouldbegintolearnfromthesamestartpointasmypeers. Linearalgebraisafundamentalareaofmathematics,andisarguablythemostpowerfulmathematical tooleverdeveloped.Itisacoretopicofstudywithinfieldsasdiverseasbusiness,economics,engineer- ing,physics,computerscience,ecology,sociology,demographyandgenetics.Foranexampleoflinear algebraatwork,oneneedlooknofurtherthantheGooglesearchengine,whichreliesonlinearalgebra toranktheresultsofasearchwithrespecttorelevance. Mypassionhasalwaysbeentoteach,andIhaveheldthepositionofSeniorLecturerinMathematics at the University of Hertfordshire for over twenty years, where I teach linear algebra to entry level undergraduates.IamalsotheauthorofEngineeringMathematicsThroughApplications,abookthatI amproudtosayisusedwidelyasthebasisforundergraduatestudiesinmanydifferentcountries.Ialso hostandregularlyupdateawebsitededicatedtomathematics. AttheUniversityofHertfordshirewehaveoveronehundredmathematicsundergraduates.Inthe pastwehavebasedourlinearalgebracoursesonvariousexistingtextbooks,butingeneralstudents havefoundthemhardtodigest;oneofmyprimaryconcernshasbeeninfindingrigorous,yetaccessible textbookstorecommendtomystudents.Becauseofthepopularityofmypreviouslypublishedbook,I havefeltcompelledtoconstructabookonlinearalgebrathatbridgestheconsiderabledividebetween schoolandundergraduatemathematics. IamsomewhatfortunateinthatIhavehadsomanystudentstoassistmeinevaluatingeachchapter. Inresponsetotheirreactions,Ihavemodified,expandedandaddedsectionstoensurethatitscontent entirelyencompassestheabilityofstudentswithalimitedmathematicalbackground,aswellasthe moreadvancedscholarsundermytutelage.Ibelievethatthishasallowedmetocreateabookthatis unparalleledinthesimplicityofitsexplanation,yetcomprehensiveinitsapproachtoeventhemost challengingaspectsofthistopic. Level This book is intended for first- and second-year undergraduates arriving with average mathematics grades. Many students find the transition between school and undergraduate mathematics difficult, andthisbookspecificallyaddressesthatgapandallowsseamlessprogression.Itassumeslimitedprior mathematicalknowledge,yetalsocoversdifficultmaterialandanswerstoughquestionsthroughtheuse ofclearexplanationandawealthofillustrations.Theemphasisofthebookisonstudentslearningfor themselvesbygraduallyabsorbingclearlypresentedtext,supportedbypatterns,graphsandassociated questions.Thetextallowsthestudenttograduallydevelopanunderstandingofatopic,withoutthe needforconstantadditionalsupportfromatutor. PedagogicalIssues Thestrengthofthetextisinthelargenumberofexamplesandthestep-by-stepexplanationofeachtopic asitisintroduced.Itiscompiledinawaythatallowsdistancelearning,withexplicitsolutionstoallof vi PREFACE thesetproblemsfreelyavailableonline<http://www.oup.co.uk/companion/singh>.Themiscellaneous exercisesattheendofeachchaptercomprisequestionsfrompastexampapersfromvariousuniversities, helpingtoreinforcethereader’sconfidence.Alsoincludedareshorthistoricalbiographiesoftheleading playersinthefieldoflinearalgebra.Thesearegenerallyplacedatthebeginningofasectiontoengage theinterestofthestudentfromtheoutset. Publishedtextbooksonthissubjecttendtoberatherstaticintheirpresentation.Bycontrast,mybook strivestobesignificantlymoredynamic,andencouragestheengagementofthereaderwithfrequent questionandanswersections.Thequestion–answerelementissprinkledliberallythroughoutthetext, consistentlytestingthestudent’sunderstandingofthemethodsintroduced,ratherthanrequiringthem torememberbyrote. Thesimpleyetconcisenatureofitscontentisspecificallydesignedtoaidtheweakerstudent,but itsrigorousapproachandcomprehensivemannermakeitentirelyappropriatereferencematerialfor mathematiciansateverylevel.IncludedintheonlineresourcewillbeaselectionofMATLABscripts, providedforthosestudentswhowishtoprocesstheirworkusingacomputer. Finally,itmustbeacknowledgedthatlinearalgebracanappearabstractwhenfirstencounteredby astudent.Toshowoffsomeofitspossibilitiesandpotential,interviewswithleadingacademicsand practitionershavebeenplacedbetweenchapters,givingreadersatasteofwhatmaybetocomeonce theyhavemasteredthispowerfulmathematicaltool. Acknowledgements IwouldparticularlyliketothankTimothyPeacockforhissignficanthelpinimprovingthistext.In additionIwanttothankSandraStarkeforherconsiderablecontributioninmakingthistextaccessible. ThankstoototheOUPteam,inparticularKeithMansfield,VikiMortimer,SmitaGuptaandClare Charles. Dedication ToShaheedBibiParamjitKaur Contents 1 LinearEquationsandMatrices 1 1.1 SystemsofLinearEquations 1 1.2 GaussianElimination 12 1.3 VectorArithmetic 27 1.4 ArithmeticofMatrices 41 1.5 MatrixAlgebra 57 1.6 TheTransposeandInverseofaMatrix 75 1.7 TypesofSolutions 91 1.8 TheInverseMatrixMethod 105 DesHighamInterview 127 2 EuclideanSpace 129 2.1 PropertiesofVectors 129 2.2 FurtherPropertiesofVectors 143 2.3 LinearIndependence 159 2.4 BasisandSpanningSet 171 ChaoYangInterview 190 3 GeneralVectorSpaces 191 3.1 IntroductiontoGeneralVectorSpaces 191 3.2 SubspaceofaVectorSpace 202 3.3 LinearIndependenceandBasis 216 3.4 Dimension 229 3.5 PropertiesofaMatrix 239 3.6 LinearSystemsRevisited 254 JanetDrewInterview 275 4 InnerProductSpaces 277 4.1 IntroductiontoInnerProductSpaces 277 4.2 InequalitiesandOrthogonality 290 4.3 OrthonormalBases 306 4.4 OrthogonalMatrices 321 AnshulGuptaInterview 338 5 LinearTransformations 339 5.1 IntroductiontoLinearTransformations 339 5.2 KernelandRangeofaLinearTransformation 352 5.3 RankandNullity 364 5.4 InverseLinearTransformations 372 5.5 TheMatrixofaLinearTransformation 389 5.6 CompositionandInverseLinearTransformations 407 PetrosDrineasInterview 429 viii CONTENTS 6 DeterminantsandtheInverseMatrix 431 6.1 DeterminantofaMatrix 431 6.2 DeterminantofOtherMatrices 439 6.3 PropertiesofDeterminants 455 6.4 LUFactorization 472 FrançoiseTisseurInterview 490 7 EigenvaluesandEigenvectors 491 7.1 IntroductiontoEigenvaluesandEigenvectors 491 7.2 PropertiesofEigenvaluesandEigenvectors 503 7.3 Diagonalization 518 7.4 DiagonalizationofSymmetricMatrices 533 7.5 SingularValueDecomposition 547 BriefSolutions 567 Index 605 ............................................................................................. 1 Linear Equations and Matrices ............................................................................................. SECTION 1.1 SystemsofLinearEquations Bytheendofthissectionyouwillbeableto ● solvealinearsystemofequations ● plotlineargraphsanddeterminethetypeofsolutions 1.1.1 Introductiontolinearalgebra Weareallfamiliarwithsimpleone-lineequations.Anequationiswheretwomathematical expressionsaredefinedasbeingequal.Given3x=6,wecanalmostintuitivelyseethatx mustequal2. However,thesolutionisn’talwaysthiseasytofind,andthefollowingexampledemon- strateshowwecanextractinformationembeddedinmorethanonelineofinformation. ImagineforamomentthatJohnhasboughttwoicecreamsandtwodrinksfor£3.00. HowmuchdidJohnpayforeachitem? Letx=costoficecreamandy=costofdrink,thentheproblemcanbewrittenas 2x+2y=3 Atthispoint,itisimpossibletofindauniquevalueforthecostofeachitem.However,youare thentoldthatJaneboughttwoicecreamsandonedrinkfor£2.50.Withthisadditionalinforma- tion,wecanmodeltheproblemasasystemofequationsandlookforuniquevaluesforthecost oficecreamsanddrinks.Theproblemcannowbewrittenas 2x+2y = 3 2x+y = 2.5 Usingabitofguesswork,wecanseethattheonlysensiblevaluesforxandythatsatisfyboth equationsarex=1andy=0.5.Thereforeanicecreammusthavecost£1.00andadrink£0.50. Ofcourse,thisisanextremelysimpleexample,thesolutiontowhichcanbefoundwith aminimumofcalculation,butlargersystemsofequationsoccurinareaslikeengineering, scienceandfinance.Inordertoreliablyextractinformationfrommultiplelinearequations, we need linear algebra. Generally, the complex scientific, or engineering problem can be solvedbyusinglinearalgebraonlinearequations.