Linear Algebra and Linear Models, Second Edition R.B. Bapat Springer Preface Themainpurposeofthepresentmonographistoprovidearigorousintroduction tothebasicaspectsofthetheoryoflinearestimationandhypothesistesting.The necessaryprerequisitesinmatrices,multivariatenormaldistribution,anddistribu- tionofquadraticformsaredevelopedalongtheway.Themonographisprimarily aimedatadvancedundergraduateandfirst-yearmaster’sstudentstakingcourses inlinearalgebra,linearmodels,multivariateanalysis,anddesignofexperiments. Itshouldalsobeofusetoresearchworkersasasourceofseveralstandardresults andproblems. Somefeaturesinwhichwedeviatefromthestandardtextbooksonthesubject areasfollows. Wedealexclusivelywithrealmatrices,andthisleadstosomenonconventional proofs. One example is the proof of the fact that a symmetric matrix has real eigenvalues.Werelyonranksanddeterminantsabitmorethanisdoneusually. Thedevelopmentinthefirsttwochaptersissomewhatdifferentfromthatinmost texts. Itisnottheintentiontogiveanextensiveintroductiontomatrixtheory.Thus, severalstandardtopicssuchasvariouscanonicalformsandsimilarityarenotfound here.Weoftenderiveonlythoseresultsthatareexplicitlyusedlater.Thelistof factsinmatrixtheorythatareelementary,elegant,butnotcoveredhereisalmost endless. Weputagreatdealofemphasisonthegeneralizedinverseanditsapplications. This amounts to avoiding the “geometric” or the “projections” approach that is favoredbysomeauthorsandtakingrecoursetoamorealgebraicapproach.Partly asapersonalbias,Ifeelthatthegeometricapproachworkswellinprovidingan vi Preface understandingofwhyaresultshouldbetruebuthaslimitationswhenitcomesto provingtheresultrigorously. Thefirstthreechaptersaredevotedtomatrixtheory,linearestimation,andtests oflinearhypotheses,respectively.Chapter4collectsseveralresultsoneigenval- uesandsingularvaluesthatarefrequentlyrequiredinstatisticsbutusuallyarenot provedinstatisticstexts.Thischapteralsoincludessectionsonprincipalcompo- nentsandcanonicalcorrelations.Chapter5preparesthebackgroundforacourse in designs, establishing the linear model as the underlying mathematical frame- work.Thesectionsonoptimalitymaybeusefulasmotivationforfurtherreading inthisresearchareainwhichthereisconsiderableactivityatpresent.Similarly, thelastchaptertriestoprovideaglimpseintotherichnessofatopicingeneralized inverses(rankadditivity)thathasmanyinterestingapplicationsaswell. Severalexercisesareincluded,someofwhichareusedinsubsequentdevelop- ments. Hints are provided for a few exercises, whereas reference to the original sourceisgiveninsomeothercases. IamgratefultoProfessorsAlokeDey,H.Neudecker,K.P.S.BhaskaraRao,and Dr. N. Eagambaram for their comments on various portions of the manuscript. ThanksarealsoduetoB.Ganeshanforhishelpingettingthecomputerprintouts atvariousstages. About the Second Edition Thisisathoroughlyrevisedandenlargedversionofthefirstedition.Besidescor- rectingtheminormathematicalandtypographicalerrors,thefollowingadditions havebeenmade: (1) Afewproblemshavebeenaddedattheendofeachsectioninthefirstfour chapters.Allthechaptersnowcontainsomenewexercises. (2) Completesolutionsorhintsareprovidedtoseveralproblemsandexercises. (3) Twonewsections,oneonthe“volumeofamatrix”andtheotheronthe“star order,”havebeenadded. NewDelhi,India R.B.Bapat Contents Preface v NotationIndex ix 1 VectorSpacesandMatrices 1 1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 VectorSpacesandSubspaces . . . . . . . . . . . . . . . . . . 4 1.3 BasisandDimension . . . . . . . . . . . . . . . . . . . . . . 5 1.4 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.5 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.6 Nonsingularity . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.7 FrobeniusInequality . . . . . . . . . . . . . . . . . . . . . . . 16 1.8 EigenvaluesandtheSpectralTheorem . . . . . . . . . . . . . 18 1.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.10 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 25 2 LinearEstimation 29 2.1 GeneralizedInverses . . . . . . . . . . . . . . . . . . . . . . . 29 2.2 LinearModel . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.3 Estimability . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.4 WeighingDesigns . . . . . . . . . . . . . . . . . . . . . . . . 38 2.5 ResidualSumofSquares . . . . . . . . . . . . . . . . . . . . 40 2.6 EstimationSubjecttoRestrictions . . . . . . . . . . . . . . . . 42 2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.8 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 48 viii Contents 3 TestsofLinearHypotheses 51 3.1 SchurComplements . . . . . . . . . . . . . . . . . . . . . . . 51 3.2 MultivariateNormalDistribution . . . . . . . . . . . . . . . . 53 3.3 QuadraticFormsandCochran’sTheorem . . . . . . . . . . . . 57 3.4 One-WayandTwo-WayClassifications . . . . . . . . . . . . . 61 3.5 GeneralLinearHypothesis . . . . . . . . . . . . . . . . . . . 65 3.6 ExtremaofQuadraticForms . . . . . . . . . . . . . . . . . . 67 3.7 MultipleCorrelationandRegressionModels . . . . . . . . . . 69 3.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.9 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 76 4 SingularValuesandTheirApplications 79 4.1 SingularValueDecomposition . . . . . . . . . . . . . . . . . 79 4.2 ExtremalRepresentations . . . . . . . . . . . . . . . . . . . . 81 4.3 Majorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.4 PrincipalComponents . . . . . . . . . . . . . . . . . . . . . . 86 4.5 CanonicalCorrelations . . . . . . . . . . . . . . . . . . . . . 88 4.6 VolumeofaMatrix . . . . . . . . . . . . . . . . . . . . . . . 89 4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.8 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 94 5 BlockDesignsandOptimality 99 5.1 ReducedNormalEquations . . . . . . . . . . . . . . . . . . . 99 5.2 TheC-Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.3 E-,A-,andD-Optimality . . . . . . . . . . . . . . . . . . . . 103 5.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.5 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 110 6 RankAdditivity 113 6.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.2 CharacterizationsofRankAdditivity . . . . . . . . . . . . . . 114 6.3 GeneralLinearModel . . . . . . . . . . . . . . . . . . . . . . 118 6.4 TheStarOrder . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 6.6 HintsandSolutions . . . . . . . . . . . . . . . . . . . . . . . 126 Notes 129 References 133 Index 137 1 Vector Spaces and Matrices 1.1 Preliminaries Inthissectionwereviewcertainbasicconcepts.Weconsideronlyrealmatrices. Althoughourtreatmentisself-contained,thereaderisassumedtobefamiliarwith basicoperationsonmatrices.Wealsoassumeknowledgeofelementaryproperties ofthedeterminant. Anm×nmatrixconsistsofmnrealnumbersarrangedinmrowsandncolumns. Wedenotematricesbyboldletters.Theentryinrowiandcolumnj ofthematrix Aisdenotedbyaij.Anm×1matrixiscalledacolumnvectoroforderm;similarly, a1×nmatrixisarowvectorofordern.Anm×nmatrixiscalledasquarematrix ifm(cid:8)n. If A,B are m×n matrices, then A+B is defined as the m×n matrix with (i,j)-entryaij +bij.IfAisamatrixandcisarealnumber,thencAisobtained bymultiplyingeachelementofAbyc. IfAism×p andBisp×n,thentheirproductC (cid:8) ABisanm×nmatrix with(i,j)-entrygivenby (cid:2)p c (cid:8) a b . ij ik kj k(cid:8)1 Thefollowingpropertieshold: (AB)C(cid:8)A(BC), A(B+C)(cid:8)AB+AC, (A+B)C(cid:8)AC+BC.