This page intentionally left blank EconometricExercises,Volume1 Matrix Algebra Matrix Algebra is the first volume of the Econometric Exercises Series. It contains ex- ercises relating to course material in matrix algebra that students are expected to know while enrolled in an (advanced) undergraduate or a postgraduate course in econometrics or statistics. The book contains a comprehensive collection of exercises, all with full answers. Butthe bookis notjusta collectionof exercises; infact, itisatextbook, though onethatisorganizedinacompletelydifferentmannerthantheusualtextbook. Thevolume canbeusedeitherasaself-containedcourseinmatrixalgebraorasasupplementarytext. Karim Abadir has held a joint Chair since 1996 in the Departments of Mathematics and EconomicsattheUniversityofYork,wherehehasbeenthefounderanddirectorofvarious degree programs. He has also taught at the American University in Cairo, the University of Oxford, and the University of Exeter. He became an Extramural Fellow at CentER (Tilburg University) in 2003. Professor Abadir is a holder of two Econometric Theory Awards, and has authored many articles intop journals, including the Annals ofStatistics, Econometric Theory, Econometrica, and the Journal of Physics A. He is Coordinating Editor (and one of the founding editors) of the Econometrics Journal, and Associate Editor of Econometric Reviews, Econometric Theory, Journal of Financial Econometrics, andPortugueseEconomicJournal. Jan Magnus is Professor of Econometrics, CentER and Department of Econometrics and Operations Research, Tilburg University, the Netherlands. He has also taught at the University of Amsterdam, The University of British Columbia, The London School of Economics, The University of Montreal, and The European University Institute among other places. His books include Matrix Differential Calculus (with H. Neudecker), Linear Structures,MethodologyandTacitKnowledge(withM.S.Morgan),andEconometrics: A FirstCourse(inRussianwithP.K.KatyshevandA.A.Peresetsky). ProfessorMagnushas written numerous articles in the leading journals, including Econometrica, The Annals of Statistics, The Journal of the American Statistical Association, Journal of Econometrics, LinearAlgebraandItsApplications,andTheReviewofIncomeandWealth. HeisaFellow oftheJournalofEconometrics,holderoftheEconometricTheoryAward,andassociateed- itorofTheJournalofEconomicMethodology,ComputationalStatisticsandDataAnalysis, andtheJournalofMultivariateAnalysis. EconometricExercises Editors: KarimM.Abadir,DepartmentsofMathematicsandEconomics, UniversityofYork,UK JanR.Magnus,CentERandDepartmentofEconometricsandOperationsResearch, TilburgUniversity,TheNetherlands PeterC.B.Phillips,CowlesFoundationforResearchinEconomics, YaleUniversity,USA TitlesintheSeries(*=planned): 1 MatrixAlgebra(K.M.AbadirandJ.R.Magnus) 2 Statistics(K.M.Abadir,R.D.H.HeijmansandJ.R.Magnus) 3 EconometricModels,I:Theory(P.Paruolo) 4 EconometricModels,I:EmpiricalApplications(A.vanSoestandM.Verbeek) * EconometricModels,II:Theory * EconometricModels,II:EmpiricalApplications * TimeSeriesEconometrics,I * TimeSeriesEconometrics,II * Microeconometrics * PanelData * BayesianEconometrics * NonlinearModels * NonparametricsandSemiparametrics * Simulation-BasedEconometricMethods * ComputationalMethods * FinancialEconometrics * Robustness * EconometricMethodology Matrix Algebra KarimM.Abadir DepartmentsofMathematicsandEconomics,UniversityofYork,UK JanR.Magnus CentERandDepartmentofEconometricsandOperationsResearch, TilburgUniversity,TheNetherlands CAMBRIDGEUNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridge.org/9780521822893 © Karim M. Abadir and Jan R. Magnus 2005 This publication is in copyright. Subject to statutory exception and to the provision of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published in print format 2005 ISBN-13 978-0-511-34440-4 eBook (EBL) ISBN-10 0-511-34440-6 eBook (EBL) ISBN-13 978-0-521-82289-3 hardback ISBN-10 0-521-82289-0 hardback ISBN-13 978-0-521-53746-9 paperback ISBN-10 0-521-53746-0 paperback Cambridge University Press has no responsibility for the persistence or accuracy of urls for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. Tomyparents,andtoKouka,Ramez,Naguib,Ne´vine ToGideonandHedda Contents Listofexercises xi PrefacetotheSeries xxv Preface xxix 1 Vectors 1 1.1 Realvectors 4 1.2 Complexvectors 11 2 Matrices 15 2.1 Realmatrices 19 2.2 Complexmatrices 39 3 Vector spaces 43 3.1 Complexandrealvectorspaces 47 3.2 Inner-productspace 61 3.3 Hilbertspace 67 4 Rank,inverse,anddeterminant 73 4.1 Rank 75 4.2 Inverse 83 4.3 Determinant 87 5 Partitionedmatrices 97 5.1 Basicresultsandmultiplicationrelations 98 5.2 Inverses 103 5.3 Determinants 109 5.4 Rank(in)equalities 119 5.5 Thesweepoperator 126 6 Systemsofequations 131 6.1 Elementarymatrices 132 6.2 Echelonmatrices 137 viii Contents 6.3 Gaussianelimination 143 6.4 Homogeneousequations 148 6.5 Nonhomogeneousequations 151 7 Eigenvalues,eigenvectors,andfactorizations 155 7.1 Eigenvaluesandeigenvectors 158 7.2 Symmetricmatrices 175 7.3 Someresultsfortriangularmatrices 182 7.4 Schur’sdecompositiontheoremanditsconsequences 187 7.5 Jordan’sdecompositiontheorem 192 7.6 Jordanchainsandgeneralizedeigenvectors 201 8 Positive (semi)definiteandidempotentmatrices 209 8.1 Positive(semi)definitematrices 211 8.2 Partitioningandpositive(semi)definitematrices 228 8.3 Idempotentmatrices 231 9 Matrixfunctions 243 9.1 Simplefunctions 246 9.2 Jordanrepresentation 255 9.3 Matrix-polynomialrepresentation 265 10 Kroneckerproduct,vec-operator,andMoore-Penroseinverse 273 10.1 TheKroneckerproduct 274 10.2 Thevec-operator 281 10.3 TheMoore-Penroseinverse 284 10.4 Linearvectorandmatrixequations 292 10.5 Thegeneralizedinverse 295 11 Patternedmatrices: commutation-andduplicationmatrix 299 11.1 Thecommutationmatrix 300 11.2 Thesymmetrizermatrix 307 11.3 Thevech-operatorandtheduplicationmatrix 311 11.4 Linearstructures 318 12 Matrixinequalities 321 12.1 Cauchy-Schwarztypeinequalities 322 12.2 Positive(semi)definitematrixinequalities 325 12.3 InequalitiesderivedfromtheSchurcomplement 341 12.4 Inequalitiesconcerningeigenvalues 343 13 Matrixcalculus 351 13.1 Basicpropertiesofdifferentials 355 13.2 Scalarfunctions 356 13.3 Vectorfunctions 360 13.4 Matrixfunctions 361 13.5 Theinverse 364 13.6 Exponentialandlogarithm 368 13.7 Thedeterminant 369