1 Cover photo credit: Copyright© 1999 Dynamic Graphics, Inc. 8 This book is printed on acid-free paper. Copyrig19h 2t0 011,9 84by ACADEMIC PRESS AllR ightRse served. No parotf t hipsu blicamtaiyob ne r eproduocret dr ansmittinea dn yf ormo rb ya ny meanse,l ectroonrmi ecc hanicianlc,l udpihnogt ocorpeyc,o rdionrag n,y i nformation storaagned r etriesvyaslt ewmi,t hopuetr missiinow nr itifnrgo mt hep ublisher. Requesftosrp ermisstioom na kec opieosfa nyp arotf t hew orks houlbde m ailed to: PermissiDoenpsa rtmeHnatr,c ouIrntc .6,2 77S eaH arboDrr ive, OrlandFol,o ri3d2a8 87-6777 Academic Press A Harcourt Science and Technology Company 525B StreeStu,i t1e9 00S,a nD iegoC,a lifor9n2i1a0 1-44U9S5A, http://www.academicpress.com Academic Press HarcouPrlta c3e2, JamestRoowand L,o ndon NW7IB Y,U K http://www.acadsesm.iccopmr e LibraorfyC ongresCsa talCoagr dN umber0:0 -107735 InternatSitoananld aBrodo kN umber0:- 12-746652-5 PRINTEDI NT HE UNITEDS TATESO F AMERICA 00 01 02 03 04 05 QW 9 8 7 6 4 5 3 2I Contents Preface to the First Edition IX Preface to the Revised Edition Xl References . . . . . . . Xlll 1 The Linear Model and Instrumental Variables Estimators 1 References . . . . . . 12 For Further Reading 12 2 Consistency 15 2.1 Limits . . . . . . . . . . . 15 . 2.2 Almost Sure Convergence . 18 2.3 Convergence in Probability 24 2.4 Convergence in rth Mean 28 References . . . . 30 . . . Laws of Large Numbers 31 3 3.1 Independent Identically Distributed Observations 32 3.2 Independent Heterogeneously Distributed Observations . 35 3.3 Dependent Identically Distributed Observations . . . . 39 3.4 Dependent Heterogeneously Distributed Observations 46 3.5 Martingale Difference Sequences 53 References . . . . . . . . . . . . . . . . . . . . . . . 62 . . Vll Contents VIII Asymptotic Normality 4 65 4.1 Convergence in Distribution 65 4.2 Hypothesis Testing . . 74 4.3 Asymptotic Efficiency 83 References . . . . . . 111 Central Limit Theory 113 5 5.1 Independent Identically Distributed Observations . . . . 114 5.2 Independent Heterogeneously Distributed Observations . 117 5.3 Dependent Identically Distributed Observations . . . . 122 5.4 Dependent Heterogeneously Distributed Observations 130 5.5 Martingale Difference Sequences 133 References . . . . . . . . . . . . . . . . . . . 136 . Estimating Asymptotic Covariance Matrices 137 6 6.1 General Structure of V . 137 6.2 Case 1: {Ztct} Uncorrenla t.ed. .. .. .. . . . . . . . 139 6.3 Case 2: {Ztct} Finitely Correlated . . . . . . 147 6.4 Case 3: { Ztct} Asymptotically Uncorrelated . 154 References . . . . . . . . . . . . . . . . . . . . 164 7 Functional Central Limit Theory and Applications 167 7.1 Random Walks and Wiener Processes 167 7.2 Weak Convergence . . . . . . . . . . 171 7.3 Functional Central Limit Theorems . . 175 7.4 Regression with a Unit Root . . . . . 178 7.5 Spurious Regression and Multivariate FCLTs 184 7.6 Cointegration and Stochastic Integrals 190 . . . . . . . . 204 References . . . . . . . . 8 Directions for FUrther Study 207 8.1 Extending the Data Generating Process 207 8.2 Nonlinear Models . . . . . . . 209 8.3 Other Estimation Techniques 209 8.4 211 Model Misspecification References . . . . 211 . . . Solution Set 213 References . 259 Index 261 Preface to the First Edition Within the framework of the classical linear model it is a fairly straight­ forward matter to establish the properties of the ordinary least squares (OLS) and generalized least squares (GLS) estimators for samples of any size. Although the classical linear model is an excellent framework for de­ veloping a feel for the statistical techniques of estimation and inference that are central to econometrics, it is not particularly well adapted to the study of economic phenomena, because economists usually cannot conduct controlled experiments. Instead, the data usually exist as the outcome of a stochastic process outside the control of the investigator. For this rea­ son, both the dependent and the explanatory variables may be stochastic, and equation disturbances may exhibit nonnormality or heteroskedasticity and serial correlation of unknown form, so that the classical assumptions are violated. Over the years a variety of useful techniques has evolved to deal with these difficulties. Many of these amount to straightforward mod­ ifications or extensions of the OLS techniques (e.g., the Cochrane-Orcutt technique, two-stage least squares, and three-stage least squares). However, the finite sample properties of these statistics are rarely easy to establish outside of somewhat limited special cases. Instead, their usefulness is jus­ tified primarily on the basis of their properties in large samples, because these properties can be fairly easily established using the powerful tools provided by laws of large numbers and central limit theory. Despite the importance of large sample theory, it has usually received fairly cursory treatment in even the best econometrics textbooks. This is IX Preface to the First Edition X really no fault of the textbooks, however, because the field of asymptotic theory has been developing rapidly. It is only recently that econometricians have discovered or established methods for treating adequately and com­ prehensively the many different techniques available for dealing with the difficulties posed by economic data. This book is intended to provide a somewhat more comprehensive and unified treatment of large sample theory than has been available previ­ ously and to relate the fundamental tools of asymptotic theory directly to many of the estimators of interest to econometricians. In addition, because economic data are generated in a variety of different contexts time series, ( cross sections, time series-cross sections , we pay particular attention to the ) similarities and differences in the to of these techniques appropriate each contexts. That it is possible to present our results in a fairly unified manner high­ lights the similarities among a variety of different techniques. It also allows us in specific instances to establish results that are somewhat more gen­ eral than those previously available. We thus include some new results in addition to those that are better known. This book is intended for use both as a reference and as a text book for graduate students taking courses in econometrics beyond the introductory level. It is therefore assumed that the reader is familiar with the basic concepts of probability and statistics as well as with calculus and linear algebra and that the reader also has a good understanding of the classical linear model. Because our goal here is to deal primarily with asymptotic theory, we do not consider in detail the meaning and scope of econometric models per se. Therefore, the material in this book can be usefully supplemented by standard econometrics texts, particularly any of those listed at the end of Chapter 1. I would like to express my appreciation to all those who have helped in the evolution of this work. In particular, I would like to thank Charles Bates, Ian Domowitz, Rob Engle, Clive Granger, Lars Hansen, David Hendry, and Murray Rosenblatt. Particular thanks are due Jeff Wooldridge for his work in producing the solution set for the exercises. I also thank the stu­ dents in UCSD, various graduate classes at who have served unwitting as and indispensable guinea pigs in the development of this material. I am deeply grateful to Annetta Whiteman, who typed this difficult manuscript with incredible swiftness and accuracy. Finally, I would like to thank the National Science Foundation for providing financial support for this work under grant SESSl-07552. Preface to the Revised Edition It is a gratifying experience to be asked to revise and update a book written over fifteen years previously. Certainly, this request would be unnecessary had the book not exhibited an unusual tenacity in serving its purpose. Such tenacity had been my fond hope for this book, and it is always gratifying to see fond hopes realized. It is also humbling and occasionally embarrassing to perform such a revision. Certain errors and omissions become painfully obvious. Thoughts of "How could I have thought that?" or "How could I have done that?" arise with regularity. Nevertheless, the opportunity is at hand to put things right, and it is satisfying to believe that one has succeeded in this. I know, ( of course, that errors still lurk, but I hope that this time they are more benign or buried more deeply, or preferably both. ) Thus, the reader of this edition will find numerous instances where defini­ tions have been corrected or clarified and where statements of results have been corrected or made more precise or complete. The exposition, too, has been polished in the hope of aiding clarity. Not only is a revision of this sort an opportunity to prior shortcom­ fix ings, but it is also an opportunity to bring the material covered up-to-date. In retrospect, the first edition of this book was more ambitious than origi­ nally intended. The fundamental research necessary to achieve the intended scope and cohesiveness of the overall vision for the work was by no means complete at the time the first edition was written. For example, the central limit theory for heterogeneous mixing processes had still not developed to XI Preface to the Revised Edition XII the desired point at that time, nor had the theories of optimal instrumental variables estimation or asymptotic covariance estimation. Indeed, the attempt made in writing the first edition to achieve its in­ tended scope and coherence revealed a host of areas where work was needed, thus providing fuel for a great deal of my own research and I like to think ( ) that of In n ven ng, of the re­ others. the years i ter i the efforts econometrics search community have succeeded wonderfully in delivering results in the areas needed and much more. Thus, the ambitions not realized in the first edition can now be achieved. If the theoretical vision presented here has not achieved a much better degree of unity, it can no longer be attributed to a lack of development of the field, but is now clearly identifiable as the author's own responsibility. As a result of these developments, the reader of this second edition will now find much updated material, particularly with regard to central limit theory, asymptotically efficient instrumental variables estimation, and esti­ mation of asymptotic covariance matrices. In particular, the original Chap­ ter 7 concerning efficient estimation with estimated error covariance ma­ ( trices and an entire section of Chapter 4 concerning efficient IV estimation ) have been removed and replaced with much more accessible and coherent results on efficient IV estimation, now appearing in Chapter 4. There is also the progress of the field to contend with. When the first edition was written, cointegration was a subject in its infancy, and the tools needed to study the asymptotic behavior of estimators for models of cointegrated processes were years away from fruition. Indeed, results of De­ Jong and Davidson essential to placing estimation for cointegrated (2000) processes cohesively in place with the theory contained in the first six chap­ ters of this book became available only months before work on this edition began. Consequently, this second edition contains a completely new Chapter 7 devoted to functional central limit theory and its applications, specifically unit root regression, spurious regression, and regression with cointegrated processes. Given the explosive growth in this area, we cannot here achieve a broad treatment of cointegration. Nevertheless, in the new Chapter 7 the reader should find all the basic tools necessary for entree into this fascinating area. The comments, suggestions, and influence of numerous colleagues over the years have had effects both subtle and patent on the material pre­ sented here. With sincere apologies to anyone inadvertently omitted, I ac­ knowledge with keen appreciation the direct and indirect contributions to the present state of this book by Takeshi Amemiya, Donald W. K. An­ drews, Charles Bates, Herman Bierens, James Davidson, Robert DeJong, References X ill Ian Domowitz, Graham Elliott, Robert Engle, A. Ronald Gallant, Arthur Goldberger, Clive W. J. Granger, James Hamilton, Bruce Hansen, Lars Hansen, Jerry Hausman, David Hendry, S0ren Johansen, Edward Leamer, James Mackinnon, Whitney Newey, Peter C. B. Phillips, Eugene Savin, Chris Sims, Maxwell Stinchcombe, James Stock, Mark Watson, Kenneth West, and Jeffrey Wooldridge. Special thanks are due Mark Salmon, who originally suggested writing this book. UCSD graduate students who helped with the revision include Jin Seo Cho, Raffaella Giacomini, Andrew Pat­ ton, Sivan Ritz, Kevin Sheppard, Liangjun Su, and Nada Wasi. I also thank sincerely Peter Reinhard Hansen, who has assisted invaluably with the cre­ ation of this revised edition, acting as electronic amanuensis and editor, and who is responsible for preparation of the revised set of solutions to the exercises. Finally, I thank Michael J. Bacci for his invaluable logistical sup­ port and the National Science Foundation for providing financial support under grant SBR-9811562. Del Mar, CA July, 2000 References DeJong R. M. and J. Davidson (2000). "The functional central limit theorem and weak convegence to stochastic integrals I: Weakly dependent processes," forthcoming in Econometric Theory, 16. CHAPTER 1 The Linear Model and Instrumental Variables Estimators The purpose of this book is to provide the reader with the tools and con­ cepts needed to study the behavior of econometric estimators and test statistics in large samples. Throughout, attention will be directed to esti­ mation and inference in the framework of a linear stochastic relationship such as X�/30 + t t 1, . . . ,n, = lt t: , = where we have observations on the scalar dependent variable yt and n the vector of explanatory variables Xt Xt , Xt , ... , Xt The scalar ( 1 2 k ) = '. stochastic disturbance is unobserved, and {30 is an unknown k 1 vector Et x of coefficients that we are interested in learning about, either through esti­ mation or through hypothesis testing. In matrix notation this relationship is written as X{30 +E, Y = where Y is ann 1 vector, X is ann k matrix with rows X�, and E is x x an 1 vector with elements Et· n x Our notation embodies a convention we follow throughout: scalars will ( be represented in standard type, while vectors and matrices will be repre­ sented in boldface. Throughout, all vectors are column vectors. ) Most econometric estimators can be viewed as solutions to an optimiza­ tion problem. For example, the ordinary least squares estimator is the value 1