The R Series Statistics B a Basics of Matrix Algebra for Statistics with R provides a guide to elementary s matrix algebra sufficient for undertaking specialized courses, such as multivariate i c Basics of data analysis and linear models. It also covers advanced topics, such as general- s ized inverses of singular and rectangular matrices and manipulation of partitioned o matrices, if you would like to delve deeper into the subject. f Matrix Algebra The book introduces the definition of a matrix and the basic rules of addition, sub- M traction, multiplication, and inversion. Later topics include determinants, calculation a of eigenvectors and eigenvalues, and differentiation of linear and quadratic forms t r with respect to vectors. The text explores how these concepts arise in statistical i for Statistics x techniques, including principal component analysis, canonical correlation analysis, A and linear modeling. l g In addition to the algebraic manipulation of matrices, the book presents numerical with R e examples that illustrate how to perform calculations by hand and using R. Many b theoretical and numerical exercises of varying levels of difficulty aid you in assess- r a ing your knowledge of the material. f Features o • Covers basic algebraic manipulation of matrices, such as basic arithmetic, r S inversion, partitioning, rank, determinants, decompositions, eigenanalysis, and t Hadamard and Kronecker products a • Shows how to implement the techniques in R using worked numerical t i examples s • Describes vector and matrix calculus, including differentiation of scalars and t i linear and quadratic forms c s • Incorporates useful tricks, such as identifying rank 1 matrices and scalar subfactors within products w • Explains how to convert an optimization problem to an eigenanalysis by i t imposing a non-restrictive constraint h • Presents the derivation of key results in linear models and multivariate R methods with step-by-step cross-referenced explanations Avoiding vector spaces and other advanced mathematics, this book shows how to manipulate matrices and perform numerical calculations in R. It prepares you for higher-level and specialized studies in statistics. F i e l l e Nick Fieller r K25114 www.crcpress.com Basics of Matrix Algebra for Statistics with R Chapman & Hall/CRC The R Series Series Editors John M. Chambers Torsten Hothorn Department of Statistics Division of Biostatistics Stanford University University of Zurich Stanford, California, USA Switzerland Duncan Temple Lang Hadley Wickham Department of Statistics RStudio University of California, Davis Boston, Massachusetts, USA Davis, California, USA Aims and Scope This book series reflects the recent rapid growth in the development and application of R, the programming language and software environment for statistical computing and graphics. R is now widely used in academic research, education, and industry. It is constantly growing, with new versions of the core software released regularly and more than 6,000 packages available. It is difficult for the documentation to keep pace with the expansion of the software, and this vital book series provides a forum for the publication of books covering many aspects of the development and application of R. The scope of the series is wide, covering three main threads: • Applications of R to specific disciplines such as biology, epidemiology, genetics, engineering, finance, and the social sciences. • Using R for the study of topics of statistical methodology, such as linear and mixed modeling, time series, Bayesian methods, and missing data. • The development of R, including programming, building packages, and graphics. The books will appeal to programmers and developers of R software, as well as applied statisticians and data analysts in many fields. The books will feature detailed worked examples and R code fully integrated into the text, ensuring their usefulness to researchers, practitioners and students. Published Titles Stated Preference Methods Using R, Hideo Aizaki, Tomoaki Nakatani, and Kazuo Sato Using R for Numerical Analysis in Science and Engineering, Victor A. Bloomfield Event History Analysis with R, Göran Broström Computational Actuarial Science with R, Arthur Charpentier Statistical Computing in C++ and R, Randall L. Eubank and Ana Kupresanin Basics of Matrix Algebra for Statistics with R, Nick Fieller Reproducible Research with R and RStudio, Second Edition, Christopher Gandrud R and MATLAB®David E. Hiebeler Nonparametric Statistical Methods Using R, John Kloke and Joseph McKean Displaying Time Series, Spatial, and Space-Time Data with R, Oscar Perpiñán Lamigueiro Programming Graphical User Interfaces with R, Michael F. Lawrence and John Verzani Analyzing Sensory Data with R, Sébastien Lê and Theirry Worch Parallel Computing for Data Science: With Examples in R, C++ and CUDA, Norman Matloff Analyzing Baseball Data with R, Max Marchi and Jim Albert Growth Curve Analysis and Visualization Using R, Daniel Mirman R Graphics, Second Edition, Paul Murrell Data Science in R: A Case Studies Approach to Computational Reasoning and Problem Solving, Deborah Nolan and Duncan Temple Lang Multiple Factor Analysis by Example Using R, Jérôme Pagès Customer and Business Analytics: Applied Data Mining for Business Decision Making Using R, Daniel S. Putler and Robert E. Krider Implementing Reproducible Research, Victoria Stodden, Friedrich Leisch, and Roger D. Peng Graphical Data Analysis with R, Antony Unwin Using R for Introductory Statistics, Second Edition, John Verzani Advanced R, Hadley Wickham Dynamic Documents with R and knitr, Second Edition, Yihui Xie Basics of Matrix Algebra for Statistics with R Nick Fieller University of Sheffield, UK CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2016 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20150602 International Standard Book Number-13: 978-1-4987-1238-5 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information stor- age or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copy- right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro- vides licenses and registration for a variety of users. For organizations that have been granted a photo- copy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To Rafe and Hal Contents Preface xvii 1 Introduction 1 1.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 FurtherReading . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Matrixalgebra . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 ElementaryR . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 GuidetoNotation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 AnOutlineGuidetoR . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4.1 WhatisR? . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4.2 InstallingR . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4.3 Risaninteractiveprogram . . . . . . . . . . . . . . . . . . 5 1.4.4 ObtaininghelpinR . . . . . . . . . . . . . . . . . . . . . 6 1.4.5 Risafunctionlanguage . . . . . . . . . . . . . . . . . . . 7 1.4.6 Risanobject-orientedlanguage . . . . . . . . . . . . . . . 7 1.4.6.1 Theclassofanobject . . . . . . . . . . . . . . . 8 1.4.7 Savingobjectsandworkspaces . . . . . . . . . . . . . . . . 8 1.4.7.1 Mistakenlysavedworkspace . . . . . . . . . . . 9 1.4.8 Risacase-sensitivelanguage . . . . . . . . . . . . . . . . 9 1.5 InputtingDatatoR . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5.1 Readingdatafromthekeyboard . . . . . . . . . . . . . . . 9 1.5.2 Readingdatafromfiles . . . . . . . . . . . . . . . . . . . . 10 1.6 SummaryofMatrixOperatorsinR . . . . . . . . . . . . . . . . . 10 1.7 ExamplesofRCommands . . . . . . . . . . . . . . . . . . . . . . 12 1.7.1 Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.7.2 Inputtingdata . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.7.3 Calculations . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.7.4 Dimensionsandlengthsofmatricesofvectors . . . . . . . 14 1.7.5 Joiningmatricestogether . . . . . . . . . . . . . . . . . . . 15 1.7.6 Diagonalsandtrace . . . . . . . . . . . . . . . . . . . . . . 16 1.7.7 Traceofproducts . . . . . . . . . . . . . . . . . . . . . . . 16 1.7.8 Transposeofproducts . . . . . . . . . . . . . . . . . . . . 16 1.7.9 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.7.10 Diagonalmatrices . . . . . . . . . . . . . . . . . . . . . . 17 1.7.11 Inverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 ix x Contents 1.7.12 Eigenanalyses. . . . . . . . . . . . . . . . . . . . . . . . . 18 1.7.13 Singularvaluedecomposition . . . . . . . . . . . . . . . . 18 1.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2 VectorsandMatrices 21 2.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Example2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.2 CreatingvectorsinR . . . . . . . . . . . . . . . . . . . . 22 2.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3 MatrixArithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.3.1 Matrixmultiplication . . . . . . . . . . . . . . . . . . . . . 25 Example2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.3.2 Example2.2inR . . . . . . . . . . . . . . . . . . . . . . 27 2.4 TransposeandTraceofSumsandProducts . . . . . . . . . . . . . 29 2.5 SpecialMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5.1 Symmetricandskew-symmetricmatrices . . . . . . . . . . 29 2.5.2 ProductswithtransposeAA(cid:48) . . . . . . . . . . . . . . . . . 30 2.5.3 Orthogonalmatrices . . . . . . . . . . . . . . . . . . . . . 30 Example2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.5.4 Normalmatrices . . . . . . . . . . . . . . . . . . . . . . . 31 2.5.5 Permutationmatrices . . . . . . . . . . . . . . . . . . . . . 31 Example2.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.5.6 Idempotentmatrices . . . . . . . . . . . . . . . . . . . . . 31 2.5.6.1 ThecenteringmatrixH . . . . . . . . . . . . . . 32 n 2.5.7 Nilpotentmatrices . . . . . . . . . . . . . . . . . . . . . . 32 2.5.8 Unipotentmatrices . . . . . . . . . . . . . . . . . . . . . . 32 2.5.9 Similarmatrices . . . . . . . . . . . . . . . . . . . . . . . 32 2.6 PartitionedMatrices . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6.1 Sub-matrices . . . . . . . . . . . . . . . . . . . . . . . . . 32 Example2.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.6.2 Manipulationofpartitionedmatrices . . . . . . . . . . . . . 33 Example2.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.6.3 ImplementationofpartitionedmatricesinR . . . . . . . . 34 2.7 AlgebraicManipulationofMatrices . . . . . . . . . . . . . . . . . 35 2.7.1 Generalexpansionofproducts . . . . . . . . . . . . . . . . 35 2.8 UsefulTricks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.8.1 Trackingdimensionsand1×1matrices . . . . . . . . . . . 35 2.8.2 Traceofproducts . . . . . . . . . . . . . . . . . . . . . . . 36 2.9 LinearandQuadraticForms . . . . . . . . . . . . . . . . . . . . . 36 Example2.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.10 CreatingMatricesinR . . . . . . . . . . . . . . . . . . . . . . . . 37 Example2.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38