APPLIED TIME SERIES ANALYSIS WITH R Second Edition APPLIED TIME SERIES ANALYSIS WITH R Second Edition Wayne A. Woodward Southern Methodist University Dallas, Texas, USA Henry L. Gray Southern Methodist University Dallas, Texas, USA Alan C. Elliott Southern Methodist University Dallas, Texas, USA CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2017 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper Version Date: 20160726 International Standard Book Number-13: 978-1-4987-3422-6 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Woodward, Wayne A. | Gray, Henry L. | Elliott, Alan C., 1952- Title: Applied time series analysis, with R / Wayne A. Woodward, Henry L. Gray and Alan C. Elliott. Description: Second edition. | Boca Raton : Taylor & Francis, 2017. | “A CRC title.” Identifiers: LCCN 2016026902 | ISBN 9781498734226 (pbk.) Subjects: LCSH: Time-series analysis. | R (Computer program language) Classification: LCC QA280 .W68 2017 | DDC 519.5/502855133--dc23 LC record available at https://lccn.loc.gov/2016026902 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Contents Preface for Second Edition Acknowledgments 1 Stationary Time Series 1.1 Time Series 1.2 Stationary Time Series 1.3 Autocovariance and Autocorrelation Functions for Stationary Time Series 1.4 Estimation of the Mean, Autocovariance, and Autocorrelation for Stationary Time Series 1.4.1 Estimation of μ 1.4.1.1 Ergodicity of 1.4.1.2 Variance of 1.4.2 Estimation of γ k 1.4.3 Estimation of ρ k 1.5 Power Spectrum 1.6 Estimating the Power Spectrum and Spectral Density for Discrete Time Series 1.7 Time Series Examples 1.7.1 Simulated Data 1.7.2 Real Data Appendix 1A: Fourier Series Appendix 1B: R Commands Exercises 2 Linear Filters 2.1 Introduction to Linear Filters 2.1.1 Relationship between the Spectra of the Input and Output of a Linear Filter 2.2 Stationary General Linear Processes 2.2.1 Spectrum and Spectral Density for a General Linear Process 2.3 Wold Decomposition Theorem 2.4 Filtering Applications 2.4.1 Butterworth Filters Appendix 2A: Theorem Poofs Appendix 2B: R Commands Exercises 3 ARMA Time Series Models 3.1 MA Processes 3.1.1 MA(1) Model 3.1.2 MA(2) Model 3.2 AR Processes 3.2.1 Inverting the Operator 3.2.2 AR(1) Model 3.2.3 AR(p) Model for p ≥ 1 3.2.4 Autocorrelations of an AR(p) Model 3.2.5 Linear Difference Equations 3.2.6 Spectral Density of an AR(p) Model 3.2.7 AR(2) Model 3.2.7.1 Autocorrelations of an AR(2) Model 3.2.7.2 Spectral Density of an AR(2) 3.2.7.3 Stationary/Causal Region of an AR(2) 3.2.7.4 ψ-Weights of an AR(2) Model 3.2.8 Summary of AR(1) and AR(2) Behavior 3.2.9 AR(p) Model 3.2.10 AR(1) and AR(2) Building Blocks of an AR(p) Model 3.2.11 Factor Tables 3.2.12 Invertibility/Infinite-Order AR Processes 3.2.13 Two Reasons for Imposing Invertibility 3.3 ARMA Processes 3.3.1 Stationarity and Invertibility Conditions for an ARMA(p,q) Model 3.3.2 Spectral Density of an ARMA(p,q) Model 3.3.3 Factor Tables and ARMA(p,q) Models 3.3.4 Autocorrelations of an ARMA(p,q) Model 3.3.5 ψ-Weights of an ARMA(p,q) 3.3.6 Approximating ARMA(p,q) Processes Using High-Order AR(p) Models 3.4 Visualizing AR Components 3.5 Seasonal ARMA(p,q) × (P ,Q ) Models S S S 3.6 Generating Realizations from ARMA(p,q) Processes 3.6.1 MA(q) Model 3.6.2 AR(2) Model 3.6.3 General Procedure 3.7 Transformations 3.7.1 Memoryless Transformations 3.7.2 AR Transformations Appendix 3A: Proofs of Theorems Appendix 3B: R Commands Exercises 4 Other Stationary Time Series Models 4.1 Stationary Harmonic Models 4.1.1 Pure Harmonic Models 4.1.2 Harmonic Signal-Plus-Noise Models 4.1.3 ARMA Approximation to the Harmonic Signal-Plus-Noise Model 4.2 ARCH and GARCH Processes 4.2.1 ARCH Processes 4.2.1.1 The ARCH(1) Model 4.2.1.2 The ARCH(q ) Model 0 4.2.2 The GARCH(p , q ) Process 0 0 4.2.3 AR Processes with ARCH or GARCH Noise Appendix 4A: R Commands Exercises 5 Nonstationary Time Series Models 5.1 Deterministic Signal-Plus-Noise Models 5.1.1 Trend-Component Models 5.1.2 Harmonic Component Models 5.2 ARIMA(p,d,q) and ARUMA(p,d,q) Processes 5.2.1 Extended Autocorrelations of an ARUMA(p,d,q) Process 5.2.2 Cyclical Models 5.3 Multiplicative Seasonal ARUMA (p,d,q) × (P , D , Q ) Process s s s s 5.3.1 Factor Tables for Seasonal Models of the Form of Equation 5.17 with s = 4 and s = 12 5.4 Random Walk Models 5.4.1 Random Walk 5.4.2 Random Walk with Drift 5.5 G-Stationary Models for Data with Time-Varying Frequencies Appendix 5A: R Commands Exercises 6 Forecasting 6.1 Mean-Square Prediction Background 6.2 Box–Jenkins Forecasting for ARMA(p,q) Models 6.2.1 General Linear Process Form of the Best Forecast Equation 6.3 Properties of the Best Forecast 6.4 π-Weight Form of the Forecast Function 6.5 Forecasting Based on the Difference Equation 6.5.1 Difference Equation Form of the Best Forecast Equation 6.5.2 Basic Difference Equation Form for Calculating Forecasts from an ARMA(p,q) Model 6.6 Eventual Forecast Function 6.7 Assessing Forecast Performance 6.7.1 Probability Limits for Forecasts 6.7.2 Forecasting the Last k Values 6.8 Forecasts Using ARUMA(p,d,q) Models 6.9 Forecasts Using Multiplicative Seasonal ARUMA Models 6.10 Forecasts Based on Signal-Plus-Noise Models Appendix 6A: Proof of Projection Theorem Appendix 6B: Basic Forecasting Routines Exercises 7 Parameter Estimation 7.1 Introduction 7.2 Preliminary Estimates 7.2.1 Preliminary Estimates for AR(p) Models 7.2.1.1 Yule–Walker Estimates 7.2.1.2 Least Squares Estimation 7.2.1.3 Burg Estimates 7.2.2 Preliminary Estimates for MA(q) Models 7.2.2.1 MM Estimation for an MA(q) 7.2.2.2 MA(q) Estimation Using the Innovations Algorithm 7.2.3 Preliminary Estimates for ARMA(p,q) Models 7.2.3.1 Extended Yule–Walker Estimates of the AR Parameters 7.2.3.2 Tsay–Tiao Estimates of the AR Parameters 7.2.3.3 Estimating the MA Parameters 7.3 ML Estimation of ARMA(p,q) Parameters 7.3.1 Conditional and Unconditional ML Estimation 7.3.2 ML Estimation Using the Innovations Algorithm 7.4 Backcasting and Estimating 7.5 Asymptotic Properties of Estimators 7.5.1 AR Case 7.5.1.1 Confidence Intervals: AR Case 7.5.2 ARMA(p,q) Case 7.5.2.1 Confidence Intervals for ARMA(p,q) Parameters 7.5.3 Asymptotic Comparisons of Estimators for an MA(1) 7.6 Estimation Examples Using Data 7.7 ARMA Spectral Estimation 7.8 ARUMA Spectral Estimation Appendix Exercises 8 Model Identification 8.1 Preliminary Check for White Noise 8.2 Model Identification for Stationary ARMA Models 8.2.1 Model Identification Based on AIC and Related Measures 8.3 Model Identification for Nonstationary ARUMA(p,d,q) Models 8.3.1 Including a Nonstationary Factor in the Model 8.3.2 Identifying Nonstationary Component(s) in a Model 8.3.3 Decision Between a Stationary or a Nonstationary Model 8.3.4 Deriving a Final ARUMA Model 8.3.5 More on the Identification of Nonstationary Components 8.3.5.1 Including a Factor (1 − B)d in the Model 8.3.5.2 Testing for a Unit Root 8.3.5.3 Including a Seasonal Factor (1 − Bs) in the Model Appendix 8A: Model Identification Based on Pattern Recognition