Introduction to Time Series Analysis and Forecasting WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A. SHEWHART and SAMUELS. WILKS Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fit;:maurice, lain M. Johnstone, Geert Molenberghs, David W Scott. Adrian F M. Smith. Ruey S. Tsay, Sanford Weisberg Editors Emeriti: Vic Barnett, 1. Stuart Hunter. Dm·id G. Kendall. Jo::efL. Teugels A complete list of the titles in this series appears at the end of this volume. Introduction to Time Series Analysis and Forecasting DOUGLAS C. MONTGOMERY Arizona State University CHERYL L. JENNINGS Bank of America MURAT KULAHCI Technical University of Denmark ffi w WILEY INTERSCIENCE A JOHN WILEY &. SONS, INC., PUBLICATION Copyright © 2008 by John Wiley & Sons. Inc. All rights reserwd. Published by John Wiley & Sons. Inc .. Hoboken. New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced. stored in a retrieval system. or transmitted in any form or by any means, electronic, mechanical, photocopying. recording. scanning. or otherwise. except as permitted under Section 107 or 108 of the 1976 United States Copyright Act. without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center. Inc .. 222 Rosewood Drive. Danvers. MA 01923.978-750-8400. fax 978-750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department. John Wiley & Sons. Inc.. Ill River Street. Hoboken. NJ 07030, 201-748-60 II, fax 201-748-6008. or online at http://www. wiley.com/go/permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accurac) or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Jl;either the publisher nor author shall be liable for any loss of profit or any other commercial damages. including but not limited to special, incidental, consequential. or other damages. For general information on our other products and services or for technical support. please contact our Customer Care Department within the United States at 877-762-2974. outside the United States at 317-572-3993 or fax 317-572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products. visit our web site at www. wiley.com. Library of Congress Cataloging-in-Publication Data: Montgomery. Douglas C. Introduction to time series analysis and forecasting I Douglas C. Montgomery. Cheryl L. Jennings, Murat Kulahci. p. em. - (Wiley series in probability and statistics) Includes bibliographical references and index. ISBN 978-0-4 71-65397-4 (cloth) I. Time-series analysis. 2. Forecasting. I. Jennings. Cheryl L. II. Kulahci. Murat. III. Title. QA280.M662 2007 519.5'5-dc22 2007019891 Printed in the United States of America 10 9 8 7 6 5 4 3 2 I Contents Preface ix 1. Introduction to Forecasting I 1.1 The Nature and Uses of Forecasts, 1.2 Some Examples of Time Series, 5 1.3 The Forecasting Process, 12 1.4 Resources for Forecasting, 14 Exercises, 15 2. Statistics Background for Forecasting 18 2.1 Introduction, 18 2.2 Graphical Displays, 19 2.2.1 Time Series Plots, 19 2.2.2 Plotting Smoothed Data, 22 2.3 Numerical Description of Time Series Data, 25 2.3.1 Stationary Time Series, 25 2.3.2 Autocovariance and Autocorrelation Functions, 28 2.4 Use of Data Transformations and Adjustments, 34 2.4.1 Transformations, 34 2.4.2 Trend and Seasonal Adjustments, 36 2.5 General Approach to Time Series Modeling and Forecasting, 46 2.6 Evaluating and Monitoring Forecasting Model Performance, 49 2.6.1 Forecasting Model Evaluation, 49 2.6.2 Choosing Between Competing Models, 57 2.6.3 Monitoring a Forecasting Model, 60 Exercises, 66 v vi CO!-ITENTS 3. Regression Analysis and Forecasting 73 3.1 Introduction, 73 3.2 Least Squares Estimation in Linear Regression Models. 75 3.3 Statistical Inference in Linear Regression. 84 3.3.1 Test for Significance of Regression. 84 3.3.2 Tests on Individual Regression Coefficients and Groups of Coefficients, 87 3.3.3 Confidence Intervals on Individual Regression Coefficients. 93 3.3.4 Confidence Intervals on the Mean Response. 94 3.4 Prediction of New Observations. 96 3.5 Model Adequacy Checking, 98 3.5.1 Residual Plots, 98 3.5.2 Scaled Residuals and PRESS. 100 3.5.3 Measures of Leverage and Influence, 105 3.6 Variable Selection Methods in Regression, 106 3.7 Generalized and Weighted Least Squares. 111 3. 7.1 Generalized Least Squares. 112 3.7.2 Weighted Least Squares, 114 3.7.3 Discounted Least Squares. 119 3.8 Regression Models for General Time Series Data, 133 3.8.1 Detecting Autocorrelation: The Durbin-Watson Test. 134 3.8.2 Estimating the Parameters in Time Series Regression Models, 139 Exercises, 161 4. Exponential Smoothing Methods 171 4.1 Introduction, 171 4.2 First-Order Exponential Smoothing, 176 4.2.1 The Initial Value, _\·0• 177 4.2.2 The Value of A, 178 4.3 Modeling Time Series Data, 180 4.4 Second-Order Exponential Smoothing. 183 4.5 Higher-Order Exponential Smoothing. 193 4.6 Forecasting, 193 4.6.1 Constant Process. 193 4.6.2 Linear Trend Process. 198 4.6.3 Estimation of a}. 207 vii CONTENTS 4.6.4 Adaptive Updating of the Discount Factor, 208 4.6.5 Model Assessment, 209 4.7 Exponential Smoothing for Seasonal Data, 210 4.7 .I Additive Seasonal Model, 210 4.7.2 Multiplicative Seasonal Model, 214 4.8 Exponential Smoothers and ARIMA Models, 217 Exercises, 220 5. Autoregressive Integrated Moving Average (ARIMA) Models 231 5.1 Introduction, 231 5.2 Linear Models for Stationary Time Series, 231 5.2.1 Stationarity, 232 5.2.2 Stationary Time Series, 233 5.3 Finite Order Moving Average (MA) Processes, 235 5.3.1 The First-Order Moving Average Process, MA(l ), 236 5.3.2 The Second-Order Moving Average Process, MA(2), 238 5.4 Finite Order Autoregressive Processes, 239 5 .4.1 First -Order Autoregressive Process, AR(l ), 240 5.4.2 Second-Order Autoregressive Process, AR(2), 242 5.4.3 General Autoregressive Process, AR(p), 246 5.4.4 Partial Autocorrelation Function, PACF, 248 5.5 Mixed Autoregressive-Moving Average CARMA) Processes, 253 5.6 Nonstationary Processes, 256 5.7 Time Series Model Building, 265 5. 7. 1 Model Identification, 265 5.7.2 Parameter Estimation, 266 5.7.3 Diagnostic Checking, 266 5.7.4 Examples of Building ARIMA Models, 267 5.8 Forecasting ARIMA Processes, 275 5.9 Seasonal Processes, 282 5.10 Final Comments, 286 Exercises, 287 6. Transfer Functions and Intervention Models 299 6.1 Introduction, 299 6.2 Transfer Function Models, 300 6.3 Transfer Function-Noise Models, 307 viii CONTENTS 6.4 Cross Correlation Function, 307 6.5 Model Specification. 309 6.6 Forecasting with Transfer Function-Noise Models. 322 6. 7 Intervention Analysis, 330 Exercises. 338 7. Survey of Other Forecasting Methods 343 7.1 Multivariate Time Series Models and Forecasting. 343 7 .1.1 Multivariate Stationary Process. 343 7.1.2 Vector ARIMA Models. 344 7.1.3 Vector AR (VAR) Models. 346 7.2 State Space Models. 350 7.3 ARCH and GARCH Models. 355 7.4 Direct Forecasting of Percentiles. 359 7.5 Combining Forecasts to Improve Prediction Performance. 365 7.6 Aggregation and Disaggregation of Forecasts, 369 7.7 Neural Networks and Forecasting. 372 7.8 Some Comments on Practical Implementation and Use of Statistical Forecasting Procedures. 375 Exercises, 378 Appendix A. Statistical Tables 387 Appendix B. Data Sets for Exercises 407 Bibliography 437 Index 443 Preface Analyzing time-oriented data and forecasting future values of a time series are among the most important problems that analysts face in many fields, ranging from finance and economics, to managing production operations, to the analysis of political and social policy sessions, to investigating the impact of humans and the policy decisions that they make on the environment. Consequently, there is a large group of people in a variety of fields including finance, economics, science, engineering, statistics, and public policy who need to understand some basic concepts of time series analysis and forecasting. Unfortunately, most basic statistics and operations management books give little if any attention to time-oriented data, and little guidance on forecasting. There are some very good high level books on time series analysis. These books are mostly written for technical specialists who are taking a doctoral-level course or doing research in the field. They tend to be very theoretical and often focus on a few specific topics or techniques. We have written this book to fill the gap between these two extremes. This book is intended for practitioners who make real-world forecasts. Our focus is on short- to medium-term forecasting where statistical methods are useful. Since many organizations can improve their effectiveness and business results by making better short-to medium-term forecasts, this book should be useful to a wide variety of professionals. The book can also be used as a textbook for an applied forecasting and time series analysis course at the advanced undergraduate or first-year graduate level. Students in this course could come from engineering, business, statistics, operations research, mathematics, computer science, and any area of application where making forecasts is important. Readers need a background in basic statistics (previous ex posure to linear regression would be helpful but not essential), and some knowledge of matrix algebra, although matrices appear mostly in the chapter on regression, and if one is interested mainly in the results, the details involving matrix manipulation can be skipped. Integrals and derivatives appear in a few places in the book, but no detailed working knowledge of calculus is required. Successful time series analysis and forecasting requires that the analyst interact with computer software. The techniques and algorithms are just not suitable to manual calculations. We have chosen to demonstrate the techniques presented using three ix