OONNSS MMeetthhooddoollooggyy aanndd SSttaattiissttiiccaall DDeevveellooppmmeenntt Guide to Seasonal Adjustment with X-12-ARIMA **DRAFT** TSAB March 2007 Guide to Seasonal Adjustment with X12ARIMA TABLE OF CONTENTS 1. Introduction .....................................................................................................................1.1 2. Introduction to seasonal adjustment .......................................................................2.1 3. Overview of the X12ARIMA Method ..........................................................................3.1 4. How to run X12ARIMA ..................................................................................................4.1 7. Length of the series ......................................................................................................7.1 8. Consistency across time .............................................................................................8.1 9. Aggregate Series ...........................................................................................................9.1 10 Revisions and updates ..............................................................................................10.1 11. The regARIMA model .................................................................................................11.1 12. Trading day ..................................................................................................................12.1 13. Easter .............................................................................................................................13.1 14. Level Shift and Additive Outliers ...........................................................................14.1 15. Which Seasonal Decomposition Model to Use ..................................................15.1 16. Moving Averages ........................................................................................................16.1 17. Seasonal Breaks .........................................................................................................17.1 18. Existence of Seasonality ..........................................................................................18.1 19. X12ARIMA Standard Output ....................................................................................19.1 20. Graphs and X-12-graph .............................................................................................20.1 21. Sliding Spans ...............................................................................................................21.1 22. History Diagnostics......................................................................................................22.1 23. Composite spec ..........................................................................................................23.1 Table of contents 25. Forecasting ...................................................................................................................25.1 26. Trend Estimation .........................................................................................................26.1 27. Quality Measures .........................................................................................................27.1 Guide to Seasonal Adjustment with X-12-ARIMA 1 INTRODUCTION 1.1 Introduction Seasonal adjustment is widely used in official statistics as a technique for enabling timely interpretation of time series data. The X-12-ARIMA seasonal adjustment package has been chosen from the many available seasonal adjustment methods as the standard one for use in official statistics in the United Kingdom (UK). This was agreed by the National Statistics Quality and Methods Programme Board in 2001, and is in line with European best practice and consistent with the Bank of England. The method is used by most of the leading national statistical institutes across the world. X-12-ARIMA is developed by the United States (US) Bureau of the Census. The software is comprehensive, with many options available for tailoring seasonal adjustment to each individual series. It therefore requires many choices to be made by its users. The main purpose of this guide is to provide practical guidance on seasonal adjustment using X-12- ARIMA. The guide explains what seasonal adjustment means, and addresses many of the statistical, organisational and presentational issues and problems associated with seasonal adjustment. This guide complements the Office for National Statistics (ONS) training courses in seasonal adjustment. A detailed explanation of the X-12-ARIMA method can be found at www.census.gov/srd/www/x12a or in Ladiray and Quenneville (2001). 1.2 What is in this guide The guide starts with a brief introduction to seasonal adjustment and to some of the main associated issues (Chapter 2). It then provides an overview of the X-12-ARIMA method (Chapter 3), and a description of how to run the program (Chapter 4). Chapter 5 describes a procedure that should be used when seasonally adjusting a series using X-12-ARIMA, while the prior considerations that users should address when seasonally adjusting a series are discussed in Chapters 6 to 10. The second part of the guide examines seasonal adjustment in more detail. Here the user will find more detailed descriptions of the regression-ARIMA model, of potential effects such as trading days, and of how X-12-ARIMA deals with them (Chapters 11 to 14). Guidance is also provided on selecting a decomposition model, moving averages, prior adjustments, and on problems such as seasonal breaks (Chapters 15 to 19). The third part of this guide deals with X-12-ARIMA output and with diagnostics or tools that may be useful in seasonal adjustment (Chapters 20 to 25). Chapters 26 to 28 look at alternative uses, other than seasonal adjustment, of the X-12-ARIMA software. 1.3 How to use this guide This guide is not intended to be read from cover to cover. Users are advised to read Part 1 before attempting any seasonal adjustment, and to refer to other parts of the guide as appropriate. Anyone who is responsible for setting up or reviewing seasonal adjustment should attend a training course aimed at seasonal adjustment practitioners. This guide complements, but is not a substitute for, such a training course. ONS provides short training courses for users of seasonal adjustment, and for 27/02/07 1-1 Introduction persons responsible for presenting data in seasonally adjusted form. More information on training courses can be obtained from Time Series Analysis Branch (TSAB). 1.4 What this guide is not This guide does not describe the detailed working of X-12-ARIMA, nor the underlying mathematics. For those who are interested in these matters, the X-12-ARIMA user manual is a good starting point. Additional technical references and papers are listed in the References below. 1.5 What does TSAB do? The Methodology Directorate (MD) provides leadership, and defines best practice for many methodological aspects of ONS work. MD also provides methodological advice to other parts of ONS and, where resources permit, to the rest of the Government Statistical Service (GSS), on methodological issues. TSAB provides expertise in time series analysis issues, and in particular seasonal adjustment. TSAB has produced this guide, and runs regular courses in seasonal adjustment. TSAB is responsible for the quality of all ONS seasonal adjustment, and manages a rolling programme of annual seasonal adjustment reviews of statistical outputs across ONS. Furthermore, TSAB provides support for practitioners and users of seasonal adjustment, and should be the main point of contact for any questions or queries regarding seasonal adjustment. 1.6 Contact details If you have a query related to time series analysis, if you require information on seasonal adjustment training, or if you have any comments on this guidance, we can be contacted at: Office for National Statistics, Time Series Analysis Branch, 1 Drummond Gate, London SW1V 2QQ or by email at: [email protected]. 1.7 Acknowledgements This guide has been developed with input and assistance from the following people: Claudia Annoni, Simon Compton, Duncan Elliott, Lee Howells, Fida Hussain, Peter Kenny, Jim Macey, Craig McLaren, Ross Meader, Nigel Stuttard, Anthony Szary, and Anita Visavadia. Please bring any errors or omissions to our attention. 1-2 27/02/07 Guide to Seasonal Adjustment with X-12-ARIMA 2 INTRODUCTION TO SEASONAL ADJUSTMENT 2.1 What is seasonal adjustment? Data that are collected over time form a time series. Many of the most well known statistics published by the Office for National Statistics are regular time series, including: the claimant count, the Retail Prices Index (RPI), Balance of Payments, and Gross Domestic Product (GDP). Those analysing time series typically seek to establish the general pattern of the data, the long term movements, and whether any unusual occurrences have had major effects on the series. This type of analysis is not straightforward when one is reliant on raw time series data, because there will normally be short-term effects, associated with the time of the year, which obscure or confound other movements. For example, retail sales rise each December due to Christmas. The purpose of seasonal adjustment is to remove systematic calendar related variation associated with the time of the year, i.e. seasonal effects. This facilitates comparisons between consecutive time periods. Figure 2-1 - Non-seasonally adjusted (NSA) and seasonally adjusted (SA) data for United Kingdom visits abroad UK visits abroad 8600 7600 6600 5600 NSA 4600 SA 3600 2600 1600 600 1994 Jan 1995 Jan 1996 Jan 1997 Jan 1998 Jan 1999 Jan 2000 Jan 2001 Jan 2002 Jan 2003 Jan 2.2 Components of a time series Time series can be thought of as combinations of three broad and distinctly different types of behaviour, each representing the impact of certain types of real world events on the data. These three components are: systematic calendar-related effects, irregular fluctuations, and trend behaviour. Systematic calendar related effects comprise seasonal effects and calendar effects. Seasonal effects are cyclical patterns that may evolve as the result of changes associated with the seasons. They may be caused by various factors, such as: (cid:131) Weather patterns: for example, the increase in energy consumption with the onset of winter; (cid:131) Administrative measures: for example, the start and end dates of the school year; 27/02/07 2-1 Introduction to seasonal adjustment (cid:131) Social / cultural / religious events: for example, retail sales increasing in the run up to Christmas; (cid:131) Variation in the length of months and quarters due to the nature of the calendar. Other calendar effects relate to factors which do not necessarily occur in the same month (or quarter) each year. They include: (cid:131) Trading day effects which are caused by months having differing numbers of each day of the week from year to year: for example, spending in hardware stores is likely to be higher in a month with five, rather than four, weekends; (cid:131) Moving holidays, which may fall in different months from year to year: for example Easter, which can occur in either March or April. Taken together these effects make up the seasonal component. Irregular fluctuations may occur due to a combination of unpredictable or unexpected factors, such as: sampling error, non-sampling error, unseasonable weather, natural disasters, or strikes. While every member of the population is affected by general economic or social conditions, each is affected somewhat differently, so there will always be some degree of random variation in a time series. The contribution of the irregular fluctuations will generally change in direction and/or magnitude from period to period. This is in marked contrast with the regular behaviour of the seasonal effects. The trend (or trend cycle) represents the underlying behaviour and direction of the series. It captures the long-term behaviour of the series as well as the various medium-term business cycles. 2.3 The seasonal adjustment process Although there are many ways in which these components could fit together in a time series, we select one of two models: (cid:131) Additive model : Y = C + S + I (cid:131) Multiplicative model: Y = C x S x I where Y is the original series, C is the trend-cycle, S is the seasonal component, and I is the irregular component. The seasonally adjusted series is formed by estimating and removing the seasonal component. (cid:131) For the additive model : Seasonally adjusted series =Y – S = C +I (cid:131) For the multiplicative model: Seasonally adjusted series = Y / S = C x I. In a multiplicative decomposition, the seasonal effects change proportionately with the trend. If the trend rises, so do the seasonal effects, while if the trend moves downward the seasonal effects diminish too. In an additive decomposition the seasonal effects remain broadly constant, no matter which direction the trend is moving in. In practice, most economic time series exhibit a multiplicative relationship and hence the multiplicative decomposition usually provides the best fit. However, a multiplicative decomposition cannot be implemented if any zero or negative values appear in the time series. 2.4 Other factors that affect seasonal adjustment There are a variety of issues that can impact on the quality of the seasonal adjustment. These include: 2-2 27/02/07 Guide to Seasonal Adjustment with X-12-ARIMA (cid:131) Outliers, which are extreme values. These usually have identifiable causes, such as strikes, war, or extreme weather conditions, which can distort the seasonal adjustment. They are normally considered to be part of the irregular component; (cid:131) Trend breaks (also known as level shifts) where the trend component suddenly increases or decreases sharply. Possible causes include changes in definitions relating to the series that is being measured, to take account of say a reclassification of products or a change in the rate of taxation; (cid:131) Seasonal breaks, where there are abrupt changes in the seasonal pattern. These issues need to be addressed before the seasonal adjustment process begins, in order to obtain the most reliable estimate of the seasonal component. 2.5 Interpreting time series outputs The original, seasonally adjusted, and trend estimates are available for users to assess. In ONS publications the focus is usually on the level of the seasonally adjusted series, the period-to- period change in the level of the seasonally adjusted series, and the period-to-period growth rate for the seasonally adjusted series. A comparison with the same period in the previous year may also be published, but this gives a historical picture of the growth rate. On average, this measure will result in a lag of six months in the identification of turning points in the series. It may be thought that this year- on-year change would be the same for the seasonally adjusted and the non-seasonally-adjusted series, but this will only be the case if the seasonal component is stable. In practice seasonality often evolves over time, and the seasonal factors should reflect this. In general, seasonally adjusted estimates will not sum to the annual estimates. This is due to a variety of factors including moving seasonality, incomplete cycles and outlier treatment. The ONS currently recommended measure of year-on-year change is based on the seasonally adjusted series, as this takes into account the impact of a change in seasonal patterns over time. People are often interested in removing the impact of the irregular component from the seasonally adjusted estimates in order to produce a trend (also known as a trend-cycle, or a short-term trend). Seasonally adjusted and trend estimates are subject to revision as additional raw data become available. As the seasonal component cannot be directly observed, it is estimated using moving averages and these can change as new data points are added to the series. 2.6 X-12-ARIMA X-12-ARIMA was developed by United States Bureau of the Census (1998) as an extended and improved version of the Statistics Canada X-11-ARIMA method. The program broadly runs through the following steps: (cid:131) The series is modified by any user defined prior adjustments; (cid:131) The program fits a regression-ARIMA model to the series in order to: detect and adjust for outliers and other distorting effects, to improve forecasts and seasonal adjustment; detect and estimates additional components such as calendar effects; and extrapolate forwards (forecast) and backwards (backcast) an extra one to three years of data; (cid:131) The program then uses a series of moving averages to decompose a time series into the three components. It does this in three iterations, getting successively better estimates of the three components. During these iterations extreme values are identified and replaced; 27/02/07 2-3 Introduction to seasonal adjustment (cid:131) A wide range of diagnostic statistics are produced, describing the final seasonal adjustment, and giving pointers to possible improvements which could be made. 2.7 Know your series In addition to being able to run a seasonal adjustment program, such as X-12-ARIMA, and to understand the outputs, a proficient user should have an appreciation of factors that are likely to affect the series being seasonally adjusted. Knowing the background to a series will give clues as to where to look for likely problems. For example: (cid:131) Is the way in which the data are collected likely to lead to any unusual effects? Are they collected on a non-calendar basis, or is there a lag between the activity being measured and when this is recorded? (cid:131) Has there been any change to the method or timing of data collection? This may cause trend or seasonal breaks. (cid:131) Is the series likely to be affected by trading days or Easter effects? (cid:131) Have there been any events which are likely to cause breaks in the series or large outliers? These could include: the Gulf war, the budget moving from March to November, Britain dropping out of the ERM, extreme weather, industrial disputes, or other events which may only affect individual series. 2-4 27/02/07 Guide to Seasonal Adjustment with X-12-ARIMA 3 OVERVIEW OF THE X-12-ARIMA METHOD 3.1 Introduction Many users regard the X-12-ARIMA software package as a black box method that can take a time series and automatically derive seasonally adjusted estimates. This chapter outlines the concepts used within X-12-ARIMA. Additional details on the use of X-12-ARIMA can be obtained from: www.census.gov/srd/www/x12a/. 3.2 The X-12-ARIMA method The X-12-ARIMA method can be described by the flowchart in Figure 3-1. Figure 3-1 The X-12-ARIMA method Figure 3-1 shows the chronological sequence in an analysis, but it may be helpful to think in terms of a historical sequence. The X-11 (Shiskin, et. al, 1967) seasonal adjustment method was developed by the United States Bureau of the Census in the 1960s, and has formed the basis for most official government seasonal adjustment since then. The X-11-ARIMA (Dagum, 1980) program is an improved version of the X-11 program and was developed by Statistics Canada in the 1980s. In recent years the United States Bureau of the Census has developed X-12-ARIMA (Findley, et. al, 1998) which uses a modelling strategy called regression-ARIMA, which can be used to identify and quantify factors which will impact on the seasonal factor estimation. If the output of the regression- ARIMA is used as a prior adjustment for the raw series, the result is a series which X-11 can handle effectively. The two parts are described below. 3.3 regression-ARIMA models This approach can model the time series and estimate any prior adjustments before seasonal adjustment. For example, regression-ARIMA models are able to detect and adjust for outliers and other distorting effects to improve the forecasts and seasonal adjustment. A regression-ARIMA model takes the form: Y  log t =β'X +Z D  t t t 01/03/07 3-1