The XMM-Newton ABC Guide: An Introduction to XMM-Newton Data Analysis NASA/GSFC XMM-Newton Guest Observer Facility Kim Weaver, Lynne Valencic, Brendan Perry, Michael Arida, K.D. Kuntz With contributions by: Steve Snowden, Ilana Harrus, Stefan Immler, Rick Shafer, Randall Smith, Martin Still Version 5.0 for XMM-SAS v 17.0 July 2018 ii Copies of this guide are available in html, postscriptand pdf formats. Contents 1 Introduction 1 1.1 ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Useful Information and References 2 2.1 The Main Websites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 XMM-Newton Help Desks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Mission Planning and Spacecraft Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.4 Public Data Archives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.5 Calibration Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.6 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.7 Analysis, Documentation, and Helpful Hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 Data 4 3.1 Useful Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3 PI Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3.1 ODF Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3.2 Pipeline Product Data – Summary Files and Groupings . . . . . . . . . . . . . . . . . . . 5 3.3.3 Pipeline Product Data – Data Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 Setting Up and Running SAS 10 4.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.2 Calibration Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3 SAS EnvironmentalParameters and Invocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3.1 SAS Helpful Hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.4 SAS Syntax and Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.4.1 Command Line Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.4.2 Table Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.4.3 Filtering Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5 Preparing the Data for Processing 14 5.1 cifbuild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.2 odfingest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.3 Welcome to the SAS GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6 An EPIC Data Processing and Analysis Primer (Imaging Mode, Command Line) 17 6.1 Rerun the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 6.2 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 6.3 Applying Standard Filters the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 6.4 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6.5 Applying Time Filters the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 6.6 Source Detection with edetect chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6.7 Extract the Source and BackgroundSpectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.8 Check for Pile Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 6.9 My Observation is Piled Up! Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 6.10 Determine the Spectrum Extraction Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 i ii 6.11 Create the Photon Redistribution Matrix (RMF) and Ancillary File (ARF) . . . . . . . . . . . . 29 7 An EPIC Data Processing and Analysis Primer (Imaging Mode, GUI) 31 7.1 Rerun the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.2 An Introduction to xmmselect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.3 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 7.4 Applying Standard Filters the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 7.5 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 7.6 Applying Time Filters the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 7.7 Source Detection with edetect chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 7.8 Extract the Source and BackgroundSpectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 7.9 Check for Pile Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 7.10 My Observation is Piled Up! Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 7.11 Determine the Spectrum Extraction Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 7.12 Create the Photon Redistribution Matrix (RMF) and Ancillary File (ARF) . . . . . . . . . . . . 43 8 An EPIC Data Processing and Analysis Primer (Timing Mode, Command Line) 45 8.1 Rerun the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 8.2 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 8.3 Applying Standard Filters to the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 8.4 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.5 Extract the Source and BackgroundSpectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.6 Check for Pile Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 8.7 My Observation is Piled Up! Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 8.8 Determine the Spectrum Extraction Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 8.9 Create the Photon Redistribution Matrix (RMF) and Ancillary File (ARF) . . . . . . . . . . . . 52 9 An EPIC Data Processing and Analysis Primer (Timing Mode, GUI) 54 9.1 Rerun the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 9.2 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 9.3 Applying Standard Filters to the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 9.4 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 9.5 Extract the Source and BackgroundSpectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 9.6 Check for Pile Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 9.7 My Data is Piled Up! Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 9.8 Determine the Spectrum Extraction Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 9.9 Create the Photon Redistribution Matrix (RMF) and Ancillary File (ARF) . . . . . . . . . . . . 61 10 An RGS Data Processing and Analysis Primer (Command Line) 63 10.1 Rerun the Pipeline on the Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 10.1.1 Potentially useful tips for using the pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . 64 10.1.2 A Nearby Bright Optical Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 10.1.3 A Nearby Bright X-ray Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 10.1.4 User-defined Source Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 10.2 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 10.3 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 10.4 Generating the Good Time Interval (GTI) File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 10.4.1 Filter on Time with gtibuild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 10.4.2 Filter on Time with tabgtigen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 10.4.3 Filter on Rate with tabgtigen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 10.4.4 Apply the new GTI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 10.5 Creating the Response Matrices (RMFs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 10.6 Combining Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 iii 11 An RGS Data Processing and Analysis Primer, GUI 71 11.1 Rerun the Pipeline on the Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 11.1.1 Potentially useful tips for using the pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . 72 11.1.2 A Nearby Bright Optical Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 11.1.3 A Nearby Bright X-ray Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 11.1.4 User-defined Source Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 11.2 An Introduction to xmmselect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 11.3 Create and Display an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 11.4 Create and Display a Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 11.5 Generating the Good Time Interval (GTI) File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 11.5.1 Filter on Time with gtibuild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 11.5.2 Filter on Time with tabgtigen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 11.5.3 Filter on Rate with tabgtigen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 11.5.4 Applying the GTI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 11.6 Creating the Response Matrices (RMFs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 11.7 Combining Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 12 An OM Data Processing and Analysis Primer 79 12.1 OM Artifacts and General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 12.2 Imaging Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 12.2.1 Rerunning the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 12.2.2 Verifying the Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 12.3 Fast Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.3.1 Rerunning the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.3.2 Verifying the Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.4 Grism Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.4.1 Rerunning the Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.4.2 Verifying the Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 13 Fitting an EPIC Spectrum in XSPEC 86 14 Fitting an RGS Spectrum in XSPEC 88 14.1 Approaches to Spectral Fitting and the Cash Statistic (cstat) . . . . . . . . . . . . . . . . . . . . 88 14.2 Rebinning the Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 14.3 Fitting a Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 15 Fitting EPIC Timing Data with Xronos 91 15.1 Making a Power Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 15.2 Finding the Period of a Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 15.3 Making a Folded Light Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 List of Tables 1 List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v 3.1 ODF data file identifiers.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2 Pipeline Processing groupings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.3 Pipeline Processing data files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1 Commonly Used Filtering Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.1 Example datasets used in this Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 12.1 Some of the important columns in the source list file. . . . . . . . . . . . . . . . . . . . . . . . . . 80 iv v Table 1: List of Acronyms ARF Ancillary Region File CAL Calibration Access Layer CCD Charge Coupled Device CCF Current Calibration File CIF Calibration Index File EPIC European Photon Imaging Camera FITS Flexible Image Transport System GO Guest Observer GOF NASA/GSFC Guest Observer Facility GSFC Goddard Space Flight Center GUI Graphical User Interface HEASARC High Energy Astrophysics Science Archive Research Center HTML Hyper Text Markup Language OAL ODF Access Layer ODF Observation Data File OM Optical Monitor PDF Portable Data Format PP Pipeline Processing System PPS Pipeline Processing PV Performance Validation RGS Reflection Grating Spectrometer RMF Redistribution Matrix File SAS Science Analysis System SOC Science Operations Center SSC Survey Science Centre SV Science Validation XMM X-ray Multi-Mirror Mission Chapter 1 Introduction ThepurposeofthisABC GuidetoXMM-Newtondataanalysisistoprovideasimplewalk-throughofbasicdata extraction and analysis tasks. Also included is a guide to references and help available to aid in the analysis of the data. We have tried to balance providing enough information to give the user a useful introduction to a variety of analysis tasks with not providing too much information, which would make a guide like this too ponderous to use. As such, there is no intention to replace the SAS Handbook, which should be consideredthe highest authority for the use of SAS. Therefore this document will not display the full versatility of the SAS tasks, and of SAS itself, but it will hopefully show a path through the forest. Chapter2provideslistsofweb-basedreferencesfortheXMM-Newtonproject,helpdesks,analysisguides, and science and calibration data. Chapter 3 provides a description of the data files provided for observation data sets. Chapter 4 discusses the installation and use of SAS. The rest of the chapters discuss the analysis of EPIC, RGS, and OM data respectively, taking the detector mode and SAS interface (command line or GUI) into account. We also include brief introductions to analysis of the reduced data with XSPEC and Xronos. This document will continue to evolve. Updated versions will be made available on our web site at: http://heasarc.gsfc.nasa.gov/docs/xmm/abc/ 1.1 ACKNOWLEDGMENTS This guide wouldnot havebeen possible withoutthe helpandcommentsfromallpeople involvedintheXMM- Newton project. In particular, we would like to thank Giuseppe Vacanti and Julian Osborne whose comments made this a more complete and accurate document. IMH wishes to thank all the OM calibration team and in particular Antonio Talavera,Matteo Guainazzi and Bing Chen for their help in the preparation of this and other documents related to the OM. SLS wishes to thank Dave Lumb, Richard Saxton, and Steve Sembay for their helpful insights into EPIC data analysis. 1 Chapter 2 Useful Information and References 2.1 The Main Websites • XMM-Newton SOC, fount of all XMM-Newton project information: https://www.cosmos.esa.int/web/xmm-newton • NASA/GSFCGOF,sourceofUSspecificinformationandamirrorsiteforsoftwareandpublicdataaccess: http://heasarc.gsfc.nasa.gov/docs/xmm/xmmgof.html • Survey Science Centre http://xmmssc-www.star.le.ac.uk/ XMM-Newton 2.2 Help Desks • The main project helpdesk is located at Vilspa and can be accessed through the SOC’s website. The helpdesk also provides an archive of previously asked questions. https://www.cosmos.esa.int/web/xmm-newton/xmm-newton-helpdesk • The NASA/GSFC GOF offers an e-mail helpdesk for both general support and for US-specific issues: [email protected] Some questions addressed to the NASA/GSFC GOF may be redirected to the Vilspa helpdesk. 2.3 Mission Planning and Spacecraft Status • Observation Log: http://xmm2.esac.esa.int/cgi-bin/obs search/selectobs cosmos The scheduling information from this data base has been extracted and incorporated into a Browse data base at GSFC: http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3browse.pl • Short-Term Timeline: https://www.cosmos.esa.int/web/xmm-newton/short-term-schedule 2 3 2.4 Public Data Archives • SOC Public Data Archive via the XSA: https://www.cosmos.esa.int/web/xmm-newton/xsa • GSFC Archive Mirror Site via Browse: http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3browse.pl 2.5 Calibration Data • XMM-Newton Calibration Page. On this page can be found links to the Current Calibration File (CCF) archive,releasenotesforCCFupdates,responseandbackgroundfiles forallinstruments,andthe current calibration status of all instruments. https://www.cosmos.esa.int/web/xmm-newton/calibration • Caldb, NASA/GSFC GOF mirror site for canned response files: ftp://legacy.gsfc.nasa.gov/caldb/data/xmm/ 2.6 Software • XMM-Newton Standard Analysis System (SAS): https://www.cosmos.esa.int/web/xmm-newton/sas • XMM-Newton Extended Source Analysis Software (ESAS): http://http://heasarc.gsfc.nasa.gov/docs/xmm/esas/cookbook/xmm-esas.html • HEASARC HEASoft Package: http://heasarc.gsfc.nasa.gov/docs/corp/software.html • CXC CIAO Package: http://asc.harvard.edu/ciao/ 2.7 Analysis, Documentation, and Helpful Hints • On-Line Users Guide to SAS: https://xmm-tools.cosmos.esa.int/external/xmm user support/documentation/sas usg/USG/ • There is a “watchout” page for current SAS bugs at: https://www.cosmos.esa.int/web/xmm-newton/sas-watchout • SAS and ESAS Threads: https://www.cosmos.esa.int/web/xmm-newton/sas-threads • Very detailed descriptions of all SAS tasks, including parameter lists, algorithms, warnings, and errors: https://xmm-tools.cosmos.esa.int/external/sas/current/doc/packagelist.html • XMM-Newton Users Handbook: https://xmm-tools.cosmos.esa.int/external/xmm user support/documentation/uhb/index.html • This Guide: http://heasarc.gsfc.nasa.gov/docs/xmm/abc/