Doc 4.14, ver. 1 | October, 2016 ALMA Science Pipeline Reference Manual: CASA 4.7.0 Interferometric and Single-Dish Data ALMA, an international astronomy facility, is a partnership of ESO (representing its member states), NSF (USA) and NINS (Japan), together with NRC (Canada), NSC and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ. 1 For further information or to comment on this document, please contact your regional Helpdesk through the ALMA User Portal at www.almascience.org. Helpdesk tickets will be directed to the appropriate ALMA Regional Center at ESO, NAOJ or NRAO. Version Date Editors 3.14v1.0 CASA 4.5.1 January 2016 Pipeline Team 4.14v1.0 CASA 4.7.0 October 2016 Pipeline Team In publications, please refer to this document as: ALMA Pipeline Team, 2016, ALMA Science Pipeline Reference Manual CASA 4.7.0, ALMA Doc 4.14v1.0 2 Table of Contents 1 PURPOSE AND SCOPE ............................................................................................... 4 2 PIPELINE TASK TYPES ................................................................................................ 5 3 PIPELINE CONTEXT ..................................................................................................... 5 4 PIPELINE TASK LIST .................................................................................................... 5 5 COMMON TASK DESCRIPTIONS ................................................................................. 8 5.1 h_init ..................................................................................................................... 8 5.2 h_resume .............................................................................................................. 9 5.3 h_save ................................................................................................................ 10 5.4 h_weblog ............................................................................................................ 10 6 INTERFEROMETRY COMMON TASK DESCRIPTIONS ............................................. 11 6.1 hif_antpos ........................................................................................................... 11 6.2 hif_applycal ......................................................................................................... 13 6.3 hif_atmflag ........................................................................................................... 15 6.4 hif_bandpass ....................................................................................................... 17 6.5 hif_bpflagchans ................................................................................................... 20 6.6 hif_cleanlist ......................................................................................................... 22 6.7 hif_clean ............................................................................................................. 24 6.8 hif_export_calstate .............................................................................................. 28 6.9 hif_exportdata...................................................................................................... 29 6.10 hif_findcont ....................................................................................................... 32 6.11 hif_gaincal ......................................................................................................... 33 6.12 hif_gainflag ........................................................................................................ 36 6.13 hif_import_calstate ............................................................................................ 39 6.14 hif_importdata ................................................................................................... 40 6.15 hif_lowgainflag .................................................................................................. 42 6.16 hif_makecleanlist ............................................................................................... 44 6.17 hif_makeimages ................................................................................................. 47 6.18 hif_makeimlist ................................................................................................... 49 6.19 hif_mstransform ................................................................................................ 53 6.20 hif_normflux ...................................................................................................... 54 6.21 hif_rawflagchans ............................................................................................... 57 6.22 hif_refant ........................................................................................................... 60 6.23 hif_restoredata .................................................................................................. 62 6.24 hif_setjy ............................................................................................................. 64 6.25 hif_setmodels .................................................................................................... 67 3 6.26 hif_show_calstate .............................................................................................. 69 6.27 hif_tclean .......................................................................................................... 69 6.28 hif_uvcontfit ...................................................................................................... 75 6.29 hif_uvcontsub .................................................................................................... 76 7 INTERFEROMETRY ALMA TASK DESCRIPTIONS ................................................... 78 7.1 hifa_antpos ........................................................................................................ 78 7.2 hifa_bandpass ..................................................................................................... 80 7.3 hifa_bpsolint ........................................................................................................ 85 7.4 hifa_flagdata ........................................................................................................ 87 7.5 hifa_flagtargets .................................................................................................. 91 7.6 hifa_fluxcalflag .................................................................................................... 92 7.7 hifa_fluxdb .......................................................................................................... 94 7.8 hifa_gaincalsnr .................................................................................................. 95 7.9 hifa_gfluxscale ................................................................................................... 97 7.10 hifa_importdata .............................................................................................. 101 7.11 hifa_linpolcal .................................................................................................. 103 7.12 hifa_spwphaseup ........................................................................................... 105 7.13 hifa_timegaincal .............................................................................................. 109 7.14 hifa_tsyscal .................................................................................................... 112 7.15 hifa_tsysflag ................................................................................................... 113 7.16 hifa_wvrgcalflag ............................................................................................. 117 7.17 hifa_wvrgcal ................................................................................................... 121 8 SINGLE-DISH TASK DESCRIPTIONS ...................................................................... 124 8.1 hsd_applycal .................................................................................................... 124 8.2 hsd_baseline .................................................................................................... 126 8.3 hsd_blflag ......................................................................................................... 129 8.4 hsd_exportdata ................................................................................................ 133 8.5 hsd_flagdata ..................................................................................................... 134 8.6 hsd_imaging ..................................................................................................... 137 8.7 hsd_importdata ................................................................................................ 139 8.8 hsd_k2jycal ...................................................................................................... 141 8.9 hsd_skycal ....................................................................................................... 143 1 Purpose and Scope The purpose of this document is to describe the tasks available for the calibration and imaging of interferometry datasets using the ALMA Science Pipeline. 4 2 Pipeline Task Types There are 4 types of Pipeline tasks. This document provides descriptions of the task types: h_, hif_, hifa_ and hsd_. Table 1: Pipeline Task Types Task pre-fix Task type Description h_ Common Pipeline tasks used in the calibration and tasks imaging of both interferometry and single- dish datasets hif_ Interferometry Pipeline tasks used in the calibration and common tasks imaging of both ALMA and EVLA interferometry datasets hifa_ Interferometry Pipeline tasks used in the calibration and ALMA tasks imaging of ALMA interferometry datasets only hsd_ Single-dish Pipeline tasks used in the calibration and tasks imaging of single-dish datasets only 3 Pipeline Context The Pipeline state is stored in its context e.g. which calibration tables need to be used at each stage. Several Pipeline tasks are associated with initialising and editing (rarely needed) the context. 4 Pipeline Task List Commissioned tasks are those used in the current ALMA standard recipes (casa_pipescript.py and casa_piperestorescript.py). Tasks which do not form part of the standard recipes are currently experimental. To assess which tasks were used in the processing of an ALMA dataset, please see the casa_pipescript.py and casa_piperestorescript.py included in ALMA deliveries for Pipeline- processed data. Table 2: Common Tasks Task Name Description 1 h_init Initialise the interferometry pipeline 2 h_resume Restore a save pipeline state from disk 3 h_save Save the pipeline state to disk 4 h_weblog Open the pipeline weblog in a browser 5 Table 3: Interferometry Common Tasks Task Name Description 1 hif_antpos Derive an antenna position calibration table 2 hif_applycal Apply the calibration(s) to the data 3 hif_atmflag Flag channels with bad atmospheric transmission 4 hif_bandpass Compute bandpass calibration solutions 5 hif_bpflagchans Flag deviant channels in bandpass calibration 6 hif_cleanlist Compute clean map 7 hif_clean Compute clean map 8 hif_export_calstate Save the pipeline calibration state to disk 9 hif_exportdata Prepare interferometry data for export 10 hif_findcont Find continuum frequency ranges 11 hif_gaincal Determine temporal gains from calibrator observations 12 hif_gainflag Flag antennas with deviant gain 13 hif_import_calstate Import a calibration state from disk 14 hif_importdata Imports data into the interferometry pipeline 15 hif_lowgainflag Flag antennas with low or high gain 16 hif_makecleanlist Compute list of clean images to be produced 17 hif_makeimages Compute clean map 18 hif_makeimlist Compute list of clean images to be produced 19 hif_mstransform 20 hif_normflux Average calibrator fluxes across measurement sets 21 hif_rawflagchans Flag deviant channels in raw data 22 hif_refant Select the best reference antennas 23 hif_restoredata Restore flagged and calibration interferometry data from a pipeline run 24 hif_setjy Fill the model column with calibrated visibilities 25 hif_setmodels Set calibrator source models 26 hif_show_calstate Show the current pipeline calibration state 27 hif_tclean Compute clean map 28 hif_uvcontfit Fit the continuum in the UV plane 29 hif_uvcontsub Subtract the fitted continuum from the data 6 Table 4: Interferometry ALMA Tasks Task Name Description 1 hifa_antpos Derive an antenna position table 2 hifa_bandpass Compute bandpass calibration solutions 3 hifa_bpsolint Compute optimal bandpass calibration solution intervals 4 hifa_flagdata Do basic flagging of a list of measurement sets 5 hifa_flagtargets Do science target flagging 6 hifa_fluxcalflag Locate line regions in solar system flux calibrator spws 7 hifa_fluxdb Connect to flux calibrator database 8 hifa_gaincalsnr Compute gaincal signal to noise ratios per spw 9 hifa_gfluxscale Derive flux density scales from standard calibrators 10 hifa_importdata Imports data into the interferometry pipeline 11 hifa_linpolcal Compute polarization calibration 12 hifa_spwphaseup Compute wide to narrow spectral window mapping and per spectral window phase offsets 13 hifa_timegaincal Determine temporal gains from calibrator observations 14 hifa_tsyscal Derive a Tsys calibration table 15 hifa_tsysflag Flag deviant system temperature measurements 16 hifa_wvrgcalflag Calculate WVR corrections 17 hifa_wvrgcal Compute the WVR calibration Table 5: Single-Dish Tasks Task Name Description 1 hsd_applycal Apply calibration tables to the science target data 2 hsd_baseline Detect and validate spectral lines, subtract a spectral baseline by masking detected lines 3 hsd_blflag Flag spectra based on the pre-defined criteria of the single dish pipeline 4 hsd_exportdata Export the data products into the product directory 5 hsd_flagdata Perform a-priori flagging of a list of measurement sets (MS) 6 hsd_imaging Apply flag tables to the data, convert Scantables to MS format, apply the K-to-Jy factor and perform imaging 7 hsd_importdata Import data into the single dish pipeline 8 hsd_k2jycal Derive Kelvin to Jansky calibration tables 9 hsd_skycal Calibrate data 7 5 Common Task Descriptions 5.1 h_init h_init must be called before any other interferometry pipeline task. The pipeline can be initialised in one of two ways: by creating a new pipeline state (h_init) or be loading a saved pipeline state (h_resume). h_init creates an empty pipeline context but does not load visiblity data into the context. hif_importdata or hsd_importdata can be used to load data. Task Description Initialise the interferometry pipeline The h_init task initialises the interferometry pipeline. Keyword arguments: ---- pipeline parameter arguments which can be set in any pipeline mode pipelinemode -- The pipeline operating mode. In 'automatic' mode the pipeline determines the values of all context defined pipeline inputs automatically. In interactive mode the user can set the pipeline context fined parameters manually. In 'getinputs' mode the user can check the settings of all pipeline parameters without running the task. default: 'automatic'. ---- pipeline context defined parameter argument which can be set only in'interactive mode' or 'getinputs' modes loglevel -- Pipeline log level threshold: (debug|info|warning|error|critical). Log messages below this threshold will not be displayed. default: 'info' plotlevel -- Pipeline plot level threshold: (all|default|summary). Toggle generation of detail plots in the web log. A level of 'all' generates all plots; 'summary' omits detail plots; 'default' generates all plots apart from for the hif_applycal task. default: 'default' output_dir -- Working directory for pipeline processing. Some pipeline processing products such as HTML logs and images will be directed to subdirectories of this path. default: './' (current directory) weblog -- Toggle web log generation. overwrite -- Overwrite existing MSs on input. --- pipeline task execution modes dryrun -- Run the commands (True) or generate the commands to be run but do not execute (False). default: True acceptresults -- Add the results of the task to the pipeline context (True) or reject them (False). default: True 8 Output results -- If pipeline mode is 'getinputs' then None is returned. Otherwise the results object for the pipeline task is returned. Examples 1. Create the pipeline context h_init() Parameter List Table 6: h_init default settings Parameter Type Default Description pipelinemode string automatic The pipeline operating mode loglevel string info Log level for pipeline messages plotlevel string default Level for pipeline plots output_dir string ./ The output working directory weblog bool True Generate the web log overwrite bool True Overwrite existing files on import dryrun bool False Run the task (False) or display the task command (True) acceptresults bool True Add the results into the pipeline context 5.2 h_resume h_resume restores a named pipeline state from disk allowing a suspended pipeline reduction session to be resumed. Task Description Restore a save pipeline state from disk h_resume restores a name pipeline state from disk allowing a suspended pipeline reduction session to be resumed. Keyword parameters: filename -- Name of the saved pipeline state. Setting filename to 'last' restores the most recently saved pipeline state whose name begins with 'context*'. default: 'last' example: filename='context.s3.2012-02-13T10:49:11' filename='last' Examples 9 1. Resume the last saved session h_resume() 2. Resume the named saved session h_resume(filename='context.s3.2012-02-13T10:49:11') Parameter List Table 7: h_resume default settings Parameter Type Default Description filename string last Filename of saved state to be restored 5.3 h_save h_save saves the current pipeline state to disk under a unique name. If no name is supplied one is generated automatically from a combination of the rootname 'context', the current stage number, and a timestamp. Task Description Save the pipeline state to disk h_save saves the current pipeline state to disk under a unique name. Keyword arguments: filename -- Name of the saved pipeline state. If filename is '' then a unique name will be generated computed from the root 'context', the current stage number, and the timestamp. default: '' Examples 1. Save the current state in the default file h_save() 2. Save the current state to a user named file h_save(filename='savestate_1') Parameter List Table 8: h_save default settings Parameter Type Default Description filename string None Name for saved state 5.4 h_weblog Task Description 10