Doc 4.14, ver. 2 | April, 2017 ALMA Science Pipeline Reference Manual: CASA 4.7.2 Interferometric and Single-Dish Data ALMA, an international astronomy facility, is a partnership of ESO (representing its member states), NSF (USA) and NINS (Japan), together with NRC (Canada), NSC and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ. 1 For further information or to comment on this document, please contact your regional Helpdesk through the ALMA User Portal at www.almascience.org. Helpdesk tickets will be directed to the appropriate ALMA Regional Center at ESO, NAOJ or NRAO. Version Date Editors 3.14v1.0 CASA 4.5.1 January 2016 Pipeline Team 4.14v1.0 CASA 4.7.0 October 2016 Pipeline Team 4.14v2.0 CASA 4.7.2 April 2017 Pipeline Team In publications, please refer to this document as: ALMA Pipeline Team, 2017, ALMA Science Pipeline Reference Manual CASA 4.7.2, ALMA Doc 4.14v2.0 2 Table of Contents 1 PURPOSE AND SCOPE ............................................................................................... 5 2 PIPELINE TASK TYPES ................................................................................................ 5 3 PIPELINE CONTEXT ..................................................................................................... 5 4 PIPELINE TASK LIST .................................................................................................... 6 5 COMMON TASK DESCRIPTIONS ................................................................................. 9 5.1 h_init ..................................................................................................................... 9 5.2 h_resume ............................................................................................................ 10 5.3 h_save ................................................................................................................ 11 5.4 h_weblog ............................................................................................................ 11 6 INTERFEROMETRY COMMON TASK DESCRIPTIONS ............................................. 12 6.1 hif_antpos ........................................................................................................... 12 6.2 hif_applycal ......................................................................................................... 14 6.3 hif_atmflag ........................................................................................................... 16 6.4 hif_bandpass ....................................................................................................... 18 6.5 hif_bpflagchans ................................................................................................... 21 6.6 hif_checkproductsize ........................................................................................... 24 6.7 hif_cleanlist ........................................................................................................ 25 6.8 hif_clean ............................................................................................................. 27 6.9 hif_export_calstate .............................................................................................. 31 6.10 hif_exportdata .................................................................................................... 32 6.11 hif_findcont ....................................................................................................... 35 6.12 hif_gaincal ......................................................................................................... 36 6.13 hif_gainflag ........................................................................................................ 39 6.14 hif_import_calstate ............................................................................................ 42 6.15 hif_importdata ................................................................................................... 42 6.16 hif_lowgainflag .................................................................................................. 45 6.17 hif_makecleanlist ............................................................................................... 47 6.18 hif_makeimages ................................................................................................. 50 6.19 hif_makeimlist ................................................................................................... 52 6.20 hif_mstransform ................................................................................................ 56 6.21 hif_normflux ...................................................................................................... 58 6.22 hif_rawflagchans ............................................................................................... 60 6.23 hif_refant ........................................................................................................... 63 6.24 hif_restoredata .................................................................................................. 65 3 6.25 hif_setjy ............................................................................................................. 67 6.26 hif_setmodels .................................................................................................... 70 6.27 hif_show_calstate .............................................................................................. 72 6.28 hif_tclean .......................................................................................................... 72 6.29 hif_uvcontfit ...................................................................................................... 78 6.30 hif_uvcontsub .................................................................................................... 79 7 INTERFEROMETRY ALMA TASK DESCRIPTIONS ................................................... 81 7.1 hifa_antpos ........................................................................................................ 81 7.2 hifa_bandpass ..................................................................................................... 83 7.3 hifa_bpsolint ........................................................................................................ 88 7.4 hifa_flagdata ........................................................................................................ 90 7.5 hifa_flagtargets .................................................................................................. 93 7.6 hifa_fluxcalflag .................................................................................................... 95 7.7 hifa_fluxdb. ......................................................................................................... 97 7.8 hifa_gaincalsnr .................................................................................................. 98 7.9 hifa_gfluxscale ................................................................................................. 100 7.10 hifa_importdata .............................................................................................. 104 7.11 hifa_linpolcal .................................................................................................. 106 7.12 hifa_spwphaseup ........................................................................................... 108 7.13 hifa_timegaincal .............................................................................................. 112 7.14 hifa_tsyscal .................................................................................................... 115 7.15 hifa_tsysflag ................................................................................................... 117 7.16 hifa_wvrgcalflag ............................................................................................. 120 7.17 hifa_wvrgcal ................................................................................................... 124 8 SINGLE-DISH TASK DESCRIPTIONS ...................................................................... 128 8.1 hsd_applycal .................................................................................................... 128 8.2 hsd_baseline .................................................................................................... 130 8.3 hsd_blflag ......................................................................................................... 132 8.4 hsd_exportdata ................................................................................................ 136 8.5 hsd_flagdata ..................................................................................................... 138 8.6 hsd_imaging ..................................................................................................... 141 8.7 hsd_importdata ................................................................................................ 142 8.8 hsd_k2jycal ...................................................................................................... 145 8.9 hsd_skycal ....................................................................................................... 147   4 1 Purpose and Scope The purpose of this document is to describe the tasks available for the calibration and imaging of interferometry datasets using the ALMA Science Pipeline. 2 Pipeline Task Types There are 4 types of Pipeline tasks. This document provides descriptions of the task types: h_, hif_, hifa_ and hsd_. Table 1: Pipeline Task Types Task pre-fix Task type Description h_ Common Pipeline tasks used in the calibration and tasks imaging of both interferometry and single- dish datasets hif_ Interferometry Pipeline tasks used in the calibration and common tasks imaging of both ALMA and EVLA interferometry datasets hifa_ Interferometry Pipeline tasks used in the calibration and ALMA tasks imaging of ALMA interferometry datasets only hsd_ Single-dish Pipeline tasks used in the calibration and tasks imaging of single-dish datasets only 3 Pipeline Context The Pipeline state is stored in its context e.g. which calibration tables need to be used at each stage. Several Pipeline tasks are associated with initialising and editing (rarely needed) the context.   5 4 Pipeline Task List Commissioned tasks are those used in the current ALMA standard recipes (casa_pipescript.py and casa_piperestorescript.py). Tasks which do not form part of the standard recipes are currently experimental. To assess which tasks were used in the processing of an ALMA dataset, please see the casa_pipescript.py and casa_piperestorescript.py included in ALMA deliveries for Pipeline- processed data. Table 2: Common Tasks Task Name Description 1 h_init Initialise the interferometry pipeline 2 h_resume Restore a save pipeline state from disk 3 h_save Save the pipeline state to disk 4 h_weblog Open the pipeline weblog in a browser 6 Table 3: Interferometry Common Tasks Task Name Description 1 hif_antpos Derive an antenna position calibration table 2 hif_applycal Apply the calibration(s) to the data 3 hif_atmflag Flag channels with bad atmospheric transmission 4 hif_bandpass Compute bandpass calibration solutions 5 hif_bpflagchans Flag deviant channels in bandpass calibration 6 hif_checkproductsize Check imaging product size 7 hif_cleanlist Compute clean map 8 hif_clean Compute clean map 9 hif_export_calstate Save the pipeline calibration state to disk 10 hif_exportdata Prepare interferometry data for export 11 hif_findcont Find continuum frequency ranges 12 hif_gaincal Determine temporal gains from calibrator observations 13 hif_gainflag Flag antennas with deviant gain 14 hif_import_calstate Import a calibration state from/to disk 15 hif_importdata Imports data into the interferometry pipeline 16 hif_lowgainflag Flag antennas with low or high gain 17 hif_makecleanlist Compute list of clean images to be produced 18 hif_makeimages Compute clean map 19 hif_makeimlist Compute list of clean images to be produced 20 hif_mstransform Select data from calibrated MS(s) to form new MS(s) for imaging 21 hif_normflux Average calibrator fluxes across measurement sets 22 hif_rawflagchans Flag deviant baseline/channels in raw data 23 hif_refant Select the best reference antennas 24 hif_restoredata Restore flagged and calibration interferometry data from a pipeline run 25 hif_setjy Fill the model column with calibrated visibilities 26 hif_setmodels Set calibrator source models 27 hif_show_calstate Show the current pipeline calibration state 28 hif_tclean Compute clean map 29 hif_uvcontfit Fit the continuum in the UV plane 30 hif_uvcontsub Subtract the fitted continuum from the data 7 Table 4: Interferometry ALMA Tasks Task Name Description 1 hifa_antpos Derive an antenna position calibration table 2 hifa_bandpass Compute bandpass calibration solutions 3 hifa_bpsolint Compute optimal bandpass calibration solution intervals 4 hifa_flagdata Do basic flagging 5 hifa_flagtargets Do science target flagging 6 hifa_fluxcalflag Locate and flag line regions in solar system flux calibrators 7 hifa_fluxdb Connect to flux calibrator database 8 hifa_gaincalsnr Compute gaincal signal to noise ratios per spw 9 hifa_gfluxscale Derive flux density scales from standard calibrators 10 hifa_importdata Imports data into the interferometry pipeline 11 hifa_linpolcal Compute polarization calibration 12 hifa_spwphaseup Compute phase calibration spw map and per spw phase offsets 13 hifa_timegaincal Determine temporal gains from calibrator observations 14 hifa_tsyscal Derive a Tsys calibration table 15 hifa_tsysflag Flag deviant system temperature measurements 16 hifa_wvrgcalflag Calculate WVR corrections 17 hifa_wvrgcal Compute the WVR calibration Table 5: Single-Dish Tasks Task Name Description 1 hsd_applycal Apply the calibration(s) to the data 2 hsd_baseline Detect and validate spectral lines, subtract baseline by masking detected lines 3 hsd_blflag Flag spectra based on predefined criteria of the single- dish pipeline 4 hsd_exportdata Prepare singledish data for export 5 hsd_flagdata Do basic flagging of a list of measurement sets 6 hsd_imaging Generate single dish images 7 hsd_importdata Import data into the single dish pipeline 8 hsd_k2jycal Derive Kelvin to Jy calibration tables 9 hsd_skycal Calibrate data 8 5 Common Task Descriptions 5.1 h_init h_init must be called before any other interferometry pipeline task. The pipeline can be initialised in one of two ways: by creating a new pipeline state (h_init) or be loading a saved pipeline state (h_resume). h_init creates an empty pipeline context but does not load visiblity data into the context. hif_importdata or hsd_importdata can be used to load data. Task Description Initialise the interferometry pipeline The h_init task initialises the interferometry pipeline. Keyword arguments: ---- pipeline parameter arguments which can be set in any pipeline mode pipelinemode -- The pipeline operating mode. In 'automatic' mode the pipeline determines the values of all context defined pipeline inputs automatically. In interactive mode the user can set the pipeline context defined parameters manually. In 'getinputs' mode the user can check the settings of all pipeline parameters without running the task. default: 'automatic'. ---- pipeline context defined parameter argument which can be set only in'interactive mode' or 'getinputs' modes loglevel -- Pipeline log level threshold: (debug|info|warning|error|critical). Log messages below this threshold will not be displayed. default: 'info' plotlevel -- Pipeline plot level threshold: (all|default|summary). Toggle generation of detail plots in the web log. A level of 'all' generates all plots; 'summary' omits detail plots; 'default' generats all plots apart from for the hif_applycal task. default: 'default' output_dir -- Working directory for pipeline processing. Some pipeline processing products such as HTML logs and images will be directed to subdirectories of this path. default: './' (current directory) weblog -- Toggle web log generation. --- pipeline task execution modes dryrun -- Run the commands (True) or generate the commands to be run but do not execute (False). default: True acceptresults -- Add the results of the task to the pipeline context (True) or reject them (False). default: True Output 9 results -- If pipeline mode is 'getinputs' then None is returned. Otherwise the results object for the pipeline task is returned. Examples 1. Create the pipeline context h_init() Parameter List Table 6: h_init default settings Parameter Type Default Description pipelinemode string automatic The pipeline operating mode loglevel string info Log level for pipeline messages plotlevel string default Level for pipeline plots output_dir string ./ The output working directory weblog bool True Generate the web log overwrite bool True Overwrite existing files on import dryrun bool False Run the task (False) or display the task command (True) acceptresults bool True Add the results into the pipeline context 5.2 h_resume h_resume restores a name pipeline state from disk allowing a suspended pipeline reduction session to be resumed. Task Description Restore a save pipeline state from disk h_resume restores a name pipeline state from disk allowing a suspended pipeline reduction session to be resumed. Keyword parameters: filename -- Name of the saved pipeline state. Setting filename to 'last' restores the most recently saved pipeline state whose name begins with 'context*'. default: 'last' example: filename='context.s3.2012-02-13T10:49:11' filename='last' Examples 1. Resume the last saved session 10