Archiving a Factor run with archivefactor¶
The results of a Factor run can be archived with:
$ archivefactor factor.parset dir_output
where factor.parset is the parset described in The Factor parset and dir_output is the output directory in which the archived files will be stored. A number of options are available and are described below:
Usage: archivefactor parset dir_output
- Options:
--version show program’s version number and exit -h, --help show this help message and exit -d DIRECTIONS, --directions=DIRECTIONS comma-separated list of directions (e.g., “-d facet1,facet3”) for which to archive the calibrated data. Use “-d all” to select all directions. -r, --resumable make an archive suitable for resuming (includes the subtracted-data MS files, but not the calibrated data unless specified with the “-d” flag) -c, --clobber clobber existing files in output directory
By default, a basic archive will contain the images, plots, and the directions file. The calibrated data for one or more directions can be archived by specifying the direction names with the -d flag. An archive that is suitable for resuming with Factor at a later date (or on a different machine) can be made by adding the -r flag. The resumable archive will include the sky models, instrument tables, state files, and subtracted-data MS files from a run and will therefore be much larger.
A full archive will include the following:
calibrated_data/- Directory containing the calibrated data MS files for one or more directions. There should be one file per time chunk, concatenated in frequency and uncompressed. These files are suitable for use in CASA, etc.
chunks/- Directory containing the time-chunked datasets that Factor uses for processing and the direction-independent sky models.
images/- Directory containing the images from the facetimage and fieldmosaic operations.
instrument_tables/- Directory containing the self-calibration solution tables.
misc/- Directory containing miscellaneous files, such as the Factor parset, the directions file, the basic Factor log, and the facet and calibrator region files.
pipeline_state/- Directory containing the state files generated by the pipeline.
plots/- Directory containing plots from self calibration.
sky_models/- Directory containing the sky models from self calibration.
state/- Directory containing files that save the state of a reduction.
Unarchiving a Factor archive with unarchivefactor¶
A resumable Factor archive can be unarchived with:
$ unarchivefactor dir_archive dir_output
where dir_archive is a resumable archive made as described in Archiving a Factor run with archivefactor (with the -r flag) and dir_output is the output directory in which the unarchived files will be written. A number of options are available and are described below:
Usage: unarchivefactor dir_archive dir_output
- Options:
--version show program’s version number and exit -h, --help show this help message and exit -s, --symlink use symlinks for subtracted data files instead of copying them -c, --clobber clobber existing files in output directory
Before resuming, you will need to copy and edit the factor parset (stored in the dir_archive/misc directory of the archive) to update the various paths. Note that you can give the chunks directory (dir_output/chunks) as the dir_ms directory in the parset if you no longer have the original input MS files.
