.. _archivefactor:

Archiving a Factor run with ``archivefactor``
---------------------------------------------

The results of a Factor run can be archived with::

    $ archivefactor factor.parset dir_output

where ``factor.parset`` is the parset described in :ref:`factor_parset` and ``dir_output`` is the output directory in which the archived files will be stored. A number of options are available and are described below:

    Usage: archivefactor parset dir_output

    Options:
      --version             show program's version number and exit
      -h, --help            show this help message and exit
      -d DIRECTIONS, --directions=DIRECTIONS
                            comma-separated list of directions (e.g., "-d
                            facet1,facet3") for which to archive the calibrated
                            data. Use "-d all" to select all directions.
      -r, --resumable       make an archive suitable for resuming (includes
                            the subtracted-data MS files, but not the calibrated
                            data unless specified with the "-d" flag)
      -c, --clobber         clobber existing files in output directory

By default, a basic archive will contain the images, plots, and the directions file. The calibrated data for one or more directions can be archived by specifying the direction names with the ``-d`` flag. An archive that is suitable for resuming with Factor at a later date (or on a different machine) can be made by adding the ``-r`` flag. The resumable archive will include the sky models, instrument tables, state files, and subtracted-data MS files from a run and will therefore be much larger.

A full archive will include the following:

``calibrated_data/``
    Directory containing the calibrated data MS files for one or more
    directions. There should be one file per time chunk, concatenated in
    frequency and uncompressed. These files are suitable for use in CASA, etc.

``chunks/``
    Directory containing the time-chunked datasets that Factor uses for processing and the direction-independent sky models.

``images/``
    Directory containing the images from the :ref:`facetimage` and :ref:`fieldmosaic` operations.

``instrument_tables/``
    Directory containing the self-calibration solution tables.

``misc/``
    Directory containing miscellaneous files, such as the Factor parset, the
    directions file, the basic Factor log, and the facet and calibrator region
    files.

``pipeline_state/``
    Directory containing the state files generated by the pipeline.

``plots/``
    Directory containing plots from self calibration.

``sky_models/``
    Directory containing the sky models from self calibration.

``state/``
    Directory containing files that save the state of a reduction.


.. _unarchivefactor:

Unarchiving a Factor archive with ``unarchivefactor``
-----------------------------------------------------

A resumable Factor archive can be unarchived with::

    $ unarchivefactor dir_archive dir_output

where ``dir_archive`` is a resumable archive made as described in :ref:`archivefactor` (with the ``-r`` flag) and ``dir_output`` is the output directory in which the unarchived files will be written. A number of options are available and are described below:

    Usage: unarchivefactor dir_archive dir_output

    Options:
      --version      show program's version number and exit
      -h, --help     show this help message and exit
      -s, --symlink  use symlinks for subtracted data files instead of copying
                     them
      -c, --clobber  clobber existing files in output directory

Before resuming, you will need to copy and edit the factor parset (stored in the ``dir_archive/misc`` directory of the archive) to update the various paths. Note that you can give the chunks directory (``dir_output/chunks``) as the ``dir_ms`` directory in the parset if you no longer have the original input MS files.