Starting a Factor run with runfactor

Factor can be run with:

$ runfactor factor.parset

where factor.parset is the parset described in The Factor parset. A number of options are available and are described below:

Usage: runfactor parset

Options:

--version	show program’s version number and exit
-h, --help	show this help message and exit
-d	enable dry-run mode
-q	enable quiet mode
-r RESET, --reset=RESET
	comma-separated list of directions to reset (e.g., “-r facet1,facet3”)
-o OPS, --ops=OPS
	comma-separated list of operations to reset for the directions specified with “-r” or “–reset” (e.g., “-o facetselfcal,facetsub”). By default, all operations are reset. Available operations are: outlierpeel, facetpeel, facetpeelimage, facetselfcal, facetsub, facetimage, fieldmosaic
-s STOP_AFTER, --stop_after=STOP_AFTER
	Stop after processing so many facetselfcal groups. (Please note the difference between directions and groups!)
-v	enable verbose mode

Factor begins a run by checking the input measurement sets. The input measurement sets are chunked in time to allow more efficient processing.

Next, Factor will check for a file describing the DDE calibrators. If not found, Factor will select the DDE calibrators automatically and generate the facet regions (saved in a text file in the working directory called factor_directions.txt). At this point, if interactive = True is set in the parset, Factor will pause to allow a check on the facets.

After intialization, Factor will begin self calibration and imaging (described in detail in Operations) of the first facet or group of facets. For the facet or facets that pass self-calibration verification, Factor subtracts the improved model using the direction-dependent instrument tables. Processing then proceeds to the self calibration of the next facet or facet group, and the steps are looped until all facets have been processed.

After self calibration is finished, imaging is done for all facets. Lastly, all facet images are mosaicked together and the primary beam attenuation is corrected to produce the final image.

Factor uses the LOFAR pipeline framework to handle the actual processing. The LOFAR pipeline framework handles the distribution of jobs and keeps track of the state of a reduction. Each Factor operation is done in a separate pipeline. See Factor structure for an overview of the various operations that Factor performs and their relation to one another, and see Operations for details of each operation and their primary data products.

Checking a Factor run with checkfactor

You can check the progress of a run with:

$ checkfactor factor.parset

Note

A number of checkfactor options are configurable in the Factor parset (see [checkfactor]).

Resuming an interrupted run

Due to the potentially long run times and the consequent non-negligible chance of some unforeseen failure occurring, Factor has been designed to allow easy resumption of a reduction from a saved state and will skip over any steps that were successfully completed previously. In this way, one can quickly resume a reduction that was halted (either by the user or due to some problem) by simply re-running Factor with the same parset.

For example, one can specify that only the first 5 directions be processed. Once these directions are done, Factor will exit. One can then alter the parset and specify that 10 directions should be done. Upon restarting, Factor will skip over the first 5 directions and start with the 6th one (and ending with the 10th one).

Note

Upon resuming a run, Factor will pick up changes to the parset and directions file. However, changes that alter the facet layout will result in incorrect results, as Factor does not handle these properly.

Resetting a direction

Factor allows for the processing of a direction to be reset. Resetting involves deleting the output products, resetting the state (which tracks which operations have been completed), and if necessary undoing the subtraction of the model resulting from self calibration and the imaging of the facet. Directions can be reset using the -r flag to runfactor. For example, to reset all operations for direction1 and direction2:

$ runfactor factor.parset -r direction1,direction2

Additionally, one or more specific operations can reset by including -o flag. For example, the following would reset only the facetimage operation for direction1 and direction2:

$ runfactor factor.parset -r direction1,direction2 -o facetimage

Note

If you want to reset all directions and all operations (i.e., to start the processing over from the very start), you can simply delete (or move) the results and state directories in the Factor working directory (see below), then restart Factor (without the -r or -o flags). Factor will then start the entire reduction again, but will skip the chunking of the input data files.