Page History
...
The science-ready data products as described in Available data products are exposed through standard Virtual Observatory (VO) protocols to facilitate their access and exploration. You can find an introduction to the usage of the VO on this page. In the following we will be refering to several VO standards (SIA, DataLink, etc.) which are all discussed in the VO documentation.
The LoTSS DR2 data itself is hosted on the SURF Data Repository (sdr) system. We recommend astronomers to use the VO interfaces described below for data discovery. Some of the data is stored on tape. This is indicated in the data description in the DataLink document that is returned by the VO query. To access files on tape, users need to obtain an account which gives them the possibility to "stage" the data from the tape to a disk from which it can be downloaded. Data that is stored on disk is directly accessible through the VO.
In particular, the protocols offered are the Tabular Access Protocol (TAP), Simple Application Messaging Protocol (SAMP) and the Simple Image Access protocol (SIA). TAP and SAMP enables queries to explore the data in a tabular form using tools such as TOPCAT. TOPCAT is an interactive graphical viewer and editor for tabular data, it enables the interactive exploration of large tables performing several types of plotting, statistics, editing and visualization of tables. SIA enables the rapid display of images and cubes through all sky atlas tools such as ALADIN. ALADIN is an interactive sky atlas allowing the user to visualize digitized astronomical images/cubes and superimpose entries from astronomical catalogues or databases.
The data published in the VO can also be accessed using a web browser at https://vo.astron.nl. This web interface provides a page on which all the collections present in the registry are listed, including the published LoTSS DR2 data sets:
...
LoTSS-DR2 Gaussian catalog cone search
...
Selecting a data collection allows the user to perform a cone search through a webform (Fig. 40). The result is either a source or Gaussian list, or a table of data products of that given class overlapping a given pointing. The size of the continuum images as well as the cubes extend beyond the 10% primary beam level for cleaning the secondary lobes of bright offset sources. To ensure that the search is done in the area of maximum sensitivity the search is performed on a maxim radius of 0.75 degrees from the center (this represents the average value of where the sensitivity drops). This value can be modified using the Max distance from center. A different output with respect to the default can be customized using More output fields selection buttonAlso the raw observations from which the survey data has been derived are accessible through the Lofar Long-Term Archvie. The main data products (images) are all directly accessible through the VO (either through the Simple Image Access, or via DataLinks to auxiliary data products). The raw data is stored on tape, which means that they need to be put on disk for access. This process is refered to as "staging". The DataLink entries to the files on tape all point to the page of the sdr which lists all the data from a specific pointing. An example of such a page is shown in Fig. 1. This page shows all data that is directly downloadable (with the status "online" or "external), data that is in the process of being staged (with the status "staging") or data that needs to be staged (with the status "offline").
| Info | ||
|---|---|---|
| ||
 Fig. 40 Query search form for continuum images. |
The result is a table in the requested output format in which every row corresponds to a data product (Fig. 41).
In each row there is a column, Product key, which is a link that allows the user to download the fits file of the image of interest. The column titles should generally be self-descriptive. However, the long human-readable description of the content of each column is a tooltip that will appear when hovering over the column name.
The selected target and the position of the individual pointings can be visualized using the Quick plot button at the top of the window of the results of the search query (Fig. 41).
In the column Related products another link connects to a page containing a list of links to additional related data that can be useful to interpret or reanalyze that given product, for which a preview is provided (Fig. 42).
Some data products are directly accessible, in which case the link on this table will initiate a direct download. However, the larger (and rawer) data products are stored on tape (in which case the text 'on tape' appears in the description). The link will then bring you to the entry of the corresponding pointing in the SURF Data repository (Fig. 43/1). If data is on tape, the status is either "online", in which case it can be downloaded by pressing the "Download" button, or "offline" in which case it can be requested to be put online by pressing the "Request" button. Please note that you need to be logged in to perform the request. If a request has been correctly performed, the status of the file will change to "staging" unill it becomes "online".
Fig. 1 Surf Data Repository view of the data from a pointing with all possible statuses (online, staging, offline and external). Click for a bigger image. |
To obtain more information about a file, you can click on the small arrow next to it. This will show a description (the same as is used in the DataLink), a persistent identifier that can be used to cite the data product itself when used in a publication, and a checksum (of the Adler32 format) that one could use to check the integrity of a downloaded product. To stage the data, you can click the "Request" button next to the "offline" status field. After confirming, a request will be made to copy the data from tape to disk where it will remain for a while for users to download it. This workflow is illustrated in Fig. 2.
| Info | ||||||
|---|---|---|---|---|---|---|
| ||||||
Fig. 2 Surf Data Repository work flow of staging. Top-left click the "request" button, Top right: click "Ok" in te popup. Then the status of the file will go to "staging" (bottom left). After waiting for a while the status will become "online". This may take some time, depending on the queue lenght at SURFs tape drives, and the file size. The page refreshes regularly but you can also manually refresh of course. Bottom right: After staging you can click the "Download" button to get the data product | ||||||
| Info | ||||||
| ||||||
 Fig. 43/1 Surf Data Repository view of the data from a pointing. Click for a bigger image. |
The source and Gaussian cone search forms each return a table with source positions and properties. As before, the long descriptions are available using tool tips. The columns "Mosaic_URL" links to the anciliary data product page of the mosaic from where the Gaussian or source was extracted (like e.g. Fig 42).
Data Usage Policy
Data access
The tables of mosaics and sources can be found in the ASTRON Virtual Observatory.
The entry of LoTSS DR2 in the SURF Data Repository.
Also you can see the HiPS of the high-resolution (primary data products) and low-resolution data.
More information can be found on the LOFAR surveys webpage and in:
Shimwell, T.W., et al. “The LOFAR Two-metre Sky Survey (LoTSS). V. Second data release" accepted for publication in Astronomy & Astrophysics, DOI 10.1051/0004-6361/202142484.
Data usage policy
Papers making use of LOFAR data should follow the LOFAR Data Policy.
This states that papers Papers making use of this data should include the following acknowledgement:
This
...
paper is based (in part) on data obtained with the International LOFAR Telescope (ILT). LOFAR (van Haarlem et al. 2013) is the Low Frequency Array designed and constructed by ASTRON. It has observing, data processing, and data storage facilities in several countries, that are owned by various parties (each with their own funding sources), and that are collectively operated by the ILT foundation under a joint scientific policy. The ILT resources have benefited from the following recent major funding sources: CNRS-INSU, Observatoire de Paris –PSL and Université d'Orléans, France; BMBF, MIWF-NRW, MPG, Germany; Science Foundation Ireland (SFI), Department of Business, Enterprise and Innovation (DBEI), Ireland; NWO, The Netherlands; The Science and Technology Facilities Council, UK; Ministry of Science and Higher Education, Poland; The Istituto Nazionale di Astrofisica (INAF), Italy.
In addition, they should include the LoTSS DR2 PID:
CHECK DOI
21.12136/95f9f5a9-406e-4270-ad8b-45e36c809561 and cite the imaging survey data release paper. This is not yet available but should be soon. Until then, it can be referenced as: Shimwell, T.W., et al. “The LOFAR Two-metre Sky Survey (LoTSS). V. Second data release" accepted for publication in Astronomy & AstrophysicAstrophysics, DOI 10.1051/0004-6361/202142484.
Overview
Content Tools
Tasks