Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
  • in query form say we use the default but more can be found using the service info and topcat/pyvo.

Introduction

This page desribes describes access to the data in ASTRON's data holdings using the Virtual Observatory standards. Those standards support access to data like catalogs and images. We introduce the most important standards on this page and give some usage exaples taylored examples tailored to the ASTRON data holdings. If you want more advanced information on how to use the VO standards, you can start with the IVOA website for astronomers and the IVOA wiki page on educational resources. For our data holdings we use the DAta Center Helper Suite (DACHS), which offers a web interface of its own. We will link to several places in DACHS in the documentation but in principle all information showed by DACHS can also be accessed using any VO tools.

...

The VO consists of several different protocols. In this section, we describe the import ant important protocols, and provide the acronyms that are generally used in the VO tooling. Most data in vo.astron.nl consist of tables, that can be accessed using the Tabular Access Protocol (TAP). Images are offered using the Simple Image Access Protocol (SIA) which in essence is a table with a field containing a link to an image. A few services offer cutout fuctionality functionality (using SODA). A cutout is essentially a service that returns an image (or more generally a data product) that only contains the data inside the cone that has been requested by the user.

...

The data published in the VO can also be accessed using a web browser at https://vo.astron.nl. The web interface can be used to perform simple queries. The power of the VO lies in the fact that several applications exist that can interact with the standards we offer. In the architecture, there is a distinction between services and tables. A service is an entity that takes user input and provides a table as output. The tables themselves are the entities that actually hold the data. Tables can also be queried directly, for instance using ADQL. The colomns In the following table, we list the services for accessing data collections in the ASTRON VO. The columns of the table are:

Service name

The (human-readable) name of the service

TypeThe type of this service. Serrvices Services of type system are not coupled to specific data collections. The services that provide access to data are either of type image or cube. The image cutout services also offer images, but those are cutouts as define above. Finally there are catalogues which are either catalogues of astrophysical objects, or catalogues which tabulate the properties of the raw data files, that can be used to request them for download (in the Apertif DR1 case).
Table nameName of the table in the TAP service, or N/A for services that are not directly querying a table.
Service descriptionHuman-readable description of the service
Link to col. descsLink to the service info page in DACHS. For services that relate to a table (i.e. anything but the system services), this will contain three tables listing columns. The first table shows the input fields, which are the fields that can be used to query the service (typically those are RA, DEC and search radius). The next table shows the default output of a query, which is the subset of the table that has a verbosity of 20 or less. The third table shows all the columns present in the data table, including the ones with verbosity levels higher than 20. In the examples we will show how to access those columns. The table columns on the service info page show the column name (i.e. the term to query on using ADQL), the table header (e,g. when querying on DACHS this ends up as the table header for a column), a human-readable description, the unit of the data in this field, and the Unified Content Descriptor (UCD) which is a machine-readable definition of the data in the column. The goal of a UCD is to make sure that clients know how to handle the data in the column. For the average user those are probably not too relevant.

...

This web interface provides a page on which all the collections present in the registry are listed.

Service nameTypeTable nameService descriptionService info
ADQL QuerysystemN/AThis provides a form that can be used to perform ADQL queries and get results in different formats.info (no columns)
LBCS Calibrator Searchcataloguelbcs.mainCatalog of calibrators from the Long-Baseline Calibrator Survey (LBCS).info
LoLSS - Image Cutout Serviceimage cutoutlolss.mosaicCutouts from the images of the LOFAR LBA Sky Survey (LoLSS).
info
LoLSS source catalogcataloguelolss.source_catalogSource catalogue of the LOFAR LBA Sky Survey (LoLSS).info
LoTSS-DR1 Cross-Matched Source Cataloguecatalogue

hetdex.main_merged

Catalogue of radio sources from the Lofar Two-metre Sky Survey First Data Release (LoTSS-DR1) that have been cross-matched with an optical or infrared counterpart.info
LoTSS-DR1 Image Archiveimagehetdex.hetdex_imagesImages of the Lofar Two-metre Sky Survey First Data Release (LoTSS-DR1).info
LoTSS-DR1 Image Cutout Serviceimage cutouthetdex.hetdex_imagesCutouts from the images of the Lofar Two-metre Sky Survey First Data Release (LoTSS-DR1).info
LoTSS-DR1 Raw Radio Catalogue Cone Searchcataloguehetdex.mainCatalogue of radio sources from the Lofar Two-metre Sky Survey First Data Release (LoTSS-DR1).
info
PDR Image Archivecataloguelotss_dr2.main_gaussesCatalogue of the Stokes I continuum mosaic gaussian components.info
LoTSS-DR2 Source catalog cone search cataloguelotss_dr2.main_sourcesCatalogue of the Stokes I continuum mosaic sources.info
LoTSS-DR2 mosaicsimagelotss_dr2.mosaicsCatalogue of the Stokes I continuum mosaic images.info
LoTSS-PDR Image Archiveimagelofartier1.img_mainImages of the Lofar Two-metre imagelofartier1.img_mainImages of the Lofar Two-metre Sky Survey Preliminary Data Release (LoTSS-PDR).info
LoTSS-PDR Image Cutout Serviceimage cutoutlofartier1.img_mainCutouts from the images of the Lofar Two-metre Sky Survey Preliminary Data Release (LoTSS-PDR).info
LoTSS-PDR Source Cataloguecataloguelofartier1.mainCatalogue of radio sources from the Lofar Two-metre Sky Survey Preliminary Data Release (LoTSS-PDR).info
M) Apertif DR1 - Continuum imagesimageapertif_dr1.continuum_imagesContinuum images of the Apertif First Data Release (Apertif DR1).info
M) Apertif DR1 - HI spectral cubescubeapertif_dr1.spectral_cubesHI spectral cubes of the Apertif First Data Release (Apertif DR1).info
M) Apertif DR1 - Polarization images and cubesimage/cubeapertif_dr1.pol_cubesStokes V image and Stokes Q and U cubes of the Apertif First Data Release (Apertif DR1).info
MSSS Verification Field Imagesimagemvf.msssvf_img_mainBoth LBA and HBA images from the Multifrequency Snapshot Sky Survey (MSSS) verification field.
info
MSSS Verification Field Sourcescataloguemvf.mainUnified source catalogue from the Multifrequency Snapshot Sky Survey (MSSS) verification field.info
S) Apertif DR1 - Field calibrated visibilitiescatalogueapertif_dr1.calibrated_visibilitiesCatalogue of the properties from the calibrated visibilities of the fields from the Apertif First Data Release (Apertif DR1).info
S) Apertif DR1 - Field raw visibilitiescatalogueapertif_dr1.raw_visibilitiesCatalogue of the properties from the raw visibilities of the fields from the from the Apertif First Data Release (Apertif DR1).info
S) Apertif DR1 - Flux calibrator raw visibilitiescatalogueapertif_dr1.flux_cal_visibilitiesCatalogue of the properties from the calibrated visibilities of the flux calibrators from the Apertif First Data Release (Apertif DR1).info
S) Apertif DR1 - Pol. calibrator raw visibilitiescatalogueapertif_dr1.pol_cal_visibilitiesCatalogue of the properties from the calibrated visibilities of the polarisation calibrators from the Apertif First Data Release (Apertif DR1).info
SAURON HI Survey Imagesimagesauron.mom0Images of the SAURON WSRT HI Survey.info
SAURON HI Survey Velocity Fieldsimagesauron.mainVelocity field images of the SAURON WSRT HI Survey.info
TGSSADR Image Archiveimagetgssadr.img_mainImages of the TIFR GMRT Sky Survey Alternative Data Release (TGSS ADR).info
TGSSADR Image Cutout Serviceimage cutouttgssadr.img_mainCutouts from the images of the TIFR GMRT Sky Survey Alternative Data Release (TGSS ADR).info
TGSSADR Source Cataloguecataloguetgssadr.mainCatalogue of the radio sources in the TIFR GMRT Sky Survey Alternative Data Release (TGSS ADR).info
The VO @ ASTRON TAP servicesystem

N/A

(info only): description on how to access the tables using the TAP protocol.info (no columns)
Apertif DR1 beam cubes (table only, no service)
apertif_dr1.beam_cubesBeam cubes from the Apertif First Data Release (Apertif DR1), no connected service but accessible via TAP and linked from the other Apertif DR1 tables through DataLink.N/A

...

The DaCHS system comes with a web view on the services offered. Even though the interface is somewhat rudimentary, it does offer the relevant functionality to query data in the ASTRON VO. The main page, listing all services (as listed in the table above) can be fount found at https://vo.astron.nl . Specific services are linked to that (either by clicking on the name of the collection, or the Q icon next to it.

Selecting a data collection allows the user to perform a cone search through a webform (Fig. 1). The result is either a catalogue (list of sources, Gaussians or visibilities), or a table of data products (images, cubes) of that given class overlapping a given pointing. In some cases, the number of query parameters can differ, in which case the field description explains what the data is that can be submitted to them.

...

The result is a table in the requested output format in which every row corresponds to a data product (Fig. 2). In each row there is a column, Product key, which is a link that allows the user to download the fits file of the image of interest. The column titles should generally be self-descriptive. However, the long human-readable description of the content of each column is a tooltip that will appear when hovering over the column name. The result is a table (like Fig. 2) which consists of a link to the data product, a thumbnail image  image that appears when hovering it (both only if querying for a data product) and relevant metadata describing the results. The Quick Plot button on top of the results page can be used to quicikly quickly plot the numerical results (e.g. making an RA/DEC overview of the images or sources). The Send via SAMP button can be used to send the result set to an application using the SAMP protocol. This essentially means that if you have another application open which supports the SAMP protocol (as do both TOPCAT and Aladin), clicking this button will make the table appear in those applications.

...

One special service is the ADQL query service, shown on Fig 4.. This contains a single box that can be used to provide custom queries using the Astronomical Data Query Language (ADQL), which in essence is a dialect of SQL. Some more information on, and examples of ADQL can be found further below.

Info
iconfalse

Anchor
dachs-adql
dachs-adql

Image Modified

Fig. 4 ADQL query form. Click for a bigger image.

...

The VO data collections can easily beThe Apertif DR1 data collection tables can be accessed using TOPCAT, an interactive graphical viewer and editor for tabular data. Since the ASTRON VO is regsitered registered to the so-called registry of registries, TOPCAT can find the relevant services in its query menus. In the VO menu of TOPCAT you can pick the SIA query menu item, which will bring up a form as shown in Fig. 5. When selecting one of the data collections the SIA URL will be automatically filel filled in. One can then either type coordinates, or an object name and click the "Resolve" button which uses Simbad to obtain coordinates for that obiectobject. When clicking OK, a table of the data products found will appeared appear in the main TOPCAT window. Alternatively, the table can be saved in various formats and used locally with other programs (e.g. python scripts etc).

...

One of the most popular tools to access data through the VO is Aladin Sky Atlas. There are both a browser and standalone desktop edition. When refering referring to Aladin in the following, this will apply to the desktop edition. Aladin is a tool that makes it possible to visualise tables and images and perform some analysis on them.

...

The SIA and TAP services are also available directly from Aladin. First of all, load a HiPS (any HiPS will do;  but it may be useful to use the HiPS from the same data release).  To find the images, you can select Others → SIA → astron.nl → <data collection name> (purple directory icons). When double-clicking the collection (or clicking t he the Load button) the popup screen. On top of the HiPS image, small shapes will appear that each represent the central pointing of each of the images (see Fig. 10; left). Note that the default colour of the shapes is red but the colour in the screenshot has been changed for contrast.  When clicking one of the shapes the meta data of the selected object appears in a table below the figure and clicking on the image URL will then initiate a download of the original image, superimposing it on top of the HiPS (Fig. 10; right). Note that the stack on the right of the Aladin screen shows all loaded layers and that the layer in which the image was loaded is showing the percentage of the image loaded.

...

Access to catalogues through TAP is handled in a way comparable to images through SIA. Open ALADIN and on the left panel for SIAP: select Others → CS → astron.nl → <data collection name> (purple directory icons). A pop-up window will open. Click load. Note that "in view" is clicked bu default, meaning that the source in view are being loaded by default. Also note that if the Field of View is too large, the loading may fail because of a overflow on the server side (ie the query resulted in too many results). After loading, each of the catalogue entries is superimposed on top of the HiPS image as, again, a small red circle (and again this coulour colour has been changed in the screen shot screenshot for contrast). You can then click the "select" button and select several sources (by drawing a rectangle around them) and the table of properties will appear beneath the HiPS. See Fig. 11 for a view of those steps.

...

In several places, ADQL was mentioned as a way to construct queries to the data and catalogues. ADQL is a query language resembling SQL, so the main concepts are familiar to those acquianted acquainted with SQL. The full  reference documentation (i.e. stnadard standard definition) contains all possible commands. Here we show some simple examples of table queries. You can perform queries using ADQL on the ADQL form (see Fig. 4), or for instance in TOPCAT (see Fig. 6). Also note that TOPCAT has some built in examples implemented. Even though the ADQL commands are case insensitive, it is

In the following examples, we will follow the common practice to write them SQL commands in upper case so that table data and variable values can easily been distinguished from commands, and names of objects (tables, columns, etc) in lower case letters. The SQL commands are however case insensitive so you are totally free to for instance write select instead of SELECT.

Basic examples

The first example is simply selected all items from the lcbsw.main table:

...

With everything we learned till now, we can start constructing complex queries. One good example of thjis this is the query to obtain all the data in Apertif DR1. This query is specific to the continuum_images data product but can be adapted to other (beam-based, processed) data products by replacing the table name, e.g., for polarization cubes/images use pol_cubes.

...