...

SELECT DISTINCT(obs_collection) FROM ivoa.obscore

TOT HIER

Export machine readable table

There are multiple ways to export a catalog of the various data products of the data release. On the vo.astron.nl pages, the results of a query can be exported to a csv file or fits table; running an empty query with a table limit of 5000 or more will return all entries.

The ADQL form is another option, and below we provide an example query that also provides information about the calibrators used for each beam. This query is specific to the continuum_images data product but can be adapted to other (beam-based, processed) data products by replacing the table name, e.g., for polarization cubes/images use pol_cubes (see Table 10 for a full list of the available tables).

...

In some cases, you may want to combine data from different tables. This can been done using so-called JOINs. The trick here is to tell the system what column in what table needs to be used to join ON.

SELECT TOP 10 * FROM apertif_dr1.continuum_images data JOIN apertif_dr1.flux_cal_visibilities flux_cal on data.obsid=flux_cal.used_for

This means that we want the first 10 entries FROM the apertif_dr1.continuum_images table, which we will call data further on in the query to save us from making typos, which should be linked to the data in the

...

apertif_dr1.flux_cal_visibilities

...

, which we call flux_cal

...

, and tell it that the link should happen for each data set that appears in both and for which the obsid value in the data table is the same as the used_for value in the flux_cal table.

Export machine readable table from Apertif DR1

With everything we learned till now, we can start constructing complex queries. One good example of thjis is the query to obtain all the data in Apertif DR1. This query is specific to the continuum_images data product but can be adapted to other (beam-based, processed) data products by replacing the table name, e.g., for polarization cubes/images use pol_cubes.

select data.*,
flux_cal.obsid as flux_calibrator_obs_id,
pol_cal.obsid as pol_calibrator_obs_id  from apertif_dr1.continuum_images data
join  apertif_dr1.flux_cal_visibilities flux_cal on data.obsid=flux_cal.used_for and data.beam_number=flux_cal.beam
join apertif_dr1.pol_cal_visibilities pol_cal on data.obsid=pol_cal.used_for and data.beam_number=pol_cal.beam
order by obsid

Python access

The data collection and the table content can be accessed directly via python using the pyvo library. Working directly in python the tables and the data products can be simply queried and outputs can be customized according to the user’s needs, without the involvement of TOPCAT or ALADIN.

An example of a SIA query and image download can be found in the python script below (it has been tested for python 3.7). The result of the query can also be plotted using python.

#To start you have to import the library pyvo (it is also possible to use astroquery if you want)
import pyvo

## Now we will query for the LoTSS PDR SIA service at ASTRON
sia_services = pyvo.registry.search(keywords='LoTSS PDR', servicetype='sia')

## This will return two services: the cutout, and the images. We want the second item in the list so:
vo_siap_service = sia_services[1]

## If we actually know the VO identifier of the service we want to query we could also do
from pyvo.registry.regtap import ivoid2service
vo_siap_service = ivoid2service('ivo://astron.nl/lofartier1/q_img/imgs')[0]
	
## And now we can query for all images that cover a specific point on the sky
all_images = vo_siap_service.search(pos=(171.1, 49.6))

## So let's see what the columns of our result are:
print("Columns of the result:")
for field in all_images.fielddescs:
  print(f"name: {field.name} - units: {field.unit}")

## all_images should be sorted by distance so we pick the first for plotting. So let's grab the first one and load it in memory
hdulist = all_images[0].getdataobj()
# this may take a while for the file to download...

from astropy.wcs import WCS
import matplotlib.pyplot as plt
from matplotlib import colors

hdu = hdulist[0]  # we take the first hdu from the FITS
wcs = WCS(hdu.header)
wcs = wcs.dropaxis(2).dropaxis(2) # axis 2 of the WCS is polarisation and we are not interested in that here
plt.subplot(projection=wcs)
plt.imshow(hdu.data[0,0, 5000:7000,5000:7000], norm=colors.LogNorm(vmin=0.005), cmap='gray_r')
plt.xlabel('RA')
plt.ylabel('DEC')
plt.show()

Of course one could also use TAP to find the data using ADQL, download the file and imaging it, For Apertif DR1 it would look like this

Python access

The data collection and the table content can be accessed directly via python using the pyvo tool. Working directly in python the tables and the data products can be simply queried and outputs can be customized according to the user’s needs, without the involvement of TOPCAT or ALADIN.

An example of a TAP query and image download can be found in the python script below (it has been tested for python 3.7). The result of the query can also be plotted using python.

#To start you have to import the library pyvo (it is also possible to use astroquery if you want)
import pyvo

## To perform a TAP query you have to connect to the service first
tap_service = pyvo.dal.TAPService('https://vo.astron.nl/__system__/tap/run/tap')

# This works also for
formfrom pyvo.registry.regtap import ivoid2service
vo_tap_service = ivoid2service('ivo://astron.nl/tap')[0]

# The TAPService object provides some introspection that allow you to check the various tables and their
# description for example to print the available tables you can execute
print('Tables present on http://vo.astron.nl')
for table in tap_service.tables:
 print(table.name)
print('-' * 10 + '\n' * 3)

# or get the column names
print('Available columns in apertif_dr1.continuum_images')
print(tap_service.tables['apertif_dr1.continuum_images'].columns)
print('-' * 10 + '\n' * 3)

## You can obviously perform tap queries accross the whole tap service as an example a cone search
print('Performing TAP query')
result = tap_service.search(
 "SELECT TOP 5 target, beam_number, accref, centeralpha, centerdelta, obsid, DISTANCE(" \
     "POINT('ICRS', centeralpha, centerdelta),"\
     "POINT('ICRS', 208.36, 52.36)) AS dist"\
 " FROM apertif_dr1.continuum_images"  \
 " WHERE 1=CONTAINS("
 "    POINT('ICRS', centeralpha, centerdelta),"\
 "    CIRCLE('ICRS', 208.36, 52.36, 0.08333333)) "\
 " ORDER BY dist ASC"
 )
print(result)

# The result can also be obtained as an astropy table
astropy_table = result.to_table()
print('-' * 10 + '\n' * 3)

## You can also download and plot the image
import astropy.io.fits as fits
from astropy.wcs import WCS
import matplotlib.pyplot as plt
import requests, os
import numpy as np

# DOWNLOAD only the first result
#
print('Downloading only the first result')
file_name = '{}_{}_{}.fits'.format(
 result[0]['obsid'].decode(),
 result[0]['target'].decode(),
 result[0]['beam_number'])
path = os.path.join(os.getcwd(), file_name)
http_result = requests.get(result[0]['accref'].decode())
print('Downloading file in', path)
with open(file_name, 'wb') as fout:
 for content in http_result.iter_content():
     fout.write(content)
hdu = fits.open(file_name)[0]
wcs = WCS(hdu.header)
# dropping unnecessary axes
wcs = wcs.dropaxis(2).dropaxis(2)
plt.subplot(projection=wcs)
plt.imshow(hdu.data[0, 0, :, :], vmax=0.0005)
plt.xlabel('RA')
plt.ylabel('DEC')
plt.show()

...