This subpackage contains modules supporting VO client-side operations.
astropy.vo.client.vos_catalog contains common utilities for accessing simple VO services.
These parameters are set via Configuration system (astropy.config):
>>> from astropy.vo.client import vos_catalog
Get all catalogs from a database named 'conesearch_good' (this contains cone search services that cleanly passed daily validations; also see Cone Search Examples):
>>> my_db = vos_catalog.get_remote_catalog_db('conesearch_good')
Downloading http://stsdas.stsci.edu/astrolib/vo_databases/conesearch_good.json
|============================================| 56/ 56k (100.00%) 00s
>>> my_db
<astropy.vo.client.vos_catalog.VOSDatabase at 0x2b0f3d0>
>>> print(my_db)
Guide Star Catalog 2.3 1
SDSS DR7 - Sloan Digital Sky Survey Data Release 7 1
SDSS DR7 - Sloan Digital Sky Survey Data Release 7 2
# ...
USNO-A2 Catalogue 1
USNO-A2.0 1
If you get timeout error, you need to use a custom timeout as follows:
>>> from astropy.utils.data import REMOTE_TIMEOUT
>>> with REMOTE_TIMEOUT.set_temp(30):
... my_db = vos_catalog.get_remote_catalog_db('conesearch_good')
Find catalog names containing 'usno*a2':
>>> my_db.list_catalogs(pattern='usno*a2')
[u'The USNO-A2.0 Catalogue (Monet+ 1998) 1', u'USNO-A2 Catalogue 1']
Get information for a catalog titled 'USNO-A2 Catalogue 1':
>>> my_cat = my_db.get_catalog('USNO-A2 Catalogue 1')
>>> my_cat
<astropy.vo.client.vos_catalog.VOSCatalog at 0x1f78150>
>>> print(my_cat)
title: USNO-A2 Catalogue
url: http://www.nofs.navy.mil/cgi-bin/vo_cone.cgi?CAT=USNO-A2&
>>> print(my_cat.dumps())
{
"capabilityClass": "ConeSearch",
"capabilityStandardID": "ivo://ivoa.net/std/ConeSearch",
"capabilityValidationLevel": "",
"contentLevel": "#University#Research#Amateur#",
# ...
"version": "",
"waveband": "#Optical#"
}
>>> my_cat.keys()
[u'validate_network_error',
u'capabilityClass',
u'updated',
# ...
u'identifier',
u'validate_xmllint']
>>> my_cat['url']
u'http://www.nofs.navy.mil/cgi-bin/vo_cone.cgi?CAT=USNO-A2&'
>>> my_cat['maxRadius']
1.0
One can also get information for a catalog using its URL. If a URL yields multiple catalogs (this can happen when the service provider re-register the URL with a different title), only the first match is returned:
>>> my_cat2 = my_db.get_catalog_by_url(
... 'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&')
>>> print(my_cat2.dumps())
{
"capabilityClass": "ConeSearch",
"capabilityStandardID": "ivo://ivoa.net/std/ConeSearch",
"capabilityValidationLevel": "",
"contentLevel": "#Research#",
# ...
"version": "15-Sep-1999",
"waveband": "#Optical#"
}
To see validation warnings generated by Validation for Simple Cone Search for the catalog above:
>>> for w in my_cat2['validate_warnings']:
... print(w)
/.../vo.xml:13:0: W22: The DEFINITIONS element is deprecated in VOTable 1.1...
To get all the matching catalogs by URL:
>>> matched_cats = [cat for key, cat in my_db.get_catalogs_by_url(
... 'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&')]
>>> for c in matched_cats:
... print(str(c))
title: The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99)
url: http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&
To get all catalogs in the database:
>>> all_cats = [cat for key, cat in my_db.get_catalogs()]
By default, pedantic is False:
>>> from astropy.io.votable.table import PEDANTIC
>>> PEDANTIC()
False
To call a given VO service; In this case, a Cone Search (also see Cone Search Examples):
>>> from astropy import coordinates as coord
>>> from astropy import units as u
>>> c = coord.ICRS.from_name('47 Tuc')
>>> c
<ICRS RA=6.02233 deg, Dec=-72.08144 deg>
>>> sr = 0.5 * u.degree
>>> sr
<Quantity 0.5 deg>
>>> result = vos_catalog.call_vo_service(
... 'conesearch_good',
... kwargs={'RA': c.ra.degree, 'DEC': c.dec.degree, 'SR': sr},
... catalog_db='The PMM USNO-A1.0 Catalogue (Monet 1997) 1')
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
WARNING: W03: ... Implictly generating an ID from a name 'RA(ICRS)'...
WARNING: W03: ... Implictly generating an ID from a name 'DE(ICRS)'...
>>> result
<astropy.io.votable.tree.Table at 0x3cc4850>
To repeat the above and suppress all the screen outputs (not recommended):
>>> import warnings
>>> with warnings.catch_warnings():
... warnings.simplefilter('ignore')
... result = vos_catalog.call_vo_service(
... 'conesearch_good',
... kwargs={'RA': c.ra.degree, 'DEC': c.dec.degree, 'SR': sr},
... catalog_db='The PMM USNO-A1.0 Catalogue (Monet 1997) 1',
... verbose=False)
astropy.vo.client.conesearch supports VO Simple Cone Search capabilities.
Available databases are generated on the server-side hosted by STScI using Validation for Simple Cone Search. The database used is controlled by astropy.vo.client.conesearch.CONESEARCH_DBNAME, which can be changed in Configurable Items below. Here are the available options:
Default. Passed validation without critical warnings and exceptions.
Has critical warnings but no exceptions. Use at your own risk.
Has some exceptions. Never use this.
Has network connection error. Never use this.
In the default setting, it searches the good Cone Search services one by one, stops at the first one that gives non-zero match(es), and returns the result. Since the list of services are extracted from a Python dictionary, the search order might differ from call to call.
There are also functions, both synchronously and asynchronously, available to return all the Cone Search query results. However, this is not recommended unless one knows what one is getting into, as it could potentially take up significant run time and computing resources.
Examples below show how to use non-default search behaviors, where the user has more control of which catalog(s) to search, et cetera.
Note
Most services currently fail to parse when pedantic=True.
Warning
When Cone Search returns warnings, user should decide whether the results are reliable by inspecting the warning codes in astropy.io.votable.exceptions.
These parameters are set via Configuration system (astropy.config):
Also depends on General VO Services Access Configurable Items.
>>> from astropy.vo.client import conesearch
Shows a sorted list of Cone Search services to be searched (to inspect them in detail, see General VO Services Access):
>>> conesearch.list_catalogs()
[u'Guide Star Catalog 2.3 1',
u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 1',
u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 2',
u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 3',
u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 4',
u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 1',
u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 2',
u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1',
u'The PMM USNO-A1.0 Catalogue (Monet 1997) 1',
u'The USNO-A2.0 Catalogue (Monet+ 1998) 1',
u'Two Micron All Sky Survey (2MASS) 1',
u'Two Micron All Sky Survey (2MASS) 2',
u'USNO-A2 Catalogue 1',
u'USNO-A2.0 1']
Select a catalog to search:
>>> my_catname = 'The PMM USNO-A1.0 Catalogue (Monet 1997) 1'
By default, pedantic is False:
>>> from astropy.io.votable.table import PEDANTIC
>>> PEDANTIC()
False
Perform Cone Search in the selected catalog above for 0.5 degree radius around 47 Tucanae with minimum verbosity, if supported. The catalog_db keyword gives control over which catalog(s) to use. If running this for the first time, a copy of the catalogs database will be downloaded to local cache. To run this again without using cached data, set cache=False:
>>> from astropy import coordinates as coord
>>> from astropy import units as u
>>> c = coord.ICRS.from_name('47 Tuc')
>>> c
<ICRS RA=6.02233 deg, Dec=-72.08144 deg>
>>> sr = 0.5 * u.degree
>>> sr
<Quantity 0.5 deg>
>>> result = conesearch.conesearch(c, sr, catalog_db=my_catname)
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
To run the command above using custom timeout of 30 seconds for each Cone Search service query:
>>> from astropy.utils.data import REMOTE_TIMEOUT
>>> with REMOTE_TIMEOUT.set_temp(30):
... result = conesearch.conesearch(c, sr, catalog_db=my_catname)
To suppress all the screen outputs (not recommended):
>>> import warnings
>>> with warnings.catch_warnings():
... warnings.simplefilter('ignore')
... result = conesearch.conesearch(c, sr, catalog_db=my_catname,
... verbose=False)
Extract Numpy array containing the matched objects. See numpy for available operations:
>>> cone_arr = result.array.data
>>> cone_arr
array([(0.499298, 4.403473, -72.124045, '0150-00088188'),
(0.499075, 4.403906, -72.122762, '0150-00088198'),
(0.499528, 4.404531, -72.045198, '0150-00088210'), ...,
(0.4988, 7.641731, -72.113156, '0150-00225965'),
(0.499554, 7.645489, -72.103167, '0150-00226134'),
(0.499917, 7.6474, -72.0876, '0150-00226223')],
dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
('USNO-A1.0', '|S13')])
>>> cone_arr.dtype.names
('_r', '_RAJ2000', '_DEJ2000', 'USNO-A1.0')
>>> cone_arr.size
36184
>>> ra_list = cone_arr['_RAJ2000']
>>> ra_list
array([ 4.403473, 4.403906, 4.404531, ..., 7.641731, 7.645489, 7.6474 ])
>>> cone_arr[0] # First row
(0.499298, 4.403473, -72.124045, '0150-00088188')
>>> cone_arr[-1] # Last row
(0.499917, 7.6474, -72.0876, '0150-00226223')
>>> cone_arr[:10] # First 10 rows
array([(0.499298, 4.403473, -72.124045, '0150-00088188'),
(0.499075, 4.403906, -72.122762, '0150-00088198'),
(0.499528, 4.404531, -72.045198, '0150-00088210'),
(0.497252, 4.406078, -72.095045, '0150-00088245'),
(0.499739, 4.406462, -72.139545, '0150-00088254'),
(0.496312, 4.410623, -72.110492, '0150-00088372'),
(0.49473, 4.415053, -72.071217, '0150-00088494'),
(0.494171, 4.415939, -72.087512, '0150-00088517'),
(0.493722, 4.417678, -72.0972, '0150-00088572'),
(0.495147, 4.418262, -72.047142, '0150-00088595')],
dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
('USNO-A1.0', '|S13')])
Sort the matched objects by angular separation in ascending order:
>>> import numpy as np
>>> sep = cone_arr['_r']
>>> i_sorted = np.argsort(sep)
>>> cone_arr[i_sorted]
array([(0.081971, 5.917787, -72.006075, '0150-00145335'),
(0.083181, 6.020339, -72.164623, '0150-00149799'),
(0.089166, 5.732798, -72.077698, '0150-00137181'), ...,
(0.499981, 7.024962, -72.477503, '0150-00198745'),
(0.499987, 6.423773, -71.597364, '0150-00168596'),
(0.499989, 6.899589, -72.5043, '0150-00192872')],
dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
('USNO-A1.0', '|S13')])
Result can also be manipulated as VOTable XML handling (astropy.io.votable) and its unit can be manipulated as Units and Quantities (astropy.units). In this example, we convert RA values from degree to arcsec:
>>> from astropy import units as u
>>> ra_field = result.get_field_by_id('_RAJ2000')
>>> ra_field.title
u'Right ascension (FK5, Equinox=J2000.0) (computed by VizieR, ...)'
>>> ra_field.unit
Unit("deg")
>>> ra_field.unit.to(u.arcsec) * ra_list
array([ 15852.5028, 15854.0616, 15856.3116, ..., 27510.2316,
27523.7604, 27530.64 ])
Perform the same Cone Search as above but asynchronously using AsyncConeSearch. Queries to individual Cone Search services are still governed by astropy.utils.data.REMOTE_TIMEOUT. Cone Search is forced to run in silent mode asynchronously, but warnings are still controlled by warnings:
>>> async_search = conesearch.AsyncConeSearch(c, sr, catalog_db=my_catname)
Check asynchronous search status:
>>> async_search.running()
True
>>> async_search.done()
False
Get search results after a 30-second wait (not to be confused with astropy.utils.data.REMOTE_TIMEOUT that governs individual Cone Search queries). If search is still not done after 30 seconds, TimeoutError is raised. Otherwise, Cone Search result is returned and can be manipulated as above. If no timeout keyword given, it waits until completion:
>>> async_result = async_search.get(timeout=30)
>>> cone_arr = async_result.array.data
>>> cone_arr.size
36184
Estimate the execution time and the number of objects for the Cone Search service URL from above. The prediction naively assumes a linear model, which might not be accurate for some cases. It also uses the normal conesearch(), not the asynchronous version. This example uses a custom timeout of 30 seconds and runs silently (except for warnings):
>>> result.url
u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&'
>>> with REMOTE_TIMEOUT.set_temp(30):
... t_est, n_est = conesearch.predict_search(
... result.url, c, sr, verbose=False, plot=True)
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
# ...
>>> t_est # Predicted execution time
19.272144977377838
>>> n_est # Predicted number of objects
37340
For debugging purpose, one can obtain the actual execution time and number of objects, and compare them with the predicted values above. The INFO message shown in controlled by astropy.logger. Keep in mind that running this for every prediction would defeat the purpose of the prediction itself:
>>> t_real, tab = conesearch.conesearch_timer(
... c, sr, catalog_db=result.url, verbose=False)
INFO: conesearch_timer took 11.5103080273 s on AVERAGE for 1 call(s). [...]
>>> t_real # Actual execution time
11.510308027267456
>>> tab.array.size # Actual number of objects
36184
One can also search in a list of catalogs instead of a single one. In this example, we look for all catalogs containing 'guide*star' in their titles and only perform Cone Search using those services. The first catalog in the list to successfully return non-zero result is used. Therefore, the order of catalog names given in catalog_db is important:
>>> gsc_cats = conesearch.list_catalogs(pattern='guide*star')
>>> gsc_cats
[u'Guide Star Catalog 2.3 1',
u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1']
>>> gsc_result = conesearch.conesearch(c, sr, catalog_db=gsc_cats)
Trying http://gsss.stsci.edu/webservices/vo/ConeSearch.aspx?CAT=GSC23&
WARNING: W25: ... failed with: timed out [...]
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
>>> gsc_result.array.size
2997
>>> gsc_result.url
u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&'
To repeat the Cone Search above with the services listed in a different order:
>>> gsc_cats_reordered = [gsc_cats[i] for i in (3, 1, 2, 0)]
>>> gsc_cats_reordered
[u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1',
u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
u'Guide Star Catalog 2.3 1']
>>> gsc_result = conesearch.conesearch(c, sr, catalog_db=gsc_cats_reordered)
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
>>> gsc_result.array.size
2997
>>> gsc_result.url
u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&'
To obtain results from all the services above:
>>> all_gsc_results = conesearch.search_all(c, sr, catalog_db=gsc_cats)
Trying http://gsss.stsci.edu/webservices/vo/ConeSearch.aspx?CAT=GSC23&
WARNING: W25: ... failed with: <urlopen error timed out> [...]
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/254/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&
Downloading ...
WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
>>> all_gsc_results
{u'http://.../220/out&': <astropy.io.votable.tree.Table at 0x2988e10>,
u'http://.../254/out&': <astropy.io.votable.tree.Table at 0x2993350>,
u'http://.../255/out&': <astropy.io.votable.tree.Table at 0x298de90>}
>>> for url, tab in all_gsc_results.items():
... print('{0} has {1} results'.format(url[-17:], tab.array.size))
source=I/254/out& has 2998 results
source=I/255/out& has 2997 results
source=I/220/out& has 2997 results
To repeat the above asynchronously:
>>> async_search_all = conesearch.AsyncSearchAll(c, sr, catalog_db=gsc_cats)
>>> async_search_all.running()
True
>>> async_search_all.done()
False
>>> all_gsc_results = async_search_all.get()
If one is unable to obtain any results using the default Cone Search database, 'conesearch_good', that only contains sites that cleanly passed validation, one can use Configuration system (astropy.config) to use another database, 'conesearch_warn', containing sites with validation warnings. One should use these sites with caution:
>>> conesearch.CONESEARCH_DBNAME.set('conesearch_warn')
>>> conesearch.list_catalogs()
Downloading http://stsdas.stsci.edu/astrolib/vo_databases/conesearch_warn.json
|===========================================| 87k/ 87k (100.00%) 00s
[u'2MASS All-Sky Catalog of Point Sources (Cutri+ 2003) 1',
u'2MASS All-Sky Point Source Catalog 1',
u'Data release 7 of Sloan Digital Sky Survey catalogs 1',
u'Data release 7 of Sloan Digital Sky Survey catalogs 2',
u'Data release 7 of Sloan Digital Sky Survey catalogs 3',
u'Data release 7 of Sloan Digital Sky Survey catalogs 4',
u'Data release 7 of Sloan Digital Sky Survey catalogs 5',
u'Data release 7 of Sloan Digital Sky Survey catalogs 6',
u'The 2MASS All-Sky Catalog 1',
u'The 2MASS All-Sky Catalog 2',
u'The USNO-B1.0 Catalog (Monet+ 2003) 1',
u'The USNO-B1.0 Catalog 1',
u'USNO-A V2.0, A Catalog of Astrometric Standards 1',
u'USNO-B1 Catalogue 1']