201.13. ShearObject table

201.13. ShearObject table¶

For the Rubin Science Platform at data.lsst.cloud.
Data Release: Data Preview 2
Container Size: large
LSST Science Pipelines version: r30.0.10
Last verified to run: 2026-07-23
Repository: github.com/lsst/tutorial-notebooks
DOI: 10.11578/rubin/dc.20250909.20

Learning objective: To understand the contents of the ShearObject table and how to access it.

LSST data products: ShearObject

Packages: lsst.rsp, lsst.daf.butler

Credit: Originally developed by the Rubin Community Science team. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

The ShearObject table contains measurements of objects detected and measured on coadded images with metadetection. The metadetection algorithm involves applying an artificial shear to images of small regions of sky and performing detection on the sheared images, as well as measurements that are used to calculate a shear response (Sheldon et al. 2020, Sheldon et al. 2023, Yamamoto et al. 2025).

• TAP table name: dp2.ShearObject
• butler table name: object_shear_all
• columns: 67

Related tutorials: The Table Access Protocol (TAP) and Butler data access services are demonstrated in the 100-level "How to" tutorials. There is a 200-level tutorial on deep_coadd images.

1.1. Import packages¶

Import standard python packages re, numpy, and matplotlib.

From the lsst package, import modules for the TAP service and the butler.

In [1]:

import re
import numpy as np
import matplotlib.pyplot as plt
from lsst.rsp import RSPDiscovery
from lsst.daf.butler import Butler

1.2. Define parameters and functions¶

Create an instance of the TAP service, and assert that it exists.

In [2]:

discovery = RSPDiscovery("dp2")
service = discovery.get_tap_client()
assert service is not None

Create an instance of the Rubin data Butler, and assert that it exists.

In [3]:

butler = Butler("dp2", collections="dp2")
assert butler is not None

2. Schema (columns)¶

To browse the table schema visit the Rubin schema browser, or use the TAP service via the Portal Aspect or as demonstrated in Section 2.1.

2.1. Retrieve table schema¶

To retrieve the table schema, define a query for the schema columns of the ShearObject table and run the query job.

In [4]:

query = "SELECT column_name, datatype, description, unit " \
        "FROM tap_schema.columns " \
        "WHERE table_name = 'dp2.ShearObject'"
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Retrieve the query results and display them as an astropy table with the to_table() attribute.

In [5]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
results

Out[5]:

Table length=67

column_name	datatype	description	unit
str64	str64	str512	str64
shearObjectId	long	Unique identifier for a ShearObject, specific to a single metacalibration counterfactual image.
tract	long	Skymap tract ID
patch	long	Skymap patch ID
cell_x	int	Column of the cell within the patch on which this measurement was made.
cell_y	int	Row of the cell within the patch on which this measurement was made.
metaStep	char	Type of artificial shear applied to image. One of 'ns', '1p', '1m', '2p', '2m'.
image_flags	int	Flags for the image on which this measurement was made.
x	float	Centroid (tract, x-axis) of the detected ShearObject.
y	float	Centroid (tract, y-axis) of the detected ShearObject.
ra	double	Detected Right Ascension of the ShearObject.	deg
dec	double	Detected Declination of the ShearObject.	deg
psfOriginal_flags	int	Flags for the original PSF measurement, ORed over the shear bands.
psfOriginal_g1	float	Reduced-shear g1 of the original PSF from adaptive moments, averaged over the shear bands.
psfOriginal_g2	float	Reduced-shear g2 of the original PSF from adaptive moments, averaged over the shear bands.
psfOriginal_T	float	Trace (<x^2> + <y^2>) measurement of the original PSF from adaptive moments, averaged over the shear bands.	arcsec**2
bmask_flags	int	bmask flags for the ShearObject.
ormask_flags	int	ored mask flags for the ShearObject.
mfrac	float	Gaussian-weighted masked fraction for the ShearObject.
gauss_psfReconvolved_flags	int	Flags for reconvolved PSF (measured with gauss algorithm).
gauss_psfReconvolved_g1	float	Reduced-shear g1 of the reconvolved PSF (measured with gauss algorithm).
gauss_psfReconvolved_g2	float	Reduced-shear g2 of the reconvolved PSF (measured with gauss algorithm).
gauss_psfReconvolved_T	float	Trace (<x^2> + <y^2>) of the reconvolved PSF (measured with gauss algorithm).	arcsec**2
...	...	...	...
i_gaussFlux_flags	int	Flags set for flux in i band measured with gauss algorithm.
i_gaussFlux	float	Flux in i band (measured with gauss algorithm).	nJy
i_gaussFluxErr	float	Flux uncertainty in i band (measured with gauss algorithm).	nJy
z_gaussFlux_flags	int	Flags set for flux in z band measured with gauss algorithm.
z_gaussFlux	float	Flux in z band (measured with gauss algorithm).	nJy
z_gaussFluxErr	float	Flux uncertainty in z band (measured with gauss algorithm).	nJy
g_pgaussFlux_flags	int	Flags set for flux in g band measured with pgauss algorithm.
g_pgaussFlux	float	Flux in g band (measured with pgauss algorithm).	nJy
g_pgaussFluxErr	float	Flux uncertainty in g band (measured with pgauss algorithm).	nJy
r_pgaussFlux_flags	int	Flags set for flux in r band measured with pgauss algorithm.
r_pgaussFlux	float	Flux in r band (measured with pgauss algorithm).	nJy
r_pgaussFluxErr	float	Flux uncertainty in r band (measured with pgauss algorithm).	nJy
i_pgaussFlux_flags	int	Flags set for flux in i band measured with pgauss algorithm.
i_pgaussFlux	float	Flux in i band (measured with pgauss algorithm).	nJy
i_pgaussFluxErr	float	Flux uncertainty in i band (measured with pgauss algorithm).	nJy
z_pgaussFlux_flags	int	Flags set for flux in z band measured with pgauss algorithm.
z_pgaussFlux	float	Flux in z band (measured with pgauss algorithm).	nJy
z_pgaussFluxErr	float	Flux uncertainty in z band (measured with pgauss algorithm).	nJy
is_cell_inner	boolean	Whether the object peak is within the inner cell boundary .
is_patch_inner	boolean	Whether the object is within the inner patch boundary.
is_tract_inner	boolean	Whether the object is within the inner tract boundary.
is_primary	boolean	Whether this object is to be selected after de-duplication.

The table displayed above has been truncated.

Option to print every column name as a list.

In [6]:

# for col in results['column_name']:
#     print(col)

Option to use the regular expressions package re to find all column names that hold the gaussFlux for the four filters.

In [7]:

# for col in results['column_name']:
#     if re.fullmatch('[griz]_gaussFlux', col):
#         print(col)

Option to search column names that contain the string defined by the temp variable.

In [8]:

# temp = 'flags'
# temp = 'gauss'
# temp = 'pgauss'
# temp = 'psf'
# for col in results['column_name']:
#     if re.search(temp, col):
#         print(col)

Delete the job, but not the results.

In [9]:

del query
job.delete()

2.2. Key columns¶

Of the 67 columns of the ShearObject table, a subset will be the most commonly used. They are mostly dimensionless unless otherwise specified.

2.2.1. ShearObject Id¶

The long integer that uniquely identifies each row of the ShearObject table.

• shearObjectId

2.2.2. Tract and patch¶

The skymap tract, patch, and cell identifiers.

• tract
• patch
• cell_x
• cell_y

2.2.3. Coordinates¶

The sky coordinates in decimal degrees.

• ra
• dec

2.2.4. Metadetection step¶

The type of artificial shear applied to the image. One of 'ns' (no shear), '1p' (positive shear applied along the first direction), '1m' (negative shear applied along the first direction), '2p' (positive shear applied along the second direction), '2m' (negative shear applied along the second direction). The first direction is along the x-axis while the second is along the 45 degrees above the x-axis counterclockwise.

• metaStep

2.2.5. Fluxes¶

The Gaussian flux and its uncertainty, in nJy, for each of the four filters.

• [f]_gaussFlux
• [f]_gaussFluxErr

Here [f] is one of g, r, i, or z.

2.2.6. Shapes¶

Reduced-shear measurements from the Gaussian algorithm.

• gauss_g1
• gauss_g2

Gaussian size in arcsec^2 and signal-to-noise measurements.

• gauss_T
• gauss_snr

2.2.7. Flags¶

Flags for the Gaussian measurements and the de-duplication selection.

• gauss_flags
• gauss_shape_flags
• gauss_object_flags
• is_primary

2.3. Descriptions and units¶

For a subset of the key columns show the table of their descriptions and units.

In [10]:

col_list = set(['shearObjectId', 'tract', 'patch', 'metaStep',
                'ra', 'dec', 'gauss_g1', 'gauss_g2',
                'gauss_snr', 'gauss_T',
                'r_gaussFlux', 'r_gaussFluxErr',
                'is_primary'])
tx = [i for i, item in enumerate(results['column_name'])
      if item in col_list]
results[tx]

Out[10]:

Table length=13

column_name	datatype	description	unit
str64	str64	str512	str64
shearObjectId	long	Unique identifier for a ShearObject, specific to a single metacalibration counterfactual image.
tract	long	Skymap tract ID
patch	long	Skymap patch ID
metaStep	char	Type of artificial shear applied to image. One of 'ns', '1p', '1m', '2p', '2m'.
ra	double	Detected Right Ascension of the ShearObject.	deg
dec	double	Detected Declination of the ShearObject.	deg
gauss_g1	float	Reduced-shear g1 measurement of the ShearObject (measured with gauss algorithm).
gauss_g2	float	Reduced-shear g2 measurement of the ShearObject (measured with gauss algorithm).
gauss_snr	float	Signal-to-noise ratio measure of the ShearObject (measured with gauss algorithm).
gauss_T	float	Trace (<x^2> + <y^2>) measurement of the ShearObject (measured with gauss algorithm).	arcsec**2
r_gaussFlux	float	Flux in r band (measured with gauss algorithm).	nJy
r_gaussFluxErr	float	Flux uncertainty in r band (measured with gauss algorithm).	nJy
is_primary	boolean	Whether this object is to be selected after de-duplication.

Clean up.

In [11]:

del col_list, tx, results

3. Data access¶

The ShearObject table is accessible via the TAP service and the Butler.

Recommended access method: TAP.

3.1. Advisory: avoid full-table queries¶

Avoid full-table queries. Always include spatial constraints.

The ShearObject table is a large, inclusive set of measurements made on coadded images for objects detected and measured during metadetection processing.

Although queries of small DP2 regions can run quickly, skipping spatial constraints is not a good habit to form because future data releases will contain much larger tables.

3.2. TAP (Table Access Protocol)¶

The ShearObject table is stored in Qserv and accessible via the TAP services using Astronomical Data Query Language (ADQL) queries.

Include spatial constraints: Qserv stores catalog data sharded by coordinate (RA, Dec), so ADQL queries that include coordinate constraints do not require a whole-catalog search and are typically faster than queries that constrain only other columns.

Use an ADQL cone or polygon search for faster queries. Do not use WHERE ... BETWEEN statements to set boundaries on RA and Dec.

3.2.1. Demo query¶

Define a query to return the "key columns" from Section 2.2.

Impose spatial constraints: search within a 0.05 degree radius of the center of the Extended Chandra Deep Field South (ECDFS) field, RA, Dec = $53.13, -28.10$.

Return primary rows for which metaStep is ns.

In [12]:

query = "SELECT shearObjectId, tract, patch, metaStep, ra, dec, " \
        "gauss_g1, gauss_g2, gauss_snr, gauss_T, " \
        "r_gaussFlux, r_gaussFluxErr, is_primary " \
        "FROM dp2.ShearObject " \
        "WHERE CONTAINS(POINT('ICRS', ra, dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 0.05)) = 1 " \
        "AND metaStep = 'ns' " \
        "AND is_primary = 1"
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Fetch the results as an astropy table.

In [13]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

1174

Option to display the table.

In [14]:

# results

Constrain the results to objects with r-band Gaussian magnitudes between 20 and 25 mag in the AB system (flux between 360 and 36000 nJy approximately).

In [15]:

tx = np.where((results['r_gaussFlux'] > 360.0)
              & (results['r_gaussFlux'] <= 36000.0)
              & (results['r_gaussFluxErr'] > 0))[0]
print(len(tx))

764

Plot the sky coordinates and the r-band Gaussian magnitude versus signal-to-noise ratio (SNR) for the subset of objects.

In [16]:

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
ax1.plot(results['ra'][tx], results['dec'][tx],
         'o', ms=2, mew=0, alpha=0.4, color='grey')
ax1.set_xlabel('Right Ascension')
ax1.set_ylabel('Declination')
ax2.plot(-2.5 * np.log10(results['r_gaussFlux'][tx]) + 31.4,
         results['r_gaussFlux'][tx] / results['r_gaussFluxErr'][tx],
         'o', ms=2, mew=0, alpha=0.4, color='grey')
ax2.set_xlabel('Gaussian Mag')
ax2.set_ylabel('SNR')
plt.tight_layout()
plt.show()

Figure 1: At left, the RA vs. Dec of retrieved shear objects is shown as a circle of grey points. At right, the r-band Gaussian magnitude vs. SNR shows how magnitude increases as SNR decreases.

Clean up.

In [17]:

job.delete()
del results, tx

3.3. Butler¶

TAP is the recommended way to access the ShearObject table, but the Butler is a convenient way to retrieve all the shear objects in a given tract.

Show that the dimensions for the object_shear_all table are the skymap and tract, and that they are required.

In [18]:

butler.get_dataset_type('object_shear_all')

Out[18]:

DatasetType('object_shear_all', {skymap, tract}, ArrowAstropy)

In [19]:

butler.get_dataset_type('object_shear_all').dimensions.required

Out[19]:

{skymap, tract}

3.3.1. Demo query¶

Include spatial constraints: The Butler object_shear_all table contents are stored and retrieved by individual tract.

Retrieve all dataset_refs for object_shear_all tables for tracts that overlap the coordinates near the center of the ECDFS field.

In [20]:

query = "tract.region OVERLAPS POINT(53.13, -28.10)"
refs = butler.query_datasets("object_shear_all", where=query)

Show the tracts that overlap the coordinates.

In [21]:

for ref in refs:
    print(ref.dataId)

{skymap: 'lsst_cells_v2', tract: 5063}

Define the columns to retrieve.

In [22]:

col_list = ['shearObjectId', 'tract', 'patch', 'metaStep',
            'ra', 'dec', 'gauss_g1', 'gauss_g2',
            'gauss_snr', 'gauss_T',
            'r_gaussFlux', 'r_gaussFluxErr',
            'is_primary']

Get the data from the butler.

In [23]:

results = butler.get(refs[0],
                     parameters={'columns': col_list})

Option to display the results.

In [24]:

# results

Constrain the results to primary rows for which metaStep is ns, and with r-band Gaussian magnitudes between 20 and 25 mag.

In [25]:

tx = np.where((results['metaStep'] == 'ns')
              & (results['is_primary'])
              & (results['r_gaussFlux'] > 360.0)
              & (results['r_gaussFlux'] <= 36000.0)
              & (results['r_gaussFluxErr'] > 0))[0]
print(len(tx))

244672

Plot the sky coordinates and Gaussian fluxes and flux errors for the subset of shear objects.

In [26]:

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
ax1.plot(results['ra'][tx], results['dec'][tx],
         'o', ms=0.5, mew=0, alpha=0.1, color='grey')
ax1.set_xlabel('Right Ascension')
ax1.set_ylabel('Declination')
ax2.plot(-2.5 * np.log10(results['r_gaussFlux'][tx]) + 31.4,
         results['r_gaussFlux'][tx] / results['r_gaussFluxErr'][tx],
         'o', ms=0.5, mew=0, alpha=0.1, color='grey')
ax2.set_xlabel('Gaussian Mag')
ax2.set_ylabel('SNR')
plt.tight_layout()
plt.show()

Figure 2: At left, the RA vs. Dec of retrieved shear objects is shown as grey points. At right, the r-band Gaussian magnitude vs. SNR shows how magnitude increases as SNR decreases.

In [27]:

del query, refs, col_list, results, tx