201.11. VisitDetector table

201.11. VisitDetector table¶

For the Rubin Science Platform at data.lsst.cloud.
Data Release: Data Preview 2
Container Size: Large
LSST Science Pipelines version: r30.0.10
Last verified to run: 2026-07-24
Repository: github.com/lsst/tutorial-notebooks
DOI: 10.11578/rubin/dc.20250909.20

Learning objective: To understand the contents of the VisitDetector table and how to access it.

LSST data products: VisitDetector

Packages: lsst.rsp, lsst.daf.butler

Credit: Originally developed by the Rubin Community Science team. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

A VisitDetector refers to an observation of the sky with an individual detector of the camera.

The VisitDetector table contains data about the individual observations for every detector of a Visit, such as the coordinates, date, time, filter, camera rotation, and exposure time. It also contains image characterization data for the associated visit_image (the processed, calibrated pixel data for one detector), such as the point-spread function (PSF) size, seeing, sky background and transparency, and magnitude limit.

The contents of the VisitDetector table are associated with the individual detectors of the camera. By contrast, the Visit contains data associated with the boresight of the entire focal plane.

• TAP table name: dp2.VisitDetector
• Butler table name: visit_detector_table
• columns: 56
• rows: 5,150,198

Related tutorials: The tutorial for the Visit table is also in this series. The TAP and Butler data access services are demonstrated in the 100-level "How to" tutorials.

1.1. Import packages¶

Import standard python packages re, numpy, matplotlib, and astropy.

From the lsst package, import modules for the TAP service and the Butler.

In [1]:

import re
import numpy as np
import matplotlib.pyplot as plt
from astropy.time import Time
from zoneinfo import ZoneInfo
from lsst.rsp import RSPDiscovery
from lsst.daf.butler import Butler
from lsst.utils.plotting import (get_multiband_plot_colors,
                                 get_multiband_plot_symbols)

1.2. Define parameters and functions¶

Create an instance of the TAP service.

In [2]:

discovery = RSPDiscovery("dp2")
tap_service = discovery.get_tap_client()

Create an instance of the Rubin data Butler.

In [3]:

butler = Butler("dp2", collections="dp2")

Define the colors and symbols to represent the LSST filters in plots.

In [4]:

filter_colors = get_multiband_plot_colors()
filter_names = filter_colors.keys()
filter_symbols = get_multiband_plot_symbols()

2. Schema (columns)¶

To browse the table schema visit the Rubin schema browser, or use the TAP service via the Portal Aspect or as demonstrated in Section 2.1.

2.1. Retrieve table schema¶

To retrieve the table schema, define a query for the schema columns of the VisitDetector table and run the query job.

In [5]:

query = "SELECT column_name, datatype, description, unit " \
        "FROM tap_schema.columns " \
        "WHERE table_name = 'dp2.VisitDetector'"
job = tap_service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Retrieve the query results and display them as an astropy table with the to_table() attribute.

In [6]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
results

Out[6]:

Table length=56

column_name	datatype	description	unit
str64	str64	str512	str64
physical_filter	char	ID of physical filter, the filter associated with a particular instrument.
band	char	Name of the band used to take the visit where this source was measured. Abstract filter that is not associated with a particular instrument.
ra	double	Right Ascension of Ccd center.	deg
dec	double	Declination of Ccd center.	deg
pixelScale	float	Measured detector pixel scale.	arcsec/pixel
zenithDistance	float	Zenith distance at observation mid-point.	deg
expTime	double	Spatially-averaged duration of visit, accurate to 10ms.	s
zeroPoint	float	Zero-point for the Ccd, estimated at Ccd center.	mag
psfSigma	float	PSF model second-moments determinant radius (center of chip)	pixel
skyBg	float	Average sky background.	adu
skyNoise	float	RMS noise of the sky background.	adu
astromOffsetMean	double	Mean offset of astrometric calibration matches (arcsec)	arcsec
astromOffsetStd	double	Standard deviation of offsets of astrometric calibration matches (arcsec)	arcsec
nPsfStar	int	Number of stars used for PSF model
psfStarDeltaE1Median	double	Median E1 residual (starE1 - psfE1) for psf stars
psfStarDeltaE2Median	double	Median E2 residual (starE2 - psfE2) for psf stars
psfStarDeltaE1Scatter	double	Scatter (via MAD) of E1 residual (starE1 - psfE1) for psf stars
psfStarDeltaE2Scatter	double	Scatter (via MAD) of E2 residual (starE2 - psfE2) for psf stars
psfStarDeltaSizeMedian	double	Median size residual (starSize - psfSize) for psf stars (pixel)	pixel
psfStarDeltaSizeScatter	double	Scatter (via MAD) of size residual (starSize - psfSize) for stars (pixel)	pixel
psfStarScaledDeltaSizeScatter	double	Scatter (via MAD) of size residual scaled by median size squared
psfTraceRadiusDelta	double	Delta (max - min) of model psf trace radius values evaluated on a grid of unmasked pixels (pixel)	pixel
...	...	...	...
obsStartMJD	double	Start of the exposure in MJD, TAI, accurate to 10ms.	d
darkTime	double	Average dark current accumulation time, accurate to 10ms.	s
xSize	long	Number of columns in the image.	pixel
ySize	long	Number of rows in the image.	pixel
llcra	double	Right Ascension of lower left corner.	deg
llcdec	double	Declination of lower left corner.	deg
ulcra	double	Right Ascension of upper left corner.	deg
ulcdec	double	Declination of upper left corner.	deg
urcra	double	Right Ascension of upper right corner.	deg
urcdec	double	Declination of upper right corner.	deg
lrcra	double	Right Ascension of lower right corner.	deg
lrcdec	double	Declination of lower right corner.	deg
effTime	double	Effective time metric	s
effTimePsfSigmaScale	double	Effective time metric -- PSF size component
effTimeSkyBgScale	double	Effective time metric -- Sky background component
effTimeZeroPointScale	double	Effective time metric -- Throughput component
magLim	double	5-sigma limiting magnitude	mag
wcsCornerMaxOffset	double	Maximum distance between the preliminary and final WCS at the corners of the detector. The WCS is set to null if this exceeds 0.5".	arcsec
wcsDetectorPointingResidual	double	Maximum difference (on the pointing-fit grid) between the final WCS position and the position predicted by camera geometry, after re-pointing using the final WCS for this detector only. The detector is rejected from the pointing fit if this exceeds 10".	arcsec
wcsVisitPointingResidual	double	Maximum difference (on the pointing-fit grid) between the final WCS position and the position predicted by camera geometry, after re-pointing using the final WCS of all non-rejected detectors in the visit. The WCS is set to null if this exceeds 60".	arcsec
wcsPreliminaryDetectorPointingResidual	double	Maximum difference (on the pointing-fit grid) between the preliminary WCS position and the position predicted by camera geometry, after re-pointing using the prelimnary WCS for this detector only. The detector is rejected from the pointing fit if this exceeds 10".	arcsec
wcsPreliminaryVisitPointingResidual	double	Maximum difference (on the pointing-fit grid) between the preliminary WCS position and the position predicted by camera geometry, after re-pointing using the preliminary WCS of all non-rejected detectors in the visit. The WCS is set to null if this exceeds 60".	arcsec

The table displayed above has been truncated.

Option to print every column name as a list.

In [7]:

# for col in results['column_name']:
#     print(col)

Option to use the regular expressions package re to search for columns for which the description contains the string temp.

In [8]:

# temp = 'time'
# temp = 'corner'
# temp = 'psf'
# for c, desc in enumerate(results['description']):
#     if re.search(temp, desc):
#         print(results['column_name'][c])

Delete the job, but not the results.

In [9]:

del query
job.delete()

2.2. Key columns¶

Descriptions for a few of the columns that are the most commonly used.

2.2.1. Identifiers¶

Integers that uniquely identify the visit and detector (CCD).

• visitId
• detector

The visitId and detector columns are important for joining tables and retrieving images from the butler.

2.2.2. Coordinates¶

The sky coordinates in decimal degrees of the center of the detector (CCD).

• ra
• dec

The RA, Dec of the four corners of the corresponding visit_image are also available. The column naming conventions are:

• "lower left corner" (llcra, llcdec)
• "lower right corner" (lrcra, lrcdec)
• "upper left corner" (ulcra, ulcdec)
• "upper right corner" (urcra, urcdec)

2.2.3. Observation metadata¶

The name of the filter (bandpass) that was used for the observation. There is also the column physical_filter which is unique to the physical filter (i.e., the coated glass plate). There is only one glass filter per band ($ugrizy$).

• band

The date and time of the observation, as a Modified Julian Date (MJD). Related columns include the start of the observation (obsStartMJD) and the exposure time (expTime).

• expMidptMJD

The airmass, $X$, can be calculated from the zenith angle, $z$, as $X = \sec(z)$.

• zenithDistance, zenith distance at the midpoint of the exposure, in degrees

2.2.4. Image characterization¶

Derived properties of the corresponding visit_image.

• magLim, the 5-sigma limiting magnitude (depth)
• psfSigma, radius of the PSF at the CCD center
• seeing, mean measured FWHM of the PSF

2.3. Descriptions and units¶

For a subset of the key columns show the table of their descriptions and units.

In [10]:

col_list = set(['visitId', 'detector', 'ra', 'dec', 'band',
                'expMidptMJD', 'magLim', 'psfSigma', 'seeing',
                'zenithDistance'])
tx = [i for i, item in enumerate(results['column_name']) if item in col_list]
results[tx]

Out[10]:

Table length=10

column_name	datatype	description	unit
str64	str64	str512	str64
band	char	Name of the band used to take the visit where this source was measured. Abstract filter that is not associated with a particular instrument.
ra	double	Right Ascension of Ccd center.	deg
dec	double	Declination of Ccd center.	deg
zenithDistance	float	Zenith distance at observation mid-point.	deg
psfSigma	float	PSF model second-moments determinant radius (center of chip)	pixel
visitId	long	Reference to the corresponding entry in the Visit table.
detector	long	Detector ID. A detector associated with a particular instrument (not an observation of that detector).
seeing	double	Mean measured FWHM of the PSF.	arcsec
expMidptMJD	double	Midpoint time for exposure at the fiducial center of the focal plane array in MJD. TAI, accurate to 10ms.	d
magLim	double	5-sigma limiting magnitude	mag

Clean up.

In [11]:

del col_list, tx, results

3. Data access¶

The VisitDetector table is available via the TAP service and the butler.

Recommended access method: TAP.

3.1. TAP (Table Access Protocol)¶

The VisitDetector table is stored in Qserv and accessible via the TAP services using ADQL queries.

3.1.2. Demo query¶

Avoid full-table queries. Although the VisitDetector table is relatively small, it is good practice to always include spatial constraints and only retrieve necessary columns.

For example, query the VisitDetector table for visit images within 2 degrees of coordinates of interest (in this case, the center of the COSMOS field is used, but it could be objectect coordinates) obtained on a specific MJD. Convert the zenith distance (zenith angle in degrees) to airmass.

In [12]:

ra_targ = 150.1
dec_targ = 2.1
search_radius = 2
query = """SELECT visitId, expMidptMJD, band, detector, seeing, magLim,
        zenithDistance, 1.0 / COS(RADIANS(zenithDistance)) AS airmass
        FROM dp2.VisitDetector
        WHERE CONTAINS(POINT('ICRS', ra, dec), CIRCLE('ICRS', {}, {}, {})) = 1
        AND expMidptMJD > 60818.7 AND expMidptMJD < 60819.7
        """.format(ra_targ, dec_targ, search_radius)
job = tap_service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Fetch the results as an astropy table.

In [13]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

20066

Option to display the table.

In [14]:

# results

As an example, plot the evolution of airmass and seeing over the night.

In [15]:

mjd_first = Time(np.min(results["expMidptMJD"]), format="mjd", scale="utc")
mjd_last = Time(np.max(results["expMidptMJD"]), format="mjd", scale="utc")
mjd_first_chile = mjd_first.datetime.astimezone(ZoneInfo("America/Santiago"))
mjd_last_chile = mjd_last.datetime.astimezone(ZoneInfo("America/Santiago"))

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
for filt in filter_names:
    fx = np.where(results['band'] == filt)[0]
    if len(fx) > 0:
        ax1.plot(results['expMidptMJD'][fx]-60819, results['airmass'][fx],
                 filter_symbols[filt], ms=2, mew=0, alpha=0.4,
                 color=filter_colors[filt], label=filt)
        ax2.plot(results['expMidptMJD'][fx]-60819, results['seeing'][fx],
                 filter_symbols[filt], ms=2, mew=0, alpha=0.4,
                 color=filter_colors[filt], label=filt)
ax1.set_xlabel('MJD-60819')
ax1.set_ylabel('airmass')
ax2.set_xlabel('MJD-60819')
ax2.set_ylabel('seeing')
ax2.legend(loc='upper right', handletextpad=0, markerscale=2.0)
plt.suptitle('Chile time: ' + str(mjd_first_chile)[0:19] + ' to ' + str(mjd_last_chile)[11:19])
plt.tight_layout()
plt.show()

Figure 1: At left, the airmass of the COSMOS field observations over the course of the night. At right, the seeing as a function of time. On this particular night, even as COSMOS was followed to higher airmass the image quality was improving, likely because these observations were taken in the early evening.

Clean up.

In [16]:

job.delete()
del query, results

3.2.2. Joinable tables¶

The VisitDetector table can be joined with any table that has a visit or visitId column (they have the same values). The detector column can also be used for joins. This includes the Visit, Source, ForcedSource, DiaSource, and ForcedSourceOnDiaObject tables.

Source¶

Search for the detector that overlaps the center of the COSMOS field at a specific time during the night, and retrieve the identifiers for the difference-image sources detected. This table join takes a couple of minutes.

In [17]:

ra_targ = 150.1
dec_targ = 2.1
search_radius = 11.0 / 60.0
mjd1 = 60819.04 - (30. / 3600. / 24.)
mjd2 = 60819.04 + (30. / 3600. / 24.)
query = """SELECT vd.visitId, vd.expMidptMJD, vd.band, vd.detector, vd.seeing, vd.magLim,
        dias.diaSourceId, dias.diaObjectId, dias.ssObjectId
        FROM dp2.VisitDetector AS vd
        JOIN dp2.DiaSource AS dias ON (vd.visitId = dias.visit AND vd.detector = dias.detector)
        WHERE CONTAINS(POINT('ICRS', vd.ra, vd.dec), CIRCLE('ICRS', {}, {}, {})) = 1
        AND vd.expMidptMJD > {} AND vd.expMidptMJD < {}
        """.format(ra_targ, dec_targ, search_radius, mjd1, mjd2)
job = tap_service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()
assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

Job phase is COMPLETED
38

Print the number of difference-image detections that were associated with a solar system object.

In [18]:

tx = np.where(results['ssObjectId'] > 0)[0]
print('Of the ', len(results), ' diaSources, ', len(tx), ' were associated with a moving object')

Of the  38  diaSources,  4  were associated with a moving object

Option to print the results of the table join.

In [19]:

# results

In [20]:

job.delete()
del query, results

3.2. Butler¶

The visit_detector_table is only available in full from the Butler (no subsetting), but it can be loaded into a Large Notebook Aspect container.

Show that the only dimension for the visit_detector_table is instrument.

In [21]:

butler.get_dataset_type('visit_detector_table')

Out[21]:

DatasetType('visit_detector_table', {instrument}, ArrowAstropy)

In [22]:

butler.get_dataset_type('visit_detector_table').dimensions.required

Out[22]:

{instrument}

Show that there is only one dataset references for the "LSSTCam" visit_detector_table.

In [23]:

dataset_refs = butler.query_datasets("visit_detector_table", instrument="LSSTCam")
print(len(dataset_refs))
print(dataset_refs[0])

1
visit_detector_table@{instrument: 'LSSTCam'} [sc=ArrowAstropy] (run=LSSTCam/runs/DRP/DP2/v30_0_0/DM-53881/stage2/20260325T031924Z id=019d2301-62b4-7db5-82f2-e5bb664c30e6)

Retrieve the entire visit_detector_table.

In [24]:

visit_detector_table = butler.get(dataset_refs[0])

Print the number of rows.

In [25]:

print(len(visit_detector_table))

5150198

Option to display the table (automatically truncated).

In [26]:

# visit_detector_table

In [27]:

del dataset_refs, visit_detector_table

In [ ]: