Skip to content

Latest commit

 

History

History
70 lines (58 loc) · 3.63 KB

README.md

File metadata and controls

70 lines (58 loc) · 3.63 KB
Raw

Science Data Model Schemas

This repository stores schema definitions for user-facing data products comprising the Science Data Model (SDM) of the Rubin Observatory. These schemas are defined in YAML files designed to be read with Felis, a tool which can convert this data to derived formats including SQL DDL and TAP_SCHEMA. These schema definitions serve as the "source of truth" for the observatory's core data models.

Schemas

The following schemas are maintained in this repository under the schemas directory:

  • imsim describes the outputs of the pipelines for LSSTCam-imSim, used to generate the data preview schemas, which are fixed at the time of their release, while imsim will continue to evolve. This schema is used by ci_imsim to verify that the tabular data of the pipeline output files is conformant with the schema definition.

  • The various Data Preview (DP) schemas such as dp02_dc2 represent content that is being actively served by the various data previews. These are created from imsim at a specific point in time.

  • apdb describes the schema of the Alert Production Database (APDB) used for Alert Production with ap_pipe and for nightly ap_verify runs within continuous integration. Previous processing runs may differ from the current schema. The user-queryable Prompt Products Database (PPDB) is expected to have a very similar schema to the APDB.

  • hsc describes the outputs of the latest data release production pipelines for HyperSuprimeCam. This schema is used by ci_hsc for verification of its output files.

  • The various cdb schemas such as cdb_latiss describe the data model of the Consolidated Database or ConsDB, an image metadata database containing summarizations of Engineering Facilities Database (EFD) telemetry by exposure and visit time windows.

Release Assets

Each release of sdm_schemas includes the following additional assets, generated automatically via GitHub Actions when a new tag is created:

  • datalink-columns.zip contains a set of YAML files with a restricted subset of the Felis schema. Currently, these identify the principal and minimal columns for a subset of the tables defined by the schema in this repository. Principal columns are those for which the principal flag is set in the TAP schema, defined in the IVOA TAP specification. The minimal columns are still experimental and in flux. These files are intended for use with the datalinker service of a Rubin Science Platform deployment.

  • datalink-snippets.zip contains a JSON manifest and a set of XML files that define VOTables following the IVOA DataLink specification and are intended to be used by the TAP service of a Rubin Science Platform deployment for adding DataLink records to the responses from a TAP query. Those DataLink records, in turn, provide links to operations that a client may wish to perform on those results, such as closely-related TAP queries.