Basic Design of the Caterpillar Pipeline

Basic Design of the Caterpillar Pipeline

• #merian #photometry
• 2020-12-10

---

• The goal of this pipeline is to take the DECam community pipeline (CP) output (the InstaCal CCD data) and generate calibrated coadd images and perform photometric modeling.

Class: Dataset:

• Main goals:

  • Summarize the metadata of the InstaCal exposures into a table.
  • Summarize the metadata from each CCD into a CCD catalog.
  • Generate individual FITS files for each CCD.

• "Dataset" here basically means the CP reduced data put into one folder.

  • It could be the single-band data collected in one night.
  • Or it could even be multiband data from the whole survey.
  • We do not imagine a super heavy duty one to handle a million fits file though. Just make sure it is enough to deal with the Merian survey (1000 deg$^2$)

Notes

• Quick tip: get the unique values of a list: data_uniq = list(set(data)).

• How to use pathlib to recursively find files

• NOAO science archive file naming convention

  • <obs_tel_instr>_<UTC_date_time>_<obstype><proctype><prodtype>.<ext>
  • There are optional fields like filters, or versions: <Obs_Tel_Inst>_yymmdd_hhmmss_<type_codes>[optional_fields].<exts>