Skip to content

Merian COSMOS QA Sample

Jump to bottom
Song Huang edited this page May 11, 2022 · 1 revision

Note

Access the COSMOS data

  • Merian data repo: /tigress/MERIAN/repo/

  • Need to be on the tiger2-sumire machine: ssh [email protected]

  • Also need to setup the permission to access the Postgres database. See the section below

  • Setup the LSST pipeline: source /projects/HSC/LSST/stack/loadLSST.bash && setup lsst_distrib

Merian COSMOS

  • The current Merian COSMOS coadd is here (Feb 2022)
Deep: DECam/runs/merian/w_2022_02/t9813_deep
Best: DECam/runs/merian/w_2022_02/t9813_best
Wide: DECam/runs/merian/w_2022_02/t9813_wide
Deep
import lsst.daf.butler as dafButler
butler = dafButler.Butler('/projects/MERIAN/repo')

n708_deep = butler.get(
	'objectTable_tract', tract=9813, instrument='DECam',
    skymap='hsc_rings_v1', collections='DECam/runs/merian/w_2022_02/t9813_deep')
  • 1816938 objects; 190 columns
  • n708_deep_use = n708_deep[n708_deep.detect_isPrimary & (n708_deep.deblend_nChild == 0)]
    • Leaves 824658 objects
    • Save it as merian_n708_deep_cosmos_202202_primary.fits
  • n708_deep_sky = n708_deep[n708_deep.merge_peak_sky]
    • Leaves 6979 objects
    • Save it as merian_n708_deep_cosmos_202202_sky.fits
Best
n708_best = butler.get(
	'objectTable_tract', tract=9813, instrument='DECam',
    skymap='hsc_rings_v1', collections='DECam/runs/merian/w_2022_02/t9813_best')
  • 2991265 objects; 190 columns
  • n708_best_use = n708_best[n708_best.detect_isPrimary & (n708_best.deblend_nChild == 0)]
    • Leaves 1277654 objects
    • Save it as merian_n708_best_cosmos_202202_primary.fits
  • n708_best_sky = n708_best[n708_best.merge_peak_sky]
    • Leaves 7468 objects
    • Save it as merian_n708_best_cosmos_202202_sky.fits
Wide
n708_wide = butler.get(
    'objectTable_tract', tract=9813, instrument='DECam',
    skymap='hsc_rings_v1', collections='DECam/runs/merian/w_2022_02/t9813_wide')
  • 2353478 objects; 190 columns
  • n708_wide_use = n708_wide[n708_wide.detect_isPrimary & (n708_wide.deblend_nChild == 0)]
    • Leaves 1069034 objects
    • Save it as merian_n708_wide_cosmos_202202_primary.fits
  • n708_wide_sky = n708_wide[n708_wide.merge_peak_sky]
    • Leaves 7219 objects
    • Save it as merian_n708_wide_cosmos_202202_sky.fits
Single visit sky object

  • Visit list:

    • Wide: 971900, 971901, 971902, 971903
    • Best: 971677, 971680, 971910, 971911, 971912, 972360, 972361, 972362, 972363, 972364, 972365
    • Deep: 971667, 971668, 971669, 971670, 971671, 971673, 971675, 971676, 971677, 971679, 971680, 971681, 971683, 971685, 971687, 971689, 971690, 971691, 971693, 971694, 971893, 971894, 971895, 971896, 971897, 971898, 971899, 971900, 971901, 971902, 971903, 971904, 971905, 971906, 971907, 971908, 971909, 971910, 971911, 971912, 972360, 972361, 972362, 972363, 972364, 972365

  • Use exposure ID as visit to get the single exposure table:

src = butler.get(
	"sourceTable_visit",
    instrument="DECam",
    visit=971900,
    collections="DECam/runs/merian/w_2022_02"
)

  • Select sky sources: sky = src[src.sky_source]

    • Each visit has 6000 sky objects
    • Save the catalog

  • Get the catalogs for a few visits:

    • 971900: merian_n708_971900_sky.fits
    • 971903: merian_n708_971903_sky.fits
    • 971677: merian_n708_971677_sky.fits
    • 972365: merian_n708_972365_sky.fits

HSC COSMOS

  • The current HSC COSMOS coadd (with new GaaP photometry; Feb 2022)

  • Example of how to access the object catalog:

import lsst.daf.butler as dafButler

butler = dafButler.Butler('/projects/MERIAN/repo')

hsc_cat = butler.get(
	'objectTable_tract', tract=9813, 
	collections='HSC/runs/RC2/w_2022_04/DM-33402',
	instrument='HSC', skymap='hsc_rings_v1')

  • hsc_cat is a Pandas.DataFrame object with 2263935 rows and 830 columns

  • Only leave the primary ones:

    • hsc_use = hsc_cat[hsc_cat.detect_isPrimary & (hsc_cat.deblend_nChild == 0)]
    • This leaves 1118505 objects.
    • Convert it into an astropy.table using Table.from_pandas().
    • Save it as hsc_cosmos_202202_primary.fits

  • Sky objects:

    • Selection: hsc_sky = hsc_cat[hsc_cat.merge_peak_sky]
    • 7634 objects
    • Save it as hsc_cosmos_202202_sky.fits

HSC x Merian Cross-Match

Check the overlap regions

  • The HSC COSMOS 9813 tract is smaller than the DECam one.

  • The n_input > 2 regions of the wide and best stacks.

    • The best reduction seems to have some issues.

  • Define a common region to match.

    • Circle: Center (150.2, 2.21), Radius: 0.71 deg

  • Use the wide and deep stacks first, saved the objects in the central regions in:

    • hsc_cosmos_202202_primary_cen.fits: 917069
    • merian_n708_wide_cosmos_202202_primary_cen.fits: 583709
    • merian_n708_deep_cosmos_202202_primary_cen.fits: 781191

Basic cross-match

  • Cross-match:
    • hsc x wide, best-matches
      • 1.0 arcsec matching radius: 372710 matches; hsc_merian_n708_wide_cosmos_cen_best_match.fits
    • hsc x wide, best-matches
      • 1.0 arcsec matching radius: 417090 matches; hsc_merian_n708_deep_cosmos_cen_best_match.fits

Match the point sources

  • Use the HSC catalogs to select point sources:

    • In the FDFC region
    • extendedness < 1 in r & i band.
    • Not saturated in griz bands using _pixelFlags_saturated
    • Central is not interpolated in griz bands using _pixelFlags_interpolatedCenter
    • End up with 62625 point sources.
    • Saved as hsc_cosmos_202202_primary_pts.fits

  • Match with Merian N708 catalogs

    • Wide using 1.5 arcsec matching distance: 24422 matched.
      • Separation is a very strong function of i-band PSF magnitudes. The matching becomes significantly worse for i > 20.5 mag stars.
      • Same with the N708 PSF magnitude, n708 < 20.5 mag
      • This also correlates with the blendednss: At n708 > 20.5 mag, the blendedness becomes increasingly higher toward the fainter-end.

  • Pick the isolated stars from the HSC point source catalog using: i_blendedness < 0.2

    • 29135 stars (47%) remain in the sample.

  • Match the isolated point sources to N708 stacks:

    • wide using 1.0 arcsec matching radius: 14410 matched; N708 ~ 20.5 mag: hsc_merian_n708_wide_cosmos_isolated_pts.fits
    • deep using 1.0 arcsec matching radius: 16086 matched; N708 ~ 21.0 mag: hsc_merian_n708_deep_cosmos_isolated_pts.fits

  • Estimate the astrometric offset using the bright and isolated matched point sources.

    • wide: 10213 point sources with i_psf < 20.5 mag.
      • Separation: median 0.044 arcsec; mean 0.056 arcsec; std 0.044 arcsec
      • RA offset: median 0.009 arcsec; mean 0.014 arcsec; std 0.049 arcsec
      • Dec offset: median 0.015 arcsec; mean 0.019 arcsec; std 0.046 arcsec
    • deep: 10952 point sources with i_psf < 21.0 mag.
      • Separation: median 0.036 arcsec; mean 0.046 arcsec; std 0.037 arcsec
      • RA offset: median 0.007 arcsec; mean 0.012 arcsec; std 0.040 arcsec
      • Dec offset: median 0.016 arcsec; mean 0.020 arcsec; std 0.037 arcsec

Initial QA

Sky objects of the Merian N708 stacks

  • Select useful sky objects using: detect_isTractInner && detect_isPatchinner && N708_ap35Flux <= 1.5e4
  • Using both the GaaP PSF flux (N708_gaapPsfFlux) and the aperture flux with 3.5 pixel radius (N708_ap35Flux), we see that there is sign of over-subtraction of sky. The level of over-subtraction is similar in all three stacks.
  • And using both fluxes, we confirm that the best stack shows smaller statistical noise. However, the deep stack doesn't show much improvement when compared to wide.

Checks the sky objects in single-visit reduction

  • Based on Lee Kelvin's suggestion, take a look at the ap09Flux of sky objects in four COSMOS single visits data. The over-subtraction is still visible.
Clone this wiki locally