Hackday mteodoro sky variance readnoise

[mairanteodoro]

Resolves RCAL-968

This PR adds a sky variance array to the mosaic file created by resample (i.e. model.var_sky). It also enables building a drizzle image weighted by the inverse of the sky variance when running resample with weight_type="ivsky".

Regression test

• 3 expected failing tests due to the new attribute var_sky added to the model.
• https://github.com/spacetelescope/RegressionTests/actions/runs/12265188362

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. 24Q4_B15 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.associations.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.mosaic_pipeline.rst
• changes/<PR#>.patch_match.rst

steps

• changes/<PR#>.dq_init.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.jump_detection.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.flux.rst
• changes/<PR#>.source_detection.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.resample.rst
• changes/<PR#>.source_catalog.rst

[codecov]

Codecov Report

Attention: Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 77.98%. Comparing base (b2aa8e0) to head (65f7acd).

Files with missing lines	Patch %	Lines
romancal/resample/resample_utils.py	93.75%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1556      +/-   ##
==========================================
+ Coverage   77.94%   77.98%   +0.04%     
==========================================
  Files         115      115              
  Lines        7622     7641      +19     
==========================================
+ Hits         5941     5959      +18     
- Misses       1681     1682       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[braingram]

I follow the addition of the new weight type but I'm not sure I see the connection between the ticket and resampling each computed var_sky into a var_sky added to the mosaic model.

If the goal is to save var_sky in the mosaic model would you open a rad and roman_datamodels PRs to update the schema to include var_sky?

[braingram]

Would you add a description here of how var_sky is computed?

[braingram]

Would you update the step spec to support the new weight type?

romancal/romancal/resample/resample_step.py

Line 55 in e250097

weight_type = option('ivm', 'exptime', None, default='ivm') # change back to None when drizpar ref update

[braingram]

I think this will crash if model doesn't have var_sky since inv_sky_variance is defined in the if on line 174 but not within the else block. How about adding something like inv_sky_variance = np.ones_like(...) to the else block.

Adding a unit test for that condition would check if the above suggestion works.

[mairanteodoro]

Nice catch! Thanks!
I completely forgot to add the default inv_sky_variance array.

[braingram]

Suggested change

	input_models = (
	input_models
	if isinstance(input_models, ModelLibrary)
	else ModelLibrary([input_models])
	)
	input_models = (
	input_models
	if isinstance(input_models, ModelLibrary)
	else ModelLibrary([input_models])
	)

If the function only accepts a ModelLibrary (based on the type annotation) then is this necessary?

[mairanteodoro]

No, it is not.

Initially, the method was supposed to handle both ModelLibrary and single datamodels, but eventually I thought that it would be better to sticking with the former only.

[schlafly]

I think this looks conceptually right; thanks! Can you put a couple of example files on /grp/roman somewhere, together with an invocation (run strun roman::resample blah.asn --weight_type skyivm ??) so that I can easily poke at some example input and output files?

[mairanteodoro]

If the goal is to save var_sky in the mosaic model would you open a rad and roman_datamodels PRs to update the schema to include var_sky?

Yes, that's the idea.