Skip to content

Commit

Permalink
Rcal 954 Remove jump code from rcal tree (#1534)
Browse files Browse the repository at this point in the history
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
ddavis-stsci and pre-commit-ci[bot] authored Dec 11, 2024
1 parent 6cb4b51 commit e2d5fb9
Show file tree
Hide file tree
Showing 37 changed files with 147 additions and 873 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ develop-eggs
.installed.cfg
distribute-*.tar.gz

# emacs save files
\#*
.#*

# Other
.cache
.tox
Expand Down
1 change: 1 addition & 0 deletions changes/1534.general.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove the jump code for the deprecated jump detection for even ramps and update the documentation
8 changes: 4 additions & 4 deletions docs/roman/associations/asn_from_list.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ To create an association, use the following command:
import romancal.associations.asn_from_list as asn_from_list
product_name = 'test_product'
items = {'r0000101001001001001_01101_0001_WFI01_uncal.asdf': 'science', 'r0000101001001001001_3_01101_0001_WFI01_uncal.asdf': 'guide_window', 'c': 'somethingelse'}
items = {'r0000101001001001001_0001_wfi01_uncal.asdf': 'science', 'r0000101001001001001_3_0001_wfi01_uncal.asdf': 'guide_window', 'c': 'somethingelse'}
asn = asn_from_list.asn_from_list([(item, type_) for item, type_ in items.items()], product_name=product_name, with_exptype=True)
asn['asn_rule']
'DMS_ELPP_Base'
Expand All @@ -29,9 +29,9 @@ an example product that has both a science and guide window exposures
would look like the following::

asn['products']
[ { 'members': [ { 'expname': 'r0000101001001001001_01101_0001_WFI01_uncal.asdf',
[ { 'members': [ { 'expname': 'r0000101001001001001_0001_wfi01_uncal.asdf',
'exptype': 'science'},
{ 'expname': 'r0000101001001001001_3_01101_0001_WFI01_uncal.asdf',
{ 'expname': 'r0000101001001001001_3_0001_wfi01_uncal.asdf',
'exptype': 'guide_window'},
{'expname': 'c', 'exptype': 'somethingelse'}],
'name': 'test_product'}]
Expand All @@ -41,7 +41,7 @@ To create a association with all the detectors for a given exposure from the com

.. code-block:: python
asn_from_list -o detector_asn.json --product-name r0000101001001001001_01101_0001_WFI data/*_cal.asdf
asn_from_list -o detector_asn.json --product-name r0000101001001001001_0001_wfi data/*_cal.asdf
where the individual calibrated detector files are in a data subdirectory.

Expand Down
8 changes: 4 additions & 4 deletions docs/roman/data_products/file_naming.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The names of the exposure level data are constructed with information from the
science header of the exposure, allowing users to map it to the observation in their corresponding
APT files. The ASDF file naming scheme for the Exposure products is::

r<ppppp><cc><aaa><sss><ooo><vvv>_<gg><s><aa>_<eeee>_<detector>_<prodType>.asdf
r<ppppp><cc><aaa><sss><ooo><vvv>_<eeee>_<detector>_<prodType>.asdf

where

Expand All @@ -25,7 +25,7 @@ where
- s: sequence ID (1=prime, >1 parallel)
- aa: activity number (base 36)
- eeee: exposure number
- detector: detector name (e.g. WFI01, WFI02, ...)
- detector: detector name (e.g. wfi01, wfi02, ...)
- prodType: product type identifier (e.g. 'uncal', 'cal')

The standard <prodType> for the pipeline are uncal and cal, for the input products and resulting
Expand All @@ -35,8 +35,8 @@ linearity, jump, darkcurrent, rampfit, assignwcs, flat, and photom.

An example Exposure Level 1 upcalibrated ASDF file name is::

r0000101001001001001_01101_0001_WFI01_uncal.asdf
r0000101001001001001_0001_wfi01_uncal.asdf

An example Exposure Level 2 product ASDF file name is::

r0000101001001001001_01101_0001_WFI01_cal.asdf
r0000101001001001001_0001_wfi01_cal.asdf
8 changes: 4 additions & 4 deletions docs/roman/datamodels/datamodels_asdf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ If you have installed the roman calibration pipeline you should also have access
to the standalone tool asdfinfo which allows access to `ASDF <https://asdf-standard.readthedocs.io/>`_ (and roman) files
from the terminal prompt,::

asdftool info r0000101001001001001_01101_0001_WFI16_cal.asdf
asdftool info r0000101001001001001_0001_wfi16_cal.asdf
root (AsdfObject)
├─asdf_library (Software)
│ ├─author (str): The ASDF Developers
Expand Down Expand Up @@ -62,10 +62,10 @@ To access the files via a python session,
import roman_datamodels as rdm
import asdf
with rdm.open('r0000101001001001001_01101_0001_WFI16_cal.asdf') as model:
with rdm.open('r0000101001001001001_0001_wfi16_cal.asdf') as model:
<Manipulate the files>
with asdf.open('r0000101001001001001_01101_0001_WFI16_cal.asdf', memmap=False) as model:
with asdf.open('r0000101001001001001_0001_wfi16_cal.asdf', memmap=False) as model:
<Manipulate the files>
Once the files are loaded you can access various attributes. Below is a table
Expand Down Expand Up @@ -146,7 +146,7 @@ asdf tree.
.. code:: python
>>> from roman_datamodels import datamodels as rdm # doctest: +SKIP
>>> d_uncal = rdm.open('r0000101001001001001_01101_0001_WFI01_uncal.asdf') # doctest: +SKIP
>>> d_uncal = rdm.open('r0000101001001001001_0001_wfi01_uncal.asdf') # doctest: +SKIP
>>> d_uncal.info() # doctest: +SKIP
root (AsdfObject)
├─asdf_library (Software)
Expand Down
6 changes: 3 additions & 3 deletions docs/roman/datamodels/models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ using python.
.. testsetup::
>>> from roman_datamodels import datamodels as rdm
>>> from roman_datamodels.maker_utils import mk_datamodel
>>> fn = 'r0019106003005004023_03203_0034_WFI01_cal.asdf'
>>> fn = 'r0019106003005004023_0034_wfi01_cal.asdf'
>>> image_model = mk_datamodel(rdm.ImageModel)

# set some metadata for the below tests
Expand All @@ -46,7 +46,7 @@ using python.
.. code-block:: python
>>> from roman_datamodels import datamodels as rdm
>>> fn = 'r0019106003005004023_03203_0034_WFI01_cal.asdf'
>>> fn = 'r0019106003005004023_0034_wfi01_cal.asdf'
>>> image_model = rdm.open(fn) # doctest: +SKIP
>>> type(image_model)
<class 'roman_datamodels.datamodels._datamodels.ImageModel'>
Expand Down Expand Up @@ -145,7 +145,7 @@ You can examine the contents of your model from within python using
meta.calibration_software_name RomanCAL
meta.calibration_software_version 9.9.0
meta.product_type l2
meta.filename r0019106003005004023_03203_0034_WFI01_cal.asdf
meta.filename r0019106003005004023_0034_wfi01_cal.asdf
meta.file_date 2020-01-01T00:00:00.000
meta.model_type ImageModel
meta.origin STSCI/SOC
Expand Down
50 changes: 0 additions & 50 deletions docs/roman/jump/arguments.rst

This file was deleted.

77 changes: 0 additions & 77 deletions docs/roman/jump/description.rst

This file was deleted.

17 changes: 0 additions & 17 deletions docs/roman/jump/even_ramp.rst

This file was deleted.

11 changes: 0 additions & 11 deletions docs/roman/jump/index.rst

This file was deleted.

15 changes: 0 additions & 15 deletions docs/roman/jump/reference_files.rst

This file was deleted.

13 changes: 0 additions & 13 deletions docs/roman/jump/uneven_ramp.rst

This file was deleted.

12 changes: 7 additions & 5 deletions docs/roman/outlier_detection/main.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ Description
:Classes: `romancal.outlier_detection.OutlierDetectionStep`
:Aliases: outlier_detection, outlier_detection_scaled, outlier_detection_stack

Processing multiple datasets together allows for the identification of bad pixels
or cosmic-rays that remain in each of the input images, many times at levels which
were not detectable by the :ref:`jump <jump_step>` step. The ``outlier_detection`` step
implements the following algorithm to identify and flag any remaining cosmic-rays or
other artifacts left over from previous calibrations:
Processing multiple datasets together allows for the identification of
bad pixels or cosmic-rays that remain in each of the input images,
many times at levels which were not detectable by jump detection in
:ref:`ramp_fitting <ramp_fitting_step>` step. The
``outlier_detection`` step implements the following algorithm to
identify and flag any remaining cosmic-rays or other artifacts left
over from previous calibrations:

- build a stack of input data

Expand Down
1 change: 0 additions & 1 deletion docs/roman/package_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
dq_init/index.rst
flatfield/index.rst
flux/index.rst
jump/index.rst
linearity/index.rst
outlier_detection/index.rst
pipeline/index.rst
Expand Down
6 changes: 3 additions & 3 deletions docs/roman/pipeline/exposure_pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ The ``exposure`` pipeline has an optional argument::

--use_ramp_jump_detection boolean default=True

When set to ``True``, the pipeline will perform :ref:`jump <jump_step>` detection as a part of the ramp
When set to ``True``, the pipeline will perform jump detection as a part of the ramp
fitting step. The data at this stage of the pipeline are still in the form of the original
3D ramps ( ngroups x ncols x nrows ) and have had all of the detector-level
correction steps applied to it, up to but not including the detection and flagging of
Cosmic-Ray (CR) hits within each ramp (integration). For this case the :ref:`jump <jump_step>`
Cosmic-Ray (CR) hits within each ramp (integration). For this case the jump detection
module in :ref:`ramp_fitting <ramp_fitting_step>` will update the dq array with the CR hits (jumps) that
are identified in the step.

Expand All @@ -62,7 +62,7 @@ Inputs
:File suffix: _uncal

The input to the ``ExposurePipeline`` can be a single raw exposure,
e.g. "r0008308002010007027_06311_0019_WFI01_uncal.asdf", which contains the
e.g. "r0008308002010007027_0019_wfi01_uncal.asdf", which contains the
original raw data from all of the detector readouts in the exposure
( ngroups x ncols x nrows ). The raw data may also be input using an association file.

Expand Down
2 changes: 1 addition & 1 deletion docs/roman/pipeline/mosaic_pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ An association of 2D calibrated image data
:File suffix: _cal

The input to the ``MosaicPipeline`` is a group of calibrated exposures,
e.g. "r0008308002010007027_06311_0019_WFI01_cal.asdf", which contains the
e.g. "r0008308002010007027_0019_wfi01_cal.asdf", which contains the
calibrated data for the the exposures. The most convenient way to pass the list of
exposures to be processed with the mosaic level pipeline is to use an association.
Instructions on how to create an input association an be found at :ref:`asn-from-list`.
Expand Down
4 changes: 2 additions & 2 deletions docs/roman/pipeline_naming_conventions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@ name. If the input file name already has a known suffix, that suffix
will be replaced. For example:
::

$ strun romancal.dq_init.DQInitStep r0008308002010007027_06311_0019_WFI01_uncal.asdf
$ strun romancal.dq_init.DQInitStep r0008308002010007027_0019_wfi01_uncal.asdf

produces an output file named
``r0008308002010007027_06311_0019_WFI01_dq_init.asdf``.
``r0008308002010007027_0019_wfi01_dq_init.asdf``.


.. _pipeline_step_suffix_definitions:
Expand Down
Loading

0 comments on commit e2d5fb9

Please sign in to comment.