Newcandidate 362rc4 (#1699)

Co-authored-by: Steve Goldman <[email protected]> Co-authored-by: Zach Burnett <[email protected]>
spacetelescope · Nov 13, 2023 · 6eb880b · 6eb880b
1 parent 0bc0126
commit 6eb880b
Show file tree

Hide file tree

Showing 12 changed files with 53 additions and 56 deletions.
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -25,7 +25,7 @@ number of the code change for that issue.  These PRs can be viewed at:
 
 - Added functionality to allow the use of a two-column poller file. This is used
   to update the WFPC2 SVM aperture header keywords from the values in the poller 
-  file. [#1683]
+  file. [#1683, #1150]
 
 - Removed the version restriction on matplotlib. [#1649]
 

diff --git a/doc/.rtd-environment.yml b/doc/.rtd-environment.yml
@@ -2,23 +2,25 @@ name: rtd311
 channels:
     - defaults
 dependencies:
-    - python
-    - setuptools
+    - astropy
     - numpy
     - numpydoc
+    - pip
+    - python=3.11
     - scipy
-    - astropy
     - graphviz
     - pip:
-      - -e ..
-      - sphinx
-      - "sphinx_rtd_theme>1.2.0"
-      - sphinx_automodapi
-      - matplotlib
-      - "spherical_geometry>=1.2.22"
+      - astroquery
       - fitsblender
+      - lxml
+      - matplotlib
       - nictools
+      - photutils
       - pyregion
+      - "spherical_geometry>=1.2.22"
+      - sphinx
+      - "sphinx_rtd_theme>1.2.0"
+      - sphinx_automodapi
       - stwcs
       - stsci.tools
       - "stsci.image>=2.3.0"
@@ -27,7 +29,4 @@ dependencies:
       - stsci.stimage
       - "tweakwcs>=0.8.0"
       - stregion
-      - astroquery
-      - photutils
-      - lxml
       - tables
diff --git a/doc/source/explanation/hap_introduction.rst b/doc/source/explanation/hap_introduction.rst
@@ -2,4 +2,19 @@ Introduction
 ------------
 .. _hap-introduction:
 
-*Coming soon!*
+The ``drizzlepac`` package can be used for many purposes, all related to aligning and combining images to create products which can provide the deepest available views of the data.  Combining the data with ``drizzlepac`` relies on the WCS solution specified in the input image headers.  These WCS solutions are expected to align the images to each other (relative astrometry) as well as align the image to the correct position on the sky (absolute astrometry).  The telemetry from HST allows the relative astrometry to be known extremely accurately (sub-milli-arcsecond level) when all images use the same guide stars and when the images were taken in the same visit.  However, data taken at different times using different guide stars have historically had errors in the alignment with a sigma of 1 arc-second (or more).  As a result, corrections to the alignment need to be made in order to successfully combine the images.
+
+The code being used in the automated HST calibration pipeline to generate the products served by the HST Archive through the MAST Portal relies on multiple methods to do the best job possible in aligning and combining data regardless of whether they came from the same visit (or instrument) or not.
+
+On December 17, 2020, MAST began production of new ACS and WFC3 products in the HST data calibration pipeline: Hubble Legacy Archive (HLA)-style mosaics comprising the data from a single HST visit which are aligned to a common astrometric reference frame. These are the first of two types of **Hubble Advanced Products (HAP)** that will be produced in the HST data pipeline and made available through the `MAST Discovery Portal <https://mast.stsci.edu/portal/Mashup/Clients/Mast/Portal.html>`_.
+
+Three levels of products are available as part of this release:
+
+  * Exposure level products contain data from a single HST exposure.
+  * Filter level products are produced from all exposures in a visit with a common filter.
+  * Total level products combine all exposures from a visit for a specific detector and are intended as a detection image for producing catalogs.
+
+The **HAP Single Visit Mosaics (SVMs)** differ from the standard HST drizzled data products, which are aligned filter-by-filter to Gaia.  SVM data products, on the other hand, are all drizzled onto the same north-up pixel grid and may have improved relative alignment across filters within a given visit, enabling easy comparison of the images through multiple filters or for images to be combined to create color mosaics. When possible, sources in the images have been aligned directly to the Gaia source catalog to improve the WCS of the images. SVM data products with both relative alignment (by filter) and absolute alignment to Gaia will contain the string 'FIT_SVM_GAIA' in the 'WCSNAME' keyword in the science extension of the image header. More discussion on HAP alignment, may be found on the webpage `Improvements in HST Astrometry <https://outerspace.stsci.edu/pages/viewpage.action?spaceKey=HAdP&title=Improvements+in+HST+Astrometry>`_.
+
+Combining data across visits to create mosaics for every observed location on the sky gets performed using
+**HAP Multi-Visit Mosaic (MVM)** processing.
diff --git a/doc/source/index.rst b/doc/source/index.rst
@@ -20,9 +20,6 @@ The Drizzlepac package is a collection packages for aligning and combining astro
     parameters
     reference
 
-
-
-
 .. DrizzlePac Release Notes
 .. ------------------------
 .. The code for this package gets released through a number of methods: namely,
@@ -72,5 +69,4 @@ The Drizzlepac package is a collection packages for aligning and combining astro
 
 
 .. Combining data across visits to create mosaics for every observed location on the sky gets performed using
-.. **HAP Multi-Visit Mosaic (MVM)** processing.
-
+.. **HAP Multi-Visit Mosaic (MVM)** processing.
diff --git a/doc/source/reference.rst b/doc/source/reference.rst
@@ -8,9 +8,10 @@ Drizzle (Image drizzling step)
 .. toctree::
     :maxdepth: 1
 
-    reference/astrodrizzle
-    reference/adrizzle
     reference/ablot
+    reference/adrizzle
+    reference/astrodrizzle
+    reference/drizcr
     reference/median
     reference/outimage
     reference/photeq
@@ -30,14 +31,12 @@ Pipeline processing
     :maxdepth: 1
 
     reference/astrometry_api
-    reference/imagefindpars
-    reference/refimagefindpars
     reference/catalogs
-    reference/drizcr
+    reference/imagefindpars
     reference/mdztab
     reference/process
+    reference/refimagefindpars
     reference/util
-    reference/utilfuncs
 
 Interactive Image alignment
 ---------------------------
@@ -79,10 +78,10 @@ Classes
 .. toctree::
     :maxdepth: 1
 
+    reference/acsobjects
     reference/baseobjects
     reference/image
     reference/imageobject
-    reference/acsobjects
     reference/nicmosobjects
     reference/stisobjects
     reference/wfc3objects

diff --git a/doc/source/reference/astrodrizzle.rst b/doc/source/reference/astrodrizzle.rst
@@ -1,15 +1,5 @@
 .. _astrodrizzle_:
 
-===================================
-Introduction to Pipeline processing
-===================================
-*Coming soon!*
-
-====================
-Overview of pipeline
-====================
-*Coming soon!*
-
 ======================================
 Primary User Interface: AstroDrizzle()
 ======================================

diff --git a/doc/source/reference/tweakback.rst b/doc/source/reference/tweakback.rst
@@ -1,8 +1,8 @@
 .. _tweakback:
 
-*********
-TWEAKBACK
-*********
+***********************************
+TWEAKBACK: Propagating The Solution
+***********************************
 
 .. moduleauthor:: Warren Hack <[email protected]>
 

diff --git a/doc/source/reference/util.rst b/doc/source/reference/util.rst
@@ -12,7 +12,6 @@ and interpretation of the ``MDRIZTAB`` reference file.
 .. toctree::
    :maxdepth: 3
 
-   utilfuncs
    wcsutils
    outimage
    mdztab

diff --git a/doc/source/reference/utilfuncs.rst b/doc/source/reference/utilfuncs.rst
diff --git a/drizzlepac/haputils/analyze.py b/drizzlepac/haputils/analyze.py
@@ -524,6 +524,7 @@ def analyze_data(input_file_list, log_level=logutil.logging.DEBUG, type=""):
                 msg_type = Messages.NOPROC.value
             else:
                 msg_type = Messages.WARN.value
+                analyze_data_good_index.append(i)
 
             process_msg = no_proc_key + '=' + str(no_proc_value)
 
@@ -543,7 +544,6 @@ def analyze_data(input_file_list, log_level=logutil.logging.DEBUG, type=""):
                               total_rms, dataset_key, status, fit_qual, headerlet_file,
                               compromised])
         process_msg = ""
-
     return output_table, analyze_data_good_index
 
 

diff --git a/drizzlepac/haputils/poller_utils.py b/drizzlepac/haputils/poller_utils.py
@@ -116,6 +116,7 @@
 #
 def interpret_obset_input(results: str, log_level):
     """
+    parses the file generated by the pipeline poller, and produces a tree listing of the output products.
 
     Parameters
     -----------
@@ -141,22 +142,32 @@ def interpret_obset_input(results: str, log_level):
     Interpret the database query for a given obset to prepare the returned
     values for use in generating the names of all the expected output products.
 
-    Input will have formated rows of one of the following three options::
+    Input will have formated rows of one of the following three options
 
     1) Full poller file
+
+    .. code-block::
+
         ib4606c5q_flc.fits,11665,B46,06,1.0,F555W,UVIS,/foo/bar/ib4606c5q_flc.fits
 
     2) Simpler poller file (list); other info taken from file header keywords
+
+    .. code-block::
+    
         ib4606c5q_flc.fits
 
     3) For updating the WFPC2 SVM aperture keyword using the poller file; it
         is important that there are no spaces within the poller aperture keyword(s)
 
+    .. code-block::
+    
         ib4606c5q_flc.fits, PC1-FIX;F160BN15
 
     Full poller files contain filename, proposal_id, program_id, obset_id, exptime, filters, detector, pathname.
 
-    The output dict will have format (as needed by further code for creating the product filenames) of::
+    The output dict will have format (as needed by further code for creating the product filenames) of
+
+    .. code-block::
 
         obs_info_dict["single exposure product 00": {"info": '11665 06 wfc3 uvis empty_aperture ib4606c5q f555w drc',
                                                      "files": ['ib4606c5q_flc.fits']}

diff --git a/pyproject.toml b/pyproject.toml
@@ -9,7 +9,7 @@ description = """
 readme = { file = 'README.md', content-type = 'text/x-rst' }
 requires-python = '>=3.9'
 license = { file = "LICENSE.txt" }
-authors = [{ name = 'Megan Sosey' }, { name = 'Warren Hack' }, { name = 'Christopher Hanley' }, { name = 'Chris Sontag' }, { name = 'Mihai Cara' }, { name = 'Steven Goldman' }]
+authors = [{ name = 'Megan Sosey' }, { name = 'Warren Hack' }, { name = 'Christopher Hanley' }, { name = 'Chris Sontag' }, { name = 'Mihai Cara' }, { name = 'Michael Dulude' }, { name = 'Michele De La Pena' }, { name = 'Steven Goldman' }]
 classifiers = [
     'Intended Audience :: Science/Research',
     "Topic :: Scientific/Engineering :: Astronomy",