diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 3e2f28828..79b939da8 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -25,7 +25,7 @@ number of the code change for that issue. These PRs can be viewed at: - Added functionality to allow the use of a two-column poller file. This is used to update the WFPC2 SVM aperture header keywords from the values in the poller - file. [#1683] + file. [#1683, #1150] - Removed the version restriction on matplotlib. [#1649] diff --git a/doc/.rtd-environment.yml b/doc/.rtd-environment.yml index a85684c17..feec004fe 100644 --- a/doc/.rtd-environment.yml +++ b/doc/.rtd-environment.yml @@ -2,23 +2,25 @@ name: rtd311 channels: - defaults dependencies: - - python - - setuptools + - astropy - numpy - numpydoc + - pip + - python=3.11 - scipy - - astropy - graphviz - pip: - - -e .. - - sphinx - - "sphinx_rtd_theme>1.2.0" - - sphinx_automodapi - - matplotlib - - "spherical_geometry>=1.2.22" + - astroquery - fitsblender + - lxml + - matplotlib - nictools + - photutils - pyregion + - "spherical_geometry>=1.2.22" + - sphinx + - "sphinx_rtd_theme>1.2.0" + - sphinx_automodapi - stwcs - stsci.tools - "stsci.image>=2.3.0" @@ -27,7 +29,4 @@ dependencies: - stsci.stimage - "tweakwcs>=0.8.0" - stregion - - astroquery - - photutils - - lxml - tables diff --git a/doc/source/explanation/hap_introduction.rst b/doc/source/explanation/hap_introduction.rst index cf45107b8..bafbcdb41 100644 --- a/doc/source/explanation/hap_introduction.rst +++ b/doc/source/explanation/hap_introduction.rst @@ -2,4 +2,19 @@ Introduction ------------ .. _hap-introduction: -*Coming soon!* \ No newline at end of file +The ``drizzlepac`` package can be used for many purposes, all related to aligning and combining images to create products which can provide the deepest available views of the data. Combining the data with ``drizzlepac`` relies on the WCS solution specified in the input image headers. These WCS solutions are expected to align the images to each other (relative astrometry) as well as align the image to the correct position on the sky (absolute astrometry). The telemetry from HST allows the relative astrometry to be known extremely accurately (sub-milli-arcsecond level) when all images use the same guide stars and when the images were taken in the same visit. However, data taken at different times using different guide stars have historically had errors in the alignment with a sigma of 1 arc-second (or more). As a result, corrections to the alignment need to be made in order to successfully combine the images. + +The code being used in the automated HST calibration pipeline to generate the products served by the HST Archive through the MAST Portal relies on multiple methods to do the best job possible in aligning and combining data regardless of whether they came from the same visit (or instrument) or not. + +On December 17, 2020, MAST began production of new ACS and WFC3 products in the HST data calibration pipeline: Hubble Legacy Archive (HLA)-style mosaics comprising the data from a single HST visit which are aligned to a common astrometric reference frame. These are the first of two types of **Hubble Advanced Products (HAP)** that will be produced in the HST data pipeline and made available through the `MAST Discovery Portal `_. + +Three levels of products are available as part of this release: + + * Exposure level products contain data from a single HST exposure. + * Filter level products are produced from all exposures in a visit with a common filter. + * Total level products combine all exposures from a visit for a specific detector and are intended as a detection image for producing catalogs. + +The **HAP Single Visit Mosaics (SVMs)** differ from the standard HST drizzled data products, which are aligned filter-by-filter to Gaia. SVM data products, on the other hand, are all drizzled onto the same north-up pixel grid and may have improved relative alignment across filters within a given visit, enabling easy comparison of the images through multiple filters or for images to be combined to create color mosaics. When possible, sources in the images have been aligned directly to the Gaia source catalog to improve the WCS of the images. SVM data products with both relative alignment (by filter) and absolute alignment to Gaia will contain the string 'FIT_SVM_GAIA' in the 'WCSNAME' keyword in the science extension of the image header. More discussion on HAP alignment, may be found on the webpage `Improvements in HST Astrometry `_. + +Combining data across visits to create mosaics for every observed location on the sky gets performed using +**HAP Multi-Visit Mosaic (MVM)** processing. diff --git a/doc/source/index.rst b/doc/source/index.rst index d4de6306e..a1d072ee2 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -20,9 +20,6 @@ The Drizzlepac package is a collection packages for aligning and combining astro parameters reference - - - .. DrizzlePac Release Notes .. ------------------------ .. The code for this package gets released through a number of methods: namely, @@ -72,5 +69,4 @@ The Drizzlepac package is a collection packages for aligning and combining astro .. Combining data across visits to create mosaics for every observed location on the sky gets performed using -.. **HAP Multi-Visit Mosaic (MVM)** processing. - +.. **HAP Multi-Visit Mosaic (MVM)** processing. \ No newline at end of file diff --git a/doc/source/reference.rst b/doc/source/reference.rst index eeae10f0d..fbf6fb80a 100644 --- a/doc/source/reference.rst +++ b/doc/source/reference.rst @@ -8,9 +8,10 @@ Drizzle (Image drizzling step) .. toctree:: :maxdepth: 1 - reference/astrodrizzle - reference/adrizzle reference/ablot + reference/adrizzle + reference/astrodrizzle + reference/drizcr reference/median reference/outimage reference/photeq @@ -30,14 +31,12 @@ Pipeline processing :maxdepth: 1 reference/astrometry_api - reference/imagefindpars - reference/refimagefindpars reference/catalogs - reference/drizcr + reference/imagefindpars reference/mdztab reference/process + reference/refimagefindpars reference/util - reference/utilfuncs Interactive Image alignment --------------------------- @@ -79,10 +78,10 @@ Classes .. toctree:: :maxdepth: 1 + reference/acsobjects reference/baseobjects reference/image reference/imageobject - reference/acsobjects reference/nicmosobjects reference/stisobjects reference/wfc3objects diff --git a/doc/source/reference/astrodrizzle.rst b/doc/source/reference/astrodrizzle.rst index 70a0fb449..21f5c9402 100644 --- a/doc/source/reference/astrodrizzle.rst +++ b/doc/source/reference/astrodrizzle.rst @@ -1,15 +1,5 @@ .. _astrodrizzle_: -=================================== -Introduction to Pipeline processing -=================================== -*Coming soon!* - -==================== -Overview of pipeline -==================== -*Coming soon!* - ====================================== Primary User Interface: AstroDrizzle() ====================================== diff --git a/doc/source/reference/tweakback.rst b/doc/source/reference/tweakback.rst index 254d8c713..5bb5b93ae 100644 --- a/doc/source/reference/tweakback.rst +++ b/doc/source/reference/tweakback.rst @@ -1,8 +1,8 @@ .. _tweakback: -********* -TWEAKBACK -********* +*********************************** +TWEAKBACK: Propagating The Solution +*********************************** .. moduleauthor:: Warren Hack diff --git a/doc/source/reference/util.rst b/doc/source/reference/util.rst index 02433a34d..8914044a4 100644 --- a/doc/source/reference/util.rst +++ b/doc/source/reference/util.rst @@ -12,7 +12,6 @@ and interpretation of the ``MDRIZTAB`` reference file. .. toctree:: :maxdepth: 3 - utilfuncs wcsutils outimage mdztab diff --git a/doc/source/reference/utilfuncs.rst b/doc/source/reference/utilfuncs.rst deleted file mode 100644 index e2c6855d2..000000000 --- a/doc/source/reference/utilfuncs.rst +++ /dev/null @@ -1,12 +0,0 @@ -***************** -Utility Functions -***************** -These functions perform various small operations within `~drizzlepac.astrodrizzle`.``AstroDrizzle``. - -.. moduleauthor:: Warren Hack - -.. automodule:: drizzlepac.util - :members: - :undoc-members: - - diff --git a/drizzlepac/haputils/analyze.py b/drizzlepac/haputils/analyze.py index dbae6751a..c1257322b 100644 --- a/drizzlepac/haputils/analyze.py +++ b/drizzlepac/haputils/analyze.py @@ -524,6 +524,7 @@ def analyze_data(input_file_list, log_level=logutil.logging.DEBUG, type=""): msg_type = Messages.NOPROC.value else: msg_type = Messages.WARN.value + analyze_data_good_index.append(i) process_msg = no_proc_key + '=' + str(no_proc_value) @@ -543,7 +544,6 @@ def analyze_data(input_file_list, log_level=logutil.logging.DEBUG, type=""): total_rms, dataset_key, status, fit_qual, headerlet_file, compromised]) process_msg = "" - return output_table, analyze_data_good_index diff --git a/drizzlepac/haputils/poller_utils.py b/drizzlepac/haputils/poller_utils.py index da7c75d90..0669bda00 100644 --- a/drizzlepac/haputils/poller_utils.py +++ b/drizzlepac/haputils/poller_utils.py @@ -116,6 +116,7 @@ # def interpret_obset_input(results: str, log_level): """ + parses the file generated by the pipeline poller, and produces a tree listing of the output products. Parameters ----------- @@ -141,22 +142,32 @@ def interpret_obset_input(results: str, log_level): Interpret the database query for a given obset to prepare the returned values for use in generating the names of all the expected output products. - Input will have formated rows of one of the following three options:: + Input will have formated rows of one of the following three options 1) Full poller file + + .. code-block:: + ib4606c5q_flc.fits,11665,B46,06,1.0,F555W,UVIS,/foo/bar/ib4606c5q_flc.fits 2) Simpler poller file (list); other info taken from file header keywords + + .. code-block:: + ib4606c5q_flc.fits 3) For updating the WFPC2 SVM aperture keyword using the poller file; it is important that there are no spaces within the poller aperture keyword(s) + .. code-block:: + ib4606c5q_flc.fits, PC1-FIX;F160BN15 Full poller files contain filename, proposal_id, program_id, obset_id, exptime, filters, detector, pathname. - The output dict will have format (as needed by further code for creating the product filenames) of:: + The output dict will have format (as needed by further code for creating the product filenames) of + + .. code-block:: obs_info_dict["single exposure product 00": {"info": '11665 06 wfc3 uvis empty_aperture ib4606c5q f555w drc', "files": ['ib4606c5q_flc.fits']} diff --git a/pyproject.toml b/pyproject.toml index 2f0652af3..0b8873015 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,7 +9,7 @@ description = """ readme = { file = 'README.md', content-type = 'text/x-rst' } requires-python = '>=3.9' license = { file = "LICENSE.txt" } -authors = [{ name = 'Megan Sosey' }, { name = 'Warren Hack' }, { name = 'Christopher Hanley' }, { name = 'Chris Sontag' }, { name = 'Mihai Cara' }, { name = 'Steven Goldman' }] +authors = [{ name = 'Megan Sosey' }, { name = 'Warren Hack' }, { name = 'Christopher Hanley' }, { name = 'Chris Sontag' }, { name = 'Mihai Cara' }, { name = 'Michael Dulude' }, { name = 'Michele De La Pena' }, { name = 'Steven Goldman' }] classifiers = [ 'Intended Audience :: Science/Research', "Topic :: Scientific/Engineering :: Astronomy",