Release v3.2.0 -> main

[jethror1]

Summary

Merging of changes for v3.2.0 release into main

Changes

• properly check for files to unarchive before running any jobs to ensure all required files are unarchived
  • patterns currently defined in utils.defaults
• improve details in readme
• catch samples with no tests codes and raise as an error
• fix total sample no run in summary report
• fix the issue with subsetting the manifest to allow skipping samples, restricts valid samplename checking to just the subset
• strip whitespace on string inputs to not cause jobs to fail from inputs like exclude samples having bonus spaces
• addition of new input -iunarchive_only to allow for just unarchiving and not running jobs
• fix issue of single gene reports having : in the report filename

Issues closed

• batch does not unarchive all files for reanalysis #195
• Ensure all files unarchived #196
• lack of test code in manifest not flagged #197
• Subset + manifest validation #198
• Mismatch in total samples run in summary report #199
• Improve description of non-reference file placeholders in the config file design section of README #206
• strip leading / trailing whitespace from user string inputs #207
• Single gene file naming #214

Consitituent PRs

• DI-930 fix #195 - correctly check all archived files #200
• DI-930 fix #197 - fix skipping samples with missing test codes #201
• DI-930 fix #198 - allow invalid samples in manifest when subsetting #202
• DI-930 fix #199 - improve summary text #203
• DI-930 fix #207 strip whitespace from inputs #208
• IN-930 - fix #196 better unarchival checking #209
• IN-930 - update .coveragerc path #210
• IN-930 - update readme #211
• IN-930 - add multiqc html pattern to defaults.py #212
• IN-930 add new input unarchive only #213
• IN-930 - fix #214 single gene report name #215
• IN-930 - update default unarchive patterns #216
• IN-930 - combine unarchiving inputs #217
• IN-930 - update dias_batch.py #219

---

This change is 

[pep8speaks]

Hello @jethror1! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file resources/home/dnanexus/dias_batch/dias_batch.py:

Line 176:80: E501 line too long (82 > 79 characters)
Line 281:80: E501 line too long (84 > 79 characters)

• In the file resources/home/dnanexus/dias_batch/tests/test_dias_batch.py:

Line 151:46: E712 comparison to True should be 'if cond is True:' or 'if cond:'
Line 163:46: E712 comparison to False should be 'if cond is False:' or 'if not cond:'

• In the file resources/home/dnanexus/dias_batch/tests/test_dx_requests.py:

Line 681:5: E303 too many blank lines (2)
Line 710:5: E303 too many blank lines (2)
Line 742:5: E303 too many blank lines (2)
Line 794:80: E501 line too long (86 > 79 characters)
Line 799:5: E303 too many blank lines (2)
Line 895:5: E303 too many blank lines (2)
Line 920:59: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
Line 937:59: E712 comparison to True should be 'if cond is True:' or 'if cond:'
Line 973:1: E302 expected 2 blank lines, found 1
Line 1025:5: E303 too many blank lines (2)
Line 1031:5: E303 too many blank lines (2)
Line 1065:5: E303 too many blank lines (2)
Line 1105:5: E303 too many blank lines (2)
Line 2365:5: E303 too many blank lines (2)

• In the file resources/home/dnanexus/dias_batch/tests/test_utils.py:

Line 388:5: E303 too many blank lines (2)
Line 833:5: E303 too many blank lines (2)
Line 854:5: E303 too many blank lines (2)

• In the file resources/home/dnanexus/dias_batch/utils/defaults.py:

Line 27:17: E231 missing whitespace after ':'
Line 37:2: W292 no newline at end of file

• In the file resources/home/dnanexus/dias_batch/utils/dx_requests.py:

Line 363:5: E303 too many blank lines (2)
Line 371:9: E125 continuation line with same indent as next logical line
Line 477:5: E303 too many blank lines (2)
Line 483:9: E125 continuation line with same indent as next logical line

Comment last updated at 2024-07-29 13:39:41 UTC

[jethror1]

Testing completed and signed off, ready for production: https://cuhbioinformatics.atlassian.net/browse/DI-931?atlOrigin=eyJpIjoiNTVmYWEzYmEwZTUyNGJkM2E5MjE4NzZiM2EyZDkxN2QiLCJwIjoiamlyYS1zbGFjay1pbnQifQ

[Addy81]

All previous PRs code-reviewed merged to release branch, testing now signed off, merging to main as part of deployment.

Reviewed 10 of 11 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @jethror1)