From 86d6274b53357371c83baeb01d1ef0ddcea04a08 Mon Sep 17 00:00:00 2001 From: zsusswein <46581799+zsusswein@users.noreply.github.com> Date: Wed, 16 Oct 2024 20:23:09 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20CDCgov/c?= =?UTF-8?q?fa-epinow2-pipeline@b859765825e0eb16c4894943dc9a0c92fd763839=20?= =?UTF-8?q?=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- index.html | 14 +++++++------- news/index.html | 3 ++- pkgdown.yml | 2 +- search.json | 2 +- 4 files changed, 11 insertions(+), 10 deletions(-) diff --git a/index.html b/index.html index 40bef7b5..a1eb06b6 100644 --- a/index.html +++ b/index.html @@ -143,11 +143,11 @@

Model-estimated quantities

Automation

-

The project has multiple GitHub Actions workflows to automate the CI/CD process. Notably, the 1_pre-Test-Model-Image-Build.yaml workflow executes jobs using a self-hosted runner, and serves as an entry point for starting the pipeline. The workflow has the following three jobs:

+

The project has multiple GitHub Actions workflows to automate the CI/CD process. Notably, the containers-and-az-pool.yaml workflow executes jobs using a self-hosted runner, and serves as an entry point for starting the pipeline. The workflow has the following three jobs:

Both container tags and pool ids are based on the branch name, making it compatible with having multiple pipelines running simultaneously.

@@ -161,21 +161,21 @@

Automation

CFAEpiNow2Pipeline (development version)

-
  • Checks if batch pool exists. Pools are named after branches. Also allows for deletion via commit message.
    • +
    • Re-organizing GitHub workflows.
      • +
    • Checks if batch pool exists. Pools are named after branches. Also allows for deletion via commit message.
    • Merges workflows 1 and 2 into a single workflow.
    • Now uses CFA Azure ACR and images in the workflows and Dockerfiles, etc.
    • Added Docker image with all the requirements to build the package.
      • diff --git a/pkgdown.yml b/pkgdown.yml index e40805c0..d9a1624d 100644 --- a/pkgdown.yml +++ b/pkgdown.yml @@ -2,7 +2,7 @@ pandoc: 3.1.11 pkgdown: 2.1.1 pkgdown_sha: ~ articles: {} -last_built: 2024-10-16T16:58Z +last_built: 2024-10-16T20:23Z urls: reference: https://cdcgov.github.io/cfa-epinow2-pipeline/reference article: https://cdcgov.github.io/cfa-epinow2-pipeline/articles diff --git a/search.json b/search.json index 9e75185f..047e6a30 100644 --- a/search.json +++ b/search.json @@ -1 +1 @@ -[{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":null,"dir":"","previous_headings":"","what":"Welcome!","title":"Welcome!","text":"Thank contributing CDC’s Open Source projects! questions doubts, don’t afraid send way. appreciate contributions, looking forward fostering open, transparent, collaborative environment.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"package-authorship","dir":"","previous_headings":"","what":"Package authorship","title":"Welcome!","text":"guidelines build recommendations R packages. Maintainer: CFA team member actively charge maintaining package. Authors: CFA team member taken lead role code development, review, testing, etc. Contributors: Team members made smaller substantial contributions codebase (40+ hours, including coding, code review, testing, etc.) Smaller contributions: Smaller contributiosn code repo predecessor, cdcent/cfa-nnh-pipelines, including small bug fixes, issuees, code review granted package authorship description file, may acknowledged NEWS.md release notes appropriate contributing, encourage also read LICENSE, README, code--conduct files, also found repository. inquiries questions answered content repository, feel free contact us.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"public-domain","dir":"","previous_headings":"","what":"Public Domain","title":"Welcome!","text":"project public domain within United States, copyright related rights work worldwide waived CC0 1.0 Universal public domain dedication. contributions project released CC0 dedication. submitting pull request agreeing comply waiver copyright interest.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"requesting-changes","dir":"","previous_headings":"","what":"Requesting Changes","title":"Welcome!","text":"pull request/merging process designed give CDC Surveillance Team space opportunity consider discuss suggested changes. policy affects CDC spaces, -line , users expected abide .","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"open-an-issue-in-the-repository","dir":"","previous_headings":"Requesting Changes","what":"Open an issue in the repository","title":"Welcome!","text":"don’t specific language submit like suggest change something addressed, can open issue repository. Team members respond issue soon possible.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"submit-a-pull-request","dir":"","previous_headings":"Requesting Changes","what":"Submit a pull request","title":"Welcome!","text":"like contribute, please submit pull request. order us merge pull request, must: * least seven days old. Pull requests may held longer necessary give people opportunity assess . * Receive +1 majority team members associated request. significant dissent team, meeting held discuss plan action pull request.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/DISCLAIMER.html","id":null,"dir":"","previous_headings":"","what":"DISCLAIMER","title":"DISCLAIMER","text":"Use service limited non-sensitive publicly available data. Users must use, share, store kind sensitive data like health status, provision payment healthcare, Personally Identifiable Information (PII) /Protected Health Information (PHI), etc. circumstance. Administrators service reserve right moderate information used, shared, stored service time. user abide disclaimer Code Conduct may subject action, including revoking access services. material embodied software provided “-” without warranty kind, express, implied otherwise, including without limitation, warranty fitness particular purpose. event shall Centers Disease Control Prevention (CDC) United States (U.S.) government liable anyone else direct, special, incidental, indirect consequential damages kind, damages whatsoever, including without limitation, loss profit, loss use, savings revenue, claims third parties, whether CDC U.S. government advised possibility loss, however caused theory liability, arising connection possession, use performance software.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"Apache License","title":"Apache License","text":"Version 2.0, January 2004 ","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_1-definitions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"1. Definitions","title":"Apache License","text":"“License” shall mean terms conditions use, reproduction, distribution defined Sections 1 9 document. “Licensor” shall mean copyright owner entity authorized copyright owner granting License. “Legal Entity” shall mean union acting entity entities control, controlled , common control entity. purposes definition, “control” means () power, direct indirect, cause direction management entity, whether contract otherwise, (ii) ownership fifty percent (50%) outstanding shares, (iii) beneficial ownership entity. “” (“”) shall mean individual Legal Entity exercising permissions granted License. “Source” form shall mean preferred form making modifications, including limited software source code, documentation source, configuration files. “Object” form shall mean form resulting mechanical transformation translation Source form, including limited compiled object code, generated documentation, conversions media types. “Work” shall mean work authorship, whether Source Object form, made available License, indicated copyright notice included attached work (example provided Appendix ). “Derivative Works” shall mean work, whether Source Object form, based (derived ) Work editorial revisions, annotations, elaborations, modifications represent, whole, original work authorship. purposes License, Derivative Works shall include works remain separable , merely link (bind name) interfaces , Work Derivative Works thereof. “Contribution” shall mean work authorship, including original version Work modifications additions Work Derivative Works thereof, intentionally submitted Licensor inclusion Work copyright owner individual Legal Entity authorized submit behalf copyright owner. purposes definition, “submitted” means form electronic, verbal, written communication sent Licensor representatives, including limited communication electronic mailing lists, source code control systems, issue tracking systems managed , behalf , Licensor purpose discussing improving Work, excluding communication conspicuously marked otherwise designated writing copyright owner “Contribution.” “Contributor” shall mean Licensor individual Legal Entity behalf Contribution received Licensor subsequently incorporated within Work.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_2-grant-of-copyright-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"2. Grant of Copyright License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable copyright license reproduce, prepare Derivative Works , publicly display, publicly perform, sublicense, distribute Work Derivative Works Source Object form.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_3-grant-of-patent-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"3. Grant of Patent License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable (except stated section) patent license make, made, use, offer sell, sell, import, otherwise transfer Work, license applies patent claims licensable Contributor necessarily infringed Contribution(s) alone combination Contribution(s) Work Contribution(s) submitted. institute patent litigation entity (including cross-claim counterclaim lawsuit) alleging Work Contribution incorporated within Work constitutes direct contributory patent infringement, patent licenses granted License Work shall terminate date litigation filed.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_4-redistribution","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"4. Redistribution","title":"Apache License","text":"may reproduce distribute copies Work Derivative Works thereof medium, without modifications, Source Object form, provided meet following conditions: () must give recipients Work Derivative Works copy License; (b) must cause modified files carry prominent notices stating changed files; (c) must retain, Source form Derivative Works distribute, copyright, patent, trademark, attribution notices Source form Work, excluding notices pertain part Derivative Works; (d) Work includes “NOTICE” text file part distribution, Derivative Works distribute must include readable copy attribution notices contained within NOTICE file, excluding notices pertain part Derivative Works, least one following places: within NOTICE text file distributed part Derivative Works; within Source form documentation, provided along Derivative Works; , within display generated Derivative Works, wherever third-party notices normally appear. contents NOTICE file informational purposes modify License. may add attribution notices within Derivative Works distribute, alongside addendum NOTICE text Work, provided additional attribution notices construed modifying License. may add copyright statement modifications may provide additional different license terms conditions use, reproduction, distribution modifications, Derivative Works whole, provided use, reproduction, distribution Work otherwise complies conditions stated License.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_5-submission-of-contributions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"5. Submission of Contributions","title":"Apache License","text":"Unless explicitly state otherwise, Contribution intentionally submitted inclusion Work Licensor shall terms conditions License, without additional terms conditions. Notwithstanding , nothing herein shall supersede modify terms separate license agreement may executed Licensor regarding Contributions.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_6-trademarks","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"6. Trademarks","title":"Apache License","text":"License grant permission use trade names, trademarks, service marks, product names Licensor, except required reasonable customary use describing origin Work reproducing content NOTICE file.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_7-disclaimer-of-warranty","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"7. Disclaimer of Warranty","title":"Apache License","text":"Unless required applicable law agreed writing, Licensor provides Work (Contributor provides Contributions) “” BASIS, WITHOUT WARRANTIES CONDITIONS KIND, either express implied, including, without limitation, warranties conditions TITLE, NON-INFRINGEMENT, MERCHANTABILITY, FITNESS PARTICULAR PURPOSE. solely responsible determining appropriateness using redistributing Work assume risks associated exercise permissions License.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_8-limitation-of-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"8. Limitation of Liability","title":"Apache License","text":"event legal theory, whether tort (including negligence), contract, otherwise, unless required applicable law (deliberate grossly negligent acts) agreed writing, shall Contributor liable damages, including direct, indirect, special, incidental, consequential damages character arising result License use inability use Work (including limited damages loss goodwill, work stoppage, computer failure malfunction, commercial damages losses), even Contributor advised possibility damages.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_9-accepting-warranty-or-additional-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"9. Accepting Warranty or Additional Liability","title":"Apache License","text":"redistributing Work Derivative Works thereof, may choose offer, charge fee , acceptance support, warranty, indemnity, liability obligations /rights consistent License. However, accepting obligations, may act behalf sole responsibility, behalf Contributor, agree indemnify, defend, hold Contributor harmless liability incurred , claims asserted , Contributor reason accepting warranty additional liability. END TERMS CONDITIONS","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"appendix-how-to-apply-the-apache-license-to-your-work","dir":"","previous_headings":"","what":"APPENDIX: How to apply the Apache License to your work","title":"Apache License","text":"apply Apache License work, attach following boilerplate notice, fields enclosed brackets [] replaced identifying information. (Don’t include brackets!) text enclosed appropriate comment syntax file format. also recommend file class name description purpose included “printed page” copyright notice easier identification within third-party archives.","code":"Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Zachary Susswein. Author, maintainer. Katelyn Gostic. Contributor. Nathan McIntosh. Contributor. Patrick Corbett. Contributor. Adam Howes. Contributor.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Susswein Z (2024). CFAEpiNow2Pipeline: EpiNow2 wrapper deployment Azure Batch. R package version 0.0.0.9000, https://cdcgov.github.io/cfa-epinow2-pipeline/.","code":"@Manual{, title = {CFAEpiNow2Pipeline: EpiNow2 wrapper for deployment to Azure Batch}, author = {Zachary Susswein}, year = {2024}, note = {R package version 0.0.0.9000}, url = {https://cdcgov.github.io/cfa-epinow2-pipeline/}, }"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":null,"dir":"","previous_headings":"","what":"Creating a Culture of Innovation","title":"Creating a Culture of Innovation","text":"aspire create culture people work joyfully, communicate openly things matter, provide great services globally. like team communities (government private sector) reflect diversity kinds, just classes protected law. Diversity fosters innovation. Diverse teams creative teams. need diversity perspective create solutions challenges face. code conduct (adapted 18F’s Code Conduct). follow Equal Employment Opportunity laws expect everyone work adhere GSA Anti-harassment Policy, even work Centers Disease Control Prevention GSA. expect every user follow code conduct laws policies mentioned .","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"be-empowering","dir":"","previous_headings":"","what":"Be Empowering","title":"Creating a Culture of Innovation","text":"Consider can encourage support others. Make room quieter voices contribute. Offer support enthusiasm great ideas. Leverage low cost experimentation support colleagues’ ideas, take care acknowledge original source. Look ways contribute collaborate, even situations normally wouldn’t. Share knowledge skills. Prioritize access input traditionally excluded civic process.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"rules-of-behavior","dir":"","previous_headings":"","what":"Rules of Behavior","title":"Creating a Culture of Innovation","text":"understand must complete security awareness records management training annually order comply latest security records management policies. understand must also follow Rules Behavior use HHS Information Resources understand must use, share, store kind sensitive data (health status, provision payment healthcare, PII, etc.) circumstance. knowingly conceal, falsify, remove information. understand can use non-sensitive /publicly available data. understand passwords create set accounts need comply CDC’s password policy. understand stewards reserves right moderate data time.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"boundaries","dir":"","previous_headings":"","what":"Boundaries","title":"Creating a Culture of Innovation","text":"Create boundaries behavior consider can create safe space helps prevent unacceptable behavior others. can’t list instances unacceptable behavior, can provide examples help guide community thinking respond experience types behavior, whether directed others. unsure something appropriate behavior, probably . person interact can define line . Impact matters intent. Ensuring behavior negative impact responsibility. Problems usually arise assume way thinking behavior norm everyone.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"here-are-some-examples-of-unacceptable-behavior","dir":"","previous_headings":"Boundaries","what":"Here are some examples of unacceptable behavior","title":"Creating a Culture of Innovation","text":"Negative offensive remarks based protected classes listed GSA Anti-harassment Policy race, religion, color, sex, national origin, age, disability, genetric information, sexual orientation, gender identity, parental status, maritual status, political affiliation well gender expression, mental illness, socioeconomic status backgrounds, neuro()typicality, physical appearance, body size, clothing. Consider calling attention differences can feel alienating. Sustained disruption meetings, talks, discussions, including chatrooms. Patronizing language behavior. Aggressive behavior, unconstructive criticism, providing correction improve conversation (sometimes referred “well actually’s”), repeatedly interrupting talking someone else, feigning surprise someone’s lack knowledge awareness topic, subtle prejudice. Referring people way misidentifies gender /rejects validity gender identity; instance using incorrect pronouns forms address (misgendering). Retaliating anyone files formal complaint someone violated codes laws.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"background","dir":"","previous_headings":"","what":"Background","title":"Creating a Culture of Innovation","text":"CDC Scientific Clearance process obtaining approvals appropriate CDC officials CDC information product released public CDC’s external public health partners. Information products require formal clearance include print, electronic, oral materials, CDC employees author co-author, whether published CDC outside CDC. CDC contractors developing content behalf CDC public CDC’s external public health partners also required put content formal clearance process. collaborative functions related projects include blogs, wikis, forums, bug tracking sites, source control others deemed necessary. individuals within CDC, adherence following policies required: * CDC “Clearance Information Products Disseminated Outside CDC Public Use” * HHS “Ensuring Quality Information Disseminated HHS agencies” collaborative materials controlled rules contained within document. allow real-time collaboration opportunities among CDC employees, CDC contractors CDC public health partners.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"credit","dir":"","previous_headings":"","what":"Credit","title":"Creating a Culture of Innovation","text":"code conduct mainly adapted 18F’s Code Conduct CDC’s Informatics Innovation Unit R&D Lab’s code conduct.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"relevant-legal-considerations","dir":"","previous_headings":"","what":"Relevant Legal Considerations","title":"Creating a Culture of Innovation","text":"Laws enforced Equal Employment Opportunity Commission Types discrimination prohibited law New proposed regulations","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"overview","dir":"","previous_headings":"","what":"Overview","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"lightweight wrapper around {EpiNow2} add functionality deployment Azure Batch. holds helper functions interface Azure services, convert input data EpiNow2’s expected input format, save expected outputs. also adds metadata logging. package meant enhance EpiNow2 package support deployment CFA’s computational environment. code open source part CFA’s goals around development, may possible support extensions additional environments.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"structure","dir":"","previous_headings":"","what":"Structure","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository holds R package, CFAEpiNow2Pipeline. repository structured standard R package. PRs pass R CMD check part CI suite pre-condition merge main. interested contributing see CONTRIBUTING.md open issue PR. package contains contains adapters wrappers run run many independent EpiNow2 models parallel cloud resources. adapters read datasets standardized formats produces outputs flat files standard names. wrapper functions enhance EpiNow2 functionality support cloud deployments, adding logging standardizing R environment. package standardizes interface EpiNow2 purposes deployment pipeline part suite models. package manage pipeline deployment kickoff, data extraction transformation, model output visualization.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"components","dir":"","previous_headings":"","what":"Components","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"package implements functions : config validated runtime, config generation specified pipeline runtime part package. Paths specified via config Parameters: Loads pre-specified -validated generation interval, delay interval, right-truncation distributions CFA data lake local environment formats use EpiNow2. Model run: Manages R environment run EpiNow2 fixed random seed, EpiNow2 initialization Stan sampling. Outputs: Provides functionality process EpiNow2 model fits standardised flat output format (described forthcoming link). Within pipeline, model fits saved entirety .rds files, well via flat output format. Logging: Steps pipeline comprehensive R-style logging, cli package Metadata: Extract comprehensive metadata model run store alongside outputs","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"output-format","dir":"","previous_headings":"","what":"Output format","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"end goals package standardize raw outputs EpiNow2 samples summaries tables, write standardized outputs, along relevant metadata, logs, etc. standard directory structure. CFA’s standard format, outputs can passed separate pipeline handles post-processing (e.g. plotting, scoring, analysis) Rt estimates several different Rt estimation models.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"directories","dir":"","previous_headings":"Output format","what":"Directories","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"nested partitioning structure outputs designed facilitate automated processes manual investigation: files organized job task IDs, allowing efficient file operations using glob patterns, also maintaining clear hierarchy aids human users navigating specific results logs. Files meant primarily machine-readable consumption (.e., draws, summaries, diagnostics) structured together make globbing easier. Files meant primarily human investigation (.e., logs, model fit object) grouped together task facilitate manual workflows. workflow, task IDs correspond location specific model runs (independent one another) jobid refers unique model run disease. example, production job contain task IDs 50 states US, job submitted testing experimentation might contain smaller number tasks/locations. /: base output directory. , example, / Docker container dedicated output directory. - job_/: directory named specific job identifier, containing outputs related job. tasks within job share top-level directory. - raw_samples/: subdirectory within job folder holds raw sample files tasks job. Task-specific draws output files live together directory enable easy globbing task-partitioned outputs. - samples_.parquet: file containing raw samples model, associated particular task identifier. file columns job_id, task_id, geo_value, disease, model, _draw, _chain, _iteration, _variable, value, reference_date. variables follow {tidybayes} specification. - summarized_quantiles/: subdirectory storing summarized quantile data. Task-specific summarized output files live together directory enable easy globbing task-partitioned outputs. - summarized_.parquet: file summarized quantiles relevant specific task identifier. file columns job_id, task_id, geo_value, disease, model, value, _lower, _upper, _width, _point, _interval, reference_date. variables follow {tidybayes} specification. - diagnostics/: subdirectory storing model fit diagnostics. Task-specific diagnostic output files live together directory enable easy globbing task-partitioned outputs. - diagnostic_.parquet: file diagnostics relevant specific task identifier. file columns diagnostic, value, job_id, task_id, geo_value, disease, model. - tasks/: directory contains subdirectories task within job. files less likely require globbing data lake manual investigation, stored togehter. - task_/: task folder identified task ID, includes several files: - model.rds: RDS file storing EpiNow2 model object fit data. - metadata.json: JSON file containing additional metadata model run task. - stdout.log: log file capturing standard output model run process. - stderr.log: log file capturing standard error output model run process. - job_metadata.json: JSON file located root job’s directory, providing metadata entire job.","code":"/ ├── job_/ │ ├── raw_samples/ │ │ ├── samples_.parquet │ ├── summarized_quantiles/ │ │ ├── summarized_.parquet │ ├── diagnostics/ │ │ ├── diagnostics_.parquet │ ├── tasks/ │ │ ├── task_/ │ │ │ ├── model.rds │ │ │ ├── metadata.json │ │ │ ├── stdout.log │ │ │ └── stderr.log │ ├── job_metadata.json"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"model-estimated-quantities","dir":"","previous_headings":"Output format","what":"Model-estimated quantities","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"EpiNow2 estimates incident cases ŷtd\\hat y_{td} timepoint t∈{1,...,T}t \\\\{1, ..., T\\} delay d∈{1,...,D}d \\\\{1, ..., D\\} D≤TD \\le T. single vintage ’re providing EpiNow2, delay dd moves inversely timepoints, d=T−t+1d = T - t + 1. observed data vector length TT ytd∈Wy_{td} \\W. supply nowcasting correction PMF ν\\nu last DD timepoints νd∈[0,1],\\nu_d \\[0, 1], ∑d=1Dνd=1\\sum_{d=1}^D\\nu_d = 1. also priors Θ\\Theta. use EpiNow2’s generative model f(y,ν,Θ)f(y, \\nu, \\Theta). EpiNow2 forward model produces expected nowcasted case count tt dd pair: γ̂td\\hat \\gamma_{td}. applies nowcasting correction ν\\nu last DD timepoints γ̂\\hat \\gamma produce expected right-truncated case count ŷ\\hat y. Note expected case counts (without right-truncation) don’t observation noise included. can apply negative binomial observation noise using EpiNow2’s estimate negative binomial overdispersion parameter ϕ̂\\hat \\phi expected case counts. posterior predictive distributions nowcasted case counts γ̃∼NB(γ̂,ϕ̂)\\tilde \\gamma \\sim \\text{NB}(\\hat \\gamma, \\hat \\phi). posterior predicted right-truncated case count ỹ∼NB(ŷ,ϕ̂)\\tilde y \\sim \\text{NB}(\\hat y, \\hat \\phi). can get 3 4 quantities pre-generated returned EpiNow2 Stan model: γ̂\\hat \\gamma: expected nowcasted case count reports[t] ŷ\\hat y: expected right-truncated case count obs_reports[t] γ̃\\tilde \\gamma: posterior-predicted nowcasted case count imputed_reports[t] ỹ\\tilde y: posterior-predicted right-truncated case count isn’t returned EpiNow2. also save RtR_t estimate time tt intrinsic growth rate time tt.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"automation","dir":"","previous_headings":"","what":"Automation","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"project multiple GitHub Actions workflows automate CI/CD process. Notably, 1_pre-Test-Model-Image-Build.yaml workflow executes jobs using self-hosted runner, serves entry point starting pipeline. workflow following three jobs: Job01-build_image_dependencies: Creates container image dependencies required build R package. job cached speed process, updates image Dockerfile-dependencies DESCRIPTION file changes. image pushed Azure container registry: cfaprdbatchcr.azurecr.io/cfa-epinow2-pipeline-dependencies:[branch name]. _01_build-model-image: Using previous image base, job installs R package pushes image Azure container registry: cfaprdbatchcr.azurecr.io/cfa-epinow2-pipeline:[branch name]. _02_create-batch-pool--submit-jobs: final job creates new Azure batch pool id cfa-epinow2-pool-[branch name] doesn’t already exist. Additionally, commit message contains string “[delete pool]”, pool deleted. container tags pool ids based branch name, making compatible multiple pipelines running simultaneously. [!IMPORTANT] CI fail branch names valid tag names containers. information, see official Azure documentation .","code":"flowchart LR START((Start))---DEPS_CACHED DEPS_CACHED{Deps
      cached?}---|No|DEPS DEPS_CACHED---|Yes|IMG subgraph DEPS[Job01-build_image_dependencies] direction TB Dockerfile-dependencies---|Generates|DEPS_IMAGE[Dependencies
      Image] end DEPS---IMG subgraph IMG[_01_build-model-image] direction TB Dockerfile---|Generates|PKG_IMG[Package
      Image] end IMG---POOL subgraph POOL[_02_create-batch-pool-and-submit-jobs] direction TB POOL_EXISTS{Is the pool
      up?} POOL_EXISTS---|No|CREATE_POOL[Create the pool] POOL_EXISTS---|Yes|SHOULD_DELETE_POOL{\"`Does the commit message
      include the phrase
      '_[delete pool]_'?`\"} SHOULD_DELETE_POOL---|Yes|DELETE_POOL[Delete the pool] SHOULD_DELETE_POOL---|No|END_POOL DELETE_POOL---END_POOL((End)) CREATE_POOL---END_POOL end"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"project-admin","dir":"","previous_headings":"","what":"Project Admin","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"@zsusswein @natemcintosh @kgostic","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"general-disclaimer","dir":"","previous_headings":"","what":"General Disclaimer","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository created use CDC programs collaborate public health related projects support CDC mission. GitHub hosted CDC, third party website used CDC partners share information collaborate software. CDC use GitHub imply endorsement one particular service, product, enterprise.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"public-domain-standard-notice","dir":"","previous_headings":"","what":"Public Domain Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository constitutes work United States Government subject domestic copyright protection 17 USC § 105. repository public domain within United States, copyright related rights work worldwide waived CC0 1.0 Universal public domain dedication. contributions repository released CC0 dedication. submitting pull request agreeing comply waiver copyright interest.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"license-standard-notice","dir":"","previous_headings":"","what":"License Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository utilizes code licensed terms Apache Software License therefore licensed ASL v2 later. source code repository free: can redistribute /modify terms Apache Software License version 2, (option) later version. source code repository distributed hope useful, WITHOUT WARRANTY; without even implied warranty MERCHANTABILITY FITNESS PARTICULAR PURPOSE. See Apache Software License details. received copy Apache Software License along program. , see http://www.apache.org/licenses/LICENSE-2.0.html source code forked open source projects inherit license.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"privacy-standard-notice","dir":"","previous_headings":"","what":"Privacy Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository contains non-sensitive, publicly available data information. material community participation covered Disclaimer Code Conduct. information CDC’s privacy policy, please visit http://www.cdc.gov//privacy.html.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"contributing-standard-notice","dir":"","previous_headings":"","what":"Contributing Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"Anyone encouraged contribute repository forking submitting pull request. (new GitHub, might start basic tutorial.) contributing project, grant world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license users terms Apache Software License v2 later. comments, messages, pull requests, submissions received CDC including GitHub page may subject applicable federal law, including limited Federal Records Act, may archived. Learn http://www.cdc.gov//privacy.html.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"records-management-standard-notice","dir":"","previous_headings":"","what":"Records Management Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository source government records copy increase collaboration collaborative potential. government records published CDC web site.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"additional-standard-notices","dir":"","previous_headings":"","what":"Additional Standard Notices","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"Please refer CDC’s Template Repository information contributing repository, public domain notices disclaimers, code conduct.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":null,"dir":"","previous_headings":"","what":"CDC GitHub Practices for Open Source Projects","title":"CDC GitHub Practices for Open Source Projects","text":"CDCGov organization GitHub designated use CDC programs publish open source code. set practices help programs release secure compliant open source projects successfully. interested using GitHub non-open source projects, please see information enterprise organization. designed practices straightforward helpful, accept feedback community updating . Required Practices, Projects don’t adhere Required Practices subject archival removal.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"getting-started","dir":"","previous_headings":"","what":"Getting Started","title":"CDC GitHub Practices for Open Source Projects","text":"can publish project, must request access added CDCgov organization. Complete steps: Review Rules Behavior. Confirm Github profile setup properly. require CDC login, don’t login, ask someone request behalf, get touch. receive email notification given access first repository setup . subsequent projects, able create repository organization using Github’s interface. template repository maintained easy way quick start repository complies guidelines. completed ’re ready follow required guidelines publish code.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"required-practices","dir":"","previous_headings":"","what":"Required Practices","title":"CDC GitHub Practices for Open Source Projects","text":"must follow practices publish real code repository. GitHub third party service used CDC collaborate public. Official CDC health messages always distributed www.cdc.gov appropriate channels, make sure plan project along official public health program cdc.gov. Add topics improve discovery use project. AI-related projects, Code.gov Implementation Guidance Federal Agencies Regarding Enterprise Data Source Code Inventories must followed setting topics. overview project, including purpose, goals team responsible. description development process README.md file. project longer active, mark archived. Public Domain Standard Notice License Standard Notice Privacy Standard Notice Contributing Standard Notice Records Management Standard Notice Additional Standard Notices need help choosing license, please review article, refer existing CDCgov projects, ask consultation support choosing license. final step publishing critical. unsure compliance, reach organization’s security officers. Make sure commit history Github repository also doesn’t things. many cases ’s easier start new repository push code sensitive information removed first commit. Enable GitHub automated security alerts configure notification repo admin see. Two-factor authentication (2FA). Project admins must secure account two-factor-authentication. Respond critical security issues communication administrators. Ignoring security issues responding communication administrators can result archiving removal. Archive old projects. ’re longer updating project moved ’s location, update README.md file let users know archive repository.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"recommended-practices","dir":"","previous_headings":"","what":"Recommended Practices","title":"CDC GitHub Practices for Open Source Projects","text":"Optional improvements make open source project successful. Establish pull request templates make easier contributors send pull requests. example SDP-V checklist PR match development practices. Agree project conventions include README.md file. Depending type project, includes folder structure data, linters, editor configuration (eg, MicrobeTrace’s .editorconfig). help improve quality project make easier others contribute project. Add support community procedures. CDC provide warranty official support open source projects, describing like questions issues assist users project. use wiki, project board, package manager, describe link . Official contribution steps make easier people outside CDC contribute project. Include references publications, presentations, sites featuring project. Add entry open.cdc.gov data, code, api, event page help people find project cdc.gov Add versions tags describing major releases milestones. example, open.cdc.gov’s releases time new version published web site geneflow’s changelog. Follow Semantic Versioning 2.0.0 creating versions project. Describe test reproducible practices install build project. example, injury_autocoding’s code section running project’s scripts). Recognize contributors existing resources helped project. example, fdns-ms-hl7-utils’ AUTHORS file. Automate build test procedures reduce effort outside contributors send pull requests (eg, Travis CI, Circle CI, GitHub Actions) Appropriately gather metrics project used incorporate feature planning process. Incorporate documentation development cycle, possible, automate generation documentation likely date useful people interested project.","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"support-and-feedback","dir":"","previous_headings":"Guidance","what":"Support and Feedback","title":"CDC GitHub Practices for Open Source Projects","text":"need additional support setting project, feedback ideas guidance please open issue send email data@cdc.gov. also accept pull requests want directly edit guidance.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"non-compliance-procedure","dir":"","previous_headings":"Guidance","what":"Non-Compliance Procedure","title":"CDC GitHub Practices for Open Source Projects","text":"Projects organization reviewed occasionally compliance Required Practices. project found compliance, contacted administrators help bring project compliance. Projects respond habitually fail meet practices archived removed organization, depending severity.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"profile-setup","dir":"","previous_headings":"Guidance","what":"Profile Setup","title":"CDC GitHub Practices for Open Source Projects","text":"Please make sure profile set properly help us work better together. Specifically, keep profile date : Name: first last name. Company: government agency contracting company. (also use GitHub personal projects, consider specifying “CDC (work) + personal projects” make clear GitHub projects may personal nature.) Location: primary work location (city, state). Photo: headshot photo, appropriate image unique . admin projects, make sure secure account two-factor authentication (2FA). Although probably already smart.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"open-source-checklist","dir":"","previous_headings":"Guidance","what":"Open Source Checklist","title":"CDC GitHub Practices for Open Source Projects","text":"’ve decided set open source project CDC. steps , common order. Create new project using template repo. Update readme.md following CDC GitHub Practices Open Source Projects Choose license. projects ASL2, license meet public health program need. See https://www.philab.cdc.gov/index.php/2012/03/27/open-source-development--public-health-informatics/ info choosing license. Remove sensitive info. Talk ADI, ADS, ISSO review clearance. approval, create GitHub user. Fill Request Repo form new repo CDCGov CDCai. get email push alert repo ready, push GitHub Add entry open.cdc.gov code page officially linked cdc.gov. helps users find use project. Keep project date, ’re finished flag archived. checklist adapted CDC Guard Rail put help people don’t access intranet.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"cdc-enterprise","dir":"","previous_headings":"Guidance","what":"CDC Enterprise","title":"CDC GitHub Practices for Open Source Projects","text":"CDCent organization used private, non-public projects CDC staff approved outside collaborators work projects, can request access GitHub Enterprise Cloud form.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"reference-links","dir":"","previous_headings":"Guidance","what":"Reference Links","title":"CDC GitHub Practices for Open Source Projects","text":"helpful links across Federal Government regarding open sourcing code. CFPB Open Tech TTS Engineering Practices Guide 18F Open Source Policy Practicing open source policy GitHub Government: agencies build software code.gov Federal Source Code Open Source Toolkit Federal Source Code Policy (M-16-21) openCDC Digital Services Playbook CDC/ATSDR Policy Releasing Sharing Data (old version, still useful reference) Clearance Information Products Disseminated Outside CDC Public Use Federal Source Code Toolkit","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert case counts in matching rows to NA — apply_exclusions","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"Mark selected points ignored model fitting. manual selection occurs externally pipeline passed pipeline exclusions file read read_exclusions(). Mechanically, exclusions applied converting specified points NAs dataset. NAs skipped model fitting EpiNow2, matched rows excluded model likelihood.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"","code":"apply_exclusions(cases, exclusions)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"cases dataframe returned read_data() exclusions dataframe returned read_exclusions()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"dataframe rows schema cases value column confirm converted NA rows match row exclusions","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":null,"dir":"Reference","previous_headings":"","what":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"Note think might wise instead specify blob prefix, list blobs, download listed blobs. let us flexibility downloading whole remote directories (like delta tables)","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"","code":"download_from_azure_blob(blob_names, local_dest, container_name)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"blob_names vector blobs donwload container_name local_dest path local directory save files container_name Azure Blob Storage container blob_names","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"NULL success","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":null,"dir":"Reference","previous_headings":"","what":"Extract diagnostic metrics from model fit and data — extract_diagnostics","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"function extracts various diagnostic metrics fitted EpiNow2 model provided data. checks low case counts computes diagnostics fitted model, including mean acceptance statistic, divergent transitions, maximum tree depth, Rhat values. diagnostics flagged exceed specific thresholds, results returned data frame.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"","code":"extract_diagnostics(fit, data, job_id, task_id, disease, geo_value, model)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"fit model fit object EpiNow2 data data frame containing input data used model fit. job_id unique identifier job task_id unique identifier task disease, geo_value, model Metadata downstream processing.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"data.frame containing extracted diagnostic metrics. data frame includes following columns: diagnostic: name diagnostic metric. value: value diagnostic metric. job_id: unique identifier job. task_id: unique identifier task. disease,geo_value,model: Metadata downstream processing.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"following diagnostics calculated: mean_accept_stat: average acceptance statistic across chains. p_divergent: proportion divergent transitions across samples. n_divergent: number divergent transitions across samples. p_max_treedepth: proportion samples hit maximum tree depth. p_high_rhat: proportion parameters Rhat values greater 1.05, indicating potential convergence issues. n_high_rhat: number parameters Rhat values greater 1.05, indicating potential convergence issues. low_case_count_flag: flag indicating low case counts data. See low_case_count_diagnostic() information diagnostic. epinow2_diagnostic_flag: combined flag indicates diagnostic thresholds exceeded. diagnostic thresholds (1) mean_accept_stat < 0.1, (2) p_divergent > 0.0075, (3) p_max_treedepth > 0.05, (4) p_high_rhat > 0.0075.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":null,"dir":"Reference","previous_headings":"","what":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"impure function depends environment variables: az_tenant_id az_subscription_id az_resource_group az_storage_account","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"","code":"fetch_blob_container(container_name)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"container_name Azure Blob Storage container associated credentials","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"Blob endpoint","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"error set.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":null,"dir":"Reference","previous_headings":"","what":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"throw informative error credential found","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"","code":"fetch_credential_from_env_var(env_var)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"env_var character, credential fetch","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"associated value","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":null,"dir":"Reference","previous_headings":"","what":"Fit an EpiNow2 model — fit_model","title":"Fit an EpiNow2 model — fit_model","text":"Fit EpiNow2 model","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Fit an EpiNow2 model — fit_model","text":"","code":"fit_model(data, parameters, seed, horizon, priors, sampler_opts)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Fit an EpiNow2 model — fit_model","text":"data, format returned read_data() parameters returned read_disease_parameters() seed random seed, used initialization EpiNow2 R sampling Stan horizon number days, integer, forecast priors list lists. first level contain key rt elements mean sd key gp element alpha_sd. sampler_opts list. Stan sampler options passed EpiNow2. required keys: cores, chains, iter_warmup, iter_sampling, max_treedepth, adapt_delta.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Fit an EpiNow2 model — fit_model","text":"fitted model object class epinow , model fitting fails, NA returned warning","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":null,"dir":"Reference","previous_headings":"","what":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"Gostic et al., 2020 simulates data stochastic SEIR model. Residence time E compartments exponentially distributed, mean 4 days (rate/inverse-scale 1/4). residence times imply gamma-distributed generation time distribution shape 2 rate 1/4. convert continuous gamma distribution PMF use {RtGam}.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"","code":"gostic_gt_pmf"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"gostic_gt_pmf numeric vector length 26 sums one within numerical tolerance","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"parametric specification, produce double-censored, left-truncated probability mass function generation interval distribution. produce PMF using {epinowcast}'s simulate_double_censored_pmf() version 0.3.0. See https://doi.org/10.1101/2024.01.12.24301247 information double-censoring biases corrections. correct output simulate_double_censored_pmf() make appropriate use {EpiNow2}. function returns numeric vector, position element corresponding one day length delay value corresponding amount discretized probability density bin. vector necessarily sum one. drop first element vector, corresponds zero-day delay. renewal framework, underpins model account zero-day delays. renormalize left-truncated vector sum one proper PMF.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":null,"dir":"Reference","previous_headings":"","what":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"dataset Gostic, Katelyn M., et al. \"Practical considerations measuring effective reproductive number, Rt.\" PLoS Computational Biology 16.12 (2020): e1008409. data simulated stochastic SEIR compartmental model.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"","code":"gostic_toy_rt"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"gostic_toy_rt data frame 301 rows 12 columns: time Timestep discrete-time stochastic SEIR simulation date Added original Gostic, 2020 dataset. date corresponding assigned time. Arbitrarily starts January 1st, 2023. S, E, , R realized state stochastic SEIR system dS, dEI, DIR stochastic transition compartments incidence true incidence compartment time t obs_cases observed number cases time t forward-convolved incidence. obs_incidence Added original Gostic, 2020 dataset. incidence column added negative-binomial observation noise. Created set.seed(123456) call rnbinom(299, mu = gostic_toy_rt[[\"incidence\"]], size = 10) Useful testing. true_r0 initial R0 system (.e., 2) true_rt known, true Rt epidemic system","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"source","dir":"Reference","previous_headings":"","what":"Source","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"https://github.com/cobeylab/Rt_estimation/tree/d9d8977ba8492ac1a3b8287d2f470b313bfb9f1d # nolint","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"synthetic dataset number desirable properties: force infection changes depending Rt, allowing sudden changes Rt. allows modeling sudden changes infection dynamics, might otherwise difficult capture. Rt estimation framework realized Rt known timepoint dataset incorporates simple generation interval reporting delay. Gostic et al. benchmark performance number Rt estimation frameworks, providing practical guidance use dataset evaluate Rt estimates. practice, found amount observation noise incidence /observed cases often undesirably low testing. Many empirical datasets much noisier. result, models built settings mind can perform poorly dataset fail converge. original dataset, add new column original incidence counts additional observation noise: obs_incidence. manually add observation noise rnbinom(299, mu = gostic_toy_rt[[\"obs_cases\"]], size = 10) random seed 123456 store obs_incidence column.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":null,"dir":"Reference","previous_headings":"","what":"Calculate low case count diagnostic flag — low_case_count_diagnostic","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"diagnostic flag TRUE either last two weeks dataset fewer aggregate 10 cases per week. aggregation excludes count confirmed outliers, set NA data.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"","code":"low_case_count_diagnostic(df)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"df dataframe returned read_data(). dataframe must include columns reference_date (date vector) confirm (number confirmed cases per day).","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"logical value (TRUE FALSE) indicating whether either last two weeks dataset fewer 10 cases per week.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"function assumes df input dataset \"completed\": implicit missingness made explicit.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":null,"dir":"Reference","previous_headings":"","what":"Format PMFs for EpiNow2 — opts_formatter","title":"Format PMFs for EpiNow2 — opts_formatter","text":"Opinionated wrappers around EpiNow2::generation_time_opts(), EpiNow2::delay_opts(), EpiNow2::dist_spec() formats generation interval, delay, right truncation parameters object ready input EpiNow2.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Format PMFs for EpiNow2 — opts_formatter","text":"","code":"format_generation_interval(pmf) format_delay_interval(pmf) format_right_truncation(pmf, data)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Format PMFs for EpiNow2 — opts_formatter","text":"pmf returned read_disease_parameters(). PMF vector NA, applying PMF model fit. data format returned read_data()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Format PMFs for EpiNow2 — opts_formatter","text":"EpiNow2::*_opts() formatted object NA message","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Format PMFs for EpiNow2 — opts_formatter","text":"Delays right truncation optional can skipped passing pmf = NA.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":null,"dir":"Reference","previous_headings":"","what":"Read in the dataset of incident case counts — read_data","title":"Read in the dataset of incident case counts — read_data","text":"row table corresponds single facilities' cases reference-date/report-date/disease tuple. want aggregate counts level geographic aggregate/report-date/reference-date/disease.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read in the dataset of incident case counts — read_data","text":"","code":"read_data( data_path, disease = c(\"COVID-19\", \"Influenza\", \"test\"), state_abb, report_date, max_reference_date, min_reference_date )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read in the dataset of incident case counts — read_data","text":"data_path path local file. contain glob must parquet format. disease One \"COVID-19\" \"Influenza\" state_abb two-letter uppercase abbreviation report_date desired single report date max_reference_date, min_reference_date first last reference dates, inclusive, timeseries","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read in the dataset of incident case counts — read_data","text":"dataframe one rows columns report_date, reference_date, state_abb, confirm","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read in the dataset of incident case counts — read_data","text":"handle two distinct cases geographic aggregates: single state: Subset facilities state aggregate state level 2. US overall: Aggregate facilities without subsetting Note apply exclusions . exclusions applied later, aggregations. means US overall, aggregate points might potentially excluded state level. recourse case exclude US overall aggregate point.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":null,"dir":"Reference","previous_headings":"","what":"Read in disease process parameters from an external file or files — read_disease_parameters","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"Read disease process parameters external file files","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"","code":"read_disease_parameters( generation_interval_path, delay_interval_path, right_truncation_path, disease, as_of_date, group, report_date )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"generation_interval_path, delay_interval_path, right_truncation_path Path local file parameter PMF. See read_interval_pmf details file schema. parameters can file different file. disease One COVID-19 Influenza as_of_date Use parameters used production date. Set current date --date version parameters set earlier date use parameters earlier time period. group optional parameter subset query parameter particular two-letter state abbrevation. Right now, parameter state-specific estimates right_truncation. report_date optional parameter subset query parameter particular report_date. Right now, parameter report date-specific estimates right_truncation. Note similar , different as_of_date. report_date used select particular value time-varying estimate. estimate may regenerated time (e.g., new data becomes available methodological update). can pull estimate date report_date generated date as_of_date.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"named list three PMFs. list elements named generation_interval, delay_interval, right_truncation. path local file provided (NA NULL), corresponding parameter estimate NA returned list.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"generation_interval_path required generation interval required parameter $R_t$ estimation. delay_interval_path right_truncation_path optional (strongly suggested).","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":null,"dir":"Reference","previous_headings":"","what":"Read exclusions from an external file — read_exclusions","title":"Read exclusions from an external file — read_exclusions","text":"Expects read CSV required columns: reference_date report_date state_abb disease","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read exclusions from an external file — read_exclusions","text":"","code":"read_exclusions(path)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read exclusions from an external file — read_exclusions","text":"path path exclusions file .csv format","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read exclusions from an external file — read_exclusions","text":"dataframe columns reference_date, report_date, state_abb, disease","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read exclusions from an external file — read_exclusions","text":"columns meaning read_data(). Additional columns allowed ignored reader.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":null,"dir":"Reference","previous_headings":"","what":"Read parameter PMF into memory — read_interval_pmf","title":"Read parameter PMF into memory — read_interval_pmf","text":"Using DuckDB parquet file. function expects file SCD2 format column names: parameter geo_value disease start_date end_date value","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read parameter PMF into memory — read_interval_pmf","text":"","code":"read_interval_pmf( path, disease = c(\"COVID-19\", \"Influenza\", \"test\"), as_of_date, parameter = c(\"generation_interval\", \"delay\", \"right_truncation\"), group = NA, report_date = NA )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read parameter PMF into memory — read_interval_pmf","text":"path path local file disease One COVID-19 Influenza as_of_date Use parameters used production date. Set current date --date version parameters set earlier date use parameters earlier time period. parameter One \"generation interval\", \"delay\", \"right-truncation group optional parameter subset query parameter particular two-letter state abbrevation. Right now, parameter state-specific estimates right_truncation. report_date optional parameter subset query parameter particular report_date. Right now, parameter report date-specific estimates right_truncation. Note similar , different as_of_date. report_date used select particular value time-varying estimate. estimate may regenerated time (e.g., new data becomes available methodological update). can pull estimate date report_date generated date as_of_date.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read parameter PMF into memory — read_interval_pmf","text":"PMF vector","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read parameter PMF into memory — read_interval_pmf","text":"start_date end_date specify date range value used. end_date may NULL (e.g. current value used production). value must contain pmf vector whose values positive sum 1. fields must consistent specifications function arguments describe , used query .parquet file. SCD2 format shorthand slowly changing dimension type 2. format normalized track change time: https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":null,"dir":"Reference","previous_headings":"","what":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"Extracts raw posterior samples Stan fit object post-processes , including merging fact table standardizing parameter names. calling process_quantiles() 50% 95% intervals returned {tidybayes} format.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"","code":"process_samples(fit, geo_value, model, disease) process_quantiles(fit, geo_value, model, disease, quantiles)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"fit EpiNow2 fit object posterior estimates. disease, geo_value, model Metadata downstream processing. quantiles vector quantiles base tidybayes::median_qi()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"data.table posterior draws quantiles, merged processed.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":null,"dir":"Reference","previous_headings":"","what":"Write model outputs to specified directories — write_model_outputs","title":"Write model outputs to specified directories — write_model_outputs","text":"Processes model fit, extracts samples quantiles, writes appropriate directories.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Write model outputs to specified directories — write_model_outputs","text":"","code":"write_model_outputs( fit, samples, summaries, output_dir, job_id, task_id, metadata = list() )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Write model outputs to specified directories — write_model_outputs","text":"fit EpiNow2 fit object posterior estimates. samples data.table returned process_samples() summaries data.table returned process_quantiles() output_dir String. base output directory path. job_id String. identifier job. task_id String. identifier task. metadata List. Additional metadata included output.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Write model outputs to specified directories — write_model_outputs","text":"Invisible NULL. function called side effects.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":null,"dir":"Reference","previous_headings":"","what":"Create output directory structure for a given job and task. — write_output_dir_structure","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"function generates necessary directory structure storing output files related job tasks, including directories raw samples summarized quantiles.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"","code":"write_output_dir_structure(output_dir, job_id, task_id)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"output_dir String. base output directory path. job_id String. identifier job. task_id String. identifier task.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"path base output directory (invisible).","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"purpose","dir":"","previous_headings":"","what":"Purpose","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"rules behavior establish privacy information security requirements use Third Party Web Applications (TPWAs) conjunction CDC GitHub.com organizations established open source projects. rules behavior developed ensure CDC confidential information technologies compromised, well protecting general CDC interests services risks associated use TPWAs allowing increased efficiencies cost savings come appropriate use third party services.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"scope","dir":"","previous_headings":"","what":"Scope","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"rules behavior related guidance apply federal employees, contractors, external collaborators access GitHub CDC directly use non-sensitive data obtained CDC. engagement TPWAs related GitHub governed rules behavior, well Rules Behavior Use HHS Information Services.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"ownership","dir":"","previous_headings":"","what":"Ownership","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"CDC assigns three stewards charge rules policy compliance: Business Steward, Security Steward, Technical Steward. business security stewards responsible establishing policy providing approval, technical steward fulfills requests users. Users requesting access GitHub approved yet need assign main backup point contact (POC) business steward, well provide justification security steward. security steward responsible security GitHub usage TPWA impact CDC network compliance CDC security policies. users, including POCs, responsible adherence policy associated processes. rule behavior provides explicit guidance, users must best safeguard CDC network services security risks.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"rules-of-behavior","dir":"","previous_headings":"","what":"Rules of Behavior","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"new users GitHub must read acknowledge rules using approved TPWAs. acknowledgment must completed annually, establishes agreement part user adhere rules. understand must complete security awareness records management training annually order comply latest security records management policies. understand must also follow Rules Behavior use HHS Information Resources. understand must use, share, store kind sensitive data (health status, provision payment healthcare, pictures, PII, etc.) TPWAs circumstance. knowingly conceal, falsify remove information.includes editing removing template language provided Github repository created. understand can use non-sensitive /publicly available data GitHub. unsure constitutes non-sensitive information, please see guidance . understand passwords create set GitHub accounts need comply CDC’s password policy. understand steward reserves right moderate data time. understand responsibilities protect systems data specified CDC policies.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"guidance-regarding-non-sensitive-and-publicly-available-information","dir":"","previous_headings":"","what":"Guidance Regarding Non-Sensitive and Publicly Available Information","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"support program collaboration use GitHub, portions GitHub projects either currently open public may become open public future. following guidelines inform assist user determining information posted GitHub sensitive. bottom line content posting appropriate post public access, posted GitHub. posting information involves CDC programs, employees, etc. GitHub, important poster ensures receive approval relevant CDC entity post information. Questions consider posting information include: reservations anyone viewing information? | Yes | post. | individuals informed information posted GitHub? | | post. | information contain details descriptions CDC security systems sensitive infrastructures? | Yes | post. | information reflect program efforts engage inform external partners public? | | post. | Examples information deemed sensitive may posted GitHub include following. Source Code Use cases User stories/requirements Process flows Program pain points Software Service Descriptions Sensitive information, posted, includes (limited ) following. Information directly attributed individual sensitive manner names pictures individuals Protected health information Project management material. includes posting discussing security documentation, implementation plans, communications regarding project specifics, etc. Opinions related programs tools, specifically may adverse impact Non-public Links CDC SharePoint internal references Non-public Details CDC internal infrastructure ’s question whether information may sensitive (detailed interview notes specific references provided program interview), guidance sought security steward prior posting information GitHub.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"enforcement","dir":"","previous_headings":"","what":"Enforcement","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"Users looking use GitHub unable follow rules behavior authorization . users violate rules behavior CDC security policies may subject action, including revoking access GitHub. Technical security stewards right enforce rules behavior based violations time.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"references","dir":"","previous_headings":"","what":"References","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"Policy Managing Use Third-Party Websites Applications Rules Behavior Use HHS Information Resources Security Awareness Training (requires login)","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/thanks.html","id":null,"dir":"","previous_headings":"","what":"Thanks and Acknowledgements","title":"Thanks and Acknowledgements","text":"Starting file way late, wanted recognize contributions made people helped repo. many , started file years ago. Chris Sandlin @cssandlin Drewry Morris @drewry","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/news/index.html","id":"cfaepinow2pipeline-development-version","dir":"Changelog","previous_headings":"","what":"CFAEpiNow2Pipeline (development version)","title":"CFAEpiNow2Pipeline (development version)","text":"Checks batch pool exists. Pools named branches. Also allows deletion via commit message. Merges workflows 1 2 single workflow. Now uses CFA Azure ACR images workflows Dockerfiles, etc. Added Docker image requirements build package. Bump pre-commit hooks Fix bug warning message incomplete data read (h/t @damonbayer) Fit EpiNow2 model using params fixed seed Removed .vscode folder repo Read apply exclusions case data Data reader processor Parameters read local parquet file files Additional CI bugs squashed Bug fixed updated, faster pre-commit checks Updated, faster pre-commit checks Azure Blob file download utilities CI running Ubuntu & working pkgdown deploy Github Pages Initial R package checks running CI Updated DESCRIPTION added guidelines package authorship Set README explanation purpose scope Removed add.R placeholder Fix bugs date casting caused DuckDB v1.1.1 release Drop unused pre-commit hooks Write outputs file Specify number samples draws iter_sampling Fix NOTE missing variable name used NSE Read new parameters schema Fix bugs parameter reading local test run Fix bugs parameter reading local test run","code":""}] +[{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":null,"dir":"","previous_headings":"","what":"Welcome!","title":"Welcome!","text":"Thank contributing CDC’s Open Source projects! questions doubts, don’t afraid send way. appreciate contributions, looking forward fostering open, transparent, collaborative environment.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"package-authorship","dir":"","previous_headings":"","what":"Package authorship","title":"Welcome!","text":"guidelines build recommendations R packages. Maintainer: CFA team member actively charge maintaining package. Authors: CFA team member taken lead role code development, review, testing, etc. Contributors: Team members made smaller substantial contributions codebase (40+ hours, including coding, code review, testing, etc.) Smaller contributions: Smaller contributiosn code repo predecessor, cdcent/cfa-nnh-pipelines, including small bug fixes, issuees, code review granted package authorship description file, may acknowledged NEWS.md release notes appropriate contributing, encourage also read LICENSE, README, code--conduct files, also found repository. inquiries questions answered content repository, feel free contact us.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"public-domain","dir":"","previous_headings":"","what":"Public Domain","title":"Welcome!","text":"project public domain within United States, copyright related rights work worldwide waived CC0 1.0 Universal public domain dedication. contributions project released CC0 dedication. submitting pull request agreeing comply waiver copyright interest.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"requesting-changes","dir":"","previous_headings":"","what":"Requesting Changes","title":"Welcome!","text":"pull request/merging process designed give CDC Surveillance Team space opportunity consider discuss suggested changes. policy affects CDC spaces, -line , users expected abide .","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"open-an-issue-in-the-repository","dir":"","previous_headings":"Requesting Changes","what":"Open an issue in the repository","title":"Welcome!","text":"don’t specific language submit like suggest change something addressed, can open issue repository. Team members respond issue soon possible.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/CONTRIBUTING.html","id":"submit-a-pull-request","dir":"","previous_headings":"Requesting Changes","what":"Submit a pull request","title":"Welcome!","text":"like contribute, please submit pull request. order us merge pull request, must: * least seven days old. Pull requests may held longer necessary give people opportunity assess . * Receive +1 majority team members associated request. significant dissent team, meeting held discuss plan action pull request.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/DISCLAIMER.html","id":null,"dir":"","previous_headings":"","what":"DISCLAIMER","title":"DISCLAIMER","text":"Use service limited non-sensitive publicly available data. Users must use, share, store kind sensitive data like health status, provision payment healthcare, Personally Identifiable Information (PII) /Protected Health Information (PHI), etc. circumstance. Administrators service reserve right moderate information used, shared, stored service time. user abide disclaimer Code Conduct may subject action, including revoking access services. material embodied software provided “-” without warranty kind, express, implied otherwise, including without limitation, warranty fitness particular purpose. event shall Centers Disease Control Prevention (CDC) United States (U.S.) government liable anyone else direct, special, incidental, indirect consequential damages kind, damages whatsoever, including without limitation, loss profit, loss use, savings revenue, claims third parties, whether CDC U.S. government advised possibility loss, however caused theory liability, arising connection possession, use performance software.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"Apache License","title":"Apache License","text":"Version 2.0, January 2004 ","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_1-definitions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"1. Definitions","title":"Apache License","text":"“License” shall mean terms conditions use, reproduction, distribution defined Sections 1 9 document. “Licensor” shall mean copyright owner entity authorized copyright owner granting License. “Legal Entity” shall mean union acting entity entities control, controlled , common control entity. purposes definition, “control” means () power, direct indirect, cause direction management entity, whether contract otherwise, (ii) ownership fifty percent (50%) outstanding shares, (iii) beneficial ownership entity. “” (“”) shall mean individual Legal Entity exercising permissions granted License. “Source” form shall mean preferred form making modifications, including limited software source code, documentation source, configuration files. “Object” form shall mean form resulting mechanical transformation translation Source form, including limited compiled object code, generated documentation, conversions media types. “Work” shall mean work authorship, whether Source Object form, made available License, indicated copyright notice included attached work (example provided Appendix ). “Derivative Works” shall mean work, whether Source Object form, based (derived ) Work editorial revisions, annotations, elaborations, modifications represent, whole, original work authorship. purposes License, Derivative Works shall include works remain separable , merely link (bind name) interfaces , Work Derivative Works thereof. “Contribution” shall mean work authorship, including original version Work modifications additions Work Derivative Works thereof, intentionally submitted Licensor inclusion Work copyright owner individual Legal Entity authorized submit behalf copyright owner. purposes definition, “submitted” means form electronic, verbal, written communication sent Licensor representatives, including limited communication electronic mailing lists, source code control systems, issue tracking systems managed , behalf , Licensor purpose discussing improving Work, excluding communication conspicuously marked otherwise designated writing copyright owner “Contribution.” “Contributor” shall mean Licensor individual Legal Entity behalf Contribution received Licensor subsequently incorporated within Work.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_2-grant-of-copyright-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"2. Grant of Copyright License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable copyright license reproduce, prepare Derivative Works , publicly display, publicly perform, sublicense, distribute Work Derivative Works Source Object form.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_3-grant-of-patent-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"3. Grant of Patent License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable (except stated section) patent license make, made, use, offer sell, sell, import, otherwise transfer Work, license applies patent claims licensable Contributor necessarily infringed Contribution(s) alone combination Contribution(s) Work Contribution(s) submitted. institute patent litigation entity (including cross-claim counterclaim lawsuit) alleging Work Contribution incorporated within Work constitutes direct contributory patent infringement, patent licenses granted License Work shall terminate date litigation filed.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_4-redistribution","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"4. Redistribution","title":"Apache License","text":"may reproduce distribute copies Work Derivative Works thereof medium, without modifications, Source Object form, provided meet following conditions: () must give recipients Work Derivative Works copy License; (b) must cause modified files carry prominent notices stating changed files; (c) must retain, Source form Derivative Works distribute, copyright, patent, trademark, attribution notices Source form Work, excluding notices pertain part Derivative Works; (d) Work includes “NOTICE” text file part distribution, Derivative Works distribute must include readable copy attribution notices contained within NOTICE file, excluding notices pertain part Derivative Works, least one following places: within NOTICE text file distributed part Derivative Works; within Source form documentation, provided along Derivative Works; , within display generated Derivative Works, wherever third-party notices normally appear. contents NOTICE file informational purposes modify License. may add attribution notices within Derivative Works distribute, alongside addendum NOTICE text Work, provided additional attribution notices construed modifying License. may add copyright statement modifications may provide additional different license terms conditions use, reproduction, distribution modifications, Derivative Works whole, provided use, reproduction, distribution Work otherwise complies conditions stated License.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_5-submission-of-contributions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"5. Submission of Contributions","title":"Apache License","text":"Unless explicitly state otherwise, Contribution intentionally submitted inclusion Work Licensor shall terms conditions License, without additional terms conditions. Notwithstanding , nothing herein shall supersede modify terms separate license agreement may executed Licensor regarding Contributions.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_6-trademarks","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"6. Trademarks","title":"Apache License","text":"License grant permission use trade names, trademarks, service marks, product names Licensor, except required reasonable customary use describing origin Work reproducing content NOTICE file.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_7-disclaimer-of-warranty","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"7. Disclaimer of Warranty","title":"Apache License","text":"Unless required applicable law agreed writing, Licensor provides Work (Contributor provides Contributions) “” BASIS, WITHOUT WARRANTIES CONDITIONS KIND, either express implied, including, without limitation, warranties conditions TITLE, NON-INFRINGEMENT, MERCHANTABILITY, FITNESS PARTICULAR PURPOSE. solely responsible determining appropriateness using redistributing Work assume risks associated exercise permissions License.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_8-limitation-of-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"8. Limitation of Liability","title":"Apache License","text":"event legal theory, whether tort (including negligence), contract, otherwise, unless required applicable law (deliberate grossly negligent acts) agreed writing, shall Contributor liable damages, including direct, indirect, special, incidental, consequential damages character arising result License use inability use Work (including limited damages loss goodwill, work stoppage, computer failure malfunction, commercial damages losses), even Contributor advised possibility damages.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"id_9-accepting-warranty-or-additional-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"9. Accepting Warranty or Additional Liability","title":"Apache License","text":"redistributing Work Derivative Works thereof, may choose offer, charge fee , acceptance support, warranty, indemnity, liability obligations /rights consistent License. However, accepting obligations, may act behalf sole responsibility, behalf Contributor, agree indemnify, defend, hold Contributor harmless liability incurred , claims asserted , Contributor reason accepting warranty additional liability. END TERMS CONDITIONS","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/LICENSE.html","id":"appendix-how-to-apply-the-apache-license-to-your-work","dir":"","previous_headings":"","what":"APPENDIX: How to apply the Apache License to your work","title":"Apache License","text":"apply Apache License work, attach following boilerplate notice, fields enclosed brackets [] replaced identifying information. (Don’t include brackets!) text enclosed appropriate comment syntax file format. also recommend file class name description purpose included “printed page” copyright notice easier identification within third-party archives.","code":"Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Zachary Susswein. Author, maintainer. Katelyn Gostic. Contributor. Nathan McIntosh. Contributor. Patrick Corbett. Contributor. Adam Howes. Contributor.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Susswein Z (2024). CFAEpiNow2Pipeline: EpiNow2 wrapper deployment Azure Batch. R package version 0.0.0.9000, https://cdcgov.github.io/cfa-epinow2-pipeline/.","code":"@Manual{, title = {CFAEpiNow2Pipeline: EpiNow2 wrapper for deployment to Azure Batch}, author = {Zachary Susswein}, year = {2024}, note = {R package version 0.0.0.9000}, url = {https://cdcgov.github.io/cfa-epinow2-pipeline/}, }"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":null,"dir":"","previous_headings":"","what":"Creating a Culture of Innovation","title":"Creating a Culture of Innovation","text":"aspire create culture people work joyfully, communicate openly things matter, provide great services globally. like team communities (government private sector) reflect diversity kinds, just classes protected law. Diversity fosters innovation. Diverse teams creative teams. need diversity perspective create solutions challenges face. code conduct (adapted 18F’s Code Conduct). follow Equal Employment Opportunity laws expect everyone work adhere GSA Anti-harassment Policy, even work Centers Disease Control Prevention GSA. expect every user follow code conduct laws policies mentioned .","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"be-empowering","dir":"","previous_headings":"","what":"Be Empowering","title":"Creating a Culture of Innovation","text":"Consider can encourage support others. Make room quieter voices contribute. Offer support enthusiasm great ideas. Leverage low cost experimentation support colleagues’ ideas, take care acknowledge original source. Look ways contribute collaborate, even situations normally wouldn’t. Share knowledge skills. Prioritize access input traditionally excluded civic process.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"rules-of-behavior","dir":"","previous_headings":"","what":"Rules of Behavior","title":"Creating a Culture of Innovation","text":"understand must complete security awareness records management training annually order comply latest security records management policies. understand must also follow Rules Behavior use HHS Information Resources understand must use, share, store kind sensitive data (health status, provision payment healthcare, PII, etc.) circumstance. knowingly conceal, falsify, remove information. understand can use non-sensitive /publicly available data. understand passwords create set accounts need comply CDC’s password policy. understand stewards reserves right moderate data time.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"boundaries","dir":"","previous_headings":"","what":"Boundaries","title":"Creating a Culture of Innovation","text":"Create boundaries behavior consider can create safe space helps prevent unacceptable behavior others. can’t list instances unacceptable behavior, can provide examples help guide community thinking respond experience types behavior, whether directed others. unsure something appropriate behavior, probably . person interact can define line . Impact matters intent. Ensuring behavior negative impact responsibility. Problems usually arise assume way thinking behavior norm everyone.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"here-are-some-examples-of-unacceptable-behavior","dir":"","previous_headings":"Boundaries","what":"Here are some examples of unacceptable behavior","title":"Creating a Culture of Innovation","text":"Negative offensive remarks based protected classes listed GSA Anti-harassment Policy race, religion, color, sex, national origin, age, disability, genetric information, sexual orientation, gender identity, parental status, maritual status, political affiliation well gender expression, mental illness, socioeconomic status backgrounds, neuro()typicality, physical appearance, body size, clothing. Consider calling attention differences can feel alienating. Sustained disruption meetings, talks, discussions, including chatrooms. Patronizing language behavior. Aggressive behavior, unconstructive criticism, providing correction improve conversation (sometimes referred “well actually’s”), repeatedly interrupting talking someone else, feigning surprise someone’s lack knowledge awareness topic, subtle prejudice. Referring people way misidentifies gender /rejects validity gender identity; instance using incorrect pronouns forms address (misgendering). Retaliating anyone files formal complaint someone violated codes laws.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"background","dir":"","previous_headings":"","what":"Background","title":"Creating a Culture of Innovation","text":"CDC Scientific Clearance process obtaining approvals appropriate CDC officials CDC information product released public CDC’s external public health partners. Information products require formal clearance include print, electronic, oral materials, CDC employees author co-author, whether published CDC outside CDC. CDC contractors developing content behalf CDC public CDC’s external public health partners also required put content formal clearance process. collaborative functions related projects include blogs, wikis, forums, bug tracking sites, source control others deemed necessary. individuals within CDC, adherence following policies required: * CDC “Clearance Information Products Disseminated Outside CDC Public Use” * HHS “Ensuring Quality Information Disseminated HHS agencies” collaborative materials controlled rules contained within document. allow real-time collaboration opportunities among CDC employees, CDC contractors CDC public health partners.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"credit","dir":"","previous_headings":"","what":"Credit","title":"Creating a Culture of Innovation","text":"code conduct mainly adapted 18F’s Code Conduct CDC’s Informatics Innovation Unit R&D Lab’s code conduct.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/code-of-conduct.html","id":"relevant-legal-considerations","dir":"","previous_headings":"","what":"Relevant Legal Considerations","title":"Creating a Culture of Innovation","text":"Laws enforced Equal Employment Opportunity Commission Types discrimination prohibited law New proposed regulations","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"overview","dir":"","previous_headings":"","what":"Overview","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"lightweight wrapper around {EpiNow2} add functionality deployment Azure Batch. holds helper functions interface Azure services, convert input data EpiNow2’s expected input format, save expected outputs. also adds metadata logging. package meant enhance EpiNow2 package support deployment CFA’s computational environment. code open source part CFA’s goals around development, may possible support extensions additional environments.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"structure","dir":"","previous_headings":"","what":"Structure","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository holds R package, CFAEpiNow2Pipeline. repository structured standard R package. PRs pass R CMD check part CI suite pre-condition merge main. interested contributing see CONTRIBUTING.md open issue PR. package contains contains adapters wrappers run run many independent EpiNow2 models parallel cloud resources. adapters read datasets standardized formats produces outputs flat files standard names. wrapper functions enhance EpiNow2 functionality support cloud deployments, adding logging standardizing R environment. package standardizes interface EpiNow2 purposes deployment pipeline part suite models. package manage pipeline deployment kickoff, data extraction transformation, model output visualization.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"components","dir":"","previous_headings":"","what":"Components","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"package implements functions : config validated runtime, config generation specified pipeline runtime part package. Paths specified via config Parameters: Loads pre-specified -validated generation interval, delay interval, right-truncation distributions CFA data lake local environment formats use EpiNow2. Model run: Manages R environment run EpiNow2 fixed random seed, EpiNow2 initialization Stan sampling. Outputs: Provides functionality process EpiNow2 model fits standardised flat output format (described forthcoming link). Within pipeline, model fits saved entirety .rds files, well via flat output format. Logging: Steps pipeline comprehensive R-style logging, cli package Metadata: Extract comprehensive metadata model run store alongside outputs","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"output-format","dir":"","previous_headings":"","what":"Output format","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"end goals package standardize raw outputs EpiNow2 samples summaries tables, write standardized outputs, along relevant metadata, logs, etc. standard directory structure. CFA’s standard format, outputs can passed separate pipeline handles post-processing (e.g. plotting, scoring, analysis) Rt estimates several different Rt estimation models.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"directories","dir":"","previous_headings":"Output format","what":"Directories","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"nested partitioning structure outputs designed facilitate automated processes manual investigation: files organized job task IDs, allowing efficient file operations using glob patterns, also maintaining clear hierarchy aids human users navigating specific results logs. Files meant primarily machine-readable consumption (.e., draws, summaries, diagnostics) structured together make globbing easier. Files meant primarily human investigation (.e., logs, model fit object) grouped together task facilitate manual workflows. workflow, task IDs correspond location specific model runs (independent one another) jobid refers unique model run disease. example, production job contain task IDs 50 states US, job submitted testing experimentation might contain smaller number tasks/locations. /: base output directory. , example, / Docker container dedicated output directory. - job_/: directory named specific job identifier, containing outputs related job. tasks within job share top-level directory. - raw_samples/: subdirectory within job folder holds raw sample files tasks job. Task-specific draws output files live together directory enable easy globbing task-partitioned outputs. - samples_.parquet: file containing raw samples model, associated particular task identifier. file columns job_id, task_id, geo_value, disease, model, _draw, _chain, _iteration, _variable, value, reference_date. variables follow {tidybayes} specification. - summarized_quantiles/: subdirectory storing summarized quantile data. Task-specific summarized output files live together directory enable easy globbing task-partitioned outputs. - summarized_.parquet: file summarized quantiles relevant specific task identifier. file columns job_id, task_id, geo_value, disease, model, value, _lower, _upper, _width, _point, _interval, reference_date. variables follow {tidybayes} specification. - diagnostics/: subdirectory storing model fit diagnostics. Task-specific diagnostic output files live together directory enable easy globbing task-partitioned outputs. - diagnostic_.parquet: file diagnostics relevant specific task identifier. file columns diagnostic, value, job_id, task_id, geo_value, disease, model. - tasks/: directory contains subdirectories task within job. files less likely require globbing data lake manual investigation, stored togehter. - task_/: task folder identified task ID, includes several files: - model.rds: RDS file storing EpiNow2 model object fit data. - metadata.json: JSON file containing additional metadata model run task. - stdout.log: log file capturing standard output model run process. - stderr.log: log file capturing standard error output model run process. - job_metadata.json: JSON file located root job’s directory, providing metadata entire job.","code":"/ ├── job_/ │ ├── raw_samples/ │ │ ├── samples_.parquet │ ├── summarized_quantiles/ │ │ ├── summarized_.parquet │ ├── diagnostics/ │ │ ├── diagnostics_.parquet │ ├── tasks/ │ │ ├── task_/ │ │ │ ├── model.rds │ │ │ ├── metadata.json │ │ │ ├── stdout.log │ │ │ └── stderr.log │ ├── job_metadata.json"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"model-estimated-quantities","dir":"","previous_headings":"Output format","what":"Model-estimated quantities","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"EpiNow2 estimates incident cases ŷtd\\hat y_{td} timepoint t∈{1,...,T}t \\\\{1, ..., T\\} delay d∈{1,...,D}d \\\\{1, ..., D\\} D≤TD \\le T. single vintage ’re providing EpiNow2, delay dd moves inversely timepoints, d=T−t+1d = T - t + 1. observed data vector length TT ytd∈Wy_{td} \\W. supply nowcasting correction PMF ν\\nu last DD timepoints νd∈[0,1],\\nu_d \\[0, 1], ∑d=1Dνd=1\\sum_{d=1}^D\\nu_d = 1. also priors Θ\\Theta. use EpiNow2’s generative model f(y,ν,Θ)f(y, \\nu, \\Theta). EpiNow2 forward model produces expected nowcasted case count tt dd pair: γ̂td\\hat \\gamma_{td}. applies nowcasting correction ν\\nu last DD timepoints γ̂\\hat \\gamma produce expected right-truncated case count ŷ\\hat y. Note expected case counts (without right-truncation) don’t observation noise included. can apply negative binomial observation noise using EpiNow2’s estimate negative binomial overdispersion parameter ϕ̂\\hat \\phi expected case counts. posterior predictive distributions nowcasted case counts γ̃∼NB(γ̂,ϕ̂)\\tilde \\gamma \\sim \\text{NB}(\\hat \\gamma, \\hat \\phi). posterior predicted right-truncated case count ỹ∼NB(ŷ,ϕ̂)\\tilde y \\sim \\text{NB}(\\hat y, \\hat \\phi). can get 3 4 quantities pre-generated returned EpiNow2 Stan model: γ̂\\hat \\gamma: expected nowcasted case count reports[t] ŷ\\hat y: expected right-truncated case count obs_reports[t] γ̃\\tilde \\gamma: posterior-predicted nowcasted case count imputed_reports[t] ỹ\\tilde y: posterior-predicted right-truncated case count isn’t returned EpiNow2. also save RtR_t estimate time tt intrinsic growth rate time tt.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"automation","dir":"","previous_headings":"","what":"Automation","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"project multiple GitHub Actions workflows automate CI/CD process. Notably, containers--az-pool.yaml workflow executes jobs using self-hosted runner, serves entry point starting pipeline. workflow following three jobs: Build dependencies image (build-dependencies-image): Creates container image dependencies required build R package. job cached speed process, updates image Dockerfile-dependencies DESCRIPTION file changes. image pushed Azure container registry: cfaprdbatchcr.azurecr.io/cfa-epinow2-pipeline-dependencies:[branch name]. Build pipeline image (build-pipeline-image): Using previous image base, job installs R package pushes image Azure container registry: cfaprdbatchcr.azurecr.io/cfa-epinow2-pipeline:[branch name]. Create Batch Pool Submit Jobs (batch-pool): final job creates new Azure batch pool id cfa-epinow2-pool-[branch name] doesn’t already exist. Additionally, commit message contains string “[delete pool]”, pool deleted. container tags pool ids based branch name, making compatible multiple pipelines running simultaneously. [!IMPORTANT] CI fail branch names valid tag names containers. information, see official Azure documentation .","code":"flowchart LR START((Start))---DEPS_CACHED DEPS_CACHED{Deps
      cached?}---|No|DEPS DEPS_CACHED---|Yes|IMG subgraph DEPS[Build dependencies image] direction TB Dockerfile-dependencies---|Generates|DEPS_IMAGE[Dependencies
      Image] end DEPS---IMG subgraph IMG[Build pipeline image] direction TB Dockerfile---|Generates|PKG_IMG[Package
      Image] end IMG---POOL subgraph POOL[Create Batch Pool and Submit Jobs] direction TB POOL_EXISTS{Is the pool
      up?} POOL_EXISTS---|No|CREATE_POOL[Create the pool] POOL_EXISTS---|Yes|SHOULD_DELETE_POOL{\"`Does the commit message
      include the phrase
      '_[delete pool]_'?`\"} SHOULD_DELETE_POOL---|Yes|DELETE_POOL[Delete the pool] SHOULD_DELETE_POOL---|No|END_POOL DELETE_POOL---END_POOL((End)) CREATE_POOL---END_POOL end"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"project-admin","dir":"","previous_headings":"","what":"Project Admin","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"@zsusswein @natemcintosh @kgostic","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"general-disclaimer","dir":"","previous_headings":"","what":"General Disclaimer","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository created use CDC programs collaborate public health related projects support CDC mission. GitHub hosted CDC, third party website used CDC partners share information collaborate software. CDC use GitHub imply endorsement one particular service, product, enterprise.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"public-domain-standard-notice","dir":"","previous_headings":"","what":"Public Domain Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository constitutes work United States Government subject domestic copyright protection 17 USC § 105. repository public domain within United States, copyright related rights work worldwide waived CC0 1.0 Universal public domain dedication. contributions repository released CC0 dedication. submitting pull request agreeing comply waiver copyright interest.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"license-standard-notice","dir":"","previous_headings":"","what":"License Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository utilizes code licensed terms Apache Software License therefore licensed ASL v2 later. source code repository free: can redistribute /modify terms Apache Software License version 2, (option) later version. source code repository distributed hope useful, WITHOUT WARRANTY; without even implied warranty MERCHANTABILITY FITNESS PARTICULAR PURPOSE. See Apache Software License details. received copy Apache Software License along program. , see http://www.apache.org/licenses/LICENSE-2.0.html source code forked open source projects inherit license.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"privacy-standard-notice","dir":"","previous_headings":"","what":"Privacy Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository contains non-sensitive, publicly available data information. material community participation covered Disclaimer Code Conduct. information CDC’s privacy policy, please visit http://www.cdc.gov//privacy.html.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"contributing-standard-notice","dir":"","previous_headings":"","what":"Contributing Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"Anyone encouraged contribute repository forking submitting pull request. (new GitHub, might start basic tutorial.) contributing project, grant world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license users terms Apache Software License v2 later. comments, messages, pull requests, submissions received CDC including GitHub page may subject applicable federal law, including limited Federal Records Act, may archived. Learn http://www.cdc.gov//privacy.html.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"records-management-standard-notice","dir":"","previous_headings":"","what":"Records Management Standard Notice","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"repository source government records copy increase collaboration collaborative potential. government records published CDC web site.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/index.html","id":"additional-standard-notices","dir":"","previous_headings":"","what":"Additional Standard Notices","title":"EpiNow2 wrapper for deployment to Azure Batch","text":"Please refer CDC’s Template Repository information contributing repository, public domain notices disclaimers, code conduct.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":null,"dir":"","previous_headings":"","what":"CDC GitHub Practices for Open Source Projects","title":"CDC GitHub Practices for Open Source Projects","text":"CDCGov organization GitHub designated use CDC programs publish open source code. set practices help programs release secure compliant open source projects successfully. interested using GitHub non-open source projects, please see information enterprise organization. designed practices straightforward helpful, accept feedback community updating . Required Practices, Projects don’t adhere Required Practices subject archival removal.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"getting-started","dir":"","previous_headings":"","what":"Getting Started","title":"CDC GitHub Practices for Open Source Projects","text":"can publish project, must request access added CDCgov organization. Complete steps: Review Rules Behavior. Confirm Github profile setup properly. require CDC login, don’t login, ask someone request behalf, get touch. receive email notification given access first repository setup . subsequent projects, able create repository organization using Github’s interface. template repository maintained easy way quick start repository complies guidelines. completed ’re ready follow required guidelines publish code.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"required-practices","dir":"","previous_headings":"","what":"Required Practices","title":"CDC GitHub Practices for Open Source Projects","text":"must follow practices publish real code repository. GitHub third party service used CDC collaborate public. Official CDC health messages always distributed www.cdc.gov appropriate channels, make sure plan project along official public health program cdc.gov. Add topics improve discovery use project. AI-related projects, Code.gov Implementation Guidance Federal Agencies Regarding Enterprise Data Source Code Inventories must followed setting topics. overview project, including purpose, goals team responsible. description development process README.md file. project longer active, mark archived. Public Domain Standard Notice License Standard Notice Privacy Standard Notice Contributing Standard Notice Records Management Standard Notice Additional Standard Notices need help choosing license, please review article, refer existing CDCgov projects, ask consultation support choosing license. final step publishing critical. unsure compliance, reach organization’s security officers. Make sure commit history Github repository also doesn’t things. many cases ’s easier start new repository push code sensitive information removed first commit. Enable GitHub automated security alerts configure notification repo admin see. Two-factor authentication (2FA). Project admins must secure account two-factor-authentication. Respond critical security issues communication administrators. Ignoring security issues responding communication administrators can result archiving removal. Archive old projects. ’re longer updating project moved ’s location, update README.md file let users know archive repository.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"recommended-practices","dir":"","previous_headings":"","what":"Recommended Practices","title":"CDC GitHub Practices for Open Source Projects","text":"Optional improvements make open source project successful. Establish pull request templates make easier contributors send pull requests. example SDP-V checklist PR match development practices. Agree project conventions include README.md file. Depending type project, includes folder structure data, linters, editor configuration (eg, MicrobeTrace’s .editorconfig). help improve quality project make easier others contribute project. Add support community procedures. CDC provide warranty official support open source projects, describing like questions issues assist users project. use wiki, project board, package manager, describe link . Official contribution steps make easier people outside CDC contribute project. Include references publications, presentations, sites featuring project. Add entry open.cdc.gov data, code, api, event page help people find project cdc.gov Add versions tags describing major releases milestones. example, open.cdc.gov’s releases time new version published web site geneflow’s changelog. Follow Semantic Versioning 2.0.0 creating versions project. Describe test reproducible practices install build project. example, injury_autocoding’s code section running project’s scripts). Recognize contributors existing resources helped project. example, fdns-ms-hl7-utils’ AUTHORS file. Automate build test procedures reduce effort outside contributors send pull requests (eg, Travis CI, Circle CI, GitHub Actions) Appropriately gather metrics project used incorporate feature planning process. Incorporate documentation development cycle, possible, automate generation documentation likely date useful people interested project.","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"support-and-feedback","dir":"","previous_headings":"Guidance","what":"Support and Feedback","title":"CDC GitHub Practices for Open Source Projects","text":"need additional support setting project, feedback ideas guidance please open issue send email data@cdc.gov. also accept pull requests want directly edit guidance.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"non-compliance-procedure","dir":"","previous_headings":"Guidance","what":"Non-Compliance Procedure","title":"CDC GitHub Practices for Open Source Projects","text":"Projects organization reviewed occasionally compliance Required Practices. project found compliance, contacted administrators help bring project compliance. Projects respond habitually fail meet practices archived removed organization, depending severity.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"profile-setup","dir":"","previous_headings":"Guidance","what":"Profile Setup","title":"CDC GitHub Practices for Open Source Projects","text":"Please make sure profile set properly help us work better together. Specifically, keep profile date : Name: first last name. Company: government agency contracting company. (also use GitHub personal projects, consider specifying “CDC (work) + personal projects” make clear GitHub projects may personal nature.) Location: primary work location (city, state). Photo: headshot photo, appropriate image unique . admin projects, make sure secure account two-factor authentication (2FA). Although probably already smart.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"open-source-checklist","dir":"","previous_headings":"Guidance","what":"Open Source Checklist","title":"CDC GitHub Practices for Open Source Projects","text":"’ve decided set open source project CDC. steps , common order. Create new project using template repo. Update readme.md following CDC GitHub Practices Open Source Projects Choose license. projects ASL2, license meet public health program need. See https://www.philab.cdc.gov/index.php/2012/03/27/open-source-development--public-health-informatics/ info choosing license. Remove sensitive info. Talk ADI, ADS, ISSO review clearance. approval, create GitHub user. Fill Request Repo form new repo CDCGov CDCai. get email push alert repo ready, push GitHub Add entry open.cdc.gov code page officially linked cdc.gov. helps users find use project. Keep project date, ’re finished flag archived. checklist adapted CDC Guard Rail put help people don’t access intranet.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"cdc-enterprise","dir":"","previous_headings":"Guidance","what":"CDC Enterprise","title":"CDC GitHub Practices for Open Source Projects","text":"CDCent organization used private, non-public projects CDC staff approved outside collaborators work projects, can request access GitHub Enterprise Cloud form.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/open_practices.html","id":"reference-links","dir":"","previous_headings":"Guidance","what":"Reference Links","title":"CDC GitHub Practices for Open Source Projects","text":"helpful links across Federal Government regarding open sourcing code. CFPB Open Tech TTS Engineering Practices Guide 18F Open Source Policy Practicing open source policy GitHub Government: agencies build software code.gov Federal Source Code Open Source Toolkit Federal Source Code Policy (M-16-21) openCDC Digital Services Playbook CDC/ATSDR Policy Releasing Sharing Data (old version, still useful reference) Clearance Information Products Disseminated Outside CDC Public Use Federal Source Code Toolkit","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert case counts in matching rows to NA — apply_exclusions","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"Mark selected points ignored model fitting. manual selection occurs externally pipeline passed pipeline exclusions file read read_exclusions(). Mechanically, exclusions applied converting specified points NAs dataset. NAs skipped model fitting EpiNow2, matched rows excluded model likelihood.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"","code":"apply_exclusions(cases, exclusions)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"cases dataframe returned read_data() exclusions dataframe returned read_exclusions()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/apply_exclusions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert case counts in matching rows to NA — apply_exclusions","text":"dataframe rows schema cases value column confirm converted NA rows match row exclusions","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":null,"dir":"Reference","previous_headings":"","what":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"Note think might wise instead specify blob prefix, list blobs, download listed blobs. let us flexibility downloading whole remote directories (like delta tables)","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"","code":"download_from_azure_blob(blob_names, local_dest, container_name)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"blob_names vector blobs donwload container_name local_dest path local directory save files container_name Azure Blob Storage container blob_names","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/download_from_azure_blob.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Download specified blobs from Blob Storage and save them in a local dir — download_from_azure_blob","text":"NULL success","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":null,"dir":"Reference","previous_headings":"","what":"Extract diagnostic metrics from model fit and data — extract_diagnostics","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"function extracts various diagnostic metrics fitted EpiNow2 model provided data. checks low case counts computes diagnostics fitted model, including mean acceptance statistic, divergent transitions, maximum tree depth, Rhat values. diagnostics flagged exceed specific thresholds, results returned data frame.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"","code":"extract_diagnostics(fit, data, job_id, task_id, disease, geo_value, model)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"fit model fit object EpiNow2 data data frame containing input data used model fit. job_id unique identifier job task_id unique identifier task disease, geo_value, model Metadata downstream processing.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"data.frame containing extracted diagnostic metrics. data frame includes following columns: diagnostic: name diagnostic metric. value: value diagnostic metric. job_id: unique identifier job. task_id: unique identifier task. disease,geo_value,model: Metadata downstream processing.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/extract_diagnostics.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Extract diagnostic metrics from model fit and data — extract_diagnostics","text":"following diagnostics calculated: mean_accept_stat: average acceptance statistic across chains. p_divergent: proportion divergent transitions across samples. n_divergent: number divergent transitions across samples. p_max_treedepth: proportion samples hit maximum tree depth. p_high_rhat: proportion parameters Rhat values greater 1.05, indicating potential convergence issues. n_high_rhat: number parameters Rhat values greater 1.05, indicating potential convergence issues. low_case_count_flag: flag indicating low case counts data. See low_case_count_diagnostic() information diagnostic. epinow2_diagnostic_flag: combined flag indicates diagnostic thresholds exceeded. diagnostic thresholds (1) mean_accept_stat < 0.1, (2) p_divergent > 0.0075, (3) p_max_treedepth > 0.05, (4) p_high_rhat > 0.0075.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":null,"dir":"Reference","previous_headings":"","what":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"impure function depends environment variables: az_tenant_id az_subscription_id az_resource_group az_storage_account","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"","code":"fetch_blob_container(container_name)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"container_name Azure Blob Storage container associated credentials","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"Blob endpoint","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_blob_container.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Load Azure Blob endpoint using credentials in environment variables — fetch_blob_container","text":"error set.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":null,"dir":"Reference","previous_headings":"","what":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"throw informative error credential found","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"","code":"fetch_credential_from_env_var(env_var)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"env_var character, credential fetch","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fetch_credential_from_env_var.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Fetch Azure credential from environment variable — fetch_credential_from_env_var","text":"associated value","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":null,"dir":"Reference","previous_headings":"","what":"Fit an EpiNow2 model — fit_model","title":"Fit an EpiNow2 model — fit_model","text":"Fit EpiNow2 model","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Fit an EpiNow2 model — fit_model","text":"","code":"fit_model(data, parameters, seed, horizon, priors, sampler_opts)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Fit an EpiNow2 model — fit_model","text":"data, format returned read_data() parameters returned read_disease_parameters() seed random seed, used initialization EpiNow2 R sampling Stan horizon number days, integer, forecast priors list lists. first level contain key rt elements mean sd key gp element alpha_sd. sampler_opts list. Stan sampler options passed EpiNow2. required keys: cores, chains, iter_warmup, iter_sampling, max_treedepth, adapt_delta.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/fit_model.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Fit an EpiNow2 model — fit_model","text":"fitted model object class epinow , model fitting fails, NA returned warning","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":null,"dir":"Reference","previous_headings":"","what":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"Gostic et al., 2020 simulates data stochastic SEIR model. Residence time E compartments exponentially distributed, mean 4 days (rate/inverse-scale 1/4). residence times imply gamma-distributed generation time distribution shape 2 rate 1/4. convert continuous gamma distribution PMF use {RtGam}.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"","code":"gostic_gt_pmf"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"gostic_gt_pmf numeric vector length 26 sums one within numerical tolerance","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_gt_pmf.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Generation interval corresponding to the sample gostic_toy_rt dataset — gostic_gt_pmf","text":"parametric specification, produce double-censored, left-truncated probability mass function generation interval distribution. produce PMF using {epinowcast}'s simulate_double_censored_pmf() version 0.3.0. See https://doi.org/10.1101/2024.01.12.24301247 information double-censoring biases corrections. correct output simulate_double_censored_pmf() make appropriate use {EpiNow2}. function returns numeric vector, position element corresponding one day length delay value corresponding amount discretized probability density bin. vector necessarily sum one. drop first element vector, corresponds zero-day delay. renewal framework, underpins model account zero-day delays. renormalize left-truncated vector sum one proper PMF.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":null,"dir":"Reference","previous_headings":"","what":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"dataset Gostic, Katelyn M., et al. \"Practical considerations measuring effective reproductive number, Rt.\" PLoS Computational Biology 16.12 (2020): e1008409. data simulated stochastic SEIR compartmental model.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"","code":"gostic_toy_rt"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"gostic_toy_rt data frame 301 rows 12 columns: time Timestep discrete-time stochastic SEIR simulation date Added original Gostic, 2020 dataset. date corresponding assigned time. Arbitrarily starts January 1st, 2023. S, E, , R realized state stochastic SEIR system dS, dEI, DIR stochastic transition compartments incidence true incidence compartment time t obs_cases observed number cases time t forward-convolved incidence. obs_incidence Added original Gostic, 2020 dataset. incidence column added negative-binomial observation noise. Created set.seed(123456) call rnbinom(299, mu = gostic_toy_rt[[\"incidence\"]], size = 10) Useful testing. true_r0 initial R0 system (.e., 2) true_rt known, true Rt epidemic system","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"source","dir":"Reference","previous_headings":"","what":"Source","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"https://github.com/cobeylab/Rt_estimation/tree/d9d8977ba8492ac1a3b8287d2f470b313bfb9f1d # nolint","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/gostic_toy_rt.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Synthetic dataset of stochastic SIR system with known Rt — gostic_toy_rt","text":"synthetic dataset number desirable properties: force infection changes depending Rt, allowing sudden changes Rt. allows modeling sudden changes infection dynamics, might otherwise difficult capture. Rt estimation framework realized Rt known timepoint dataset incorporates simple generation interval reporting delay. Gostic et al. benchmark performance number Rt estimation frameworks, providing practical guidance use dataset evaluate Rt estimates. practice, found amount observation noise incidence /observed cases often undesirably low testing. Many empirical datasets much noisier. result, models built settings mind can perform poorly dataset fail converge. original dataset, add new column original incidence counts additional observation noise: obs_incidence. manually add observation noise rnbinom(299, mu = gostic_toy_rt[[\"obs_cases\"]], size = 10) random seed 123456 store obs_incidence column.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":null,"dir":"Reference","previous_headings":"","what":"Calculate low case count diagnostic flag — low_case_count_diagnostic","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"diagnostic flag TRUE either last two weeks dataset fewer aggregate 10 cases per week. aggregation excludes count confirmed outliers, set NA data.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"","code":"low_case_count_diagnostic(df)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"df dataframe returned read_data(). dataframe must include columns reference_date (date vector) confirm (number confirmed cases per day).","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"logical value (TRUE FALSE) indicating whether either last two weeks dataset fewer 10 cases per week.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/low_case_count_diagnostic.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Calculate low case count diagnostic flag — low_case_count_diagnostic","text":"function assumes df input dataset \"completed\": implicit missingness made explicit.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":null,"dir":"Reference","previous_headings":"","what":"Format PMFs for EpiNow2 — opts_formatter","title":"Format PMFs for EpiNow2 — opts_formatter","text":"Opinionated wrappers around EpiNow2::generation_time_opts(), EpiNow2::delay_opts(), EpiNow2::dist_spec() formats generation interval, delay, right truncation parameters object ready input EpiNow2.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Format PMFs for EpiNow2 — opts_formatter","text":"","code":"format_generation_interval(pmf) format_delay_interval(pmf) format_right_truncation(pmf, data)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Format PMFs for EpiNow2 — opts_formatter","text":"pmf returned read_disease_parameters(). PMF vector NA, applying PMF model fit. data format returned read_data()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Format PMFs for EpiNow2 — opts_formatter","text":"EpiNow2::*_opts() formatted object NA message","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/opts_formatter.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Format PMFs for EpiNow2 — opts_formatter","text":"Delays right truncation optional can skipped passing pmf = NA.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":null,"dir":"Reference","previous_headings":"","what":"Read in the dataset of incident case counts — read_data","title":"Read in the dataset of incident case counts — read_data","text":"row table corresponds single facilities' cases reference-date/report-date/disease tuple. want aggregate counts level geographic aggregate/report-date/reference-date/disease.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read in the dataset of incident case counts — read_data","text":"","code":"read_data( data_path, disease = c(\"COVID-19\", \"Influenza\", \"test\"), state_abb, report_date, max_reference_date, min_reference_date )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read in the dataset of incident case counts — read_data","text":"data_path path local file. contain glob must parquet format. disease One \"COVID-19\" \"Influenza\" state_abb two-letter uppercase abbreviation report_date desired single report date max_reference_date, min_reference_date first last reference dates, inclusive, timeseries","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read in the dataset of incident case counts — read_data","text":"dataframe one rows columns report_date, reference_date, state_abb, confirm","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_data.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read in the dataset of incident case counts — read_data","text":"handle two distinct cases geographic aggregates: single state: Subset facilities state aggregate state level 2. US overall: Aggregate facilities without subsetting Note apply exclusions . exclusions applied later, aggregations. means US overall, aggregate points might potentially excluded state level. recourse case exclude US overall aggregate point.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":null,"dir":"Reference","previous_headings":"","what":"Read in disease process parameters from an external file or files — read_disease_parameters","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"Read disease process parameters external file files","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"","code":"read_disease_parameters( generation_interval_path, delay_interval_path, right_truncation_path, disease, as_of_date, group, report_date )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"generation_interval_path, delay_interval_path, right_truncation_path Path local file parameter PMF. See read_interval_pmf details file schema. parameters can file different file. disease One COVID-19 Influenza as_of_date Use parameters used production date. Set current date --date version parameters set earlier date use parameters earlier time period. group optional parameter subset query parameter particular two-letter state abbrevation. Right now, parameter state-specific estimates right_truncation. report_date optional parameter subset query parameter particular report_date. Right now, parameter report date-specific estimates right_truncation. Note similar , different as_of_date. report_date used select particular value time-varying estimate. estimate may regenerated time (e.g., new data becomes available methodological update). can pull estimate date report_date generated date as_of_date.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"named list three PMFs. list elements named generation_interval, delay_interval, right_truncation. path local file provided (NA NULL), corresponding parameter estimate NA returned list.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_disease_parameters.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read in disease process parameters from an external file or files — read_disease_parameters","text":"generation_interval_path required generation interval required parameter $R_t$ estimation. delay_interval_path right_truncation_path optional (strongly suggested).","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":null,"dir":"Reference","previous_headings":"","what":"Read exclusions from an external file — read_exclusions","title":"Read exclusions from an external file — read_exclusions","text":"Expects read CSV required columns: reference_date report_date state_abb disease","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read exclusions from an external file — read_exclusions","text":"","code":"read_exclusions(path)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read exclusions from an external file — read_exclusions","text":"path path exclusions file .csv format","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read exclusions from an external file — read_exclusions","text":"dataframe columns reference_date, report_date, state_abb, disease","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_exclusions.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read exclusions from an external file — read_exclusions","text":"columns meaning read_data(). Additional columns allowed ignored reader.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":null,"dir":"Reference","previous_headings":"","what":"Read parameter PMF into memory — read_interval_pmf","title":"Read parameter PMF into memory — read_interval_pmf","text":"Using DuckDB parquet file. function expects file SCD2 format column names: parameter geo_value disease start_date end_date value","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read parameter PMF into memory — read_interval_pmf","text":"","code":"read_interval_pmf( path, disease = c(\"COVID-19\", \"Influenza\", \"test\"), as_of_date, parameter = c(\"generation_interval\", \"delay\", \"right_truncation\"), group = NA, report_date = NA )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read parameter PMF into memory — read_interval_pmf","text":"path path local file disease One COVID-19 Influenza as_of_date Use parameters used production date. Set current date --date version parameters set earlier date use parameters earlier time period. parameter One \"generation interval\", \"delay\", \"right-truncation group optional parameter subset query parameter particular two-letter state abbrevation. Right now, parameter state-specific estimates right_truncation. report_date optional parameter subset query parameter particular report_date. Right now, parameter report date-specific estimates right_truncation. Note similar , different as_of_date. report_date used select particular value time-varying estimate. estimate may regenerated time (e.g., new data becomes available methodological update). can pull estimate date report_date generated date as_of_date.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read parameter PMF into memory — read_interval_pmf","text":"PMF vector","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/read_interval_pmf.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read parameter PMF into memory — read_interval_pmf","text":"start_date end_date specify date range value used. end_date may NULL (e.g. current value used production). value must contain pmf vector whose values positive sum 1. fields must consistent specifications function arguments describe , used query .parquet file. SCD2 format shorthand slowly changing dimension type 2. format normalized track change time: https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":null,"dir":"Reference","previous_headings":"","what":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"Extracts raw posterior samples Stan fit object post-processes , including merging fact table standardizing parameter names. calling process_quantiles() 50% 95% intervals returned {tidybayes} format.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"","code":"process_samples(fit, geo_value, model, disease) process_quantiles(fit, geo_value, model, disease, quantiles)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"fit EpiNow2 fit object posterior estimates. disease, geo_value, model Metadata downstream processing. quantiles vector quantiles base tidybayes::median_qi()","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/sample_processing_functions.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Process posterior samples from a Stan fit object (raw draws). — sample_processing_functions","text":"data.table posterior draws quantiles, merged processed.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":null,"dir":"Reference","previous_headings":"","what":"Write model outputs to specified directories — write_model_outputs","title":"Write model outputs to specified directories — write_model_outputs","text":"Processes model fit, extracts samples quantiles, writes appropriate directories.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Write model outputs to specified directories — write_model_outputs","text":"","code":"write_model_outputs( fit, samples, summaries, output_dir, job_id, task_id, metadata = list() )"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Write model outputs to specified directories — write_model_outputs","text":"fit EpiNow2 fit object posterior estimates. samples data.table returned process_samples() summaries data.table returned process_quantiles() output_dir String. base output directory path. job_id String. identifier job. task_id String. identifier task. metadata List. Additional metadata included output.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_model_outputs.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Write model outputs to specified directories — write_model_outputs","text":"Invisible NULL. function called side effects.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":null,"dir":"Reference","previous_headings":"","what":"Create output directory structure for a given job and task. — write_output_dir_structure","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"function generates necessary directory structure storing output files related job tasks, including directories raw samples summarized quantiles.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"","code":"write_output_dir_structure(output_dir, job_id, task_id)"},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"output_dir String. base output directory path. job_id String. identifier job. task_id String. identifier task.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/reference/write_output_dir_structure.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create output directory structure for a given job and task. — write_output_dir_structure","text":"path base output directory (invisible).","code":""},{"path":[]},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"purpose","dir":"","previous_headings":"","what":"Purpose","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"rules behavior establish privacy information security requirements use Third Party Web Applications (TPWAs) conjunction CDC GitHub.com organizations established open source projects. rules behavior developed ensure CDC confidential information technologies compromised, well protecting general CDC interests services risks associated use TPWAs allowing increased efficiencies cost savings come appropriate use third party services.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"scope","dir":"","previous_headings":"","what":"Scope","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"rules behavior related guidance apply federal employees, contractors, external collaborators access GitHub CDC directly use non-sensitive data obtained CDC. engagement TPWAs related GitHub governed rules behavior, well Rules Behavior Use HHS Information Services.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"ownership","dir":"","previous_headings":"","what":"Ownership","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"CDC assigns three stewards charge rules policy compliance: Business Steward, Security Steward, Technical Steward. business security stewards responsible establishing policy providing approval, technical steward fulfills requests users. Users requesting access GitHub approved yet need assign main backup point contact (POC) business steward, well provide justification security steward. security steward responsible security GitHub usage TPWA impact CDC network compliance CDC security policies. users, including POCs, responsible adherence policy associated processes. rule behavior provides explicit guidance, users must best safeguard CDC network services security risks.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"rules-of-behavior","dir":"","previous_headings":"","what":"Rules of Behavior","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"new users GitHub must read acknowledge rules using approved TPWAs. acknowledgment must completed annually, establishes agreement part user adhere rules. understand must complete security awareness records management training annually order comply latest security records management policies. understand must also follow Rules Behavior use HHS Information Resources. understand must use, share, store kind sensitive data (health status, provision payment healthcare, pictures, PII, etc.) TPWAs circumstance. knowingly conceal, falsify remove information.includes editing removing template language provided Github repository created. understand can use non-sensitive /publicly available data GitHub. unsure constitutes non-sensitive information, please see guidance . understand passwords create set GitHub accounts need comply CDC’s password policy. understand steward reserves right moderate data time. understand responsibilities protect systems data specified CDC policies.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"guidance-regarding-non-sensitive-and-publicly-available-information","dir":"","previous_headings":"","what":"Guidance Regarding Non-Sensitive and Publicly Available Information","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"support program collaboration use GitHub, portions GitHub projects either currently open public may become open public future. following guidelines inform assist user determining information posted GitHub sensitive. bottom line content posting appropriate post public access, posted GitHub. posting information involves CDC programs, employees, etc. GitHub, important poster ensures receive approval relevant CDC entity post information. Questions consider posting information include: reservations anyone viewing information? | Yes | post. | individuals informed information posted GitHub? | | post. | information contain details descriptions CDC security systems sensitive infrastructures? | Yes | post. | information reflect program efforts engage inform external partners public? | | post. | Examples information deemed sensitive may posted GitHub include following. Source Code Use cases User stories/requirements Process flows Program pain points Software Service Descriptions Sensitive information, posted, includes (limited ) following. Information directly attributed individual sensitive manner names pictures individuals Protected health information Project management material. includes posting discussing security documentation, implementation plans, communications regarding project specifics, etc. Opinions related programs tools, specifically may adverse impact Non-public Links CDC SharePoint internal references Non-public Details CDC internal infrastructure ’s question whether information may sensitive (detailed interview notes specific references provided program interview), guidance sought security steward prior posting information GitHub.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"enforcement","dir":"","previous_headings":"","what":"Enforcement","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"Users looking use GitHub unable follow rules behavior authorization . users violate rules behavior CDC security policies may subject action, including revoking access GitHub. Technical security stewards right enforce rules behavior based violations time.","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/rules_of_behavior.html","id":"references","dir":"","previous_headings":"","what":"References","title":"Rules of Behavior and Posting Guidelines for the Use of GitHub as a Third-Party Web Application","text":"Policy Managing Use Third-Party Websites Applications Rules Behavior Use HHS Information Resources Security Awareness Training (requires login)","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/thanks.html","id":null,"dir":"","previous_headings":"","what":"Thanks and Acknowledgements","title":"Thanks and Acknowledgements","text":"Starting file way late, wanted recognize contributions made people helped repo. many , started file years ago. Chris Sandlin @cssandlin Drewry Morris @drewry","code":""},{"path":"https://cdcgov.github.io/cfa-epinow2-pipeline/news/index.html","id":"cfaepinow2pipeline-development-version","dir":"Changelog","previous_headings":"","what":"CFAEpiNow2Pipeline (development version)","title":"CFAEpiNow2Pipeline (development version)","text":"Re-organizing GitHub workflows. Checks batch pool exists. Pools named branches. Also allows deletion via commit message. Merges workflows 1 2 single workflow. Now uses CFA Azure ACR images workflows Dockerfiles, etc. Added Docker image requirements build package. Bump pre-commit hooks Fix bug warning message incomplete data read (h/t @damonbayer) Fit EpiNow2 model using params fixed seed Removed .vscode folder repo Read apply exclusions case data Data reader processor Parameters read local parquet file files Additional CI bugs squashed Bug fixed updated, faster pre-commit checks Updated, faster pre-commit checks Azure Blob file download utilities CI running Ubuntu & working pkgdown deploy Github Pages Initial R package checks running CI Updated DESCRIPTION added guidelines package authorship Set README explanation purpose scope Removed add.R placeholder Fix bugs date casting caused DuckDB v1.1.1 release Drop unused pre-commit hooks Write outputs file Specify number samples draws iter_sampling Fix NOTE missing variable name used NSE Read new parameters schema Fix bugs parameter reading local test run Fix bugs parameter reading local test run","code":""}]