From c0040e37880a9fc7aa97f61b73fa199c37b38f7e Mon Sep 17 00:00:00 2001 From: m-goggins Date: Mon, 25 Nov 2024 11:15:21 -0800 Subject: [PATCH 1/6] update "developer documentation" --- query-connector/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/query-connector/README.md b/query-connector/README.md index 5e6af20f1..9098064a8 100644 --- a/query-connector/README.md +++ b/query-connector/README.md @@ -77,7 +77,7 @@ The Query Connector will eventually require other inputs from other DIBBs servic ### Developer Documentation -Can be found in [api-documentation.md](api-documentation.md). +A Postman collection demonstrating use of the API can be found [here](https://github.com/CDCgov/dibbs-query-connector/blob/main/query-connector/src/app/assets/DIBBs_Query_Connector_API.postman_collection.json). ### Architecture Diagram From 4cbe280a03234f87902ab73ef87affb0a4d18136 Mon Sep 17 00:00:00 2001 From: m-goggins Date: Mon, 25 Nov 2024 14:53:05 -0800 Subject: [PATCH 2/6] update README --- query-connector/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/query-connector/README.md b/query-connector/README.md index 9098064a8..a2b5a7858 100644 --- a/query-connector/README.md +++ b/query-connector/README.md @@ -79,6 +79,14 @@ The Query Connector will eventually require other inputs from other DIBBs servic A Postman collection demonstrating use of the API can be found [here](https://github.com/CDCgov/dibbs-query-connector/blob/main/query-connector/src/app/assets/DIBBs_Query_Connector_API.postman_collection.json). +### Query Connector Data for Query Building + +When initializing, the Query Connector makes the value sets associated with 200+ reportable conditions available to users tasked with building queries for their jurisdiction. To group value sets by condition and to group the conditions by type, the Query Connector obtains and organizes data from the eRSD and the VSAC in the following way: + +1. The Query Connector retrieves the 200+ reportable conditions from the eRSD as well as the value sets associated value set IDs. +2. Using the value set IDs from the eRSD, the Query Connector retrieves the value sets from the VSAC, i.e., the LOINC, SNOMED, etc. codes associated with each value set ID. +3. The Query Connector then organizes and makes the value sets organized by condition available to users who are building queries. The conditions are additionally organized by category, e.g., sexually transmitted diseases or respiratory conditions, using a mapping curated by HLN Consulting. + ### Architecture Diagram ```mermaid From e4d5b00f2bdb01c5c9b6ca1f206a1cbec2d2289a Mon Sep 17 00:00:00 2001 From: m-goggins Date: Tue, 26 Nov 2024 08:25:35 -0800 Subject: [PATCH 3/6] move VS_DUMP_INFO into main README --- query-connector/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/query-connector/README.md b/query-connector/README.md index a2b5a7858..dd709b852 100644 --- a/query-connector/README.md +++ b/query-connector/README.md @@ -87,6 +87,27 @@ When initializing, the Query Connector makes the value sets associated with 200+ 2. Using the value set IDs from the eRSD, the Query Connector retrieves the value sets from the VSAC, i.e., the LOINC, SNOMED, etc. codes associated with each value set ID. 3. The Query Connector then organizes and makes the value sets organized by condition available to users who are building queries. The conditions are additionally organized by category, e.g., sexually transmitted diseases or respiratory conditions, using a mapping curated by HLN Consulting. +#### Query Building Data in `dev` mode + +In order to make the dev process as low-lift as possible, we want to avoid executing the `db-creation` scripts when booting up the application in dev mode via `npm run dev` or `npm run dev-win`. To that end, we've created a `pg_dump` file containing all the value sets, concepts, and foreign key mappings that would be extracted from a fresh pull of the eRSD and processed through our creation functions. This file, `vs_dump.sql` has been mounted into the docker volume of our postgres DB when running in dev mode as an entrypoint script. This means it will be automatically executed when the DB is freshly spun up. You shouldn't need to do anything to facilitate this mounting or file running. + +#### Updating the pg_dump + +If the DB extract file ever needs to be updated, you can use the following simple process: + +1. Start up the application on your local machine using a regular `docker compose up`, and wait for the DB to be ready. +2. Load the eRSD and value sets into the DIBBs DB by using the `Create Query` button on the `/queryBuilding` page. Optionally, use DBeaver to verify that value sets exist in the database. +3. In a fresh terminal window, run + +``` +pg_dump -U postgres -f vs_dump.sql -h localhost -p 5432 tefca_db +``` + +If the above doesn't work, try replacing `localhost` with `0.0.0.0`. + +4. Enter the DB password when prompted. +5. The extract file, `vs_dump.sql`, should now be created. It should automatically be located in `/query-connector`, but if it isn't, put `vs_dump.sql` there. + ### Architecture Diagram ```mermaid From 66ee9b5749f42a5c54e83cb738e7bec6e1725e22 Mon Sep 17 00:00:00 2001 From: m-goggins Date: Tue, 26 Nov 2024 08:25:50 -0800 Subject: [PATCH 4/6] delete VS_DUMP_INFO --- query-connector/VS_DUMP_INFO.md | 19 ------------------- 1 file changed, 19 deletions(-) delete mode 100644 query-connector/VS_DUMP_INFO.md diff --git a/query-connector/VS_DUMP_INFO.md b/query-connector/VS_DUMP_INFO.md deleted file mode 100644 index d6c3ab06f..000000000 --- a/query-connector/VS_DUMP_INFO.md +++ /dev/null @@ -1,19 +0,0 @@ -## ValueSet SQL Dump Information - -In order to make the dev process as low-lift as possible, we want to avoid executing the `db-creation` scripts when booting up the application in dev mode via `npm run dev`. To that end, we've created a `pg_dump` file containing all the valusets, concepts, and foreign key mappings that would be extracted from a fresh pull of the eRSD and processed through our creation functions. This file, `vs_dump.sql` has been mounted into the docker volume of our postgres DB when running in dev mode as an entrypoint script. This means it will be automatically executed when the DB is freshly spun up. You shouldn't need to do anything to facilitate this mounting or file running. - -## Updating the pg_dump - -If the DB extract file ever needs to be updated, you can use the following simple process: - -1. Start up the application on your local machine using a regular `docker compose up`, and wait for the DB to be ready. -2. Load the eRSD and valuesets into the DIBBs DB by using the `Create Query` button on the `/queryBuilding` page. Optionally, use DBeaver to verify that valuesets exist in the database. -3. In a fresh terminal window, run - -``` -pg_dump -U postgres -f vs_dump.sql -h localhost -p 5432 tefca_db -``` - -If the above doesn't work, try replacing `localhost` with `0.0.0.0`. -4. Enter the DB password when prompted. -5. The extract file should now be created. It should automatically be located in `/query-connector`, but if it isn't, put it there. \ No newline at end of file From c0a6606d200546534f2e200564a980d587fac9f3 Mon Sep 17 00:00:00 2001 From: m-goggins Date: Tue, 26 Nov 2024 09:27:07 -0800 Subject: [PATCH 5/6] incoporate brandon's suggestions --- query-connector/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/query-connector/README.md b/query-connector/README.md index dd709b852..def722df1 100644 --- a/query-connector/README.md +++ b/query-connector/README.md @@ -81,11 +81,11 @@ A Postman collection demonstrating use of the API can be found [here](https://gi ### Query Connector Data for Query Building -When initializing, the Query Connector makes the value sets associated with 200+ reportable conditions available to users tasked with building queries for their jurisdiction. To group value sets by condition and to group the conditions by type, the Query Connector obtains and organizes data from the eRSD and the VSAC in the following way: +When initializing the backend database for the first time, the Query Connector makes the value sets associated with 200+ reportable conditions available to users tasked with building queries for their jurisdiction. To group value sets by condition and to group the conditions by type, the Query Connector obtains and organizes data from the eRSD and the VSAC in the following way: -1. The Query Connector retrieves the 200+ reportable conditions from the eRSD as well as the value sets associated value set IDs. +1. The Query Connector retrieves the 200+ reportable conditions from the eRSD as well as the value sets' associated IDs. 2. Using the value set IDs from the eRSD, the Query Connector retrieves the value sets from the VSAC, i.e., the LOINC, SNOMED, etc. codes associated with each value set ID. -3. The Query Connector then organizes and makes the value sets organized by condition available to users who are building queries. The conditions are additionally organized by category, e.g., sexually transmitted diseases or respiratory conditions, using a mapping curated by HLN Consulting. +3. The Query Connector then organizes these value sets according to the conditions with which they're associated, making the result available to users interested in building queries. The conditions are additionally organized by category, e.g., sexually transmitted diseases or respiratory conditions, using a mapping curated by HLN Consulting. #### Query Building Data in `dev` mode From 26e4f86fcee61d75ce2e76619ebefc7e78ae73cd Mon Sep 17 00:00:00 2001 From: m-goggins Date: Tue, 26 Nov 2024 09:27:46 -0800 Subject: [PATCH 6/6] update VSAC description --- query-connector/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/query-connector/README.md b/query-connector/README.md index def722df1..652829503 100644 --- a/query-connector/README.md +++ b/query-connector/README.md @@ -84,7 +84,7 @@ A Postman collection demonstrating use of the API can be found [here](https://gi When initializing the backend database for the first time, the Query Connector makes the value sets associated with 200+ reportable conditions available to users tasked with building queries for their jurisdiction. To group value sets by condition and to group the conditions by type, the Query Connector obtains and organizes data from the eRSD and the VSAC in the following way: 1. The Query Connector retrieves the 200+ reportable conditions from the eRSD as well as the value sets' associated IDs. -2. Using the value set IDs from the eRSD, the Query Connector retrieves the value sets from the VSAC, i.e., the LOINC, SNOMED, etc. codes associated with each value set ID. +2. Using the value set IDs from the eRSD, the Query Connector retrieves the value set's comprehensive information from the VSAC, i.e., the LOINC, SNOMED, etc. codes associated with each value set ID. 3. The Query Connector then organizes these value sets according to the conditions with which they're associated, making the result available to users interested in building queries. The conditions are additionally organized by category, e.g., sexually transmitted diseases or respiratory conditions, using a mapping curated by HLN Consulting. #### Query Building Data in `dev` mode