Preserve trigger code elements within sections (#2005)

* updated the loinc json object to include trigger code metadata; updated function and implementation

* we're separating out trigger code data but not cleanly and not exactly right yet

* updated the test function for loading json object

* adding documentation to help guide further dev work

* moved file name and added more details to notes

* fixed typo

* big refactor; if no params or sections_to_include are passed response is valid from schematron

* testing out mermaid in the container readme

* Update README.md

* finally figured out solution to trigger code and clinical services filtering; valid output created

* tests passing locally for test_refine.py

* adding step to remove any section that isn't required

* removed util functions we're no longer using; adding new clinical services dict test

* test_refiner tests passing locally

* remove _analyze_structure; as it's unused now

* move _get_entries_for_section to tests/test_refine.py; as it's used only in tests now

* adding missing docstrings

* updated mermaid chart

* remove Callable, args, kwargs

* continue to remove Callable, args, kwargs

* moved description to README, made correction in Dockerfile, and linked README to eICR-Notes

* try adding fontawesome link

* need space before fenced code div

* no inline css so fontawesome icons won't work for mermaid on github it seems

* make sure description is changed to README in main

* this should hopefully get the integration test to pass

containers/message-refiner/Dockerfile

@@ -7,7 +7,7 @@ RUN pip install -r requirements.txt
 
 COPY ./app /code/app
 COPY ./assets /code/assets
-COPY ./description.md /code/description.md
+COPY ./README.md /code/README.md
 
 EXPOSE 8080
-CMD uvicorn app.main:app --host 0.0.0.0 --port 8080
+CMD uvicorn app.main:app --host 0.0.0.0 --port 8080

containers/message-refiner/description.md → containers/message-refiner/README.md

@@ -48,3 +48,54 @@ To build the Docker image for the message refiner from source instead of downloa
 ### The API
 
 When viewing these docs from the `/redoc` endpoint on a running instance of the message refiner or the PHDI website, detailed documentation on the API will be available below.
+
+### Architecture Diagram
+
+```mermaid
+flowchart LR
+
+subgraph requests["Requests"]
+direction TB
+subgraph GET["fas:fa-download <code>GET</code>"]
+hc["<code>/</code>\n(health check)"]
+example["<code>/example-collection</code>\n(Example Requests)"]
+end
+subgraph PUT["fas:fa-upload <code>PUT</code>"]
+ecr["<code>/ecr</code>\n(refine eICR)"]
+end
+end
+
+subgraph service[REST API Service]
+direction TB
+subgraph mr["fab:fa-docker container"]
+refiner["fab:fa-python <code>message-refiner<br>HTTP:8080/</code>"]
+end
+subgraph tcr["fab:fa-docker container"]
+tcr-service["fab:fa-python <code>trigger-code-reference<br>HTTP:8081/</code>"] <==> db["fas:fa-database SQLite DB"]
+end
+mr <==> |<code>/get-value-sets</code>| tcr
+end
+
+subgraph response["Responses"]
+subgraph JSON["fa:fa-file-alt <code>JSON</code>"]
+rsp-hc["fa:fa-file-code <code>OK</code> fa:fa-thumbs-up"]
+rsp-example["fa:fa-file-code Postman Collection"]
+end
+subgraph XML["fas:fa-chevron-left fas:fa-chevron-right <code>XML</code>"]
+rsp-ecr["fas:fa-file-code Refined eICR"]
+end
+end
+
+hc -.-> mr -.-> rsp-hc
+example --> mr --> rsp-example
+ecr ===> mr ===> rsp-ecr
+
+```
+
+### Additional notes on eICR Refinement
+
+For further details on `<section>`, `<entry>`, and `<templateId>` elements, please see [eICR-Notes.md](eICR-Notes.md) for an explanation of trigger code `<templateId>`s, which sections they're in, and the `<observation>` data that should be returned in the refined eICR output.
+
+```
+
+```

containers/message-refiner/app/main.py

@@ -15,7 +15,7 @@
 from app.refine import refine
 from app.refine import validate_message
 from app.refine import validate_sections_to_include
-from app.utils import create_clinical_xpaths
+from app.utils import create_clinical_services_dict
 from app.utils import read_json_from_assets
 
 settings = get_settings()
@@ -26,7 +26,7 @@
 app = BaseService(
     service_name="Message Refiner",
     service_path="/message-refiner",
-    description_path=Path(__file__).parent.parent / "description.md",
+    description_path=Path(__file__).parent.parent / "README.md",
     include_health_check_endpoint=False,
     openapi_url="/message-refiner/openapi.json",
 ).start()
@@ -156,10 +156,9 @@ async def refine_ecr(
                 content=error_message, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY
             )
 
-    clinical_services_xpaths = None
+    clinical_services = None
     if conditions_to_include:
         responses = await get_clinical_services(conditions_to_include)
-        # confirm all API responses were 200
         if set([response.status_code for response in responses]) != {200}:
             error_message = ";".join(
                 [str(response) for response in responses if response.status_code != 200]
@@ -168,9 +167,11 @@ async def refine_ecr(
                 content=error_message, status_code=status.HTTP_502_BAD_GATEWAY
             )
         clinical_services = [response.json() for response in responses]
-        clinical_services_xpaths = create_clinical_xpaths(clinical_services)
 
-    data = refine(validated_message, sections, clinical_services_xpaths)
+        # create a simple dictionary structure for refine.py to consume
+        clinical_services = create_clinical_services_dict(clinical_services)
+
+    data = refine(validated_message, sections, clinical_services)
 
     return Response(content=data, media_type="application/xml")