Skip to content

Commit

Permalink
More on mermaid
Browse files Browse the repository at this point in the history
Signed-off-by: Mandy Chessell <[email protected]>
  • Loading branch information
mandy-chessell committed Oct 21, 2024
1 parent 1e9be0b commit fd2a10d
Show file tree
Hide file tree
Showing 8 changed files with 41 additions and 19 deletions.
14 changes: 7 additions & 7 deletions site/docs/connectors/unity-catalog/unity-catalog.drawio
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@
<mxPoint x="660" y="380" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-53" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=27;position=0.32;" parent="1" vertex="1">
<mxGeometry x="227" y="368.88" width="76.5" height="81.62" as="geometry" />
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-53" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=41;position=0.43;base=19;position2=0.69;" parent="1" vertex="1">
<mxGeometry x="227" y="359.88" width="76.5" height="90.62" as="geometry" />
</mxCell>
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-56" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=27;position=0.32;position2=0.11;" parent="1" vertex="1">
<mxGeometry x="71.25" y="365.88" width="76.5" height="74.75" as="geometry" />
Expand All @@ -145,17 +145,17 @@
<mxCell id="IWJ7ZKurHC-wRTvsE-lp-2" value="" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="197" y="368.38" width="20" height="10" as="geometry" />
</mxCell>
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-55" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=122;position=0.44;position2=0.22;" parent="1" vertex="1">
<mxGeometry x="127.75" y="350.88" width="76.5" height="185.24" as="geometry" />
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-55" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=122;position=0.44;position2=0.02;" parent="1" vertex="1">
<mxGeometry x="118.75" y="346.88" width="76.5" height="172.12" as="geometry" />
</mxCell>
<mxCell id="a5z0X3CHtMkCVGNf4Bgy-57" value="Functions" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" parent="1" vertex="1">
<mxGeometry x="124.25" y="491.62" width="80" height="30" as="geometry" />
<mxGeometry x="115.25" y="480" width="80" height="30" as="geometry" />
</mxCell>
<mxCell id="IWJ7ZKurHC-wRTvsE-lp-4" value="" style="shape=callout;whiteSpace=wrap;html=1;perimeter=calloutPerimeter;fillColor=#ffff88;strokeColor=#36393d;rotation=-180;size=122;position=0.44;position2=1;" vertex="1" parent="1">
<mxGeometry x="208.25" y="378.38" width="76.5" height="185.24" as="geometry" />
<mxGeometry x="208.25" y="378.38" width="76.5" height="173.62" as="geometry" />
</mxCell>
<mxCell id="IWJ7ZKurHC-wRTvsE-lp-5" value="Models" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="216" y="518" width="60" height="30" as="geometry" />
<mxGeometry x="216" y="510" width="60" height="30" as="geometry" />
</mxCell>
<mxCell id="IWJ7ZKurHC-wRTvsE-lp-6" value="Schema" style="rounded=0;whiteSpace=wrap;html=1;rotation=90;verticalAlign=top;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="1">
<mxGeometry x="179.75" y="157.75" width="53" height="167.5" as="geometry" />
Expand Down
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
43 changes: 33 additions & 10 deletions site/docs/user-interfaces/mermaid/overview.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
<!-- Copyright Contributors to the Egeria project. -->

# Mermaid diagrams
# Graph Visualization

[Mermaid](https://mermaid.js.org/) is a markdown extension for representing diagram using structured text that can then be rendered in a JavaScript environment. Selected [Open Metadata View Services (OMVSs)](/services/omvs) return a Mermaid markdown string that can be used to render a visual representation of the elements returned on the same request. This markdown can be rendered in a JavaScript environment that supports the Mermaid markdown, such as [JupyterLab Notebooks](#jupyterlab-notebooks) and [Draw.io](#drawio).
Egeria visualizes collections of related metadata elements as graphs using Mermaid.
[Mermaid](https://mermaid.js.org/) is a markdown extension for representing diagram using structured text that can then be rendered in a JavaScript environment. Selected [Open Metadata View Services (OMVSs)](/services/omvs) REST API calls return a Mermaid markdown string that can be used to render a visual representation of the elements returned on the same request.

## Asset graphs

Expand Down Expand Up @@ -46,32 +47,54 @@ The asset is on the far left-hand side. Each box on the graph is a metadata elem
}
```


## Lineage graphs

[Asset Catalog OMVS](/services/omvs/asset-catalog/overview) also supports a REST API called *getAssetLineageGraph(assetGUID)* which returns a graph of metadata elements that are linked via lineage relationships with the requested asset. In addition it returns a Mermaid string that can be used to visualise this structure. Below is a screen capture of a rendered lineage graph.

![Lineage graph showing data flows](lineage-graph-coco-sus.png)

The graph above shows [DataFlow](/types/7/0750-Data-Passing) relationships between systems. The next image shows a lineage graph of the data sources that make up the resources in a Unity Catalog schema. It is showing [DataContentForDataSet](/types/2/0210-Data-Stores) relationships.

![lineage graph showing data dependencies](lineage-graph-unity-default.png)

The full list of relationships supported are:

* [DataFlow](/types/7/0750-Data-Passing) - how is data flowing
* [ProcessCall](/types/7/0750-Data-Passing) - which system is calling another
* [ControlFlow](/types/7/0750-Data-Passing) - how is control passed between processes that are under the control of a workflow manager
* [LineageMapping](/types/7/0770-Lineage_Mapping) - how do elements catalogued from different lineage systems relate to one another
* [DataContentForDataSet](/types/2/0210-Data-Stores) - how are data sources combined to create a data set
* [DerivedSchemaTypeQueryTarget](/types/5/0512-Derived-Schema-Elements) - what are the queries used to create derived data fields and views

You can specify any number of these relationship type on the query. If you specify no relationships, it searches for any of these relationships.

## Governance Action Process graphs

[Governance Action Processes](/concepts/governance-action-process) describe a linked series of steps that run in an [engine host](/concepts/engine-host). Their definition is maintained in the open metadata repositories. In addition, when a governance action process runs, a [*GovernanceActionProcessInstance*](/type/4/0462-Governance-Action-Processes) element is created and linked to the [engine actions](/concepts/engine-action) that controls the execution of the process.
[Governance Action Processes](/concepts/governance-action-process) describe a linked series of steps that run in an [engine host](/concepts/engine-host). Their definition is maintained in the open metadata repositories. In addition, when a governance action process runs, a [*GovernanceActionProcessInstance*](/types/4/0462-Governance-Action-Processes) element is created and linked to the [engine actions](/concepts/engine-action) that controls the execution of the process.

The [Automated Curation OMVS](/services/omvs/automated-curation/overview) has a REST API called *getGovernanceActionProcessGraph(processGUID)*. It can either be passed the unique identifier of a governance action process element, or a governance action process instance element. The mermaid graph returned for these processes is as follows. This first image shows the sample *DailyGovernanceActionProcess* which determines the day of the week and runs a specific task for that day. Each box is a step in the process. The relationships show possible links to the next step to run. The label on the relationship shows which [guard](/concepts/guard) must be produced to cause the linked step to run. The *wait_time* is the minutes to wait before running the step and *mult_trig* indicates whether the step can be triggered multiple times in the same process instance, or not.

![DailyGovernanceActionProcess definition](daily-goverance-action-process-definition.png)
![DailyGovernanceActionProcess definition](daily-governance-action-process-definition.png)

The next image show a single execution of this process. Notice that the shape is different because each time it runs, it only outputs a single task.

![DailyGovernanceActionProcess instance](daily-governance-action-process-instance.png)

## Lineage graphs

*Work in progress*

## Rendering mermaid diagrams

Mermaid is a mature standard that has widespread support. Below are some examples of where it can be used.

### JupyterLab Notebooks
### Jupyter Notebooks

If you are using *JupyterLab Notebooks* with [pyegeria](user-interfaces/python-widgets/overview/#installation) installed, such as with the [egeria-workspaces](https://github.com/odpi/egeria-workspaces), it is possible to render Mermaid strings using the `render_mermaid(mermaid_string)` function call.
If you are using *Jupyter Notebooks* with [pyegeria](user-interfaces/python-widgets/overview/#installation) installed, such as with the [egeria-workspaces](https://github.com/odpi/egeria-workspaces), it is possible to render Mermaid strings using the `render_mermaid(mermaid_string)` function call.

### Draw.io

The *draw.io* diagramming tool used extensively in this site can render Mermaid diagrams. It is described in the [draw.io help](https://www.drawio.com/blog/mermaid-diagrams).

### Mermaid live editor

The Mermaid community provide a web page where you can paste a mermaid string, and it will render it dynamically. The live editor can be found at [https://mermaid.live/edit](https://mermaid.live/edit)

--8<-- "snippets/abbr.md"
3 changes: 1 addition & 2 deletions site/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ nav:
- User Interfaces:
- User Interface overview: user-interfaces/index.md
- Python Widgets: user-interfaces/python-widgets/overview.md
- Mermaid Diagrams: user-interfaces/mermaid/overview.md
- Graph Visualization: user-interfaces/mermaid/overview.md
- Explorers: user-interfaces/brain-explorers/overview.md
- Features:
- Features Overview: features/index.md
Expand Down Expand Up @@ -694,7 +694,6 @@ nav:
- Favorite Collections: concepts/favorite-things-collection.md
- Feedback: concepts/feedback.md
- File Type: concepts/file-type.md
- Glossary Term: practices/common-data-definitions/anatomy-of-a-glossary.md
- GUID: concepts/guid.md
- Governance Action: concepts/governance-action.md
- Governance Action Engine: concepts/governance-action-engine.md
Expand Down

0 comments on commit fd2a10d

Please sign in to comment.