Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend rewrite for v1.0 #106

Merged
merged 88 commits into from
Mar 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
8e7a118
WIP
jayqi Feb 24, 2024
e20f216
Working?
jayqi Feb 26, 2024
b6b27a2
WIP
jayqi Feb 26, 2024
980c210
Progress
jayqi Mar 4, 2024
eb127ff
Renaming typing; don't use entrypoints; don't use Registry class
jayqi Mar 5, 2024
a6d507d
Working test against static assets
jayqi Mar 6, 2024
d838f33
Typechecking
jayqi Mar 6, 2024
03e11af
graphviz options
jayqi Mar 9, 2024
39043c1
Classifiers
jayqi Mar 9, 2024
47d57a3
Typechecking
jayqi Mar 9, 2024
73ad605
Handle forward references
jayqi Mar 9, 2024
b3f96a8
Remove unnecessary forward ref stuff
jayqi Mar 9, 2024
25ec0a1
Forward ref exception and logging
jayqi Mar 13, 2024
6c8cf6a
Rearrange plugins
jayqi Mar 13, 2024
108f8a9
Test plugins
jayqi Mar 13, 2024
79eb24a
Pydantic comprehensive forward references
jayqi Mar 14, 2024
84cc94d
attrs and dataclasses forward ref WIP
jayqi Mar 14, 2024
d7bded8
Comprehensive attrs and dataclasses testing for forward refs
jayqi Mar 15, 2024
ea3c486
Clean up exceptions
jayqi Mar 16, 2024
4693082
Tests
jayqi Mar 16, 2024
c9ab353
Tests and coverage
jayqi Mar 16, 2024
3e3bd1b
Docstrings
jayqi Mar 16, 2024
f28e207
Fix lint
jayqi Mar 17, 2024
bc770d2
Update edge cardinality/modality
jayqi Mar 18, 2024
e870dd8
Add note about path
jayqi Mar 18, 2024
d62a276
Correct double-sided arrows
jayqi Mar 18, 2024
be88e58
Fix typing
jayqi Mar 18, 2024
8bddacc
Persist test outputs
jayqi Mar 18, 2024
bf09d05
Parameterize outputs name by OS and Python version
jayqi Mar 18, 2024
5ed3046
Fix tests for older Python versions
jayqi Mar 18, 2024
5d3490a
Merge branch 'main' into refactor2
jayqi Mar 19, 2024
c920335
Requirements
jayqi Mar 19, 2024
fcd67fb
Rename test assets
jayqi Mar 20, 2024
3b1006a
Add py.typed marker
jayqi Mar 20, 2024
50bc0fb
Typechecking
jayqi Mar 20, 2024
9e65986
Use PyPI sortedcontainers-pydantic
jayqi Mar 20, 2024
2503738
Test extras
jayqi Mar 20, 2024
98e7687
Print out font availability
jayqi Mar 20, 2024
5ec4dfe
Use Liberation Serif as default primary font
jayqi Mar 20, 2024
bf97db1
Fix fallback
jayqi Mar 20, 2024
a5ef331
Remove font changes, skip PNG test for CI
jayqi Mar 20, 2024
17440c5
Set fontname anyways for more robustness
jayqi Mar 20, 2024
27e50c6
Add cli test_draw to outputs
jayqi Mar 20, 2024
dbfb1a5
Docs WIP
jayqi Mar 21, 2024
a286e39
Rename graphviz stuff
jayqi Mar 24, 2024
2de3a20
Update JSON test assets
jayqi Mar 24, 2024
9b3f5c2
Generate new docs images
jayqi Mar 24, 2024
4f6407d
Rename dataclass predicate function
jayqi Mar 25, 2024
6e5ff7e
Docs
jayqi Mar 25, 2024
f49a0ff
Docs stuff
jayqi Mar 25, 2024
bddd75e
Regenerate stuff
jayqi Mar 25, 2024
26a1469
Fix lint
jayqi Mar 25, 2024
0cdf666
Print diagnostics
jayqi Mar 25, 2024
904c890
Fix empty list defaults
jayqi Mar 25, 2024
7583e90
Another list fix
jayqi Mar 25, 2024
6481667
More docs
jayqi Mar 25, 2024
baed976
Rerun notebooks
jayqi Mar 26, 2024
11eb697
Remove unneeded regex for admonitions
jayqi Mar 26, 2024
e70ed8f
Write to outputs dir
jayqi Mar 26, 2024
2b2ecbe
Use Annotated typer syntax
jayqi Mar 26, 2024
ce1a8bb
Update docstring
jayqi Mar 26, 2024
c506686
Fix lint
jayqi Mar 26, 2024
8cb4d46
Docs, typing, exceptions
jayqi Mar 26, 2024
a1fb9d1
Some typing fixes
jayqi Mar 26, 2024
adbfb4a
Raises docstrings
jayqi Mar 26, 2024
15d55c1
Working pretty print methods
jayqi Mar 27, 2024
2e4d682
Replace imghdr with filetype
jayqi Mar 27, 2024
5d8b63b
Better IPython and rich printing
jayqi Mar 27, 2024
74d4d3e
Fix typo
jayqi Mar 27, 2024
6081633
Windows still horrible. Drop for now.
jayqi Mar 27, 2024
eaac28d
Polish
jayqi Mar 27, 2024
0c4cd61
Test coverage
jayqi Mar 27, 2024
7007871
Patch notes
jayqi Mar 28, 2024
fe0c9c7
Move all DOT functionality to EntityRelationshipDiagram class
jayqi Mar 29, 2024
363cae7
Revert "Move all DOT functionality to EntityRelationshipDiagram class"
jayqi Mar 29, 2024
2fcf6ca
ModelInfo and Edge class uses type annotation
jayqi Mar 29, 2024
057ad2a
Hide cell prompts in Jupyter notebooks
jayqi Mar 29, 2024
b1faae0
Subclassing test
jayqi Mar 30, 2024
60f35fe
Clean up
jayqi Mar 30, 2024
c56f775
Update docs
jayqi Mar 30, 2024
bf749cf
Lint
jayqi Mar 30, 2024
a496000
Handle Annotated
jayqi Mar 30, 2024
4999c81
Prep for release
jayqi Mar 30, 2024
46ca23b
Add release note for manyness fix
jayqi Mar 30, 2024
0c1195e
Move README v1 note
jayqi Mar 30, 2024
c514fda
Fix linebreak
jayqi Mar 30, 2024
f628ac3
GitHub alert doesn't support titles
jayqi Mar 30, 2024
a7cf975
Shorten note
jayqi Mar 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ jobs:
PACKAGE_VERSION=$(.nox/dev/bin/python -c "import erdantic; print(erdantic.__version__)")
echo "Package version: [$PACKAGE_VERSION]"
[ ${{ github.event.release.tag_name }} == "v$PACKAGE_VERSION" ] || { exit 1; }
echo "::set-output name=major_minor_version::v${PACKAGE_VERSION%.*}"
echo "full_version=v$PACKAGE_VERSION" >> $GITHUB_ENV
echo "major_minor_version=v${PACKAGE_VERSION%.*}" >> $GITHUB_ENV

- name: Build package
run: |
Expand All @@ -57,7 +58,19 @@ jobs:
password: ${{ secrets.PYPI_PROD_PASSWORD }}
skip_existing: false

- name: Stage docs on gh-pages
- name: Stage docs on gh-pages (release candidate)
if: contains(steps.version.outputs.full_version, 'rc')
working-directory: docs
run: |
git fetch origin gh-pages --depth=1
git config user.name github-actions[bot]
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
mike deploy --push \
${{ steps.version.outputs.major_minor_version }}rc \
--title="${{ steps.version.outputs.major_minor_version }}rc"

- name: Stage docs on gh-pages (stable)
if: ! contains(steps.version.outputs.full_version, 'rc')
working-directory: docs
run: |
git fetch origin gh-pages --depth=1
Expand Down
21 changes: 17 additions & 4 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Lint
run: |
nox -s lint --verbose

- name: Typecheck
run: |
nox -s typecheck --verbose
Expand All @@ -41,7 +41,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]

steps:
Expand All @@ -62,6 +62,14 @@ jobs:
run: |
nox -s tests-${{ matrix.python-version }} --verbose

- name: Upload test outputs
uses: actions/upload-artifact@v4
if: success() || failure()
with:
name: test-outputs-${{ matrix.os }}-python-${{ matrix.python-version }}
path: tests/_outputs


- name: Upload coverage to codecov
uses: codecov/codecov-action@v3
with:
Expand All @@ -88,11 +96,16 @@ jobs:
pipx install nox
pipx install uv

- name: Build distribution and test installation
shell: bash
- name: Build distribution
run: |
nox -s build

- name: Test wheel with extras matrix
run: |
nox -s test_wheel --verbose

- name: Test sdist
run: |
nox -s test_sdist --verbose

docs:
Expand Down
10 changes: 6 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
.vscode
docs/docs/index.md
docs/docs/changelog.md
docs/docs/cli.md
docs/docs/examples/diagram.png

docs/docs/examples/
docs/notebooks/diagram.png
docs/notebooks/diagram.svg
docs/site

tests/_outputs

### Python

# Byte-compiled / optimized / DLL files
Expand Down
48 changes: 45 additions & 3 deletions HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,52 @@
# erdantic Changelog

## v0.8.0 (Unreleased)
## v1.0.0rc1 Release Candidate (2024-03-30)

- Removed support for Python 3.7. ([PR #102](https://github.com/drivendataorg/erdantic/pull/102))
- Changed rendering of type names to use the [typenames](https://github.com/jayqi/typenames) library. This should generally produce with same rendered outputs, with the following exception:
_This is a pre-release version for v1.0.0._

> [!IMPORTANT]
> This release features significant changes to erdantic, primarily to the backend process of analyzing models and representing data. If you have been primarily using the CLI or the convenience functions `create`, `draw`, and `to_dot`, then your code may continue to work without any changes. If you are doing something more advanced, you may need to update your code.

### CLI changes

- Deprecated `--termini` option. Use the new `--terminal-model` option instead. The shorthand option `-t` remains the same. The `--termini` option still works but will emit a deprecation warning.

### Convenience function changes

- Deprecated `termini` argument for `create`, `draw`, and `to_dot` functions. Use the new `terminal_models` argument instead. The `termini` argument still works but will emit a deprecation warning.
- Added `graph_attr`, `node_attr`, and `edge_attr` arguments to the `draw` and `to_dot` functions that allow you to override attributes on the generated pygraphviz object for the diagram.

### Visual changes

A few changes have been made to the visual content of rendered diagrams.

- Changed the extraction of type names to use the [typenames](https://github.com/jayqi/typenames) library. This should generally produce identical rendered outputs as before, with the following exception:
- Removed the special case behavior for rendering enum classes. Enums now just show the class name without inheritance information.
- Changed collection fields (e.g., `List[TargetModel]`) to display as a "many" relationship (crow) instead of a "zero-or-many" relationship (odot + crow), treating the modality of the field as unspecified. A field will only be displayed as "zero-or-many" (odot + crow) if it is explicitly optional, like `Optional[List[TargetModel]]`.
- Fixed incorrect representation of manyness for type annotations where the outermost annotation wasn't a collection type. ([Issue #105](https://github.com/drivendataorg/erdantic/issues/105))

### Support for attrs

- Added support for [attrs](https://www.attrs.org/en/stable/index.html) classes, i.e., classes decorated by `attrs.define`. The source code for attrs support can be found in the new module `erdantic.plugins.attrs`.
- Added new example module `erdantic.examples.attrs`.

### Backend changes

Significant changes have been made to the library backend to more strongly separate the model analysis process, the extracted data, and the diagram rendering process. We believe this more structured design facilitates customizing diagrams and simplifies the implementation for each data modeling framework. Please see the new documentation pages ["Customizing diagrams"](http://erdantic.drivendata.org/stable/customizing/) and ["Extending or modifying erdantic"](http://erdantic.drivendata.org/stable/extending/) for details on the new design.

A summary of some key changes is below:

- Removed the adapter base classes `Model` and `Field` and the conrete adapters `DataClassModel`, `DataClassField`, `PydanticModel`, and `PydanticField`.
- Added new Pydantic models `ModelInfo` and `FieldInfo` to replace the adapter system. These new models hold static data that have been extracted from models that erdantic analyzed.
- Removed the adapter system and associated objects such as `model_adapter_registry` and `register_model_adapter`.
- Added new plugin system to replace the adapter system as the way that modeling frameworks are supported. Plugins must implement two functions—a predicate function and a field extractor function—and be registered using `register_plugin`. All objects related to plugins can be found in the new `erdantic.plugins` module and its submodules.
- Renamed `erdantic.typing` module to `erdantic.typing_utils`.

### Other

- Added [PEP 561 `py.typed` marker file](https://peps.python.org/pep-0561/#packaging-type-information) to indicate that the package supports type checking.
- Added IPython special method for pretty-print string representations of `EntityRelationshipDiagram` instances.
- Removed support for Python 3.7. ([PR #102](https://github.com/drivendataorg/erdantic/pull/102))

## v0.7.0 (2024-02-11)

Expand Down
23 changes: 16 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,19 @@
[![tests](https://github.com/drivendataorg/erdantic/workflows/tests/badge.svg?branch=main)](https://github.com/drivendataorg/erdantic/actions?query=workflow%3Atests+branch%3Amain)
[![codecov](https://codecov.io/gh/drivendataorg/erdantic/branch/main/graph/badge.svg)](https://codecov.io/gh/drivendataorg/erdantic)

> [!NOTE]
> erdantic v1.0 is coming soon and involves big backend changes. See the [changelog](./HISTORY.md) for more information.

**erdantic** is a simple tool for drawing [entity relationship diagrams (ERDs)](https://en.wikipedia.org/wiki/Data_modeling#Entity%E2%80%93relationship_diagrams) for Python data model classes. Diagrams are rendered using the venerable [Graphviz](https://graphviz.org/) library. Supported data modeling frameworks are:

- [Pydantic V2](https://docs.pydantic.dev/latest/)
- [Pydantic V1 legacy](https://docs.pydantic.dev/latest/migration/#continue-using-pydantic-v1-features)
- [attrs](https://www.attrs.org/en/stable/)
- [dataclasses](https://docs.python.org/3/library/dataclasses.html) from the Python standard library

Features include a convenient CLI, automatic native rendering in Jupyter notebooks, and easy extensibility to other data modeling frameworks. Docstrings are even accessible as tooltips for SVG outputs. Great for adding a simple and clean data model reference to your documentation.
You can use erdantic either as a convenient CLI or as a Python library. Great for adding a simple and clean data model reference to your documentation.

<object type="image/svg+xml" data="https://raw.githubusercontent.com/drivendataorg/erdantic/main/docs/docs/examples/pydantic.svg" width="100%" typemustmatch><img alt="Example diagram created by erdantic" src="https://raw.githubusercontent.com/drivendataorg/erdantic/main/docs/docs/examples/pydantic.svg"></object>
<object type="image/svg+xml" data="./docs/docs/assets/example_diagram.svg" width="100%" typemustmatch><img alt="Example diagram created by erdantic" src="./docs/docs/assets/example_diagram.svg"></object>

## Installation

Expand All @@ -39,15 +43,17 @@ You can get the development version from GitHub with:
pip install git+https://github.com/drivendataorg/erdantic.git#egg=erdantic
```

## Quick Usage
## Quick usage

First, make sure that the data model classes that you want to include in your diagram are importable. This means the code with your models should either be available on your [`sys.path`](https://docs.python.org/3/library/sys_path_init.html) or installed into the same virtual environment as erdantic.

The fastest way to produce a diagram like the above example is to use the erdantic CLI. Simply specify the full dotted path to your data model class and an output path. The rendered format is interpreted from the filename extension.
The fastest way to produce a diagram like the above example is to use the erdantic CLI. Simply specify the full dotted import path to your model and an output file path. The rendered format is interpreted from the filename extension.

```bash
erdantic erdantic.examples.pydantic.Party -o diagram.png
```

You can also import the erdantic Python library and use its functions.
You can also import the erdantic Python library. This lets you inspect the diagram data and potentially modify it. You will have greater ability to customize the diagram in Python.

```python
import erdantic as erd
Expand All @@ -58,8 +64,11 @@ erd.draw(Party, out="diagram.png")

# Or create a diagram object that you can inspect and do stuff with
diagram = erd.create(Party)
diagram.models
#> [PydanticModel(Adventurer), PydanticModel(Party), PydanticModel(Quest), PydanticModel(QuestGiver)]
list(diagram.models.keys())
#> [ 'erdantic.examples.pydantic.Adventurer',
#> 'erdantic.examples.pydantic.Party',
#> 'erdantic.examples.pydantic.Quest',
#> 'erdantic.examples.pydantic.QuestGiver']
diagram.draw("diagram.png")
```

Expand Down
3 changes: 0 additions & 3 deletions docs/docs/api-reference/base.md

This file was deleted.

3 changes: 3 additions & 0 deletions docs/docs/api-reference/convenience.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.convenience

::: erdantic.convenience
3 changes: 3 additions & 0 deletions docs/docs/api-reference/core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.core

::: erdantic.core
3 changes: 0 additions & 3 deletions docs/docs/api-reference/dataclasses.md

This file was deleted.

3 changes: 0 additions & 3 deletions docs/docs/api-reference/erd.md

This file was deleted.

3 changes: 3 additions & 0 deletions docs/docs/api-reference/examples/attrs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.examples.attrs

::: erdantic.examples.attrs
7 changes: 7 additions & 0 deletions docs/docs/api-reference/examples/pydantic_v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# erdantic.examples.pydantic

::: erdantic.examples.pydantic
selection:
filters:
- "!^_"
- "!^Config$"
3 changes: 3 additions & 0 deletions docs/docs/api-reference/plugins/attrs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.plugins.attrs

::: erdantic.plugins.attrs
3 changes: 3 additions & 0 deletions docs/docs/api-reference/plugins/dataclasses.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.plugins.dataclasses

::: erdantic.plugins.dataclasses
3 changes: 3 additions & 0 deletions docs/docs/api-reference/plugins/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.plugins

::: erdantic.plugins
3 changes: 3 additions & 0 deletions docs/docs/api-reference/plugins/pydantic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.plugins.pydantic

::: erdantic.plugins.pydantic
3 changes: 0 additions & 3 deletions docs/docs/api-reference/pydantic.md

This file was deleted.

3 changes: 3 additions & 0 deletions docs/docs/api-reference/typing_utils.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# erdantic.typing_utils

::: erdantic.typing_utils
Binary file added docs/docs/assets/edge-many-unspecified.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/assets/edge-many-zero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/assets/edge-one-one.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/assets/edge-one-zero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/assets/erdantic_diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading