Updates pre-commit and actions with doc dry run, updated model docume…

…ntation strings, updated dependencies and CONTRIBUTING.md.

.github/workflows/check.yml

@@ -76,4 +76,8 @@ jobs:
  - name: Run mypy
  run: |
  poetry install --with mypy
- poetry run mypy .
+ poetry run mypy .
+ - name: Run sphinx
+ run: |
+ poetry install --with docs
+ poetry run sphinx-build -M dummy ./docs ./docs/_build -W -a

.pre-commit-config.yaml

@@ -52,3 +52,8 @@ repos:
  entry: poetry run mypy --no-namespace-packages --exclude esgf_playground_utils/models/__init__.py
  language: system
  types: [ file, python ]
+ - id: docs
+ name: docs
+ entry: bash -c "poetry run sphinx-build -M dummy ./docs ./docs/_build -W -q -a"
+ language: system
+ types: [ file, python ]

CONTRIBUTING.md

@@ -36,6 +36,7 @@ When you commit, the following checks will be run:
 - mypy (python static type analysis)
 - bandit (python SAST analyis)
 - xenon (McCabe cyclomatc complexity analysis)
+- sphinx (dry-run documentation build)
 
 You can disable the `pre-commit hooks` per commit with the flag `--no-verify` however all checks will be preformed in the CI.
 
@@ -49,7 +50,7 @@ foo@bar:~$ poetry run pre-commit run -a
 
 The CI Enviroment runs the same checks as the `pre-commit hooks` on push[^2] plus the following additional checks:
 
-- audit (checks all dependencies for vulnerabilities) 
+- audit (checks all dependencies for vulnerabilities)
 
 [^1]: These pre-commit hooks will attempt to fix the issue in place
 [^2]: On CI, no checks perform code changes in place

esgf_playground_utils/models/kafka.py

@@ -11,60 +11,138 @@
 
 
 class _Payload(BaseModel):
+ """
+ Base model for payloads in a Kafka message, provides the required ``collection_id`` attribute.
+
+ .. warning::
+
+ This model should not be used directly.
+
+ """
+
  collection_id: str
 
 
 class CreatePayload(_Payload):
+ """
+ Model describing a ``CREATE`` payload. This must be sent as a ``POST`` request.
+ """
+
  method: Literal["POST"]
  item: Item
 
 
 class RevokePayload(_Payload):
+ """
+ Model describing a ``REVOKE`` payload. This must be sent as a ``PATCH`` or ``DELETE`` request.
+
+ .. note::
+
+ It is intended that the ``PATCH`` request is interpreted as a "soft" delete, simply updating the item to
+ signify that it is revoked.
+
+ The ``DELETE`` request will operate as a "hard" delete strictly removing the item from the STAC index.
+
+ .. danger::
+
+ The behaviour of either of these actions is not yet fully defined.
+ """
+
  method: Literal["PATCH", "DELETE"]
  item_id: str
 
 
 class UpdatePayload(_Payload):
+ """
+ Model describing a ``UPDATE`` payload. This must be sent as a ``PATCH`` or ``PUT`` request.
+
+ .. danger::
+
+ The behaviour of the ``PATCH`` request is not yet fully defined. Specifically, it is not clear
+ if the ``PATCH`` request will require a specific ``item_id``, and as such will require an alternative
+ model.
+ """
+
  method: Literal["PUT", "PATCH"]
  item: Item
 
 
 class Data(BaseModel):
+ """
+ Model describing the ``DATA`` component of a Kafka message. This contains the payload itself.
+
+ .. note::
+
+ Whilst the ``type`` and ``version`` attributes are available, it is not expected that these will change for
+ a significant length of time.
+ """
+
  type: Literal["STAC"]
  version: Literal["1.0.0"]
  payload: Union[CreatePayload, RevokePayload, UpdatePayload]
 
 
 class Auth(BaseModel):
+ """
+ Model describing the ``AUTH`` component of a Kafka message.
+
+ .. note::
+
+ This is not an authorisation token or other verified identity. It is the simply an indication of the institute
+ providing the message.
+ """
+
  client_id: str
  server: str
 
 
 class Publisher(BaseModel):
+ """
+ Model describing the ``PUBLISHER`` component of a Kafka message. This is the name and version of the software used
+ to publish the Kafka message.
+ """
+
  package: str
  version: str
 
 
 class Metadata(BaseModel):
+ """
+ Multiple metadata attributes required for ESGF but not part of the STAC payload.
+ """
+
  auth: Auth
  publisher: Publisher
  time: datetime
  schema_version: str
 
 
 class KafkaEvent(BaseModel):
+ """
+ The full content of a Kafka message, containing both the STAC payload, the request description and the ESGF
+ mandated metadata.
+ """
+
  metadata: Metadata
  data: Data
 
 
 class ErrorType(str, Enum):
+ """
+ Enum describing the source of the error that occurred.
+ """
+
  payload = "payload"
  stac_server = "stac_server"
  kafka = "kafka"
  unknown = "unknown"
 
 
 class Error(BaseModel):
+ """
+ Error event published to the Kafka error queue.
+ """
+
  original_payload: str
  node: str
  traceback: str