Skip to content

Commit

Permalink
docs: cron stac validate jobs
Browse files Browse the repository at this point in the history
  • Loading branch information
paulfouquet committed May 21, 2024
1 parent 9be85a5 commit 0e75498
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 7 deletions.
17 changes: 14 additions & 3 deletions workflows/cron/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
# Contents:

- [cron STAC validate](#cron-STAC-validate)
- [cron-stac-validata](#cron-stac-validate)
- [cron-stac-validate-all-data](#cron-stac-validate-all-data)

# cron STAC validate
# STAC validation

## cron-stac-validate

Workflow that validates the STAC metadata using [`stac-validate`](https://teams.microsoft.com/v2/?meetingjoin=true#/l/meetup-join/19:meeting_MDc1MWEzNzYtYTI4Yy00OWZmLWJhMzUtYjA1ZmU1ODBmNTg5@thread.v2/0?context=%7b%22Tid%22%3a%222134e961-7e38-4c34-a22b-10da5466b725%22%2c%22Oid%22%3a%2263d2d811-1d35-49f7-b9a3-c60e9b9a9ed1%22%7d&anon=true&deeplinkId=1c8a1674-d597-4c2d-ab7e-6ed968f086b7) and verify that the [STAC links](https://github.com/radiantearth/stac-spec/blob/master/collection-spec/collection-spec.md#link-object) are valid (using their checksums).

- schedule: every day at 5am
- schedule: **every day at 5am**

## cron-stac-validate-all-data

It also validate that the data - assets - is valid (using their checksums). Verifying all data checksum is costly, so this workflow is ran less often than the [cron STAC validate](#cron-stac-validate).

> **_NOTE:_** Due to the parallelism design, this workflow does not validate the root parent `catalog.json` in order to validate each `collection.json` separately. This is not an issue as the `catalog.json` does not contain any `asset` and is already validated by the [cron-stac-validata](#cron-stac-validate) job.
- schedule: **To decide**
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
name: cron-stac-validate-checksums
name: cron-stac-validate-all-data
labels:
linz.govt.nz/category: stac
spec:
schedule: '0 07 1 * *' # 7 AM every 1st of the month
schedule: '0 05 1 * *' # 5 AM every 1st of the month
timezone: 'NZ'
startingDeadlineSeconds: 3600 # Allow 1 hour delay if the workflow-controller clashes during the starting time.
concurrencyPolicy: 'Allow'
Expand Down
4 changes: 2 additions & 2 deletions workflows/cron/cron-stac-validate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
name: test-cron-stac-validate
name: cron-stac-validate
labels:
linz.govt.nz/category: stac
spec:
schedule: 56 13 * * * # 7 AM every day
schedule: '0 05 * * *' # 5 AM every day
timezone: 'NZ'
startingDeadlineSeconds: 3600 # Allow 1 hour delay if the workflow-controller clashes during the starting time.
concurrencyPolicy: 'Allow'
Expand Down

0 comments on commit 0e75498

Please sign in to comment.