Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(basemaps): Update the cogify workflow to support elevation imports. BM-998 #509

Merged
merged 5 commits into from
May 30, 2024
Merged

feat(basemaps): Update the cogify workflow to support elevation imports. BM-998 #509

Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
120a3a8
feat: basemaps import dem process
blacha Apr 3, 2024
d6afcff
Update workflows/basemaps/imagery-import-cogify.yml
blacha May 23, 2024
ba7ddf7
Merge branch 'master' into feat/basemaps-dem
Wentao-Kuang May 23, 2024
f1af3f8
Update some default paramters and skip create-overview parameter
Wentao-Kuang May 23, 2024
f7f5e11
Update workflows/basemaps/imagery-import-cogify.yml
Wentao-Kuang May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions workflows/basemaps/imagery-import-cogify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,15 @@ spec:
description: Ticket ID e.g. 'AIP-55'
value: ''

- name: preset
description: Import preset configuration, WebP for 4 band RGBA LERC for 1 band DEM/DSM
value: 'webp'
enum:
- 'webp'
- 'lerc_10mm'
- 'lerc_1mm'
- 'lerc_1m'
blacha marked this conversation as resolved.
Show resolved Hide resolved
l0b0 marked this conversation as resolved.
Show resolved Hide resolved
Wentao-Kuang marked this conversation as resolved.
Show resolved Hide resolved

- name: region
description: Region of the dataset
value: 'new-zealand'
Expand Down Expand Up @@ -84,13 +93,15 @@ spec:
- 'Scanned Aerial Imagery'
- 'Satellite Imagery'
- 'Event'
- 'Elevation'

- name: target
description: Target location for output COGs
value: 's3://linz-basemaps/'
enum:
- 's3://linz-basemaps/'
- 's3://linz-basemaps-staging/'
- 's3://linz-workflowsnp-scratch/'

- name: tile_matrix
description: Output tile matrix, ";" separated list
Expand All @@ -112,13 +123,21 @@ spec:
description: How many items to pass to each create-cog job
value: '20'

- name: create_overview
description: 'Create overview after importing imagery.'
value: 'true'
enum:
- 'true'
- 'false'

templates:
# Main entrypoint into the workflow
- name: main
inputs:
parameters:
Wentao-Kuang marked this conversation as resolved.
Show resolved Hide resolved
- name: source
- name: target
- name: preset
- name: tile_matrix
- name: cutline
- name: cutline_blend
Expand All @@ -135,6 +154,8 @@ spec:
value: '{{ inputs.parameters.source }}'
- name: target
value: '{{ inputs.parameters.target }}'
- name: preset
value: '{{ inputs.parameters.preset }}'
- name: tile_matrix
value: '{{ item }}'
- name: cutline
Expand Down Expand Up @@ -162,6 +183,7 @@ spec:
- name: cutline
- name: cutline_blend
- name: group_size
- name: preset
dag:
tasks:
# generate a tile covering from the source imagery
Expand All @@ -173,6 +195,8 @@ spec:
value: '{{ inputs.parameters.source }}'
- name: target
value: '{{ inputs.parameters.target }}'
- name: preset
value: '{{ inputs.parameters.preset }}'
- name: tile_matrix
value: '{{ inputs.parameters.tile_matrix }}'
- name: cutline
Expand Down Expand Up @@ -209,9 +233,11 @@ spec:
- name: covering_grouped
from: '{{ tasks.group.outputs.artifacts.output }}'

# TODO: overviews are only supported in RGBA pipelines
# once all COGs are created generate a more overviews to increase tile rendering performance
- name: create-overview
template: create-overview
when: '{{workflow.parameters.create_overview}} == true'
depends: create-cog
arguments:
parameters:
Expand Down Expand Up @@ -247,6 +273,7 @@ spec:
- name: tile_matrix
- name: cutline
- name: cutline_blend
- name: preset
container:
image: ghcr.io/linz/basemaps/cli:{{ workflow.parameters.version_basemaps_cli }}
resources:
Expand All @@ -258,6 +285,7 @@ spec:
value: s3://linz-bucket-config/config.basemaps.json
args:
- 'cover'
- '--preset={{ inputs.parameters.preset }}'
- '--tile-matrix={{ inputs.parameters.tile_matrix }}'
- "{{= sprig.empty(inputs.parameters.cutline) ? '' : '--cutline=' + inputs.parameters.cutline }}"
- '--cutline-blend={{ inputs.parameters.cutline_blend }}'
Expand Down
Loading