Skip to content

Commit

Permalink
fix(cogify): lower concurrency and set resource limits to memory = 2x…
Browse files Browse the repository at this point in the history
… cpu (linz#176)

### Motivation

We are still getting pods killed by `OOMKilled`, possibly caused by too
many gdal_translates happening at the same time.

### Modification

Remove memory `limits` and double the requested memory
Lower concurrency of gdal_translates to 2 per pod rather than 4 per pod
  • Loading branch information
blacha authored Sep 18, 2023
1 parent 461a7a9 commit f391f68
Showing 1 changed file with 4 additions and 5 deletions.
9 changes: 4 additions & 5 deletions workflows/basemaps/imagery-import-cogify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -226,11 +226,9 @@ spec:
container:
resources:
requests:
memory: 15.6Gi
cpu: 15000m
ephemeral-storage: 100Gi
limits:
memory: 15.6Gi
memory: 30Gi
cpu: 15000m # AWS gives 2x cpu cores = memory for most instances
ephemeral-storage: 98Gi # 2 pods per 200GB of storage
image: ghcr.io/linz/basemaps/cli:{{ workflow.parameters.version_basemaps_cli }}
command: [node, /app/node_modules/.bin/cogify]
env:
Expand All @@ -239,6 +237,7 @@ spec:
args:
- "create"
- "--from-file={{= inputs.artifacts.covering_grouped.path }}{{inputs.parameters.covering_grouped_id}}.json"
- "--concurrency=2"

# Create a basemaps configuration file to view the imagery
- name: create-config
Expand Down

0 comments on commit f391f68

Please sign in to comment.