Skip to content

Commit

Permalink
refactor: apply formatter
Browse files Browse the repository at this point in the history
  • Loading branch information
blacha committed Oct 11, 2023
1 parent e4a181e commit ca8c348
Show file tree
Hide file tree
Showing 36 changed files with 922 additions and 970 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,21 @@ jobs:
- name: Use Node.js 18.x
uses: actions/[email protected]
with:
node-version: "18.x"
node-version: '18.x'

- name: Install
run: npm install --ci

- name: format
run: npm run lint -- --fix=false # ensure eslint is not configured to --fix

- name: format
run: npm run format -- --fix=false # ensure eslint is not configured to --fix

- name: Setup kubectl
uses: azure/setup-kubectl@v3
with:
version: "latest"
version: 'latest'

- name: AWS Configure
uses: aws-actions/[email protected]
Expand Down
17 changes: 6 additions & 11 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,7 @@ container:
env:
- name: AWS_ROLE_CONFIG_PATH
value: s3://linz-bucket-config/config.json
args:
[
"stac-validate",
"--recursive",
"{{inputs.parameters.location}}flat/collection.json",
]
args: ['stac-validate', '--recursive', '{{inputs.parameters.location}}flat/collection.json']
```
## <a name='WorkflowPerformanceSettings'></a>Workflow Performance Settings
Expand Down Expand Up @@ -122,9 +117,9 @@ Two example workflow tasks requesting specific resources (memory and CPU) from t
```yaml
- name: standardise-validate
retryStrategy:
limit: "2"
limit: '2'
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
inputs:
parameters:
- name: file
Expand All @@ -138,13 +133,13 @@ Two example workflow tasks requesting specific resources (memory and CPU) from t
ephemeral-storage: 3Gi
volumeMounts:
- name: ephemeral
mountPath: "/tmp"
mountPath: '/tmp'
```

```yaml
- name: copy
retryStrategy:
limit: "2"
limit: '2'
inputs:
parameters:
- name: file
Expand Down Expand Up @@ -173,7 +168,7 @@ _Note: the Argo Workflows flowchart UI refers to pods as “nodes”._
In the standardising workflow, the `aws-list` task creates an output of a list of lists. Each `standardise-validate` task pod receives a list of files to process as an input.

```yaml
withParam: "{{tasks.aws-list.outputs.parameters.files}}"
withParam: '{{tasks.aws-list.outputs.parameters.files}}'
```

For example, `aws-list` output with a `group` size of 4 set:
Expand Down
8 changes: 4 additions & 4 deletions config/semaphores.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ metadata:
name: semaphores
namespace: argo
data:
standardising: "2" # Limit of how many standardising workflow instances can run at the same time
bulk: "4" # Limit of how many bulk workflow instances can run at the same time
bulkcopy: "8" # Limit of how many publish-copy workflow instances can run at the same time
basemaps_import: "10" # Limit of how many basemaps import workflow instances can run at the same time
standardising: '2' # Limit of how many standardising workflow instances can run at the same time
bulk: '4' # Limit of how many bulk workflow instances can run at the same time
bulkcopy: '8' # Limit of how many publish-copy workflow instances can run at the same time
basemaps_import: '10' # Limit of how many basemaps import workflow instances can run at the same time
2 changes: 1 addition & 1 deletion docs/concurrency.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ metadata:
name: semaphores
namespace: argo
data:
standardising: "2"
standardising: '2'
```
which shows a limit of `2` added to the key `standardising`. This can then
Expand Down
46 changes: 23 additions & 23 deletions docs/training_workshop/ARGO_WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
templates:
- name: main
Expand All @@ -129,9 +129,9 @@ spec:
template: say-hello-template
- name: say-hello-template
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["hello world"]
args: ['hello world']
```
Example workflow file: [wf_hello_world.yaml](example_workflows/wf_hello_world.yaml)
Expand All @@ -142,7 +142,7 @@ You will see the following specified in the example workflows for this workshop.
```yaml
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
```
Note: prefixing the name of the workflow with `test-` prevents alerts for the workflow being sent to the `#alert-argo-workflows` Slack channel.
Expand Down Expand Up @@ -174,7 +174,7 @@ spec:
arguments:
parameters:
- name: message
value: "hello world"
value: 'hello world'
```

The parameter can be referenced in the workflow using Argo variables as `"{{workflow.parameters.message}}"`
Expand Down Expand Up @@ -269,17 +269,17 @@ argo submit topo-workflows/imagery/standardising-publish-import.yaml -n argo -f
_params.yaml_:

```yaml
source: "s3://linz-imagery-source-example/aerial-imagery/new-zealand/christchurch_urban_2021_0.05m_RGB/"
target: "s3://linz-imagery-example/canterbury/christchurch_2021_0.05m/rgb/2193/"
scale: "500"
group: "29"
cutline: "s3://linz-imagery-cutline-example/historical-imagery-cutlines/2023-01-16_84fd68f/SNC50451-combined.fgb"
title: "Christchurch 0.05m Urban Aerial Photos (2021)"
description: "Orthophotography within the Canterbury region captured in the 2021 flying season."
producer: "Aerial Surveys"
licensor: "Toitū Te Whenua Land Information New Zealand"
start-datetime: "2021-11-02"
end-datetime: "2021-12-02"
source: 's3://linz-imagery-source-example/aerial-imagery/new-zealand/christchurch_urban_2021_0.05m_RGB/'
target: 's3://linz-imagery-example/canterbury/christchurch_2021_0.05m/rgb/2193/'
scale: '500'
group: '29'
cutline: 's3://linz-imagery-cutline-example/historical-imagery-cutlines/2023-01-16_84fd68f/SNC50451-combined.fgb'
title: 'Christchurch 0.05m Urban Aerial Photos (2021)'
description: 'Orthophotography within the Canterbury region captured in the 2021 flying season.'
producer: 'Aerial Surveys'
licensor: 'Toitū Te Whenua Land Information New Zealand'
start-datetime: '2021-11-02'
end-datetime: '2021-12-02'
```

> **_Resources:_** [Argo CLI documentation](https://argoproj.github.io/argo-workflows/cli/argo/) - [Kubernetes Cheat Sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/)
Expand Down Expand Up @@ -343,14 +343,14 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
arguments:
parameters:
- name: message1
value: "hello world 1"
value: 'hello world 1'
- name: message2
value: "hello world 2"
value: 'hello world 2'
templates:
- name: main
dag:
Expand All @@ -360,21 +360,21 @@ spec:
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message1}}"
value: '{{workflow.parameters.message1}}'
- name: say-hello-task2
template: say-hello-template
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message2}}"
value: '{{workflow.parameters.message2}}'
- name: say-hello-template
inputs:
parameters:
- name: message
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["{{inputs.parameters.message}}"]
args: ['{{inputs.parameters.message}}']
```

Example workflow file: [wf_hello_world_args_tasks.yaml](example_workflows/wf_hello_world_args_tasks.yaml)
Expand Down
6 changes: 3 additions & 3 deletions docs/training_workshop/example_workflows/wf_hello_world.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
templates:
- name: main
Expand All @@ -16,6 +16,6 @@ spec:
template: say-hello-template
- name: say-hello-template
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["hello world"]
args: ['hello world']
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
arguments:
parameters:
- name: message
value: "hello world"
value: 'hello world'
templates:
- name: main
dag:
Expand All @@ -21,12 +21,12 @@ spec:
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message}}"
value: '{{workflow.parameters.message}}'
- name: say-hello-template
inputs:
parameters:
- name: message
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["{{inputs.parameters.message}}"]
args: ['{{inputs.parameters.message}}']
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
arguments:
parameters:
- name: message1
value: "hello world 1"
value: 'hello world 1'
- name: message2
value: "hello world 2"
value: 'hello world 2'
templates:
- name: main
dag:
Expand All @@ -23,18 +23,18 @@ spec:
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message1}}"
value: '{{workflow.parameters.message1}}'
- name: say-hello-task2
template: say-hello-template
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message2}}"
value: '{{workflow.parameters.message2}}'
- name: say-hello-template
inputs:
parameters:
- name: message
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["{{inputs.parameters.message}}"]
args: ['{{inputs.parameters.message}}']
24 changes: 12 additions & 12 deletions docs/training_workshop/example_workflows/wf_hello_world_dag.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ metadata:
namespace: argo
spec:
nodeSelector:
karpenter.sh/capacity-type: "spot"
karpenter.sh/capacity-type: 'spot'
entrypoint: main
arguments:
parameters:
- name: message1
value: "hello world 1"
value: 'hello world 1'
- name: message2
value: "hello world 2"
value: 'hello world 2'
templates:
- name: main
dag:
Expand All @@ -23,33 +23,33 @@ spec:
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message1}}"
value: '{{workflow.parameters.message1}}'
- name: say-hello-task2
template: say-hello-template
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message2}}"
depends: "say-hello-task1"
value: '{{workflow.parameters.message2}}'
depends: 'say-hello-task1'
- name: say-hello-task3
template: say-hello-template
arguments:
parameters:
- name: message
value: "say hello task 3"
depends: "say-hello-task1"
value: 'say hello task 3'
depends: 'say-hello-task1'
- name: say-hello-task4
template: say-hello-template
arguments:
parameters:
- name: message
value: "say hello task 4"
depends: "say-hello-task2 && say-hello-task3"
value: 'say hello task 4'
depends: 'say-hello-task2 && say-hello-task3'
- name: say-hello-template
inputs:
parameters:
- name: message
container:
image: "019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3"
image: '019359803926.dkr.ecr.ap-southeast-2.amazonaws.com/topo-imagery:v3'
command: [echo]
args: ["{{inputs.parameters.message}}"]
args: ['{{inputs.parameters.message}}']
Loading

0 comments on commit ca8c348

Please sign in to comment.