Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Iceland, Faroe and Svalbard/Spitsbergen, use Sonny's LiDAR #14

Merged
merged 19 commits into from
Jul 30, 2023
Merged

Add Iceland, Faroe and Svalbard/Spitsbergen, use Sonny's LiDAR #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
cedef42
Fix Iceland, Faroe and Svalbard, use Sonny's LiDAR
jonaseberle Jul 6, 2023
3fa0622
use partly successful work with fail-fast=false
jonaseberle Jul 9, 2023
d649299
add Spitsbergen/Svalbard to eur_tile_list
jonaseberle Jul 9, 2023
286ef0b
move the often failing sha256 step into "build"
jonaseberle Jul 9, 2023
7ef9eaa
Make .zip.info depend on the .zip
jonaseberle Jul 9, 2023
23a6d2f
add "compilage" to .gitignore
jonaseberle Jul 9, 2023
f41c0ae
Use .zip.info as build target
jonaseberle Jul 9, 2023
60d0954
cleanup
jonaseberle Jul 9, 2023
b6ea982
Add GITHUB_TOKEN to all workflow steps
jonaseberle Jul 9, 2023
66c674e
Support debugging by more concise output
jonaseberle Jul 10, 2023
8e964cd
Support quick retries by deleting Ortho4XP with distclean only
jonaseberle Jul 10, 2023
277c3e0
work on output, logic
jonaseberle Jul 10, 2023
2856ccd
Add "build single .zip" workflow for quicker testing
jonaseberle Jul 10, 2023
5c2d628
remove make concurrency in runners again
jonaseberle Jul 10, 2023
3ad4769
Show elevation data when starting
jonaseberle Jul 11, 2023
192b32a
fix single tile workflow
jonaseberle Jul 11, 2023
c0b8e57
Let Makefile.tiles handle the Orto4XP setup, too
jonaseberle Jul 11, 2023
24a9f2e
Improve the output, show available elevation data
jonaseberle Jul 12, 2023
25d677a
Merge branch 'sonny-lidar' into feat/use-sonny-lidar-dtm
kubilus1 Jul 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions .github/workflows/build-tiles.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Test
name: Build one .zip (and upload) (sub-workflow)

on:
workflow_call:
Expand All @@ -22,6 +22,7 @@ jobs:
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}

runs-on: ubuntu-22.04

Expand All @@ -44,19 +45,16 @@ jobs:
TILENAME=${{ inputs.setname }} make -f Makefile.tiles ${{ inputs.setname }}_tile_list_chunks
TILENAME=${{ inputs.setname }} make -n -f Makefile.tiles ${{ inputs.zip }}

- name: Setup Ortho4XP
run: make Ortho4XP

- name: Build
id: build
run: |
TILENAME=${{ inputs.setname }} make -f Makefile.tiles ${{ inputs.setname }}_tile_list_chunks
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }} && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }}.info && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256

- name: Show work
if: steps.build.outputs.done == 'true'
run: |
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256
echo '### ${{ inputs.tileset }} : ${{ inputs.zip }}' >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
#unzip -l ${{ inputs.zip }} *.dsf >> $GITHUB_STEP_SUMMARY
Expand Down Expand Up @@ -91,6 +89,10 @@ jobs:
if: needs.build.outputs.done == 'false'
needs: build
runs-on: ubuntu-22.04
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}

outputs:
done: ${{ steps.rebuild.outputs.done }}
Expand Down Expand Up @@ -120,12 +122,12 @@ jobs:
id: rebuild
run: |
TILENAME=${{ inputs.setname }} make -f Makefile.tiles ${{ inputs.setname }}_tile_list_chunks
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }} && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }}.info && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256

- name: Show work
if: steps.rebuild.outputs.done == 'true'
run: |
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256
echo '### ${{ inputs.tileset }} : ${{ inputs.zip }}' >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
#unzip -l ${{ inputs.zip }} *.dsf >> $GITHUB_STEP_SUMMARY
Expand Down Expand Up @@ -160,6 +162,10 @@ jobs:
if: needs.rebuild.outputs.done == 'false'
needs: rerun
runs-on: ubuntu-22.04
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}

outputs:
done: ${{ steps.rebuild.outputs.done }}
Expand Down Expand Up @@ -189,12 +195,12 @@ jobs:
id: rebuild
run: |
TILENAME=${{ inputs.setname }} make -f Makefile.tiles ${{ inputs.setname }}_tile_list_chunks
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }} && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.setname }} timeout 300m make -f Makefile.tiles ${{ inputs.zip }}.info && echo "done=true" >> "$GITHUB_OUTPUT" || echo "done=false" >> "$GITHUB_OUTPUT"
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256

- name: Show work
if: steps.rebuild.outputs.done == 'true'
run: |
TILENAME=${{ inputs.tileset }} make -f Makefile.tiles ${{ inputs.zip }}.sha256
echo '### ${{ inputs.tileset }} : ${{ inputs.zip }}' >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
#unzip -l ${{ inputs.zip }} *.dsf >> $GITHUB_STEP_SUMMARY
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/push-main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Main
name: Build tileset (and upload)

on:
workflow_dispatch:
Expand All @@ -24,6 +24,10 @@ on:
jobs:
ziplist:
runs-on: ubuntu-22.04
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}
outputs:
chunks: ${{ steps.chunks.outputs.chunks }}
steps:
Expand All @@ -44,6 +48,8 @@ jobs:

strategy:
matrix: ${{ fromJSON(needs.ziplist.outputs.chunks) }}
# might still need several restarts of failed jobs, but prevents us from losing successful work
fail-fast: false

uses: ./.github/workflows/build-tiles.yml
with:
Expand All @@ -59,6 +65,10 @@ jobs:
environment: release

runs-on: ubuntu-22.04
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}
steps:
- name: Checkout
uses: actions/checkout@v3
Expand Down
41 changes: 41 additions & 0 deletions .github/workflows/run-one-tile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Build one tile (and create artifact)

on:
workflow_dispatch:
inputs:
tile:
type: string
default: '+62-008'

jobs:
build:
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}

runs-on: ubuntu-22.04

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Install reqs
run: |
sudo apt-get update
sudo apt-get install -y python3 python3-pip python3-tk zip libgeos-dev
python3 -m pip install -r requirements.txt
make --version

- name: Build
id: build
run: |
timeout 20m make -f Makefile.tiles z_aosingle_${{ inputs.tile }}.zip

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: z_aosingle_${{ inputs.tile }}.zip
path: ./z_aosingle_${{ inputs.tile }}.zip
retention-days: 2

2 changes: 1 addition & 1 deletion .github/workflows/run-one.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Run one
name: Build one .zip (and upload)

on:
workflow_dispatch:
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/test-main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
name: Test
name: Build test tileset

on:
push:

jobs:
ziplist:
runs-on: ubuntu-22.04
env:
DEBIAN_FRONTEND: noninteractive
TZ: America/NewYork
GH_TOKEN: ${{ github.token }}
outputs:
chunks: ${{ steps.chunks.outputs.chunks }}
steps:
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,7 @@
*.swp
Ortho4XP
*_tile_list.*
/var/
/.venv/
/z_*_*/
/z_*_*.zip
87 changes: 0 additions & 87 deletions Makefile
View file
Open in desktop

This file was deleted.

Loading