Skip to content

Commit

Permalink
speed up fizz github CI by using the system deps
Browse files Browse the repository at this point in the history
Summary:

In previous changed noticed the CI was quite slow as it was building things like boost every time.  Regenerated it with:

```
./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --src-dir=. --output-dir=.github/workflows fizz
```

Test Plan:

CI
  • Loading branch information
ahornby committed Jul 21, 2023
1 parent 45567ee commit 0c0fcde
Show file tree
Hide file tree
Showing 2 changed files with 90 additions and 84 deletions.
86 changes: 45 additions & 41 deletions .github/workflows/getdeps_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,89 +15,93 @@ jobs:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Update system package info
run: sudo apt-get update
- name: Install system deps
run: sudo python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive fizz
- name: Fetch ninja
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests ninja
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests ninja
- name: Fetch cmake
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests cmake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests cmake
- name: Fetch boost
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests boost
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests boost
- name: Fetch double-conversion
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests double-conversion
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests double-conversion
- name: Fetch fmt
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests fmt
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests fmt
- name: Fetch gflags
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests gflags
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests gflags
- name: Fetch glog
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests glog
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests glog
- name: Fetch googletest
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests googletest
- name: Fetch libevent
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libevent
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libevent
- name: Fetch lz4
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests lz4
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests lz4
- name: Fetch snappy
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests snappy
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests snappy
- name: Fetch zstd
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests zstd
- name: Fetch zlib
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests zlib
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests zlib
- name: Fetch autoconf
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests autoconf
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests autoconf
- name: Fetch automake
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests automake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests automake
- name: Fetch libtool
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libtool
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libtool
- name: Fetch libsodium
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libsodium
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libsodium
- name: Fetch xz
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests xz
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests xz
- name: Fetch folly
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests folly
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests folly
- name: Build ninja
run: python3 build/fbcode_builder/getdeps.py build --no-tests ninja
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests ninja
- name: Build cmake
run: python3 build/fbcode_builder/getdeps.py build --no-tests cmake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests cmake
- name: Build boost
run: python3 build/fbcode_builder/getdeps.py build --no-tests boost
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests boost
- name: Build double-conversion
run: python3 build/fbcode_builder/getdeps.py build --no-tests double-conversion
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests double-conversion
- name: Build fmt
run: python3 build/fbcode_builder/getdeps.py build --no-tests fmt
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests fmt
- name: Build gflags
run: python3 build/fbcode_builder/getdeps.py build --no-tests gflags
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests gflags
- name: Build glog
run: python3 build/fbcode_builder/getdeps.py build --no-tests glog
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests glog
- name: Build googletest
run: python3 build/fbcode_builder/getdeps.py build --no-tests googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests googletest
- name: Build libevent
run: python3 build/fbcode_builder/getdeps.py build --no-tests libevent
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libevent
- name: Build lz4
run: python3 build/fbcode_builder/getdeps.py build --no-tests lz4
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests lz4
- name: Build snappy
run: python3 build/fbcode_builder/getdeps.py build --no-tests snappy
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests snappy
- name: Build zstd
run: python3 build/fbcode_builder/getdeps.py build --no-tests zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests zstd
- name: Build zlib
run: python3 build/fbcode_builder/getdeps.py build --no-tests zlib
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests zlib
- name: Build autoconf
run: python3 build/fbcode_builder/getdeps.py build --no-tests autoconf
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests autoconf
- name: Build automake
run: python3 build/fbcode_builder/getdeps.py build --no-tests automake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests automake
- name: Build libtool
run: python3 build/fbcode_builder/getdeps.py build --no-tests libtool
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libtool
- name: Build libsodium
run: python3 build/fbcode_builder/getdeps.py build --no-tests libsodium
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libsodium
- name: Build xz
run: python3 build/fbcode_builder/getdeps.py build --no-tests xz
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests xz
- name: Build folly
run: python3 build/fbcode_builder/getdeps.py build --no-tests folly
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests folly
- name: Build fizz
run: python3 build/fbcode_builder/getdeps.py build --src-dir=. fizz --project-install-prefix fizz:/usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. fizz --project-install-prefix fizz:/usr/local
- name: Copy artifacts
run: python3 build/fbcode_builder/getdeps.py fixup-dyn-deps --strip --src-dir=. fizz _artifacts/linux --project-install-prefix fizz:/usr/local --final-install-prefix /usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fixup-dyn-deps --strip --src-dir=. fizz _artifacts/linux --project-install-prefix fizz:/usr/local --final-install-prefix /usr/local
- uses: actions/upload-artifact@v2
with:
name: fizz
path: _artifacts
- name: Test fizz
run: python3 build/fbcode_builder/getdeps.py test --src-dir=. fizz --project-install-prefix fizz:/usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. fizz --project-install-prefix fizz:/usr/local
88 changes: 45 additions & 43 deletions .github/workflows/getdeps_mac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,93 +15,95 @@ jobs:
runs-on: macOS-latest
steps:
- uses: actions/checkout@v2
- name: Install system deps
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive fizz
- name: Fetch ninja
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests ninja
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests ninja
- name: Fetch cmake
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests cmake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests cmake
- name: Fetch boost
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests boost
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests boost
- name: Fetch double-conversion
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests double-conversion
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests double-conversion
- name: Fetch fmt
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests fmt
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests fmt
- name: Fetch gflags
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests gflags
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests gflags
- name: Fetch glog
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests glog
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests glog
- name: Fetch googletest
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests googletest
- name: Fetch lz4
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests lz4
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests lz4
- name: Fetch openssl
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests openssl
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests openssl
- name: Fetch snappy
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests snappy
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests snappy
- name: Fetch zstd
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests zstd
- name: Fetch zlib
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests zlib
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests zlib
- name: Fetch libevent
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libevent
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libevent
- name: Fetch autoconf
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests autoconf
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests autoconf
- name: Fetch automake
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests automake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests automake
- name: Fetch libtool
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libtool
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libtool
- name: Fetch libsodium
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests libsodium
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests libsodium
- name: Fetch xz
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests xz
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests xz
- name: Fetch folly
run: python3 build/fbcode_builder/getdeps.py fetch --no-tests folly
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests folly
- name: Build ninja
run: python3 build/fbcode_builder/getdeps.py build --no-tests ninja
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests ninja
- name: Build cmake
run: python3 build/fbcode_builder/getdeps.py build --no-tests cmake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests cmake
- name: Build boost
run: python3 build/fbcode_builder/getdeps.py build --no-tests boost
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests boost
- name: Build double-conversion
run: python3 build/fbcode_builder/getdeps.py build --no-tests double-conversion
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests double-conversion
- name: Build fmt
run: python3 build/fbcode_builder/getdeps.py build --no-tests fmt
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests fmt
- name: Build gflags
run: python3 build/fbcode_builder/getdeps.py build --no-tests gflags
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests gflags
- name: Build glog
run: python3 build/fbcode_builder/getdeps.py build --no-tests glog
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests glog
- name: Build googletest
run: python3 build/fbcode_builder/getdeps.py build --no-tests googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests googletest
- name: Build lz4
run: python3 build/fbcode_builder/getdeps.py build --no-tests lz4
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests lz4
- name: Build openssl
run: python3 build/fbcode_builder/getdeps.py build --no-tests openssl
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests openssl
- name: Build snappy
run: python3 build/fbcode_builder/getdeps.py build --no-tests snappy
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests snappy
- name: Build zstd
run: python3 build/fbcode_builder/getdeps.py build --no-tests zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests zstd
- name: Build zlib
run: python3 build/fbcode_builder/getdeps.py build --no-tests zlib
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests zlib
- name: Build libevent
run: python3 build/fbcode_builder/getdeps.py build --no-tests libevent
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libevent
- name: Build autoconf
run: python3 build/fbcode_builder/getdeps.py build --no-tests autoconf
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests autoconf
- name: Build automake
run: python3 build/fbcode_builder/getdeps.py build --no-tests automake
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests automake
- name: Build libtool
run: python3 build/fbcode_builder/getdeps.py build --no-tests libtool
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libtool
- name: Build libsodium
run: python3 build/fbcode_builder/getdeps.py build --no-tests libsodium
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests libsodium
- name: Build xz
run: python3 build/fbcode_builder/getdeps.py build --no-tests xz
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests xz
- name: Build folly
run: python3 build/fbcode_builder/getdeps.py build --no-tests folly
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --no-tests folly
- name: Build fizz
run: python3 build/fbcode_builder/getdeps.py build --src-dir=. fizz --project-install-prefix fizz:/usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. fizz --project-install-prefix fizz:/usr/local
- name: Copy artifacts
run: python3 build/fbcode_builder/getdeps.py fixup-dyn-deps --src-dir=. fizz _artifacts/mac --project-install-prefix fizz:/usr/local --final-install-prefix /usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fixup-dyn-deps --src-dir=. fizz _artifacts/mac --project-install-prefix fizz:/usr/local --final-install-prefix /usr/local
- uses: actions/upload-artifact@v2
with:
name: fizz
path: _artifacts
- name: Test fizz
run: python3 build/fbcode_builder/getdeps.py test --src-dir=. fizz --project-install-prefix fizz:/usr/local
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. fizz --project-install-prefix fizz:/usr/local

0 comments on commit 0c0fcde

Please sign in to comment.