Add clone-dependencies.sh script
#5
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, builds of our Python SDK clone several C++ dependencies directly from the `main` branch of their canonical git repositories and build them from source. Sometimes, though, `main` might be broken, causing our nightly builds to spuriously fail. Instead, we should clone from tagged releases of our dependencies, pinning the versions so we know that our builds will not fail due to development issues in our dependencies. This patch refactors the `bin/build-*.sh` scripts to enable us to do this more easily. First, it creates a new `bin/clone-dependencies.sh` script, which specifies the tagged versions of and clones the dependencies `bde-tools`, `bde`, `ntf-core`, and `blazingmq`. This consolidates the dependency cloning that is common between all the `bin/build-*.sh` scripts. This script is not intended to be run by users directly; instead, this patch secondarily removes the dependency cloning from each build script, and sources the `bin/clone-dependencies.sh` script. The existing dependency cloning logic allows for a user to manually clone these dependencies in the correct location before running the script, and those clones will be used instead. This patch preserves this functionality, which allows users to still build and test the Python SDK against the `main` branches of these dependencies, at the cost of doing the cloning and checking out themselves. To update the release tags that the SDK is built against, the environment variables at the top of `.bin/clone-dependencies.sh` should be modified. Testing ------- This patch was tested by building and testing the SDK with the `./bin/build-macos-universal.sh` script on an M2 Mac system, and manually ensuring that all dependencies were correctly cloned to the correct tag. Signed-off-by: Patrick M. Niedzielski <[email protected]>
This patch brings the `build-macos-universal.sh` script to parity with the `build-manylinux.sh` script in installing the BlazingMQ broker to `DIR_INSTALL` after building it. This is necessary to run integration tests on an OSX GitHub Actions runner, and better automates the process for running these tests manually, bringing the process up to line with the process on Linux machines. Signed-off-by: Patrick M. Niedzielski <[email protected]>
hallfox
reviewed
Nov 28, 2023
hallfox
approved these changes
Nov 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, builds of our Python SDK clone several C++ dependencies directly from the
mainbranch of their canonical git repositories and build them from source. Sometimes, though,mainmight be broken, causing our nightly builds to spuriously fail. Instead, we should clone from tagged releases of our dependencies, pinning the versions so we know that our builds will not fail due to development issues in our dependencies.This patch refactors the
bin/build-*.shscripts to enable us to do this more easily. First, it creates a newbin/clone-dependencies.shscript, which specifies the tagged versions of and clones the dependenciesbde-tools,bde,ntf-core, andblazingmq. This consolidates the dependency cloning that is common between all thebin/build-*.shscripts. This script is not intended to be run by users directly; instead, this patch secondarily removes the dependency cloning from each build script, and sources thebin/clone-dependencies.shscript.The existing dependency cloning logic allows for a user to manually clone these dependencies in the correct location before running the script, and those clones will be used instead. This patch preserves this functionality, which allows users to still build and test the Python SDK against the
mainbranches of these dependencies, at the cost of doing the cloning and checking out themselves.To update the release tags that the SDK is built against, the environment variables at the top of
.bin/clone-dependencies.shshould be modified.Testing
This patch was tested by building and testing the SDK with the
./bin/build-macos-universal.shscript on an M2 Mac system, and manually ensuring that all dependencies were correctly cloned to the correct tag.