Skip to content

Commit

Permalink
Fix building of wheels in monorepo and update docs
Browse files Browse the repository at this point in the history
The main issue is that our previous hack to detect the component name
from the folder no longer works, since it'll now be an unprefixed
"client", so have the Makefile add the prefixing in.

Documentation has been updated as well.
  • Loading branch information
legoktm committed Feb 8, 2024
1 parent 6e39c77 commit 0e0335e
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 18 deletions.
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ requirements: ## Creates requirements files for the Python projects

.PHONY: build-wheels
build-wheels: ## Builds the wheels and adds them to the wheels subdirectory
./scripts/verify-sha256sum-signature $$(basename ${PKG_DIR})
./scripts/verify-sha256sum-signature securedrop-$$(basename ${PKG_DIR})
./scripts/build-sync-wheels
./scripts/sync-sha256sums $$(basename ${PKG_DIR})
./scripts/sync-sha256sums securedrop-$$(basename ${PKG_DIR})
@echo Done!

.PHONY: test
Expand Down
31 changes: 15 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ _If you don't need to deploy a package and just want to test locally, you can st
## Updating our bootstrapped build tools
We use [build](https://pypa-build.readthedocs.io/en/latest/) toolchain to build our reproducible wheels.
If we have to update the tool, use the following steps
We use the [build](https://pypa-build.readthedocs.io/en/latest/) toolchain to build our reproducible wheels.
If we have to update the tools, use the following steps
```shell
# Ensure you are running in a cleanly boostrapped virtual environment
Expand Down Expand Up @@ -95,35 +95,31 @@ Make sure that your GPG public key is stored in `pubkeys/`, so CI can verify the
## Updating Python wheels
Maintainers of `securedrop-client` and `securedrop-proxy` must ensure that
the requirements files which are used for build of these packages (`build-requirements.txt`)
using `make requirements` are kept up to date in latest `main` of those repositories.
If new dependencies were added in the `build-requirements.txt` of that
repo that are not in the `wheels` subdirectory for the package in this repository,
then the maintainer needs to do the following (we are taking `securedrop-client` project
as an example):
When adding a new production dependency to a component, new wheels will need to be built
plus updates to `build-requirements.txt`. This should be done after you have updated the
dependencies in the component's `pyproject.toml` and `poetry.lock` files.

### 0. Enable the virtualenv

You can create a fresh virtualenv and install the build tools from our bootstrapped wheels.
Create a fresh virtualenv and install the build tools from our bootstrapped wheels.

```shell
rm -rf .venv
make install-deps
```

Remember that the following steps needs to be done from the same virtual environment.
The following steps needs to be done from the same virtual environment.

### 1. Create updated build-requirements.txt for the project

From the `securedrop-builder` directory,
From the `securedrop-builder` directory, run the following, where `<component>`
is what you're trying to update dependencies for, e.g. "client", "proxy", etc.
```shell
PKG_DIR=/home/user/code/securedrop-client make requirements
PKG_DIR=/home/user/code/securedrop-client/<component> make requirements
```
This will create the proper `build-requirements.txt` file in the project directory along with the binary wheel
This will create/update the `build-requirements.txt` file in the project directory along with the binary wheel
hashes from our own Python package index server.
If we are missing any wheels from our cache/build/server, it will let you know with a following message.
Expand Down Expand Up @@ -151,7 +147,10 @@ PKG_DIR=/home/user/code/securedrop-client make build-wheels
This above command will let you know about any new wheels + sources. It will
build/download sources from PyPI (by verifying it against the sha256sums from
the `requirements.txt` of the project).
the `poetry.lock` of the project).
If your package contains compiled code (e.g. C or Rust extensions), it must be
built for all Debian versions we support.
### 3. Commit changes to the wheels directory (if only any update of wheels)
Expand Down

0 comments on commit 0e0335e

Please sign in to comment.