Skip to content

Commit

Permalink
Merge branch 'master' into fix_setup_mag_in_case_of_non_public
Browse files Browse the repository at this point in the history
  • Loading branch information
valentin-pinkau authored Dec 3, 2024
2 parents bd69718 + 1eee692 commit ee7feeb
Show file tree
Hide file tree
Showing 102 changed files with 86,181 additions and 101,294 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,10 @@ jobs:
with:
# Install a specific version of uv.
version: "0.4.22"


- name: Install proxay
run: npm install -g proxay

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

Expand Down
13 changes: 11 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,16 +133,25 @@ Internal workflows for scalable minds:

The `webknossos` folder contains examples, which are not part of the package, but are tested via `tests/test_examples.py` and added to the documentation (see `docs/src/webknossos-py/examples`).

To run the `./test.sh` script it is necessary to install `proxay`. This is either done with [NPM](https://www.npmjs.com) or [yarn](https://yarnpkg.com/getting-started/install):
```bash
npm install --global proxay

# or if you're using yarn
yarn global add proxay
```

The tests also contain functionality for the WEBKNOSSOS client. There a two modes to run the tests:

1. `./test.sh --refresh-snapshots`, sending network requests to a WEBKNOSSOS instance:
This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [vcr.py](https://vcrpy.readthedocs.io), see next point:
2. `./test.sh` replays responses from previous network snapshots using [vcr.py](https://vcrpy.readthedocs.io) via [pytest-recording](https://github.com/kiwicom/pytest-recording). No additional network requests are allowed in this mode.
This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [proxay](https://github.com/airtasker/proxay), see next point:
2. `./test.sh` replays responses from previous network snapshots using [proxay](https://github.com/airtasker/proxay).

`./test.sh --store-durations` updates the durations for
[`pytest-split`](https://jerry-git.github.io/pytest-split),
which is used in the CI to split the tests for different runners.


#### `cluster_tools` package

For testing the `slurm` setup a `docker compose` setup is available. Please see the [respective Readme](https://github.com/scalableminds/webknossos-libs/blob/master/cluster_tools/README.md) for details.
Expand Down
10 changes: 4 additions & 6 deletions cluster_tools/dockered-slurm/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "2.2"

services:
mysql:
image: mysql:5.7
Expand All @@ -16,7 +14,7 @@ services:

slurmdbd:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmdbd"]
command: [ "slurmdbd" ]
container_name: slurmdbd
hostname: slurmdbd
volumes:
Expand All @@ -32,7 +30,7 @@ services:

slurmctld:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmctld"]
command: [ "slurmctld" ]
container_name: slurmctld
environment:
USER: "root"
Expand All @@ -53,7 +51,7 @@ services:

c1:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmd"]
command: [ "slurmd" ]
hostname: c1
container_name: c1
volumes:
Expand All @@ -71,7 +69,7 @@ services:

c2:
image: scalableminds/slurm-docker-cluster:master__11274637426
command: ["slurmd"]
command: [ "slurmd" ]
hostname: c2
container_name: c2
volumes:
Expand Down
2 changes: 0 additions & 2 deletions docs/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit ee7feeb

Please sign in to comment.