Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TS-1197] feat: use nextest #16

Merged
merged 10 commits into from
Jul 25, 2024
6 changes: 5 additions & 1 deletion .github/workflows/server-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ jobs:
scheduler/target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

# Install Nextest
- name: Install nextest
uses: taiki-e/install-action@nextest

- name: Run migrations
run: |
cd scheduler
Expand Down Expand Up @@ -84,4 +88,4 @@ jobs:
- name: Run server tests
run: |
cd scheduler
cargo test --all
cargo nextest run --workspace
7 changes: 4 additions & 3 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ export DATABASE_URL := "postgresql://postgres:postgres@localhost:45432/nettusche
# Install minimal tools
install_tools:
cargo install sqlx-cli --no-default-features --features postgres
cargo install cargo-nextest

# Install all tools
install_all_tools: install_tools
Expand All @@ -26,9 +27,9 @@ _setup_client_node:
dev: _setup_db
cd scheduler && cargo run

# Test
test: _setup_db
cd scheduler && cargo test --all
# Run the tests on a temporary DB container
test:
bash ./scripts/run_tests.sh

# Lint
lint: _setup_db
Expand Down
8 changes: 8 additions & 0 deletions scheduler/.config/nextest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Test group for running some tests serially
[test-groups]
serial-integration = { max-threads = 1 }

# Tests that contain `serial_` are run serially
[[profile.default.overrides]]
filter = 'test(serial_)'
test-group = 'serial-integration'
8 changes: 4 additions & 4 deletions scheduler/crates/api/src/event/get_upcoming_reminders.rs
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ mod tests {
#[actix_web::main]
#[serial_test::serial]
#[test]
async fn get_upcoming_reminders() {
async fn serial_get_upcoming_reminders() {
let mut ctx = setup_context().await;
ctx.sys = Arc::new(StaticTimeSys1 {});

Expand Down Expand Up @@ -271,7 +271,7 @@ mod tests {
#[actix_web::main]
#[serial_test::serial]
#[test]
async fn updating_event_also_updates_reminders() {
async fn serial_updating_event_also_updates_reminders() {
let mut ctx = setup_context().await;
ctx.sys = Arc::new(StaticTimeSys1 {});

Expand Down Expand Up @@ -334,7 +334,7 @@ mod tests {
#[actix_web::main]
#[serial_test::serial]
#[test]
async fn deleting_event_reminder_setting_also_deletes_reminders() {
async fn serial_deleting_event_reminder_setting_also_deletes_reminders() {
let mut ctx = setup_context().await;
ctx.sys = Arc::new(StaticTimeSys1 {});

Expand Down Expand Up @@ -389,7 +389,7 @@ mod tests {
#[actix_web::main]
#[serial_test::serial]
#[test]
async fn deleting_event_also_deletes_reminders() {
async fn serial_deleting_event_also_deletes_reminders() {
let mut ctx = setup_context().await;
ctx.sys = Arc::new(StaticTimeSys1 {});

Expand Down
77 changes: 77 additions & 0 deletions scripts/run_tests.sh

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] Could we just use testcontainers for this?

Copy link
Collaborator Author

@GuillaumeDecMeetsMore GuillaumeDecMeetsMore Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I forgot to post my comment about this 😓

I wanted to use testcontainers, but the problems is that Rust by default doesn't have a concept of global "beforeAll" and "afterAll"

nextest does have the feature for beforeAll (as an experimental feature), but from what I understand, it's a "one time thing". So, using testcontainers there or a shell script would mean that the containers would start and be killed before running the tests (because the script/app/context is killed before).

(technically, it would work fine for starting the containers. The issue would be on how to garbage collect them)

Note that with NodeJS, I believe it works by keeping a connection opened to the Ryuk container. This is possible because the NodeJS context ("main context") stays alive as long as the tests are running (and the tests are ran inside this context via the use of the vm module). Once the tests are finished, the app "exits", Ryuk detects that the connection is closed and therefore it can kill the containers.

Here, for Rust, from what I understand, each file of test is like a binary executed independently, so we don't have that "main context" (or it's not accessible from what I searched).

So wrapping everything in a shell script seemed the easiest and less complex 😅

Also worth noting that the garbage collection in testcontainers is not finished testcontainers/testcontainers-rs#577

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me!

Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#! /bin/bash

# Script to run all the (Rust) tests of the scheduler project
# It launches a temporary PostgreSQL container, runs the migrations, runs the tests and then stops and removes the container

# Function to clean up the containers
CLEANUP_CALLED=false
cleanup() {
if [ "$CLEANUP_CALLED" = true ]; then
return
fi
echo "Cleaning up..."
CLEANUP_CALLED=true

if [ -n "$NC_PID" ] && kill -0 $NC_PID 2>/dev/null; then
kill $NC_PID >/dev/null 2>&1
fi

if [ "$(docker ps -q -f name=$RANDOM_NAME)" ]; then
docker stop $RANDOM_NAME >/dev/null 2>&1
fi

if [ "$(docker ps -q -f name=ryuk)" ]; then
docker stop ryuk >/dev/null 2>&1
fi
}

# Set up a trap to call the cleanup function on EXIT, SIGINT, and SIGTERM
trap cleanup EXIT SIGINT SIGTERM

# Search for a free port to bind the temporary PG container
BASE_PORT=1234
INCREMENT=1

PORT=$BASE_PORT
IS_FREE=$(netstat -taln | grep $PORT)

while [[ -n "$IS_FREE" ]]; do
PORT=$((PORT + INCREMENT))
IS_FREE=$(netstat -taln | grep $PORT)
done

# Generate a random name for the temporary PG container
RANDOM_NAME="pg_test_$(date +%s)"

LABEL="nittei_testing=true"

cd scheduler && cargo build --workspace

# Launch the resource reaper (like testcontainers)
docker run -d --name ryuk --rm -v /var/run/docker.sock:/var/run/docker.sock -e RYUK_VERBOSE=true -e RYUK_PORT=8080 -p 8080:8080 testcontainers/ryuk:0.8.1 >/dev/null 2>&1

# Keep the connection open and send the label to Ryuk
TIMEOUT=60
(
echo "label=${LABEL}"
# Keep the connection open to Ryuk and read the ACK response
while [ $((TIMEOUT--)) -gt 0 ]; do
sleep 1
done
) | nc localhost 8080 &
>/dev/null 2>&1
NC_PID=$!

# Launch a PG container
docker run --rm -d -l ${LABEL} --name $RANDOM_NAME -p $PORT:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=scheduler postgres:13 >/dev/null 2>&1

# Set the DATABASE_URL environment variable
export DATABASE_URL="postgres://postgres:postgres@localhost:${PORT}/scheduler"

# Run the migrations
cd crates/infra && sqlx migrate run && cd ../..

# Run the tests
cargo nextest run --workspace && cd ..

# The cleanup function will be called automatically