Skip to content

Commit

Permalink
Merge branch 'main' into feat-healthchecks
Browse files Browse the repository at this point in the history
  • Loading branch information
ppawlowski authored Nov 28, 2024
2 parents 9becbef + b451284 commit dd6804b
Show file tree
Hide file tree
Showing 8 changed files with 584 additions and 54 deletions.
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
DOMAIN=
APPLICATION_DOMAIN=""

### TLS certificates configuration
TLS_ENABLED=""
TLS_CERTIFICATE=""
TLS_KEY=""
APP_TLS_CERTIFICATE=""
APP_TLS_KEY=""

### Database configuration
DB_HOST=""
Expand All @@ -17,3 +20,6 @@ EMAIL_PORT=587
EMAIL_SECURE=""
EMAIL_USER=""
EMAIL_PASSWORD=""

### Docker Driver options
DOCKER_DRIVER_PRIVATE_CA_PATH=""
163 changes: 163 additions & 0 deletions .github/workflows/test-docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
name: Test Docker Compose

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
default-stack:
name: Test default stack
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Create .env file for default settings
run: |
cp .env.example .env
sed -i 's/DOMAIN=.*/DOMAIN=ci-example.com/' .env
- name: Create stack
uses: hoverkraft-tech/[email protected]
with:
compose-file: "./docker-compose.yml"
up-flags: "-d --quiet-pull"

- name: Check readiness
run: |
has_healthcheck() {
local container=$1
local health_status=$(docker inspect --format='{{if .Config.Healthcheck}}true{{else}}false{{end}}' "$container")
[ "$health_status" = "true" ]
}
check_containers() {
containers=$(docker compose ps -q)
for container in $containers; do
container_name=$(docker inspect --format '{{.Name}}' "$container" | sed 's/\///')
container_ip=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$container")
if has_healthcheck "$container"; then
echo "Container has healthcheck defined"
status=$(docker inspect --format "{{.State.Health.Status}}" "$container")
if [ "$status" != "healthy" ]; then
echo "❌ Container $container_name is not healthy (status: $status)"
return 1
fi
else
running=$(docker inspect --format "{{.State.Running}}" "$container")
if [ "$running" != "true" ]; then
echo "❌ Container $container_name is not running"
return 1
fi
fi
echo "✅ Container $container_name is ready"
done
return 0
}
# Wait for containers with timeout
TIMEOUT=300 # 5 minutes timeout
ELAPSED=0
SLEEP_TIME=10
until check_containers; do
if [ $ELAPSED -ge $TIMEOUT ]; then
echo "❌ Timeout waiting for containers to be ready"
docker compose ps
docker compose logs
exit 1
fi
echo "⏳ Waiting for containers... ($ELAPSED seconds elapsed)"
sleep $SLEEP_TIME
ELAPSED=$((ELAPSED + SLEEP_TIME))
done
echo "✅ All containers are ready!"
docker compose ps
- name: Tear down the stack
if: always()
run: docker compose down


quick-start-stack:
name: Test quick-start stack
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Create .env file for default settings
run: |
cp .env.example .env
sed -i 's/DOMAIN=.*/DOMAIN=ci-example.com/' .env
- name: Create stack
uses: hoverkraft-tech/[email protected]
with:
compose-file: "./docker-compose-quick-start.yml"
up-flags: "-d --quiet-pull"

- name: Check readiness
run: |
has_healthcheck() {
local container=$1
local health_status=$(docker inspect --format='{{if .Config.Healthcheck}}true{{else}}false{{end}}' "$container")
[ "$health_status" = "true" ]
}
check_containers() {
containers=$(docker compose ps -q)
for container in $containers; do
container_name=$(docker inspect --format '{{.Name}}' "$container" | sed 's/\///')
container_ip=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$container")
if has_healthcheck "$container"; then
echo "Container has healthcheck defined"
status=$(docker inspect --format "{{.State.Health.Status}}" "$container")
if [ "$status" != "healthy" ]; then
echo "❌ Container $container_name is not healthy (status: $status)"
return 1
fi
else
running=$(docker inspect --format "{{.State.Running}}" "$container")
if [ "$running" != "true" ]; then
echo "❌ Container $container_name is not running"
return 1
fi
fi
echo "✅ Container $container_name is ready"
done
return 0
}
# Wait for containers with timeout
TIMEOUT=300 # 5 minutes timeout
ELAPSED=0
SLEEP_TIME=10
until check_containers; do
if [ $ELAPSED -ge $TIMEOUT ]; then
echo "❌ Timeout waiting for containers to be ready"
docker compose ps
docker compose logs
exit 1
fi
echo "⏳ Waiting for containers... ($ELAPSED seconds elapsed)"
sleep $SLEEP_TIME
ELAPSED=$((ELAPSED + SLEEP_TIME))
done
echo "✅ All containers are ready!"
docker compose ps
- name: Tear down the stack
if: always()
run: docker compose down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
#### 2.11.0: Release

- Ensure the broker service name doesn't change (#168)
- ci: Introduce workflow for testing docker compose (#166)
- Update Node-RED version in Docker compose (#162)
- chore: Refactor docker-compose to simplify installation experience (#160)
- First pass at TeamBroker (#165) @hardillb
- fix: Remove interpolation when creating TLS certificates (#164) @ppawlowski

#### 2.10.0: Release

- Introduce quick-start compose file (#158) @ppawlowski
Expand Down
48 changes: 39 additions & 9 deletions UPGRADE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,17 @@ This allows for easier management of the platform and better separation of conce

```bash
curl -o docker-compose-new.yml https://raw.githubusercontent.com/flowfuse/docker-compose/main/docker-compose.yml
curl -o docker-compose-tls.override.new.yml https://raw.githubusercontent.com/flowfuse/docker-compose/main/docker-compose-tls.override.yml
curl -o .env https://raw.githubusercontent.com/flowfuse/docker-compose/main/.env.example
```

3. **Move configurations to the new approach**

* Copy content of `./etc/flowforge.yml` file to `docker-compose-new.yml` file, to `configs.flowfuse.content` section. Remove all commented lines. Maintain indentation.
* Make sure, that `broker.url` is seto fo `mqtt://broker:1883`. Update if needed.
* Copy content of `./etc/flowforge-storage.yml` file to `docker-compose-new.yml` file, to `configs.flowfuse_storage.content` section. Remove all commented lines. Maintain indentation.
* Set the `DOMAIN` variable in the `.env` file to the domain used by your instance of FlowFuse platform.
* If FlowFuse application is running outside of the `DOMAIN` scope, set it as a value of `APPLICATION_DOMAIN` variable in the `.env` file.
* If application should be accessible via seured connection (HTTPS), set `TLS_ENABLED` variable to `true` in `.env` file.
* If custom certificates are used, copy their content to `.env` file, to `TLS_CERTIFICATE` and `TLS_KEY` variables. They should look like this:

```bash
Expand All @@ -68,6 +70,28 @@ This allows for easier management of the platform and better separation of conce
"
```

* If custom certificates are used and FlowFuse application is running on a different domain than other stack components (defined in `APPLICATION_DOMAIN` variable),
use `APP_TLS_CERTIFICATE` and `APP_TLS_KEY` variabls to provide certificate and it's key. They should look like this:
```bash
APP_TLS_CERTIFICATE="
-----BEGIN CERTIFICATE-----
MIIFfzCCBKegAwIBAgISA0
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFfzCCBKegAwIBAgISA0
...
-----END CERTIFICATE-----
"
APP_TLS_KEY="
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQD
...
-----END PRIVATE KEY-----
"
```
4. **Migrate database files**
Move the database files from host to the new volume. This will allow you to keep the existing data.
Expand All @@ -90,38 +114,44 @@ This allows for easier management of the platform and better separation of conce
docker run --rm -v flowfuse_db:/data -v $(pwd)/db:/backup alpine sh -c "cp -a /backup/. /data/"
```
5. **Start FlowFuse**
5. **Rename files**
Start the new FlowFuse platform using the new Docker Compose file.
In order to maintain the same file structurem, rename the compose files.
* With automatic TLS certificate generation:
```bash
docker compose -f docker-compose-new.yml -f docker-compose-tls.new.override.yml --profile autossl -p flowfuse up -d
mv docker-compose.yml docker-compose-old.yml
mv docker-compose-new.yml docker-compose.yml
```
* With custom TLS certificate:
6. **Start FlowFuse**
Start the new FlowFuse platform using the new Docker Compose file.
* With automatic TLS certificate generation:
```bash
docker compose -f docker-compose.new.yml -f docker-compose-tls..new.override.yml -p flowfuse up -d
docker compose -f docker-compose.yml --profile autotls -p flowfuse up -d
```
* In all other cases
```bash
docker compose -p flowfuse up -d
```
6. **Verify the migration**
7. **Verify the migration**
Verify that the new FlowFuse platform is working correctly and it is accessible using the domain set in the `.env` file.
Login credentials should remain the same as before the migration, as well as platform configuration.
Restart the Node-RED instances if they appear in `Starting` state.
7. **Cleanup**
8 **Cleanup**
After verifying that the new FlowFuse platform is working correctly, you can remove the old configuration files.
```bash
rm ./etc/flowforge.yml ./etc/flowforge.yml.bak
rm ./etc/flowforge-storage.yml.bak ./etc/flowforge-storage.yml
rm -rf ./db ./db.bak
rm -f ./docker-compose-old.yml
```
Loading

0 comments on commit dd6804b

Please sign in to comment.