diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 62e3b66a5..3e3cbf660 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,71 +1 @@ -# Zeppelin development environment -Zeppelin's development environment runs entirely within a Docker container. -Below you can find instructions for setting up the environment and getting started with development! - -**Note:** If you'd just like to run the bot for your own server, see 👉 **[PRODUCTION.md](./PRODUCTION.md)** 👈 - -## Starting the development environment - -### Using VSCode devcontainers -1. Install Docker -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` -4. In VSCode: Install the `Remote - Containers` plugin -5. In VSCode: Run `Remote-Containers: Open Folder in Container...` and select the Zeppelin folder - -### Using VSCode remote SSH plugin -1. Install Docker -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` -4. Run `docker compose -f docker-compose.development.yml up` to start the development environment -5. In VSCode: Install the `Remote - SSH` plugin -6. In VSCode: Run `Remote-SSH: Connect to Host...` - * As the address, use `ubuntu@127.0.0.1:3002` (where `3002` matches `DOCKER_DEV_SSH_PORT` in `.env`) - * Use the password specified in `.env` as `DOCKER_DEV_SSH_PASSWORD` -7. In VSCode: Once connected, click `Open folder...` and select `/home/ubuntu/zeppelin` - -### Using JetBrains Gateway -1. Install Docker -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` -4. Run `docker compose -f docker-compose.development.yml up` to start the development environment -5. Choose `Connect via SSH` and create a new connection: - * Username: `ubuntu` - * Host: `127.0.0.1` - * Port: `3002` (matching the `DOCKER_DEV_SSH_PORT` value in `.env`) -6. Click `Check Connection and Continue` and enter the password specified in `.env` as `DOCKER_DEV_SSH_PASSWORD` when asked -7. In the next pane: - * IDE version: WebStorm, PHPStorm, or IntelliJ IDEA - * Project directory: `/home/ubuntu/zeppelin` -8. Click `Download and Start IDE` - -### Using any other IDE with SSH development support -1. Install Docker -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` -4. Run `docker compose -f docker-compose.development.yml up` to start the development environment -5. Use the following credentials for connecting with your IDE: - * Host: `127.0.0.1` - * Port: `3002` (matching the `DOCKER_DEV_SSH_PORT` value in `.env`) - * Username: `ubuntu` - * Password: As specified in `.env` as `DOCKER_DEV_SSH_PASSWORD` - -## Starting the project - -### Starting the backend (bot + api) -These commands are run inside the dev container. You should be able to open a terminal in your IDE after connecting. - -1. `cd ~/zeppelin/backend` -2. `npm ci` -3. `npm run migrate-dev` -4. `npm run watch` - -### Starting the dashboard -These commands are run inside the dev container. You should be able to open a terminal in your IDE after connecting. - -1. `cd ~/zeppelin/dashboard` -2. `npm ci` -3. `npm run watch-build` - -### Opening the dashboard -Browse to https://localhost:3300 to view the dashboard +Moved to [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) diff --git a/MANAGEMENT.md b/MANAGEMENT.md index a6d12ef55..758c2c3ad 100644 --- a/MANAGEMENT.md +++ b/MANAGEMENT.md @@ -1,34 +1 @@ -# Management -After starting Zeppelin -- either in the [development](./DEVELOPMENT.md) or [production](./PRODUCTION.md) environment -- you have several tools available to manage it. - -## Note -Make sure to add yourself to the list of staff members (`STAFF`) in `.env` and allow at least one server by default (`DEFAULT_ALLOWED_SERVERS`). Then, invite the bot to the server. - -In all examples below, `@Bot` refers to a user mention of the bot user. Make sure to run the commands on a server with the bot, in a channel that the bot can see. - -In the command parameters, `` refers to a required parameter (don't include the `< >` symbols) and `[this]` refers to an optional parameter (don't include the `[ ]` symbols). `` refers to being able to list multiple values, e.g. `value1 value2 value3`. - -## Allow a server to invite the bot -Run the following command: -``` -@Bot allow_server [userId] -``` -When specifying a user ID, that user will be given "Bot manager" level access to the server's dashboard, allowing them to manage access for other users. - -## Disallow a server -Run the following command: -``` -@Bot disallow_server -``` - -## Grant access to a server's dashboard -Run the following command: -``` -@Bot add_dashboard_user -``` - -## Remove access to a server's dashboard -Run the following command: -``` -@Bot remove_dashboard_user -``` +Moved to [docs/MANAGEMENT.md](docs/MANAGEMENT.md) diff --git a/PRODUCTION.md b/PRODUCTION.md index 8c57494b3..33939f0d4 100644 --- a/PRODUCTION.md +++ b/PRODUCTION.md @@ -1,67 +1 @@ -# Zeppelin production environment -Zeppelin's production environment uses Docker. There are a few different ways to run Zeppelin in production: - -1. **Standalone** - * The easiest way to get Zeppelin up and running. This setup comes with a built-in database and web server. -2. **Lightweight** - * In case you don't want to use the built-in database and web server. This setup only runs the bot, API, and dashboard themselves. You'll have to provide your own database connection options and set up a proxy server for the API and dashboard. -3. **Manual** - * If you only want to run a specific service, you can use Zeppelin's Dockerfile directly. - -## Standalone - -### Setup -1. Install Docker on the machine running the bot -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` (including the "PRODUCTION - STANDALONE" section) - -**Note:** The dashboard and API are exposed with a self-signed certificate. It is recommended to set up a proxy with a proper certificate in front of them. A popular option for this is [Cloudflare Tunnel](https://www.cloudflare.com/products/tunnel/). - -### Running the bot -`docker compose -f docker-compose.standalone.yml up -d` - -### Shutting the bot down -`docker compose -f docker-compose.standalone.yml down` - -### Updating the bot -1. Shut the bot down -2. Update the files (e.g. `git pull`) -3. Rebuild: `docker compose -f docker-compose.standalone.yml build` -4. Run the bot again - -### Viewing logs -`docker compose -f docker-compose.standalone.yml logs -t -f` - -## Lightweight - -### Setup -1. Install Docker on the machine running the bot -2. Make a copy of `.env.example` called `.env` -3. Fill in the missing values in `.env` (including the "PRODUCTION - LIGHTWEIGHT" section) - -### Running the bot -`docker compose -f docker-compose.lightweight.yml up -d` - -### Shutting the bot down -`docker compose -f docker-compose.lightweight.yml down` - -### Updating the bot -1. Shut the bot down -2. Update the files (e.g. `git pull`) -3. Rebuild: `docker compose -f docker-compose.lightweight.yml build` -4. Run the bot again - -### Viewing logs -`docker compose -f docker-compose.lightweight.yml logs -t -f` - -## Manual -1. Build the Zeppelin image: `docker build --tag 'zeppelin' .` -2. Run the service: - * Bot: `docker run zeppelin npm run start-bot` - * API: `docker run zeppelin npm run start-api` - * Dashboard: `docker run zeppelin npm run start-dashboard` - -If you're using an application platform such as Railway, you can simply point it to Zeppelin's repository and it should pick up the Dockerfile from there. -For the start command, you can use the same commands as above: `npm run start-bot`, `npm run start-api`, `npm run start-dashboard`. - -**Note:** You'll need to provide the necessary env variables. For example, `docker run --env-file .env zeppelin` +Moved to [docs/PRODUCTION.md](docs/PRODUCTION.md) diff --git a/README.md b/README.md index 52f60f073..1800bba33 100644 --- a/README.md +++ b/README.md @@ -23,11 +23,11 @@ See https://zeppelin.gg/ for more details. For information on how to use the bot, see https://zeppelin.gg/docs ## Development -See [DEVELOPMENT.md](./DEVELOPMENT.md) for instructions on running the development environment. +See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for instructions on running the development environment. -Once you have the environment up and running, see [MANAGEMENT.md](./MANAGEMENT.md) for how to manage your bot. +Once you have the environment up and running, see [docs/MANAGEMENT.md](docs/MANAGEMENT.md) for how to manage your bot. ## Production -See [PRODUCTION.md](./PRODUCTION.md) for instructions on how to run the bot in production. +See [docs/PRODUCTION.md](docs/PRODUCTION.md) for instructions on how to run the bot in production. -Once you have the environment up and running, see [MANAGEMENT.md](./MANAGEMENT.md) for how to manage your bot. +Once you have the environment up and running, see [docs/MANAGEMENT.md](docs/MANAGEMENT.md) for how to manage your bot. diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 000000000..9b562b3b9 --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,74 @@ +# Zeppelin development environment + +⚠️ **Updating from before 30 Mar 2024? See [MIGRATE_DEV.md](./MIGRATE_DEV.md) for instructions.** + +Zeppelin's development environment runs entirely within a Docker container. +Below you can find instructions for setting up the environment and getting started with development! + +**Note:** If you'd just like to run the bot for your own server, see 👉 **[PRODUCTION.md](./PRODUCTION.md)** 👈 + +## Starting the development environment + +### Using VSCode devcontainers +1. Install Docker +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` +4. In VSCode: Install the `Remote - Containers` plugin +5. In VSCode: Run `Remote-Containers: Open Folder in Container...` and select the Zeppelin folder + +### Using VSCode remote SSH plugin +1. Install Docker +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` +4. Run `docker compose -f docker-compose.development.yml up` to start the development environment +5. In VSCode: Install the `Remote - SSH` plugin +6. In VSCode: Run `Remote-SSH: Connect to Host...` + * As the address, use `ubuntu@127.0.0.1:3002` (where `3002` matches `DEVELOPMENT_SSH_PORT` in `.env`) + * Use the password specified in `.env` as `DEVELOPMENT_SSH_PASSWORD` +7. In VSCode: Once connected, click `Open folder...` and select `/home/ubuntu/zeppelin` + +### Using JetBrains Gateway +1. Install Docker +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` +4. Run `docker compose -f docker-compose.development.yml up` to start the development environment +5. Choose `Connect via SSH` and create a new connection: + * Username: `ubuntu` + * Host: `127.0.0.1` + * Port: `3002` (matching the `DEVELOPMENT_SSH_PORT` value in `.env`) +6. Click `Check Connection and Continue` and enter the password specified in `.env` as `DEVELOPMENT_SSH_PASSWORD` when asked +7. In the next pane: + * IDE version: WebStorm, PHPStorm, or IntelliJ IDEA + * Project directory: `/home/ubuntu/zeppelin` +8. Click `Download and Start IDE` + +### Using any other IDE with SSH development support +1. Install Docker +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` +4. Run `docker compose -f docker-compose.development.yml up` to start the development environment +5. Use the following credentials for connecting with your IDE: + * Host: `127.0.0.1` + * Port: `3002` (matching the `DEVELOPMENT_SSH_PORT` value in `.env`) + * Username: `ubuntu` + * Password: As specified in `.env` as `DEVELOPMENT_SSH_PASSWORD` + +## Starting the project +These commands are run inside the dev container. You should be able to open a terminal in your IDE after connecting. + +### 1. Install dependencies + +1. `npm ci` + +### Starting the backend (bot + api) + +1. `cd ~/zeppelin/backend` +2. `npm run watch` + +### Starting the dashboard + +1. `cd ~/zeppelin/dashboard` +2. `npm run watch` + +### Opening the dashboard +Browse to https://localhost:3300 to view the dashboard diff --git a/docs/MANAGEMENT.md b/docs/MANAGEMENT.md new file mode 100644 index 000000000..a6d12ef55 --- /dev/null +++ b/docs/MANAGEMENT.md @@ -0,0 +1,34 @@ +# Management +After starting Zeppelin -- either in the [development](./DEVELOPMENT.md) or [production](./PRODUCTION.md) environment -- you have several tools available to manage it. + +## Note +Make sure to add yourself to the list of staff members (`STAFF`) in `.env` and allow at least one server by default (`DEFAULT_ALLOWED_SERVERS`). Then, invite the bot to the server. + +In all examples below, `@Bot` refers to a user mention of the bot user. Make sure to run the commands on a server with the bot, in a channel that the bot can see. + +In the command parameters, `` refers to a required parameter (don't include the `< >` symbols) and `[this]` refers to an optional parameter (don't include the `[ ]` symbols). `` refers to being able to list multiple values, e.g. `value1 value2 value3`. + +## Allow a server to invite the bot +Run the following command: +``` +@Bot allow_server [userId] +``` +When specifying a user ID, that user will be given "Bot manager" level access to the server's dashboard, allowing them to manage access for other users. + +## Disallow a server +Run the following command: +``` +@Bot disallow_server +``` + +## Grant access to a server's dashboard +Run the following command: +``` +@Bot add_dashboard_user +``` + +## Remove access to a server's dashboard +Run the following command: +``` +@Bot remove_dashboard_user +``` diff --git a/docs/MIGRATE_DEV.md b/docs/MIGRATE_DEV.md new file mode 100644 index 000000000..2518088cf --- /dev/null +++ b/docs/MIGRATE_DEV.md @@ -0,0 +1,10 @@ +# Migrating from a version before 30 Mar 2024 +Zeppelin's development environment was restructured on 30 Mar 2024. Here's a list of changes to keep in mind when updating to the new version: +* Env variables in `backend/bot.env` and `backend/api.env` have been consolidated into `.env` at the root directory + * It is recommended to create a fresh `.env` file based on `.env.example` +* MySQL data is no longer symlinked to `docker/development/data`. This means that when you start the dev env for the first time, the database will also be created fresh. + * The data is now saved to a named Docker volume instead + * If you need to move over the old data, check the `volumes` section of the `mysql` service in [docker-compose.development.yml](../docker-compose.development.yml) for instructions. +* The recommended dashboard watch command has changed from `npm run watch-build` to `npm run watch` +* If you had made changes to the home folder of the devenv (i.e. `/home/ubuntu` inside the `devenv` container), e.g. by adding SSH keys to `.ssh`, these will need to be re-applied + * For SSH specifically, it is recommended to use SSH agent forwarding rather than copying key files directly to the container. VS Code and Jetbrains Gateway handle this for you automatically. diff --git a/docs/MIGRATE_PROD.md b/docs/MIGRATE_PROD.md new file mode 100644 index 000000000..5a7dde8cb --- /dev/null +++ b/docs/MIGRATE_PROD.md @@ -0,0 +1,11 @@ +# Migrating from a version before 30 Mar 2024 +Zeppelin's production environment was restructured on 30 Mar 2024. Here's a list of changes to keep in mind when updating to the new version: +* The docker compose file for the production environment is now called `docker-compose.standalone.yml`. There is also a `docker-compose.lightweight.yml` file for different use cases, see [PRODUCTION.md](PRODUCTION.md) for details. +* Env variables in `backend/bot.env` and `backend/api.env` have been consolidated into `.env` at the root directory + * It is recommended to create a fresh `.env` file based on `.env.example` +* MySQL data is no longer symlinked to `docker/production/data`. This means that when you start the bot for the first time, the database will also be created fresh. + * To migrate your data, connect to the database and import a database dump + * If you did not take a backup of your data before updating, check the `volumes` section of the `mysql` service in [docker-compose.production.yml](../docker-compose.production.yml) for instructions on loading the old data folder +* When the production Docker image is being built, files from the bot's folder are now *copied* rather than linked. This means that if you make changes to the files, you need to rebuild the services to see the changes. + +If you need help with any of these steps, please join us on the Zeppelin self-hosting community The Hangar at [https://discord.gg/uTcdUmF6Q7](https://discord.gg/uTcdUmF6Q7)! diff --git a/docs/PRODUCTION.md b/docs/PRODUCTION.md new file mode 100644 index 000000000..2cd95e0ad --- /dev/null +++ b/docs/PRODUCTION.md @@ -0,0 +1,71 @@ +# Zeppelin production environment + +⚠️ **Updating from before 30 Mar 2024? See [MIGRATE_PROD.md](./MIGRATE_PROD.md) for instructions.** + +Zeppelin's production environment uses Docker. There are a few different ways to run Zeppelin in production: + +1. **Standalone** + * The easiest way to get Zeppelin up and running. This setup comes with a built-in database and web server. +2. **Lightweight** + * In case you don't want to use the built-in database and web server. This setup only runs the bot, API, and dashboard themselves. You'll have to provide your own database connection options and set up a proxy server for the API and dashboard. +3. **Manual** + * If you only want to run a specific service, you can use Zeppelin's Dockerfile directly. + +## Standalone + +### Setup +1. Install Docker on the machine running the bot +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` (including the "PRODUCTION - STANDALONE" section) + +**Note:** The dashboard and API are exposed with a self-signed certificate. It is recommended to set up a proxy with a proper certificate in front of them. A popular option for this is [Cloudflare Tunnel](https://www.cloudflare.com/products/tunnel/). + +### Running the bot +`docker compose -f docker-compose.standalone.yml up -d` + +### Shutting the bot down +`docker compose -f docker-compose.standalone.yml down` + +### Updating the bot +1. Shut the bot down +2. Update the files (e.g. `git pull`) +3. Rebuild: `docker compose -f docker-compose.standalone.yml build` +4. Run the bot again + +### Viewing logs +`docker compose -f docker-compose.standalone.yml logs -t -f` + +## Lightweight + +### Setup +1. Install Docker on the machine running the bot +2. Make a copy of `.env.example` called `.env` +3. Fill in the missing values in `.env` (including the "PRODUCTION - LIGHTWEIGHT" section) + +### Running the bot +`docker compose -f docker-compose.lightweight.yml up -d` + +### Shutting the bot down +`docker compose -f docker-compose.lightweight.yml down` + +### Updating the bot +1. Shut the bot down +2. Update the files (e.g. `git pull`) +3. Rebuild: `docker compose -f docker-compose.lightweight.yml build` +4. Run the bot again + +### Viewing logs +`docker compose -f docker-compose.lightweight.yml logs -t -f` + +## Manual +1. Build the Zeppelin image: `docker build --tag 'zeppelin' .` +2. Run the service: + * Bot: `docker run zeppelin npm run start-bot` + * API: `docker run zeppelin npm run start-api` + * Dashboard: `docker run zeppelin npm run start-dashboard` + +If you're using an application platform such as Railway, you can simply point it to Zeppelin's repository and it should pick up the Dockerfile from there. +For the start command, you can use the same commands as above: `npm run start-bot`, `npm run start-api`, `npm run start-dashboard`. +Make sure to also run migrations when you update the bot. + +**Note:** You'll need to provide the necessary env variables. For example, `docker run --env-file .env zeppelin`