From 2ab575b84426df5aff0b2c1299070178815f452c Mon Sep 17 00:00:00 2001 From: Jon Van Oast Date: Mon, 19 Aug 2024 12:02:52 -0600 Subject: [PATCH 1/3] docs WIP --- README.md | 2 +- devops/README.md | 55 ++++++++++++++++++++++++++++++------------------ 2 files changed, 35 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index e3130363fe..3bdafd1b26 100755 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ You will want to work in a branch when doing any feature development you want to ` git pull upstream main` ### Set Up Development Environment with Docker -For easiest development, you will need to set up your development environment to work with Docker. See `devops/development/README.md` for detailed instructions. +For easiest development, you will need to set up your development environment to work with Docker. See [`devops/README.md`](devops/README.md) for detailed instructions. ### Deploy frontend To setup frontend, we need to deploy the React build to Wildbook, please follow the detailed instructions provided in the `frontend/README.md` file within the project directory. diff --git a/devops/README.md b/devops/README.md index 04f383eea0..2299561b72 100644 --- a/devops/README.md +++ b/devops/README.md @@ -1,45 +1,58 @@ # Wildbook Installation -We provide two different containerized Wildbooks using Docker. Unless you are setting up a production instance of Wildbook, use the `development` image. Wildbook can also be set up as a local tomcat application. +We provide two different containerized Wildbooks using Docker. Unless you are setting up a production instance of Wildbook, use the **Development Docker** option. +Wildbook can also be set up as a local tomcat application. [do we want to mention/encourage this local tomcat? would need its own set of docs -- setting up psql and opensearch manually etc] -## Development Docker Image +## Development Docker -This image and all support materials are found in the `development/` subdirectory. Run docker images necessary to launch _wildbook.war_ file developed via java/maven +All support materials are found in the `devops/development/` subdirectory. This will launch docker containers sufficient for you to deploy a `wildbook.war` file which you have developed +via your own Java environment. -This will launch docker containers sufficient for you to deploy a `wildbook.war` file which you have developed -via your own Java environment. +### Overview of docker containers deployed -### WBIA / ML - -Presently, this deployment does not start a local WBIA (image analysis) docker container. This feature will be added in future development. +- **db** - postgresql database for storing Wildbook data +- **wildbook** -- the tomcat (java server) container which runs Wildbook +- **opensearch** -- runs OpenSearch to support searching in Wildbook +- **smtp** -- handles outgoing email (for password reset etc.) It is beyond the scope of this current document to address setting up email relays on the open internet. +Environment variables for this are set in `.env` file - see below. +- ~~**wbia**~~ - Presently, this deployment _does not_ start a local WBIA (image analysis) docker container. This feature will be added in future development. ### Setup and running -1. `sudo sysctl -w vm.max_map_count=262144` (this only needs to be run once on your system) -1. In this folder, create a `.env` file and copy the contents of `_env.template` to it. By default, no changes should be needed, but you can edit this new file. -1. In your terminal, create your base directory (value of `WILDBOOK_BASE_DIR`) and the required subdirectories. The default is `~/wildbook-dev/`): +1. Run `sudo sysctl -w vm.max_map_count=262144` (A requirement for OpenSearch, it only needs to be run once on your system.) +1. In `devops/development/` folder, create a `.env` file with a copy the contents of `_env.template`. By default, no changes should be needed, but you can edit this new file if needed. +1. In your terminal, create your base directory (value of `WILDBOOK_BASE_DIR` from `.env` file above) and the required subdirectories. The default is `~/wildbook-dev/`). For example: ``` mkdir -p ~/wildbook-dev/webapps/wildbook mkdir ~/wildbook-dev/logs ``` -1. deploy your `.war` file in the `wildbook/` directory, using `jar`: +1. Deploy your `.war` file (see section below) in the above `wildbook/` directory, using `jar`: ``` cd ~/wildbook-dev/webapps/wildbook jar -xvf /path/to/wildbook-xxx.war ``` -1. return to the `devops/development/` directory in the wildbook repo -1. run `docker-compose up [-d]`, which will launch latest postgresql and tomcat9 docker images -1. To verify successful launch, open in browser http://localhost:81/ when tomcat has started +1. Return to the `devops/development/` directory in the wildbook repo +1. Run `docker-compose up [-d]`, which will launch all of the aforementioned docker images +1. To verify successful launch, open in browser http://localhost:81/ when tomcat has started. Default login of username/password `tomcat`/`tomcat123` should work. -### When developing +### Development environment setup for compiling Wildbook -As you compile new war files, they can be deployed into the `wildbook` dir (as in step 3 above) and then tomcat restarted with: +To run Wildbook in the development docker environment, even to try out the software, you will need a "war file" which is made by compiling the Wildbook java project. +This requires some software to be set up on your development machine: -``` -docker-compose restart wildbook -``` +- Java JDK (`openjdk`) and `build-essential` linux package, as well as `maven` [probably a link to generic setup doc elsewhere?] +- node and npm for React build [likewise, link to generic help?], more details in [frontend/README.md](../frontend/README.md). + +#### Compiling + +Once the above requirements are met, the war file can be created by running `mvn clean install`. This will create the war file to be used in `target/wildbook-X.Y.Z.war` (with current version number). + +If you make code changes and compile new war files, they can be deployed into the `wildbook` dir (as in step 3 above) and then tomcat restarted with +`docker-compose restart wildbook`. + +--- -## Deploy Docker Image - DRAFT +## Deploy (e.g. Production) Docker Image - DRAFT This image and all support materials are found in the `deploy/` subdirectory. Run Wildbook and required docker images for production installations only. From 9bda4da1b4401dfe5e0d97a78c08481e62c49c9e Mon Sep 17 00:00:00 2001 From: Tanya <62726866+TanyaStere42@users.noreply.github.com> Date: Mon, 19 Aug 2024 14:31:00 -0700 Subject: [PATCH 2/3] roadmap updates removed references for WBIA docker as that has been removed from roadmap. removed local tomcat support reference as that has been removed from roadmap --- devops/README.md | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) diff --git a/devops/README.md b/devops/README.md index 2299561b72..91ca3b32d6 100644 --- a/devops/README.md +++ b/devops/README.md @@ -1,7 +1,6 @@ # Wildbook Installation We provide two different containerized Wildbooks using Docker. Unless you are setting up a production instance of Wildbook, use the **Development Docker** option. -Wildbook can also be set up as a local tomcat application. [do we want to mention/encourage this local tomcat? would need its own set of docs -- setting up psql and opensearch manually etc] ## Development Docker @@ -15,7 +14,7 @@ via your own Java environment. - **opensearch** -- runs OpenSearch to support searching in Wildbook - **smtp** -- handles outgoing email (for password reset etc.) It is beyond the scope of this current document to address setting up email relays on the open internet. Environment variables for this are set in `.env` file - see below. -- ~~**wbia**~~ - Presently, this deployment _does not_ start a local WBIA (image analysis) docker container. This feature will be added in future development. +- ~~**wbia**~~ - Presently, this _does not_ start a local WBIA (image analysis) docker container. The current roadmap is focused on removing WBIA as a requirement as we modernize our machine learning tech. ### Setup and running @@ -32,12 +31,12 @@ Environment variables for this are set in `.env` file - see below. jar -xvf /path/to/wildbook-xxx.war ``` 1. Return to the `devops/development/` directory in the wildbook repo -1. Run `docker-compose up [-d]`, which will launch all of the aforementioned docker images +1. Run `docker-compose up [-d]`, which launches all of the aforementioned docker images 1. To verify successful launch, open in browser http://localhost:81/ when tomcat has started. Default login of username/password `tomcat`/`tomcat123` should work. ### Development environment setup for compiling Wildbook -To run Wildbook in the development docker environment, even to try out the software, you will need a "war file" which is made by compiling the Wildbook java project. +To run Wildbook in the development docker environment, even to try out the software, you need a "war file" which is made by compiling the Wildbook java project. This requires some software to be set up on your development machine: - Java JDK (`openjdk`) and `build-essential` linux package, as well as `maven` [probably a link to generic setup doc elsewhere?] @@ -45,7 +44,7 @@ This requires some software to be set up on your development machine: #### Compiling -Once the above requirements are met, the war file can be created by running `mvn clean install`. This will create the war file to be used in `target/wildbook-X.Y.Z.war` (with current version number). +Once the above requirements are met, the war file can be created by running `mvn clean install`. This creates the war file to be used in `target/wildbook-X.Y.Z.war` (with current version number). If you make code changes and compile new war files, they can be deployed into the `wildbook` dir (as in step 3 above) and then tomcat restarted with `docker-compose restart wildbook`. @@ -58,13 +57,12 @@ This image and all support materials are found in the `deploy/` subdirectory. Ru **THIS IS CURRENTLY UNDER DEVELOPMENT - DRAFT ONLY** -This will launch an instance of Wildbook for the sake of testing or using in production. +This launches an instance of Wildbook for the sake of testing or using in production. It can be used to deploy on a VM/host on the internet or locally. ### Docker images used -The following docker containers should launch if started with `docker-compose [-d] up`. Some of these -are "optional" as noted below. +The following docker containers should launch if started with `docker-compose [-d] up`. Some of these are "optional" as noted below. - latest postegresql - tomcat9 / latest wildbook @@ -82,13 +80,5 @@ Currently, nginx is not configured to support ssl/https certs. There are some no - copy `_env.template` to `.env` and edit this new file with your own values - `docker-compose up ....` -## Local Tomcat -**TODO #503: draft local tomcat instructions** - -If you are running tomcat locally (not using docker), in order to access it as `http://localhost:8080/` (rather than with `/wildbook` trailing directory), -you should modify the `...` block of tomcat's `conf/server.xml` to contain the following: - -``` - - -``` +## Local tomcat +If you want to run Wildbook on non-dockerized tomcat, the system will likely build, but functionality will be restricted (i.e., search will not work) and additional functionality will likely break as we continue modernizing the stack. That being said, good luck! From b597c4e3b0d83330501a47e905a7acd800973523 Mon Sep 17 00:00:00 2001 From: Tanya <62726866+TanyaStere42@users.noreply.github.com> Date: Mon, 19 Aug 2024 14:34:44 -0700 Subject: [PATCH 3/3] tense shift minor language choice that I wanted to fix --- devops/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devops/README.md b/devops/README.md index 91ca3b32d6..1e43eb0595 100644 --- a/devops/README.md +++ b/devops/README.md @@ -4,7 +4,7 @@ We provide two different containerized Wildbooks using Docker. Unless you are se ## Development Docker -All support materials are found in the `devops/development/` subdirectory. This will launch docker containers sufficient for you to deploy a `wildbook.war` file which you have developed +All support materials are found in the `devops/development/` subdirectory. This launches docker containers sufficient for you to deploy a `wildbook.war` file which you have developed via your own Java environment. ### Overview of docker containers deployed