Skip to content

Commit

Permalink
Add reverse proxy (#63)
Browse files Browse the repository at this point in the history 
* Initial work for integrating caddy for reverse proxy

* Remove annotator image name

* Reverse proxy implementation forwarding 9001 -> 9000

* Remove the port from the host

* Testing. Changing all ports to 9000

* Add supervisord conf.

* Disable log rotation in supervisor.

* Expose port 80 on Cadfile.

* Caddyfile: point to annotator in localhost.

* Config port 9000 to caddy and 9001 to python app.

* Add caddyfile and supervisor.

* Update the multistage build process

* Correct local file copying in docker

* Add caddyfile and supervisor.

* Move the relevant docker connfiguration files into a directory

* Remove the caddy service as it's baked into the Dockerfile

* Change Dockerfile to use main branch

* Update README.md

* Keep 9000 port as the default for the main service.

* Keep 9000 port as the default for the main service.

* Configure supervisor logs.

* Configure supervisor logs.

* Revert the deployment port value

* Add the --port command to supervisor when starting the server

* Change ports exposed to be 9000

* Update the ports

* Add logs to Caddy server.

* Update config logs for python app in supervisor.

---------

Co-authored-by: jschaff <[email protected]>
Co-authored-by: Everaldo <[email protected]>
  • Loading branch information
@ctrl-schaff @everaldorodrigo
3 people authored Dec 2, 2024
1 parent 562033e commit fa6f375
Show file tree
Hide file tree
Showing 5 changed files with 119 additions and 204 deletions.
270 changes: 71 additions & 199 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ we've broken it out into it's own service
### Installation


##### environment setup
##### Environment Setup
```shell
git clone https://github.com/biothings/biothings_annotator
python3 -m venv biothings_annotator
Expand All @@ -22,23 +22,19 @@ pip install -e git+https://github.com/biothings/biothings_annotator#egg=biothing

### Structure
```shell
├── biothings_annotator
│   ├── annotator.py <- annotation logic
│   ├── application <- web service logic
│   │   ├── configuration
│   │   ├── views
│   │   ├── __init__.py
│   │   └── launcher.py
│   ├── biolink.py
│   ├── exceptions.py
│   ├── __init__.py
│   ├── __main__.py <- entrypoint
│   ├── transformer.py
│   └── utility.py
biothings_annotator
├── annotator <- annotation backend logic
└── application <- web service logic
├── cli
├── configuration
├── exceptions
├── middleware
└── views
```
`biothings_annotator` is the main module for the python package.
Separated between the annotator logic and web handler logic.
The annotator logic primarily exists within `annotator.py` and `transformer.py`. Whereas the web server application is defined entirely within `application` directory.

`biothings_annotator` is as a package is separated into the annotator logic and web handler logic.
The annotator logic primarily exists within `annotator/annotator.py` and `annotator/transformer.py`.
Whereas the web server application is defined entirely within `application` directory.


### Command-line Interface
Expand All @@ -49,6 +45,7 @@ installation run the following to command to start the annotator service:
python3 -m biothings_annotator
```

###### Command-line Implementation
The `__main__.py` file points to the application module where it will run the `main` function within
the launcher. The majority of our implementation exists within the
`biothings_annotator/application/cli` module. We store the main command line handling logic in
Expand All @@ -67,7 +64,7 @@ cli.parse()
cli.run()
```

The`attach` method aggregates all of the `ArgumentParser` instances stored in what `sanic`
The `attach` method aggregates all of the `ArgumentParser` instances stored in what `sanic`
defines as `Group` objects. In order to add our custom arguments we define our own implementations
of the `Group` argument parsers to be called during the `attach` method.

Expand Down Expand Up @@ -151,7 +148,16 @@ python3 -m biothings_annotator --host "172.84.29.248" --port 9384 --workers 12 -
```


### Docker
### Builds

```shell
docker
├── configuration
│   ├── Caddyfile
│   └── supervisord.conf
└── Dockerfile
```

We have a Dockerfile and service through docker-compose for the biothings-annotator service. The
Docker file lives in `~/docker/Dockerfile` and defines two build stages. The first pulls down the
repository and creates a wheel for the python package. There are two optional arguments for
Expand All @@ -167,200 +173,66 @@ and installs the wheels generated from the previous builder stage
The entrypoint is set as:

```docker
ENTRYPOINT ["/home/annotator/venv/bin/python", "-m", "biothings_annotator", "--conf=/build/annotator/application/configuration/sanic.json"]
ENTRYPOINT ["supervisord"]
CMD ["-c", "/etc/supervisor/supervisord.conf"]
```

Which will call the `__main__.py` entrypoint of the package itself. This should start the sanic web
service for hosting the annotation service. For configuration of the service itself, modify the
configuration found under `biothings_annotator/application/configuration/sanic.json`

This leverages supervisord to launch to different services. The first is the annotator web server

###### Commands
```shell
[program:python_app]
command=/home/annotator/venv/bin/python -m biothings_annotator --conf=/home/annotator/configuration/default.json
```

- docker build (default arguments)
This command will call the `__main__.py` entrypoint of the package itself. This should start the sanic web
service for hosting the annotation service. For configuration of the service itself, modify the
configuration found under `biothings_annotator/application/configuration/sanic.json`.

```shell
~/biothings_annotator$ docker build ./docker
[+] Building 28.0s (18/18) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.11kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.10-slim 0.6s
=> [internal] load metadata for docker.io/library/python:3.10 0.6s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [stage-1 1/9] FROM docker.io/library/python:3.10-slim@sha256:3b37199fbc5a730a551909b3efa7b29105c859668b7502451c163f2a4a7ae1ed 0.0s
=> [builder 1/4] FROM docker.io/library/python:3.10@sha256:506eee363017f0b9c7f06f4839e7db90d1001094882e8cff08c8261ba2e05be2 0.0s
=> [stage-1 2/9] RUN apt update -q -y && apt install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/* 10.4s
=> CACHED [builder 2/4] WORKDIR /build/annotator 0.0s
=> [builder 3/4] RUN git clone -b main --recursive https://github.com/biothings/biothings_annotator.git . 1.0s
=> [builder 4/4] RUN pip wheel --wheel-dir=/build/wheels /build/annotator 6.3s
=> [stage-1 3/9] RUN apt-get update -y && apt-get install vim -y && apt-get install sudo -y && apt-get install telnet -y 11.0s
=> [stage-1 4/9] RUN useradd -m annotator && usermod -aG sudo annotator 0.4s
=> [stage-1 5/9] RUN python -m venv /home/annotator/venv 2.9s
=> [stage-1 6/9] COPY --from=builder --chown=annotator:annotator /build/wheels /home/annotator/wheels 0.1s
=> [stage-1 7/9] RUN /home/annotator/venv/bin/pip install /home/annotator/wheels/*.whl && rm -rf /home/annotator/wheels 2.3s
=> [stage-1 8/9] COPY --from=builder --chown=annotator:annotator /build/annotator /home/annotator/biothings_annotator 0.0s
=> [stage-1 9/9] WORKDIR /home/annotator/ 0.0s
=> exporting to image 0.2s
=> => exporting layers 0.2s
=> => writing image sha256:11c4e8ad86724c8c223b7f07ade148c4701c43acff5d73fa26e06e139a230019 0.0s
```

- docker build (with arguments)
The second service launched is caddy. We use caddy in this case as a reverse proxy.

```shell
~/biothings_annotator$ docker build --build-arg ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git --build-arg ANNOTATOR_BRANCH=main ./docker/
[+] Building 25.9s (18/18) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.11kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.10-slim 0.3s
=> [internal] load metadata for docker.io/library/python:3.10 0.5s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [stage-1 1/9] FROM docker.io/library/python:3.10-slim@sha256:3b37199fbc5a730a551909b3efa7b29105c859668b7502451c163f2a4a7ae1ed 0.0s
=> [builder 1/4] FROM docker.io/library/python:3.10@sha256:506eee363017f0b9c7f06f4839e7db90d1001094882e8cff08c8261ba2e05be2 0.0s
=> CACHED [builder 2/4] WORKDIR /build/annotator 0.0s
=> [stage-1 2/9] RUN apt update -q -y && apt install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/* 8.7s
=> [builder 3/4] RUN git clone -b main --recursive https://github.com/biothings/biothings_annotator.git . 1.0s
=> [builder 4/4] RUN pip wheel --wheel-dir=/build/wheels /build/annotator 6.6s
=> [stage-1 3/9] RUN apt-get update -y && apt-get install vim -y && apt-get install sudo -y && apt-get install telnet -y 10.4s
=> [stage-1 4/9] RUN useradd -m annotator && usermod -aG sudo annotator 0.5s
=> [stage-1 5/9] RUN python -m venv /home/annotator/venv 2.9s
=> [stage-1 6/9] COPY --from=builder --chown=annotator:annotator /build/wheels /home/annotator/wheels 0.0s
=> [stage-1 7/9] RUN /home/annotator/venv/bin/pip install /home/annotator/wheels/*.whl && rm -rf /home/annotator/wheels 2.4s
=> [stage-1 8/9] COPY --from=builder --chown=annotator:annotator /build/annotator /home/annotator/biothings_annotator 0.1s
=> [stage-1 9/9] WORKDIR /home/annotator/ 0.1s
=> exporting to image 0.2s
=> => exporting layers 0.2s
=> => writing image sha256:dee96b398b55ba335494490eddfd7735e3fc6cced1ba971a7d2509a64809cc24 0.0s
[program:caddy]
command=caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
```

- docker compose build (same output as above)
The default port for the web server is `9001`, and the caddy proxy forwards to that upstream web server via port
`9000` for deployment purposes. The main purposes at the moment for caddy are for compression and
potential future usage as a load balancer

```shell
~/biothings_annotator$ docker compose build
~/biothings_annotator$ docker compose build --build-arg ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git --build-arg ANNOTATOR_BRANCH=main
```

- docker run
###### docker build

```shell
~/biothings_annotator$ docker run biothings_annotator-annotator

docker run biothings_annotator-annotator
INFO:sanic-application:global sanic configuration:
{
"network": {
"host": "0.0.0.0",
"port": 9999
},
"application": {
"settings": {}
}
}
INFO:sanic-application:generated sanic application from loader: <Sanic TEST-SANIC>
DEBUG:sanic.root:

Sanic
Build Fast. Run Fast.


INFO:sanic.root:Sanic v24.6.0
INFO:sanic.root:Goin' Fast @ http://127.0.0.1:9999
INFO:sanic.root:app: TEST-SANIC
Main 21:39:25 DEBUG:
Sanic
Build Fast. Run Fast.
Main 21:39:25 INFO: Sanic v24.6.0
Main 21:39:25 INFO: Goin' Fast @ http://127.0.0.1:9999
Main 21:39:25 INFO: app: TEST-SANIC
Main 21:39:25 INFO: mode: debug, single worker
Main 21:39:25 INFO: server: sanic, HTTP/1.1
INFO:sanic.root:mode: debug, single worker
INFO:sanic.root:server: sanic, HTTP/1.1
INFO:sanic.root:python: 3.10.14
INFO:sanic.root:platform: Linux-6.5.0-1025-oem-x86_64-with-glibc2.36
INFO:sanic.root:auto-reload: enabled
Main 21:39:25 INFO: python: 3.10.14
Main 21:39:25 INFO: platform: Linux-6.5.0-1025-oem-x86_64-with-glibc2.36
Main 21:39:25 INFO: auto-reload: enabled
Main 21:39:25 INFO: packages: sanic-routing==23.12.0
INFO:sanic.root:packages: sanic-routing==23.12.0
Main 21:39:25 DEBUG: Creating multiprocessing context using 'spawn'
DEBUG:sanic.root:Creating multiprocessing context using 'spawn'
Main 21:39:25 DEBUG: Starting a process: Sanic-Server-0-0
DEBUG:sanic.root:Starting a process: Sanic-Server-0-0
DEBUG:sanic.root:Starting a process: Sanic-Reloader-0
Main 21:39:25 DEBUG: Starting a process: Sanic-Reloader-0
DEBUG:sanic.root:Process ack: Sanic-Server-0-0 [15]
Srv 0 21:39:26 DEBUG: Process ack: Sanic-Server-0-0 [15]
Srv 0 21:39:26 INFO: Starting worker [15]
INFO:sanic.server:Starting worker [15]
```
- default
- `~/biothings_annotator$ docker build ./docker`
- `~/biothings_annotator$ docker build . -f docker/Dockerfile`

- docker compose up
- build arguments
- `~/biothings_annotator$ docker build --build-arg ANNOTATOR_BRANCH=main ./docker/`
- `~/biothings_annotator$ docker build --build-arg ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git ./docker/`
- `~/biothings_annotator$ docker build --build-arg ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git --build-arg ANNOTATOR_BRANCH=main ./docker/`

- tag
- `~/biothings_annotator$ docker build ./docker/ --tag=annotator`

- without caching
- `~/biothings_annotator$ docker build ./docker/ --no-cache`


###### docker compose build
- default
- `~/biothings_annotator$ docker compose build`

- build arguments
- `~/biothings_annotator$ docker compose build --build-arg ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git --build-arg ANNOTATOR_BRANCH=main ./docker/`


###### docker run
- `~/biothings_annotator$ docker run <annotator-image-name>`


###### docker compose up
- `~/biothings_annotator$ docker run <annotator-image-name>`

```shell
biothings) jschaff@tsri-ubuntu:~/workspace/biothings/biothings_annotator$ docker compose up
[+] Running 2/0
✔ Container biothings-annotator Recreated 0.0s
! annotator Published ports are discarded when using host network mode 0.0s
Attaching to biothings-annotator
biothings-annotator | INFO:sanic-application:global sanic configuration:
biothings-annotator | {
biothings-annotator | "network": {
biothings-annotator | "host": "0.0.0.0",
biothings-annotator | "port": 9999
biothings-annotator | },
biothings-annotator | "application": {
biothings-annotator | "settings": {}
biothings-annotator | }
biothings-annotator | }
biothings-annotator | INFO:sanic-application:generated sanic application from loader: <Sanic TEST-SANIC>
biothings-annotator | DEBUG:sanic.root:
biothings-annotator |
biothings-annotator | Sanic
biothings-annotator | Build Fast. Run Fast.
biothings-annotator |
biothings-annotator |
biothings-annotator | INFO:sanic.root:Sanic v24.6.0
biothings-annotator | INFO:sanic.root:Goin' Fast @ http://127.0.0.1:9999
biothings-annotator | INFO:sanic.root:app: TEST-SANIC
biothings-annotator | INFO:sanic.root:mode: debug, single worker
biothings-annotator | INFO:sanic.root:server: sanic, HTTP/1.1
biothings-annotator | INFO:sanic.root:python: 3.10.14
biothings-annotator | Main 21:41:22 DEBUG:
biothings-annotator |
biothings-annotator | Sanic
biothings-annotator | Build Fast. Run Fast.
biothings-annotator |
biothings-annotator |
biothings-annotator | Main 21:41:22 INFO: Sanic v24.6.0
biothings-annotator | Main 21:41:22 INFO: Goin' Fast @ http://127.0.0.1:9999
biothings-annotator | Main 21:41:22 INFO: app: TEST-SANIC
biothings-annotator | Main 21:41:22 INFO: mode: debug, single worker
biothings-annotator | Main 21:41:22 INFO: server: sanic, HTTP/1.1
biothings-annotator | Main 21:41:22 INFO: python: 3.10.14
biothings-annotator | Main 21:41:22 INFO: platform: Linux-6.5.0-1025-oem-x86_64-with-glibc2.36
biothings-annotator | Main 21:41:22 INFO: auto-reload: enabled
biothings-annotator | Main 21:41:22 INFO: packages: sanic-routing==23.12.0
biothings-annotator | INFO:sanic.root:platform: Linux-6.5.0-1025-oem-x86_64-with-glibc2.36
biothings-annotator | INFO:sanic.root:auto-reload: enabled
biothings-annotator | INFO:sanic.root:packages: sanic-routing==23.12.0
biothings-annotator | Main 21:41:22 DEBUG: Creating multiprocessing context using 'spawn'
biothings-annotator | DEBUG:sanic.root:Creating multiprocessing context using 'spawn'
biothings-annotator | Main 21:41:22 DEBUG: Starting a process: Sanic-Server-0-0
biothings-annotator | DEBUG:sanic.root:Starting a process: Sanic-Server-0-0
biothings-annotator | DEBUG:sanic.root:Starting a process: Sanic-Reloader-0
biothings-annotator | Main 21:41:22 DEBUG: Starting a process: Sanic-Reloader-0
biothings-annotator | Srv 0 21:41:23 DEBUG: Process ack: Sanic-Server-0-0 [15]
biothings-annotator | DEBUG:sanic.root:Process ack: Sanic-Server-0-0 [15]
biothings-annotator | INFO:sanic.server:Starting worker [15]
biothings-annotator | Srv 0 21:41:23 INFO: Starting worker [15]
```


### Tests
Expand Down
5 changes: 3 additions & 2 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
name: biothings-annotator
services:
annotator:
build:
Expand All @@ -7,6 +8,6 @@ services:
- ARG ANNOTATOR_REPO=https://github.com/biothings/biothings_annotator.git
- ARG ANNOTATOR_BRANCH=main
container_name: biothings-annotator
network_mode: "host"
ports:
- 9999:9999
- 9000:9000

Loading

0 comments on commit fa6f375

Please sign in to comment.