Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

House keeping #16

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
bb06f26
update frontend dependencies. Update eslint and tsconfig
jonasrenault Oct 7, 2024
7046ade
update backend dependencies
jonasrenault Oct 7, 2024
24b98ff
add ruff and mypy
jonasrenault Oct 7, 2024
3418a5b
fix some ruff issues (sorting)
jonasrenault Oct 7, 2024
00d1ab1
fix ruff issues
jonasrenault Oct 7, 2024
6064a24
update config
jonasrenault Oct 7, 2024
0e12549
fix mypy errors
jonasrenault Oct 7, 2024
5a8b752
fix mypy issues
jonasrenault Oct 7, 2024
2e92250
fix frontend lint issues
jonasrenault Oct 7, 2024
207a0e9
fix security issue with fast-api-sso
jonasrenault Oct 7, 2024
0ae898e
add pre-commit config file
jonasrenault Oct 7, 2024
e41ec3b
remove facebook sso
jonasrenault Oct 8, 2024
3378093
remove deprecated Grid component. Use Grid2 instead
jonasrenault Oct 8, 2024
b622162
fix tests by changing testing api url
jonasrenault Oct 8, 2024
c1c4961
update node and mongodb versions in tests workflow
jonasrenault Oct 8, 2024
2b9f34e
make black use pyproject.toml in github actions
jonasrenault Oct 8, 2024
44e5da6
Add config to black
jonasrenault Oct 8, 2024
f96f8cd
change working dir for python lint
jonasrenault Oct 8, 2024
cccefc7
refactor docker-compose setup. Update traefik to v3.2
jonasrenault Oct 9, 2024
186d2b8
add docker watch for frontend container
jonasrenault Oct 9, 2024
ba2ab48
add node_modules ignore to docker watch
jonasrenault Oct 9, 2024
f40474c
refactor config to use root level .env file
jonasrenault Oct 9, 2024
abd7522
update docker doc in README
jonasrenault Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 22 additions & 19 deletions .env
Original file line number Diff line number Diff line change
@@ -1,41 +1,44 @@
### Domain name for the app
DOMAIN=localhost

### Environment: development, test, production
ENVIRONMENT=development

### Project name and stack name
PROJECT_NAME=farm-docker
STACK_NAME=farm-docker

TRAEFIK_PUBLIC_NETWORK=traefik-public
TRAEFIK_TAG=farm-docker
TRAEFIK_PUBLIC_TAG=traefik-public
TRAEFIK_PUBLIC_NETWORK_IS_EXTERNAL=false
### Email adress used by traefik to obtain Let's Encrypt certificates
[email protected]

### Docker package registry and images
DOCKER_IMAGE_BACKEND=farmd-backend
DOCKER_IMAGE_FRONTEND=farmd-frontend
DOCKER_PACKAGE_REPOSITORY=ghcr.io/jonasrenault

# Backend
BACKEND_CORS_ORIGINS=["http://localhost", "http://localhost:5173", "http://localhost:3000", "http://localhost:8080", "https://localhost", "https://localhost:5173", "https://localhost:3000", "https://localhost:8080", "http://dev.farm-docker.com", "https://stag.farm-docker.com", "https://farm-docker.com"]
PROJECT_NAME=farm-docker
### Backend variables
BACKEND_CORS_ORIGINS=["http://localhost", "http://localhost:5173", "https://localhost", "https://localhost:5173"]
SECRET_KEY=98153798f1616ba9e65c2cbcdb3fd3e3a6297b2002f6936b72823fd21ce609d9
[email protected]
FIRST_SUPERUSER_PASSWORD=admin

### SSO variables
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
FACEBOOK_CLIENT_ID=
FACEBOOK_CLIENT_SECRET=
# HOSTNAME used to build google redirect uri. Should point to backend
SSO_CALLBACK_HOSTNAME=http://localhost
# Callback URL we redirect to after google login. Should point to frontend/sso-login-callback
SSO_LOGIN_CALLBACK_URL=http://localhost/sso-login-callback

# Frontend
### HOSTNAME used to build google redirect uri. Should point to backend
SSO_CALLBACK_HOSTNAME=http://localhost:8000
### Callback URL we redirect to after google login. Should point to frontend/sso-login-callback
SSO_LOGIN_CALLBACK_URL=http://localhost:5173/sso-login-callback

### Frontend variables
VITE_BACKEND_API_URL=http://localhost/api/v1/
VITE_PWD_SIGNUP_ENABLED=true
VITE_GA_TRACKING_ID=

# MongoDB
MONGO_HOST=db
### MongoDB variables
MONGO_HOST=localhost
MONGO_DB=farmd
MONGO_PORT=27017
MONGO_USER=mongodbadmin
MONGO_PASSWORD=wpnimdabdognom
# uncomment following lines to set mongodb user and password
# MONGO_USER=mongodbadmin
# MONGO_PASSWORD=wpnimdabdognom
12 changes: 7 additions & 5 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ on:
- main

env: # environment variables (available in any part of the action)
NODE_VERSION: 20
PYTHON_VERSION: 3.11
MONGODB_VERSION: 6.0
NODE_VERSION: 21
PYTHON_VERSION: 3.12
MONGODB_VERSION: 8.0

jobs:
run-js-linters:
Expand All @@ -33,7 +33,7 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
cache: "npm"
cache-dependency-path: frontend/package-lock.json

- name: Install Node.js dependencies
Expand All @@ -55,6 +55,8 @@ jobs:

- name: Run black
uses: psf/black@stable
with:
options: "--config backend/pyproject.toml"

test-backend:
name: Run backend unit tests
Expand All @@ -75,7 +77,7 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'poetry'
cache: "poetry"

- name: Start MongoDB
uses: supercharge/[email protected]
Expand Down
64 changes: 39 additions & 25 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,26 +1,40 @@
default_language_version:
python: python3.12

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: debug-statements
- repo: https://github.com/psf/black
rev: 23.10.1
hooks:
- id: black
# It is recommended to specify the latest version of Python
# supported by your project here, or alternatively use
# pre-commit's default_language_version, see
# https://pre-commit.com/#top_level-default_language_version
language_version: python3.11
- repo: https://github.com/python-poetry/poetry
rev: '1.5.1'
hooks:
- id: poetry-check
args: ["--directory", "backend"]
- id: poetry-lock
args: ["--directory", "backend"]
- id: poetry-export
args: ["-f", "requirements.txt", "-o", "backend/requirements.txt", "--directory", "backend", "--with", "dev"]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: check-added-large-files
- id: check-toml
- id: check-yaml
args: [--allow-multiple-documents]
- id: end-of-file-fixer
- id: trailing-whitespace

- repo: https://github.com/psf/black
rev: 24.8.0
hooks:
- id: black
name: black
args:
- "--config"
- "./backend/pyproject.toml"

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.9
hooks:
# Run the linter.
- id: ruff
# Run the formatter.
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.11.2
hooks:
- id: mypy
name: mypy
args:
- "--config"
- "./backend/pyproject.toml"
- "--ignore-missing-imports"
38 changes: 10 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ To run the application manually in a terminal, see both the [backend](backend/RE

## Running the application with Docker

The project contains Docker configuration files to run the application with Docker compose. Two docker-compose files are provided with configuration for `dev` and for `production` environments. The Docker configuration is largely adapted from Tiangolo's [Full stack FastAPI cookiecutter](https://github.com/tiangolo/full-stack-fastapi-postgresql) project.
The project contains Docker configuration files to run the application with Docker compose. Two docker-compose files are provided with configuration for `dev` and for `production` environments. The Docker configuration is largely adapted from Tiangolo's [Full stack FastAPI template](https://github.com/fastapi/full-stack-fastapi-template) project.

### Local development with Docker

Expand All @@ -39,16 +39,14 @@ The local development file for docker is [docker-compose.yml](./docker-compose.y
Start the stack with Docker Compose:

```bash
docker compose up -d --build
docker compose watch
```

The `--build` arg can be omitted after the images have been built at least once.

Now you can open your browser and interact with these URLs:
You can then open your browser and interact with these URLs:

* Frontend, served with vite with hot reload of code: http://localhost

* Backend, JSON based web API based on OpenAPI, with hot code reloading: http://localhost/api/
* Backend, JSON based web API based on OpenAPI, with hot code reloading: http://localhost/api/v1

* Automatic interactive documentation with Swagger UI (from the OpenAPI backend): http://localhost/docs

Expand All @@ -68,30 +66,12 @@ To check the logs of a specific service, add the name of the service, e.g.:
docker compose logs backend
```

### Docker Compose settings for development

When running the application with docker in development, both the frontend and backend directories are mounted as volumes to their corresponding docker containers to enable hot reload of code changes. This allows you to test your changes right away, without having to build the Docker image again. It should only be done during development, for production you should build the Docker image with a recent and stable version of the code.

For the backend, there is a command override that runs `/start-reload.sh` (included in the base image) instead of the default `/start.sh` (also included in the base image). It starts a single server process (instead of multiple, as would be for production) and reloads the process whenever the code changes. Have in mind that if you have a syntax error and save the Python file, it will break and exit, and the container will stop. After that, you can restart the container by fixing the error and running the `docker-compose up -d` command again. The backend [Dockerfile](backend/Dockerfile) is in the backend directory.

For the frontend, when in development, the frontend docker container starts with the `npm run dev -- --host` command, while in production the frontend app is built into static files and the app is served by an nginx server. The [nginx configuration file](frontend/nginx.conf) is in the frontend dir.

### Accessing the containers

To get inside a container with a `bash` session you can start the stack with:

```console
$ docker compose up -d
```

and then `exec` inside the running container:
To get access to a bash session inside a container (e.g. the `backend`):

```console
$ docker compose exec backend bash
```

This will give you access to a bash session in the `backend` container. Change the name of the container to the one you want to access.


### Docker Compose settings for production

Expand All @@ -103,13 +83,15 @@ docker compose -f docker-compose.prod.yml up -d

**Note:** This will not work out of the box, mainly because the `docker-compose-prod.yml` configures a traefik proxy with ssl enabled that will try to fetch ssl certificates from Let's Encrypt, which will not work unless you specify a valid hostname accessible on the internet. However, to deploy the application in production on a server, you only need to set the required env variables in the [.env](./.env) file.

When using the production configuration, the frontend app is built into static files and the app is served by an nginx server. The [nginx configuration file](frontend/nginx.conf) is in the frontend dir.

### Docker Compose files and env vars

Both the [docker-compose.yml](./docker-compose.yml) and [docker-compose-prod.yml](./docker-compose.prod.yml) files use the [.env](./.env) file containing configurations to be injected as environment variables in the containers.

The docker-compose files are designed to support several environments (i.e. development, building, testing, production) simply by setting the appropriate variable values in the `.env` file.
The docker-compose files are designed to support several environments (i.e. development, testing, production) simply by setting the appropriate variable values in the `.env` file.

The [.env](./.env) file contains all the configuration variables. The values set in the `.env` file will override those that are set in the frontend and backend `.env` files for local development. For exemple, the backend app also has a [.env.dev](backend/.env.dev) file which is read to populate the backend's [config](backend/app/config/config.py) module. When the application is run with docker though, the env variables in the projet root's [.env](./.env) file will override the env variables set in the backend and frontend's respective .env files. In order to be able to keep working both with docker and manually, you only have to make sure that the required variables are set both in the root `.env` file, and in the backend and frontend `.env` files.
The [.env](./.env) file contains all the configuration variables. The values set in the `.env` file will override those that are set in the frontend `.env` files for local development.

The `.env` file that is commited to the github repository contains example values which are ok to use for testing and development, but which should be changed when running the application in production (admin passwords, secret keys, client ids, etc.). During deployment in production, the .env file is replaced with one containing the appropriate values.

Expand All @@ -119,7 +101,7 @@ To setup SSO and enable the `Sign-In with Google` button, you must first obtain

Create a new project, and from the `APIs & Services` menu, first create an `OAuth consent screen` for you application, then add an `OAuth 2.0 Client Id` in the `Credentials` menu. Select `Web application` as the application type. In the `Authorized redirect URIs`, add your hostname with the `api/v1/login/google/callback` endpoint. For instance, if testing locally while running the backend app with `uvicorn`, add `http://localhost:8000/api/v1/login/google/callback` (use `http://localhost/api/v1/login/google/callback` if running the application in dev with docker). If your application is hosted on a domain name, add it to the list of URIs (remember to update `http` to `https` when using SSL).

Once you've create a client-id and token in your Google cloud console, copy those into your `.env` file's (either directly in the backend [.env](./backend/.env.dev) or in the root [.env](./.env) if using docker) `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` variables.
Once you've create a client-id and token in your Google cloud console, copy those into your `.env` file's `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` variables.

## Setting up automatic build of the docker images in github

Expand Down
10 changes: 10 additions & 0 deletions backend/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Python
__pycache__
app.egg-info
*.pyc
.mypy_cache
.pytest_cache
.ruff_cache
.coverage
htmlcov
.venv
25 changes: 0 additions & 25 deletions backend/.env.dev

This file was deleted.

2 changes: 2 additions & 0 deletions backend/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,5 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

.ruff_cache
27 changes: 14 additions & 13 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
FROM tiangolo/uvicorn-gunicorn-fastapi:latest
FROM python:3.12-slim

ENV PYTHONUNBUFFERED=true

WORKDIR /app

ENV POETRY_HOME=/opt/poetry
# ENV POETRY_VIRTUALENVS_IN_PROJECT=true
ENV PATH="$POETRY_HOME/bin:$PATH"
RUN curl -sSL https://install.python-poetry.org | python3 -
RUN poetry config virtualenvs.create false
# Copy poetry.lock* in case it doesn't exist in the repo
COPY ./pyproject.toml ./poetry.lock* /app/
### install poetry
RUN pip install poetry && poetry config virtualenvs.in-project true

### install dependencies and project
ADD pyproject.toml README.md ./
ADD app /app/app
RUN poetry install --no-interaction --no-ansi

# Allow installing dev dependencies to run tests
ARG INSTALL_DEV=false
RUN bash -c "if [ $INSTALL_DEV == 'true' ] ; then poetry install --no-root ; else poetry install --no-root --no-dev ; fi"
### add executables to path
ENV PATH="/app/.venv/bin:$PATH"

COPY . /app
ENV PYTHONPATH=/app
### default cmd: run fastapi with 4 workers
CMD ["fastapi", "run", "--workers", "4", "app/main.py"]
14 changes: 7 additions & 7 deletions backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,22 @@ This directory contains the backend API app. It is built with [FastAPI](https://

## Install

The project uses poetry to manage dependencies. You can use your own environment, or use poetry to manage the virtuel env and the project depencencies.
The project uses poetry to manage dependencies and run the backend application. You can use an other tool to manage your virtual environment, such as `pip` or [uv](https://docs.astral.sh/uv/), but you'll need to extract the dependencies from the [pyproject.toml](./pyproject.toml) file.

If using poetry, for convenience, run `poetry config virtualenvs.in-project true` before installing depencies. This will install the dependencies in the project directory and make it easier to manage in vscode.

You can then install the dependencies with `poetry install --with dev`.

To activate the virtual env, use `poetry shell`.

## Running the server

Run this command to start a development server :
To start a development server, run

```console
uvicorn app.main:app --reload
poetry run fastapi dev app/main.py
```

from the `backend` directory (remove the `poetry run` prefix if using another dependency management tool).

This will start a server running on `http://127.0.0.1:8000`. The API will be available on the API's base prefix, which by default is `/api/v1`.

Navigate to `http://localhost:8000/api/v1/` to access the root API path.
Expand All @@ -36,10 +36,10 @@ Navigate to `http://localhost:8000/redoc` to access the API's alternative doc bu
Run

```console
pytest
poetry run pytest
```

to run the unit test for the backend app.
to run the unit tests for the backend app.

## Configuration

Expand Down
Loading
Loading