dis-wagtail

The Wagtail CMS for managing and publishing content for the Office for National Statistics (ONS)

---

Table of Contents

• Getting Started
  • Pre-requisites
  • Setup
    • Using Docker
• Development
  • Front-end tooling
  • Adding Python packages
  • Run Tests with Coverage
  • Linting and Formatting
    • Python
    • Front-end
    • pre-commit
    • Megalinter
  • Django Migrations
• Contributing
• License

For further developer documentation see docs

Getting Started

To get a local copy up and running, follow the steps below.

Pre-requisites

Ensure you have the following installed:

1. Python: Version specified in .python-version. We recommend using pyenv for managing Python versions.
2. Poetry: This is used to manage package dependencies and virtual environments.
3. Colima for running the project in Docker containers.
4. PostgreSQL for the database. Provided as container via docker-compose.yml when using the Docker setup.
5. Node and nvm (Node Version Manager) for front-end tooling.
6. Operation System: Ubuntu/MacOS

Setup

1. Clone the repository

   git clone https://github.com/ONSdigital/dis-wagtail.git

2. Install dependencies

   Poetry is used to manage dependencies in this project. For more information, read the Poetry documentation.

   To install all dependencies, including development dependencies, run:

   make install-dev

   To install only production dependencies, run:

   make install

Using Docker

Follow these steps to set up and run the project using Docker.

1. Build and Start the Container

   # build the container
   make docker-build
   # start the container
   make docker-start

2. Migrations and Superuser Creation

   If this is your first time setting up the project, you’ll need to run migrations to set up the database schema and create an administrative user. Note: dj is an alias for django-admin

   # ssh into the web container
   make docker-shell
   
   # Run database migrations
   dj migrate
   
   # Create a superuser for accessing the admin interface
   dj createsuperuser

3. Start Django Inside the Container

   Once the containers are running, you need to manually start Django from within the web container. This allows for running both the Django server and any additional background services (e.g., schedulers).

   # Start both Django and the scheduler using Honcho
   honcho start
   
   # To run just the web server. alias: djrun
   # This is not needed if you used `honcho start`
   dj runserver 0.0.0.0:8000

You can then access the admin at http://0.0.0.0:8000/admin/ or http://localhost:8000/admin/.

Development

Get started with development by running the following commands. Before proceeding, make sure you have the development dependencies installed using the make install-dev command.

A Makefile is provided to simplify common development tasks. To view all available commands, run:

make

Front-end tooling

While the end goal is to have all front-end elements in the Design System, the new design introduces a number of components that we need to build and contributed to the DS. In order to aid development and avoid being blocked by the DS, we will use modern front-end tooling for that.

Here are the common commands:

# Install front-end dependencies.
npm install
# Start the Webpack build in watch mode, without live-reload.
npm run start
# Start the Webpack server build on port 3000 only with live-reload.
npm run start:reload
# Do a one-off Webpack development build.
npm run build
# Do a one-off Webpack production build.
npm run build:prod

Adding Python packages

Python packages can be installed using poetry in the web container:

make docker-shell
poetry add wagtailmedia

Run Tests with Coverage

To run the tests and check coverage, run:

make test

During tests, the cms.settings.test settings module is used. When running test without using make test, ensure this settings module is used.

Linting and Formatting

Various tools are used to lint and format the code in this project.

Python

The project uses Ruff and pylint for linting and formatting of the Python code.

The tools are configured using the pyproject.toml file.

To lint the Python code, run:

make lint

To auto-format the Python code, and correct fixable linting issues, run:

make format

Front-end

# lint and format custom CSS/JS
npm run lint
# only CSS
npm run lint:css
# only JS
npm run lint:js
# check css, js, markdown and yaml formatting
npm run lint:format
# format
npm run format

pre-commit

Note that this project has configuration for pre-commit. To set up locally:

# if you don't have it yet, globally
pip install pre-commit

# in the project directory, initialize pre-commit
pre-commit install

# Optional, run all checks once for this, then the checks will run only on the changed files
pre-commit run --all-files

The detect-secrets pre-commit hook requires a baseline secrets file to be included. If you need to,
you can update this file, e.g. when adding dummy secrets for unit tests:

poetry run detect-secrets scan > .secrets.baseline

MegaLinter (Lint/Format non-python files)

MegaLinter is utilised to lint the non-python files in the project. It offers a single interface to execute a suite of linters for multiple languages and formats, ensuring adherence to best practices and maintaining consistency across the repository without the need to install each linter individually.

MegaLinter examines various file types and tools, including GitHub Actions, Shell scripts, Dockerfile, etc. It is configured using the .mega-linter.yml file.

To run MegaLinter, ensure you have Docker installed on your system.

Note: The initial run may take some time to download the Docker image. However, subsequent executions will be considerably faster due to Docker caching. 🚀

To start the linter and automatically rectify fixable issues, run:

make megalint

Django Migrations

Wagtail is built on Django and changes to its models may require generating and running schema migrations. For full details see the Django documentation on migrations

Below are the commands you will most commonly use, note that these have to be run inside the container.

# Check if you need to generate any new migrations after changes to the model
poetry run django-admin makemigrations --check

# Generate migrations
poetry run django-admin makemigrations

# Apply migrations. Needed if new migrations have been generated (either by you, or via upstream code)
poetry run django-admin migrate

Contributing

See CONTRIBUTING.md for details.

License

See LICENSE for details.