This project handles receiving images from the user and generating smaller versions. It is using docker to be able to run it either together with the API machine or on a separate machine.
The original image uploaded by the user is:
- optionally rotate the image according to the EXIF orientation value
- uniquely renamed using a timestamp and random number;
- stored locally in an "incoming" directory;
- converted to smaller sizes.
The image is uploaded immediately to S3. The user receives the renamed filename.
The user associates the filename to a document, which is stored in the API. At that time, a request is sent to image backend to move original and resized images from the incoming bucket to the public bucket. This step ensures the image is associated with an authenticated user.
Configuration should be set by environment variables:
STORAGE_BACKEND: (required) s3 or local
s3: requiresINCOMING_BUCKETandACTIVE_BUCKET, should be used in production.local: requiresINCOMING_FOLDERandACTIVE_FOLDER, should be used for tests and development.
TEMP_FOLDER: (required) Local folder to store images temporarily.
INCOMING_FOLDER: Local folder for incoming files.
ACTIVE_FOLDER: Local folder for active files.
INCOMING_BUCKET: Name bucket for incoming files.
INCOMING_PREFIX: Prefix of the incoming bucket connection options.
ACTIVE_BUCKET: Name bucket for active files.
ACTIVE_PREFIX: Prefix of the active bucket connection options.
PREFIX_ENDPOINT: Endpoint url for corresponding prefix.
PREFIX_ACCESS_KEY_ID: API key for corresponding prefix.
PREFIX_SECRET_KEY: Secret key for corresponding prefix.
PREFIX_DEFAULT_REGION: Default region for corresponding prefix.
S3_SIGNATURE_VERSION: S3 signature version ('s3' or 's3v4'), see docs.
API_SECRET_KEY: API secret key, needed to publish images on the active
bucket.
V5_DATABASE_URL: Address of the V5 database for the migration script.
ROUTE_PREFIX: Path prefix for serving the photo backend API.
RESIZING_CONFIG: Configuration of the thumbnail names and sizes serialized in JSON. See c2corg_images/init.py for a description of the format.
AUTO_ORIENT_ORIGINAL: 1 to rotate the uploaded image according to the EXIF orientation. Default is 0.
CACHE_CONTROL: Cache-Control value to be set to all the images uploaded to s3. Default is public, max-age=3600.
Here is an example configuration with S3 backend on exoscale:
STORAGE_BACKEND: s3
TEMP_FOLDER: /srv/images/temp
INCOMING_FOLDER:
ACTIVE_FOLDER:
INCOMING_BUCKET: c2corg_demov6_incoming
INCOMING_PREFIX: EXO
ACTIVE_BUCKET: c2corg_demov6_active
ACTIVE_PREFIX: EXO
EXO_ENDPOINT: https://sos.exo.io
EXO_ACCESS_KEY_ID: xxx
EXO_SECRET_KEY: xxx
API_SECRET_KEY: xxx
V5_DATABASE_URL: postgresql://www-data:www-data@postgres/c2corgThe files which were not activated are automatically expired by S3.
make run
The migration retrieves a list of images from the v5 database. The connection to the database can be defined as environment variable, for example:
V5_DATABASE_URL: postgresql://www-data:www-data@postgres/c2corgThe migration script iterates through v5 images. For each original image found:
- If the image already exists in publication bucket, nothing is done (only migrate the new ones).
- If the image does not exists in the v6 bucket:
- the original image is copied locally,
- resized images are produced according to configuration,
- original and resized images are pushed on publication bucket.
To run the migration script:
docker-compose exec images migrate
docker-compose exec images migrate --help to get options list.
This can be used to change the size or quality of resized images.
This script iterates through published images. For each original image found:
- the original image is copied locally,
- resized images are produced according to configuration,
- resized images are pushed in publication bucket, overwriting old ones.
To regenerate the resized images:
docker-compose exec images resize
docker-compose exec images resize --help to get options list.
The project is built by Github actions.