Running the Aivideo Core Product on Two Server

Overview

This documentation describes the setup of running our aivideo application across two servers with shared file storage using Network File System (NFS). The application is distributed between Server 1 and Server 2:

Server 1: Hosts the main application, RabbitMQ, PostgreSQL, Flower to monitor the celery jobs. Server 2: Hosts a Celery worker for the jobs and FFmpeg itself. It uses NFS to access files uploaded on Server 1.

Server Configuration

Server 1 Setup

Services Running:

Main Application RabbitMQ PostgreSQL Flower Steps to Configure Server 1:

1. Install Required Software:

Ensure your application, RabbitMQ, PostgreSQL, and Flower are installed and configured.

2. Configure PostgreSQL to Allow Remote Connections: Edit the PostgreSQL configuration file (/etc/postgresql//main/postgresql.conf) and set listen_addresses to '*'. Update the pg_hba.conf file to allow connections from Server 2.

host    all             all             <Server 2 IP>/32            md5

b. Restart PostgreSQL:

sudo systemctl restart postgresql

3. Set Up NFS Server:

sudo apt update
sudo apt-get install nfs-kernel-server

b. Configure the NFS exports by editing /etc/exports:

/home/teampython/aivideo_be/media <Server 2 IP>(rw,sync,no_subtree_check)

c. Export the shares:

sudo exportfs -a

d. Restart the NFS service:

sudo systemctl restart nfs-kernel-server

Server 2 Setup

Services Running:

Celery Worker FFmpeg

Steps to Configure Server 2:

1. Install Required Software:

Install FFmpeg and Celery:

sudo apt update
sudo apt install ffmpeg
pip install celery

2. Set Up NFS Client:

a. Install NFS client utilities:

sudo apt install nfs-common

b. Create a mount point:

sudo mkdir -p /mnt/aivideo_be/media

c. Mount the NFS share:

sudo mount <Server 1 IP>:/home/teampython/aivideo_be/media /mnt/aivideo_be/media

d. Verify the mount:

df -h

e. Edit /etc/fstab on Server 2:

<Server 1 IP>:/home/teampython/aivideo_be/uploads /mnt/aivideo_be/uploads nfs defaults 0 0

3. Configure Celery with PM2

Note: Ensure the Celery configuration points to the appropriate broker and backend.