Ploutos is a Django based web interface to track an organisation's spending on DNAnexus. It gives information on:
- Running Totals - Daily and monthly billing charges calculated according to the running total DNANexus provides.
- Storage Cost - An estimated cost of storage per month (live vs archived) across all projects (default) while allowing you to view this on a more granular level by searching for projects that start or end with a certain string or on a single project level.
- File Type Storage - View the size and counts of files by file type (BAM, FASTQ and VCF) across all projects or similarly on a more granular level. Also displays the total size of files in DNAnexus for the current day in TiB delineated by their state (live vs archived).
- Compute Costs - Daily and monthly charges for running apps and workflows on DNAnexus. This can be filtered by project name (startswith) and assay type (project name endswith). It can also be filtered by timeperiod (i.e. July 2021 - Jan 2022).
- Leaderboard - Monthly and daily compute costs by users. This can be filtered by project name (startswith) and username (endswith). It can also be filtered by timeperiod (i.e. July 2021 - Jan 2022).
Built with:
The required Python package dependencies to query the API, populate the database and view the dashboard can be installed via:
pip install -r requirements.txt
A config file is required. An example env is in the repo
DB_USERNAME - mysql database username
DB_PASSWORD - mysql database password
MYSQL_ROOT_PASSWORD - (optional) depending on your db setup
MYSQL_ROOT_HOST - (optional) depending on your db setup
HOSTS - depending on your db setup
DB_PORT - default port 3306
DB_NAME - database name
SECRET_KEY - django [secret key](https://www.educative.io/answers/how-to-generate-a-django-secretkey)
DNANEXUS_TOKEN - dnanexus auth token
ORG - dnanexus organization
LIVE_STORAGE_COST_MONTH - storage cost (DNANexus)
ARCHIVED_STORAGE_COST_MONTH - archived storage cost (DNANexus)
PLOUTOS_DEBUG - debug mode (default False)
PROJ_COLOUR_DICT=PROJ1_rgb(00,00,00)|PROJ2_rgb(00,00,00)
ASSAY_COLOUR_DICT=ASSAY_rgb(00,00,00)|ASSAY2_rgb(00,00,00)
LIVE_STORAGE_COST_MONTH & ARCHIVED_STORAGE_COST_MONTH are the costs of storage as determined in the org's contract with DNAnexus
PROJ_COLOUR_DICT & ASSAY_COLOUR_DICT set the desired colours for the plots.
To migrate the Django models to the Ploutos database:
python manage.py makemigrations
python manage.py migrate
There is cron installed in the web container to run cron schedule. Crontab is copied to the container automatically when building the docker image. To edit the cron, access the web container and use command crontab -e. To start the cron for populating the database daily:
# access the web container
docker exec -it <container id> bash
service cron start
# comment out a line in /etc/pam.d/cron using vim
vim /etc/pam.d/cron
# comment out **session required module pam_loginuid.so**
The script populate_db.py is run daily to grab data from the DNAnexus API and populate the database. This can be run manually:
python manage.py runscript populate_db
The server can be run with python manage.py runserver.
