ThemisIO

ThemisIO is a first of its kind software-defined I/O system for supercomputers. It enables policy-driven I/O capacity sharing on supercomputers. At its core, ThemisIO disassociates I/O control (i.e., I/O request processing order) from processing by incorporating job metadata such as user, job id, and job size (i.e., node count) ThemisIO can precisely balance the I/O cycles between applications via time slicing to enforce processing isolation, enabling a variety of fair sharing policies. ThemisIO can precisely allocate I/O resources to jobs so that every job gets at least its fair share of the I/O capacity as defined by the sharing policy. ThemisIO can decrease the slowdown of real applications due to I/O interference by two to three orders of magnitude when using fair sharing polices compared to the first-in-first-out (FIFO) baseline.

Fig.1 - Size-fair, 4-node job competing with 1-node job

Fig.2 - Job-fair, 4-node job competing with 1-node job

Fig.3 - User-fair, Two 2-node jobs competing with a 1-node job

Benchmark

Fig.4 - IOR benchmark for one server

Fig.5 - IOR benchmark for multiple servers

Build Instruction

Compile server and wrapper.so,
git clone https://github.com/bbThemis/ThemisIO
cd ThemisIO
mkdir obj
make
cd src/client
./compile.sh

You need to revise the impi path in Makefile.

Run a server,
cd ThemisIO
./server

Run on client side
export MYFS_CONF="/full_path/ThemisIO/myfs.param"
export LD_PRELOAD="/full_path/ThemisIO/wrapper.so"

ls -l /myfs
touch /myfs/a
ls -l /myfs

There are still many bugs. You can "unset LD_PRELOAD" whenever you get issues. I normally add "LD_PRELOAD=xxxx" before the command I need to test.

Run Client within a container

Assume you are on a system with the server running already (see above for instructions of launching a server), you may run the client within our pre-built container. To pull the container with Docker, run:

docker pull ghcr.io/bbthemis/themisio:client

This can also be done with other container runtimes, for example the Apptainer:

apptainer pull themisio.sif docker://ghcr.io/bbthemis/themisio:client

Then, you may run a shell within the container (using apptainer as example here):

apptainer run themisio.sif bash

You can then access the filesystem with:

export MYFS_CONF="/full_path/ThemisIO/myfs.param"
export LD_PRELOAD="/ThemisIO-client/wrapper.so"
ls -l /myfs
touch /myfs/a
ls -l /myfs

You may also build a new container using our client container as the base to run any applications with ThemisIO support.