Skip to content

Commit

Permalink
Add stuff to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
galenlynch committed Nov 3, 2024
1 parent 8afae0d commit 313b39e
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 2 deletions.
63 changes: 61 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,71 @@
![Coverage](https://img.shields.io/badge/coverage-97%25-brightgreen?logo=codecov)
![Python](https://img.shields.io/badge/python->=3.10-blue?logo=python)

This project is used to encode behavior videos and, depending on the user
request, will compress those videos while ensuring that they are formatted for
correct display across many devices. This may include performing common image
pre-processing steps that are common across many commercial video products, but
have to be done post-hoc for our behavior videos.

## Goals

This will attempt to compress videos so that the results:

* Retain the majority of the detail of the input video
* Take up as little space as possible for a target amount of visual detail
* Are in a format that can be widely viewed across devices, and streamed by
browsers
* Have pixel data in a color space that most players can properly display

This video compression is often lossy, and the original videos are not kept, so
this library will attempt to perform the highest-quality video compression once.
The _speed_ of this compression is strictly secondary to the _quality_ of the
compression, as measured by the visual detail retained and the compression
ratio. See
[this section](#brief-benchmarks-on-video-compression-with-cpu-based-encoders-and-gpu-based-encoders)
for more details.


Additionally, this package should provide an easy to use interface that:

* Presents users with a curated set of compression settings, which have been
rigorously tested
* Allow users to also provide their own compression settings, if they have
specific requirements

## Non-goals

* Spending as little time and compute resources as possible to transform videos,
at the expense of compression quality.

## Usage
- The BehaviorVideoJob.run_job method in the transform_videos should be the primary method to call for processing video files.
- On a merge to main, this package will be published as a singularity container, which can easily be run on a SLURM cluster.

## Brief benchmarks on video compression with CPU-based encoders and GPU-based encoders

A perhaps surprising fact is that video encoders implementing the same algorithm
written for different compute resources
_do not have the same visual performance_, in that for a given compression ratio
they do not retain the same amount of visual detail.

This can be seen in the plot below, where the GPU encoder and CPU encoders
retain different amounts of visual detail, as assessed with VMAF.

![visual performance vs compress ratio](/assets/compression-vs-quality.png)

This also shows that different presets of the same encoder, like _CPU Slow_ and
_CPU Fast_, also have different performance. For comrpession ratios greater than
100, it often makes sense to take your time and use a slow preset of a CPU-based
encoder to retain as much visual information for a given amount of compression.

While it may be tempting to select a faster preset, or faster compute resource
like GPU for dramatic speedups shown below, the lossy compression and intent to
delete the original means that taking time to do it right once might well be
worth it.

![throughput vs compress ratio](/assets/compression-vs-speed.png)

## Development

To develop the code, run
Expand Down Expand Up @@ -84,10 +143,10 @@ The table below, from [semantic release](https://github.com/semantic-release/sem
### Documentation
To generate the rst files source files for documentation, run
```bash
sphinx-apidoc -o doc_template/source/ src
sphinx-apidoc -o doc_template/source/ src
```
Then to create the documentation HTML files, run
```bash
sphinx-build -b html doc_template/source/ doc_template/build/html
```
More info on sphinx installation can be found [here](https://www.sphinx-doc.org/en/master/usage/installation.html).
More info on sphinx installation can be found [here](https://www.sphinx-doc.org/en/master/usage/installation.html).
Binary file added assets/compression-vs-quality.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/compression-vs-speed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 313b39e

Please sign in to comment.