Ulti Clips

A tool to annotate videos of ultimate from recorded speech, and then edit the videos to clips matching selected parameters.

For example, if the audio includes "turnover" whenever one happens, the user can later make a compilation of all the turnovers in their video library.

Current state

The MWP of using speech recordings and AI to split the video into peaces that are glued together with the excess removed from between points is working.

Getting Started

Dependencies

Using Conda for managing Python virtual environments is highly encouraged.

Conda documentation on installation.

Video editing

We use a Python interface to ffmpeg to process the video files.

ffmpeg download page

Processing audio files

WhisperX is used to convert audio files into text annotations.

Installing WhisperX

Mostly following the original WhisperX setup:

Tested for PyTorch 2.0, Python 3.10 (use other versions at your own risk!) GPU execution requires the NVIDIA libraries cuBLAS 11.x and cuDNN 8.x to be installed on the system. Please refer to the CTranslate2 documentation.

You don't need GPU execution. It does speed things up, but a 2015 laptop handles speech-to-text with the medium model in almost real time.

1. Create Python3.10 environment

conda create --name whisperx python=3.10

conda activate whisperx

2. Install PyTorch

Linux and Windows CUDA11.8:

conda install pytorch==2.0.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia

CPU only (laptop):

conda install pytorch==2.0.0 torchaudio==2.0.0 cpuonly -c pytorch

See other methods here.

3. Install the WhisperX project repo

pip install git+https://github.com/m-bain/whisperx.git

If already installed, update package to most recent commit

pip install git+https://github.com/m-bain/whisperx.git --upgrade

Running the speech-to-text conversion

• How to run the program
• Step-by-step bullets

code blocks for commands

Running the video processing

• How to run the program
• Step-by-step bullets

code blocks for commands

Authors

Teemu Säilynoja

License

This project is licensed under the [NAME HERE] License - see the LICENSE.md file for details