Skip to content

Latest commit

 

History

History
268 lines (200 loc) · 10.8 KB

README.md

File metadata and controls

268 lines (200 loc) · 10.8 KB
Raw

Audacity OpenVINO module build for Windows 🔨

Hi! The following is the process that we use when building the Audacity modules for Windows.

High-Level Overview

Before we get into the specifics, at a high-level we will be doing the following:

  • Cloning & building whisper.cpp with OpenVINO support (For transcription audacity module)
  • Cloning & building Audacity 3.4.2 without modifications (just to make sure 'vanilla' build works fine)
  • Adding our OpenVINO module src's to the Audacity source tree, and re-building it.

Dependencies

Here are some of the dependencies that you need to grab. If applicable, I'll also give the cmd's to set up your environment here.

  • CMake (https://cmake.org/download/)

  • Visual Studio (MS VS 2019 / 2022 Community Edition is fine)

  • python3 / pip - Audacity requires conan 2.0+ to be installed, and the recommended way to do that is through pip.

  • OpenVINO - You can use public version from here. Setup your cmd.exe shell environment by running setupvars.bat:

    call "C:\path\to\w_openvino_toolkit_windows_xxxx\setupvars.bat"

  • OpenVINO Tokenizers Extension - Download package from here. Make sure that you download the version that matches the version of OpenVINO that you are using. For example, we are using openvino_tokenizers_windows_2024.0.0.0_x86_64.zip Download the zip package, and copy the DLLs into your w_openvino_toolkit_windows_xxxx\runtime\bin\intel64\Release folder.

  • Libtorch (C++ distribution of pytorch)- This is a dependency for many of the pipelines that we ported from pytorch (musicgen, htdemucs, etc). We are currently using this version: libtorch-win-shared-with-deps-2.1.1+cpu.zip. After extracting the package, setup environment like this:

    set LIBTORCH_ROOTDIR=C:\path\to\libtorch-shared-with-deps-2.1.1+cpu\libtorch
set Path=%LIBTORCH_ROOTDIR%\lib;%Path%

  • OpenCL - To optimize performance for GPUs, we (lightly) use OpenCL with interoperability (i.e. remote tensor) APIs for OpenVINO. So, we need to download the OpenCL SDK. You can download this version: OpenCL-SDK-v2023.04.17-Win-x64.zip. After extracting the package, setup environment like this:

    set OCL_ROOT=C:\path\to\OpenCL-SDK-v2023.04.17-Win-x64
set Path=%OCL_ROOT%\bin;%Path%

    (Note -- 'OpenCL-SDK-v2023.04.17-Win-x64' folder pointed to by OCL_ROOT should be the one that contains 'bin', 'include', 'lib', etc. subdirectories.)

Sub-Component builds

We're now going to build whisper.cpp. You should have a cmd.exe (not powershell!) shell running, and environment setup for above dependencies. To recap:

:: OpenVINO
call "C:\path\to\w_openvino_toolkit_windows_xxxx\setupvars.bat"

:: OpenCV
set OpenCV_DIR=C:\path\to\opencv\build
set Path=%OpenCV_DIR%\x64\vc16\bin;%Path%

:: Libtorch
set LIBTORCH_ROOTDIR=C:\path\to\libtorch-shared-with-deps-2.1.1+cpu\libtorch
set Path=%LIBTORCH_ROOTDIR%\lib;%Path%

Whisper.cpp

:: Clone it  & check out specific v1.5.4 release tag.
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
git checkout v1.5.4
cd ..

:: Create build folder
mkdir whisper-build
cd whisper-build

:: Run CMake, specifying that you want to enable OpenVINO support.
:: Note: Replace visual studio version if needed
cmake ..\whisper.cpp -G "Visual Studio 16 2019" -A x64 -DWHISPER_OPENVINO=ON

:: Build it:
cmake --build . --config Release

:: Install built whisper collateral into a local 'installed' directory:
cmake --install . --config Release --prefix .\installed

With the build / install complete, the Audacity build will find the built collateral via the WHISPERCPP_ROOTDIR. So you can set it like this:

set WHISPERCPP_ROOTDIR=C:\path\to\whisper-build\installed
set Path=%WHISPERCPP_ROOTDIR%\bin;%Path%

(I'll remind you later about this though)

Audacity

Okay, moving on to actually building Audacity. Just a reminder, we're first going to just build Audacity without any modifications. Once that is done, we'll copy our openvino-module into the Audacity src tree, and built that.

Audacity initial (vanilla) build

:: For Audacity 3.4.0+, conan 2.0+ is required. Install it like this:
pip install conan

:: clone Audacity
git clone https://github.com/audacity/audacity.git

:: Check out Audacity-3.4.2 tag, as this is the specific version that
:: our modules are compatible with.
cd audacity
git checkout Audacity-3.4.2
cd ..

mkdir audacity-build
cd audacity-build

:: Run cmake (grab a coffee & a snack... this takes a while)
cmake ..\audacity -G "Visual Studio 16 2019" -A x64 -DAUDACITY_BUILD_LEVEL=2

:: build it 
cmake --build . --config Release

When this is done, you can run Audacity like this (from audacity-build directory):

Release\Audacity.exe

Audacity OpenVINO module build

Now we'll run through the steps to actually build the OpenVINO-based Audacity module.

First, clone this repo. This is of course where the actual Audacity module code lives.

:: clone it
git clone https://github.com/intel/openvino-plugins-ai-audacity.git

cd openvino-plugins-ai-audacity

:: Check out appropriate branch or tag 
git checkout v3.4.2-R1

We need to copy the mod-openvino folder into the Audacity source tree. i.e. Copy openvino-plugins-ai-audacity\mod-openvino folder to audacity\modules.

We now need to edit audacity\modules\CMakeLists.txt to add mod-openvino as a build target. You just need to add a add_subdirectory(mod-openvino) someplace in the file. For example:

...
foreach( MODULE ${MODULES} )
   add_subdirectory("${MODULE}")
endforeach()

#YOU CAN ADD IT HERE
add_subdirectory(mod-openvino)

if( NOT CMAKE_SYSTEM_NAME MATCHES "Darwin" )
   if( NOT "${CMAKE_GENERATOR}" MATCHES "Visual Studio*")
      install( DIRECTORY "${_DEST}/modules"
               DESTINATION "${_PKGLIB}" )
   endif()
endif()
...

Okay, now we're going to (finally) build the module. Here's a recap of the environment variables that you should have set:

:: OpenVINO
call "C:\path\to\w_openvino_toolkit_windows_xxxx\setupvars.bat"

:: Libtorch
set LIBTORCH_ROOTDIR=C:\path\to\libtorch-shared-with-deps-2.0.1+cpu\libtorch
set Path=%LIBTORCH_ROOTDIR%\lib;%Path%

:: Whisper.cpp 
set WHISPERCPP_ROOTDIR=C:\path\to\whisper-build\installed
set Path=%WHISPERCPP_ROOTDIR%\bin;%Path%

:: OpenCL
set OCL_ROOT=C:\path\to\OpenCL-SDK-v2023.04.17-Win-x64
set Path=%OCL_ROOT%\bin;%Path%

Okay, on to the build:

:: cd back to the same Audacity folder you used to build Audacity before
cd audacity-build

:: and build the new target, mod-openvino.
:: (Note: CMake will automatically re-run since you modified CMakeLists.txt)
cmake --build . --config Release --target mod-openvino

If it all builds correctly, you should see mod-openvino.dll sitting in audacity-build\bin\Release\modules.

You can go ahead and run audacity-build\bin\Release\Audacity.exe

Once Audacity is open, you need to go to Edit -> Preferences. And on the left side you'll see a Modules tab, click that. And here you (hopefully) see mod-openvino entry set to New. You need to change it to Enabled, as shown in the following picture.

Once you change to Enabled, close Audacity and re-open it. When it comes back up, you should now see the OpenVINO modules listed.

Installing the OpenVINO models

In order for the OpenVINO effects to work, you need to install the OpenVINO models. At runtime, the plugins will look for these models in a openvino-models directory.
Here are the commands that you can use (from cmd.exe) to create this directory, and populate it with the required models.

:: Create an empty 'openvino-models' directory to start with
mkdir openvino-models

:: Since many of these models will come from huggingdface repo's, let's make sure git lfs is installed
git lfs install

::************
::* MusicGen *
::************
mkdir openvino-models\musicgen

:: clone the HF repo
git clone https://huggingface.co/Intel/musicgen-static-openvino

:: unzip the 'base' set of models (like the EnCodec, tokenizer, etc.) into musicgen folder
tar -xf musicgen-static-openvino\musicgen_small_enc_dec_tok_openvino_models.zip -C openvino-models\musicgen

:: unzip the mono-specific set of models
tar -xf musicgen-static-openvino\musicgen_small_mono_openvino_models.zip -C openvino-models\musicgen

:: unzip the stereo-specific set of models
tar -xf musicgen-static-openvino\musicgen_small_stereo_openvino_models.zip -C openvino-models\musicgen

:: Now that the required models are extracted, feel free to delete the cloned 'musicgen-static-openvino' directory.
rmdir musicgen-static-openvino /s /q

::*************************
::* Whisper Transcription *
::*************************

:: clone the HF repo
git clone https://huggingface.co/Intel/whisper.cpp-openvino-models

:: Extract the individual model packages into openvino-models directory
tar -xf whisper.cpp-openvino-models\ggml-base-models.zip -C openvino-models
tar -xf whisper.cpp-openvino-models\ggml-small-models.zip -C openvino-models
tar -xf whisper.cpp-openvino-models\ggml-small.en-tdrz-models.zip -C openvino-models

:: Now that the required models are extracted, feel free to delete the cloned 'whisper.cpp-openvino-models' directory.
rmdir whisper.cpp-openvino-models /s /q

::********************
::* Music Separation *
::********************

:: clone the HF repo
git clone https://huggingface.co/Intel/demucs-openvino

:: Copy the demucs OpenVINO IR files
copy /y demucs-openvino\htdemucs_v4.bin openvino-models
copy /y demucs-openvino\htdemucs_v4.xml openvino-models

:: Now that the required models are extracted, feel free to delete the cloned 'demucs-openvino' directory.
rmdir demucs-openvino /s /q

::*********************
::* Noise Suppression *
::*********************

:: Clone the deepfilternet HF repo
git clone https://huggingface.co/Intel/deepfilternet-openvino

:: extract deepfilter2 models
tar -xf deepfilternet-openvino\deepfilternet2.zip -C openvino-models

:: extract deepfilter3 models
tar -xf deepfilternet-openvino\deepfilternet3.zip -C openvino-models

:: TODO: Add remaining models!

Now that you have generated the openvino-models directory, copy it to the same folder as Audacity.exe (e.g. audacity-build\bin\Release\).

Need Help? 🙋‍♂️

For any questions about this build procedure, feel free to submit an issue here