MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX delivers highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions along with Convolution Neural Net Model Compiler & Optimizer supporting ONNX, and Khronos NNEF™ exchange formats. The toolkit allows for rapid prototyping and deployment of optimized computer vision and machine learning inference workloads on a wide range of computer hardware, including small embedded x86 CPUs, APUs, discrete GPUs, and heterogeneous servers.

• AMD OpenVX
• AMD OpenVX Extensions
  • 360 Video Stitch Extension
  • Media Extension
  • Neural Net Extension
  • OpenCV Extension
  • RPP Extension
  • WinML Extension
• Applications
• Neural Net Model Compiler & Optimizer
• RALI
• Samples
• Toolkit
• Utilities
  • Inference Generator
  • Loom Shell
  • RunCL
  • RunVX
• Prerequisites
• Build & Install MIVisionX
• Verify the Installation
• Docker
• Release Notes

AMD OpenVX

AMD OpenVX is a highly optimized open source implementation of the Khronos OpenVX™ computer vision specification. It allows for rapid prototyping as well as fast execution on a wide range of computer hardware, including small embedded x86 CPUs and large workstation discrete GPUs.

Khronos OpenVX™ 1.0.1 conformant implementation is available in MIVisionX Lite

AMD OpenVX Extensions

The OpenVX framework provides a mechanism to add new vision functions to OpenVX by 3rd party vendors. This project has below mentioned OpenVX modules and utilities to extend amd_openvx project, which contains the AMD OpenVX Core Engine.

• amd_loomsl: AMD Radeon Loom stitching library for live 360 degree video applications
• amd_media: vx_amd_media is an OpenVX AMD media extension module for encode and decode
• amd_nn: OpenVX neural network module
• amd_opencv: OpenVX module that implements a mechanism to access OpenCV functionality as OpenVX kernels
• amd_rpp: OpenVX extension providing an interface to some of the RPP's (Radeon Performance Primitives) functions. This extension is used to enable RALI to perform image augmentation.
• amd_winml: WinML extension will allow developers to import a pre-trained ONNX model into an OpenVX graph and add hundreds of different pre & post processing vision / generic / user-defined functions, available in OpenVX and OpenCV interop, to the input and output of the neural net model. This will allow developers to build an end to end application for inference.

Applications

MIVisionX has several applications built on top of OpenVX modules, it uses AMD optimized libraries to build applications which can be used to prototype or used as models to develop a product.

• Bubble Pop: This sample application creates bubbles and donuts to pop using OpenVX & OpenCV functionality.
• Cloud Inference Application: This sample application does inference using a client-server system.
• Digit Test: This sample application is used to recognize hand written digits.
• Image Augmentation: This sample application demonstrates the basic usage of RALI's C API to load JPEG images from the disk and modify them in different possible ways and displays the output images.
• MIVisionX Inference Analyzer: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze and summarize images.
• MIVisionX OpenVX Classsification: This sample application shows how to run supported pre-trained caffe models with MIVisionX RunTime.
• MIVisionX Validation Tool: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze, summarize and validate models.
• MIVisionX WinML Classification: This sample application shows how to run supported ONNX models with MIVisionX RunTime on Windows.
• MIVisionX WinML YoloV2: This sample application shows how to run tiny yolov2(20 classes) with MIVisionX RunTime on Windows.
• External Applications

Neural Net Model Compiler & Optimizer

Neural Net Model Compiler & Optimizer converts pre-trained neural net models to MIVisionX runtime code for optimized inference.

RALI

The Radeon Augmentation Library - RALI is designed to efficiently decode and process images and videos from a variety of storage formats and modify them through a processing graph programmable by the user.

Toolkit

MIVisionX Toolkit, is a comprehensive set of helpful tools for neural net creation, development, training, and deployment. The Toolkit provides you with helpful tools to design, develop, quantize, prune, retrain, and infer your neural network work in any framework. The Toolkit is designed to help you deploy your work to any AMD or 3rd party hardware, from embedded to servers.

MIVisionX provides you with tools for accomplishing your tasks throughout the whole neural net life-cycle, from creating a model to deploying them for your target platforms.

Utilities

• inference_generator: generate inference library from pre-trained CAFFE models
• loom_shell: an interpreter to prototype 360 degree video stitching applications using a script
• RunVX: command-line utility to execute OpenVX graph described in GDF text file
• RunCL: command-line utility to build, execute, and debug OpenCL programs

Prerequisites

• CPU: SSE4.1 or above CPU, 64-bit

• GPU: GFX7 or above [optional]

• APU: Carrizo or above [optional]

  Note: Some modules in MIVisionX can be built for CPU ONLY. To take advantage of Advanced Features And Modules we recommend using AMD GPUs or AMD APUs.

Windows

• Windows 10
• Windows SDK
• Visual Studio 2017 and above
• Install the latest AMD drivers
• Install OpenCL SDK
• Install OpenCV 3.4
  • Set OpenCV_DIR environment variable to OpenCV/build folder
  • Add %OpenCV_DIR%\x64\vc14\bin or %OpenCV_DIR%\x64\vc15\bin to your PATH

Linux

• Install ROCm
• CMake 2.8 or newer download
• ROCm CMake, MIOpenGEMM & MIOpen for Neural Net Extensions (vx_nn)
• Qt Creator for Cloud Inference Client
• Protobuf for inference generator & model compiler
  • install libprotobuf-dev and protobuf-compiler needed for vx_nn
• OpenCV 3.4
  • Set OpenCV_DIR environment variable to OpenCV/build folder
• FFMPEG n4.0.4 - Optional
  • FFMPEG is required for amd_media & mv_deploy modules
• RALI Prerequisites

Prerequisites setup script for Linux - MIVisionX-setup.py

For the convenience of the developer, we here provide the setup script which will install all the dependencies required by this project.

MIVisionX-setup.py builds all the prerequisites required by MIVisionX. The setup script creates a deps folder and installs all the prerequisites, this script only needs to be executed once. If the directory option is not given, the script will install the deps folder in the home directory(~/) by default, else in the user-specified location.

Prerequisites for running the script

1. Ubuntu 16.04 / 18.04 or CentOS 7.5 / 7.6
2. ROCm supported hardware
3. ROCm

usage:

python MIVisionX-setup.py --directory [setup directory - optional (default:~/)]
                          --installer [Package management tool - optional (default:apt-get) [options: Ubuntu:apt-get;CentOS:yum]]
                          --opencv    [OpenCV Version - optional (default:3.4.0)]
                          --miopen    [MIOpen Version - optional (default:2.5.0)]
                          --miopengemm[MIOpenGEMM Version - optional (default:1.1.5)]
                          --protobuf  [ProtoBuf Version - optional (default:3.12.0)]
                          --rpp       [RPP Version - optional (default:0.5)]
                          --ffmpeg    [FFMPEG Installation - optional (default:no) [options:yes/no]]
                          --rali      [MIVisionX RALI Dependency Install - optional (default:yes) [options:yes/no]]
                          --neural_net[MIVisionX Neural Net Dependency Install - optional (default:yes) [options:yes/no]]
                          --reinstall [Remove previous setup and reinstall (default:no)[options:yes/no]]

Note:

• use --installer yum for CentOS
• ROCm upgrade with sudo apt upgrade requires the setup script rerun.
• use X Window / X11 for remote GUI app control

Refer to Wiki page for developer instructions.

Build & Install MIVisionX

Windows

Using .msi packages

• MIVisionX-installer.msi: MIVisionX
• MIVisionX_WinML-installer.msi: MIVisionX for WinML

Using Visual Studio 2017 on 64-bit Windows 10

• Install Windows Prerequisites

• Use MIVisionX.sln to build for x64 platform

  NOTE: vx_nn is not supported on Windows in this release

Linux

Using apt-get / yum

Prerequisites

1. Ubuntu 16.04 / 18.04 or CentOS 7.5 / 7.6
2. ROCm supported hardware
3. ROCm

Ubuntu

sudo apt-get install mivisionx

CentOS

sudo yum install mivisionx

Note:

• vx_winml is not supported on linux
• source code will not available with apt-get / yum install
• executables placed in /opt/rocm/mivisionx/bin and libraries in /opt/rocm/mivisionx/lib
• OpenVX and module header files into /opt/rocm/mivisionx/include
• model compiler, toolkit, & samples placed in /opt/rocm/mivisionx
• Package (.deb & .rpm) install requires OpenCV v3.4.0 to execute AMD OpenCV extensions

Using MIVisionX-setup.py and CMake on Linux with ROCm

• Install ROCm
• Use the below commands to set up and build MIVisionX

git clone https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX.git
cd MIVisionX

python MIVisionX-setup.py --directory [setup directory - optional (default:~/)]
                          --installer [Package management tool - optional (default:apt-get) [options: Ubuntu:apt-get;CentOS:yum]]
                          --opencv    [OpenCV Version - optional (default:3.4.0)]
                          --miopen    [MIOpen Version - optional (default:2.5.0)]
                          --miopengemm[MIOpenGEMM Version - optional (default:1.1.5)]
                          --protobuf  [ProtoBuf Version - optional (default:3.12.0)]
                          --rpp       [RPP Version - optional (default:0.5)]
                          --ffmpeg    [FFMPEG Installation - optional (default:no) [options:yes/no]]
                          --rali      [MIVisionX RALI Dependency Install - optional (default:yes) [options:yes/no]]
                          --neural_net[MIVisionX Neural Net Dependency Install - optional (default:yes) [options:yes/no]]
                          --reinstall [Remove previous setup and reinstall (default:no)[options:yes/no]]

Note: use --installer yum for CentOS

mkdir build
cd build
cmake ../
make -j8
sudo make install

Note:

• vx_winml is not supported on Linux
• the installer will copy
  • executables into /opt/rocm/mivisionx/bin
  • libraries into /opt/rocm/mivisionx/lib
  • OpenVX and module header files into /opt/rocm/mivisionx/include

Using CMake on Linux with ROCm

• Install ROCm
• git clone, build and install other ROCm projects (using cmake and % make install ) in the below order for vx_nn.
  • rocm-cmake
  • MIOpenGEMM
  • MIOpen -- make sure to use -DMIOPEN_BACKEND=OpenCL option with cmake
• install protobuf
• install OpenCV
• install RPP
• install FFMPEG n4.0.4 - Optional
• build and install (using cmake and % make install )
  • executables will be placed in bin folder
  • libraries will be placed in lib folder
  • the installer will copy all executables into /opt/rocm/mivisionx/bin and libraries into /opt/rocm/mivisionx/lib
  • the installer also copies all the OpenVX and OpenVX module header files into /opt/rocm/mivisionx/include folder
• add the installed library path to LD_LIBRARY_PATH environment variable (default /opt/rocm/mivisionx/lib )
• add the installed executable path to PATH environment variable (default /opt/rocm/mivisionx/bin )

Verify the Installation

Linux

• The installer will copy
  • executables into /opt/rocm/mivisionx/bin
  • libraries into /opt/rocm/mivisionx/lib
  • OpenVX and OpenVX module header files into /opt/rocm/mivisionx/include
  • Apps, Samples, Documents, Model Compiler, and Toolkit are placed into /opt/rocm/mivisionx
• Run below sample to verify the installation

Canny Edge Detection

  export PATH=$PATH:/opt/rocm/mivisionx/bin
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/mivisionx/lib
  runvx /opt/rocm/mivisionx/samples/gdf/canny.gdf 

Note: More samples are available here

Windows

• MIVisionX.sln builds the libraries & executables in the folder MIVisionX/x64
• Use RunVX to test the build

./runvx.exe PATH_TO/MIVisionX/samples/gdf/skintonedetect.gdf

Docker

MIVisionX provides developers with docker images for Ubuntu 16.04 / 18.04 and CentOS 7.5 / 7.6. Using docker images developers can quickly prototype and build applications without having to be locked into a single system setup or lose valuable time figuring out the dependencies of the underlying software.

MIVisionX Docker

• Ubuntu 16.04
• Ubuntu 18.04
• CentOS 7.5
• CentOS 7.6

Docker Workflow Sample on Ubuntu 16.04

Prerequisites

• Ubuntu 16.04
• rocm supported hardware

Workflow

• Step 1 - Install rocm-dkms

sudo apt update
sudo apt dist-upgrade
sudo apt install libnuma-dev
sudo reboot

wget -qO - http://repo.radeon.com/rocm/apt/debian/rocm.gpg.key | sudo apt-key add -
echo 'deb [arch=amd64] http://repo.radeon.com/rocm/apt/debian/ xenial main' | sudo tee /etc/apt/sources.list.d/rocm.list
sudo apt update
sudo apt install rocm-dkms
sudo reboot

• Step 2 - Setup Docker

sudo apt-get install curl
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
apt-cache policy docker-ce
sudo apt-get install -y docker-ce
sudo systemctl status docker

• Step 3 - Get Docker Image

sudo docker pull mivisionx/ubuntu-16.04

• Step 4 - Run the docker image

sudo docker run -it --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host mivisionx/ubuntu-16.04

Note:

• Map host directory on the docker image

  • map the localhost directory to be accessed on the docker image.
  • use -v option with docker run command: -v {LOCAL_HOST_DIRECTORY_PATH}:{DOCKER_DIRECTORY_PATH}
  • usage:

  sudo docker run -it -v /home/:/root/hostDrive/ --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host mivisionx/ubuntu-16.04
  

• Display option with docker

  • Using host display

  xhost +local:root
  sudo docker run -it --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host --env DISPLAY=unix$DISPLAY --  privileged --volume $XAUTH:/root/.Xauthority --volume /tmp/.X11-unix/:/tmp/.X11-unix mivisionx/ubuntu-16.04:latest
  

  • Test display with MIVisionX sample

  export PATH=$PATH:/opt/rocm/mivisionx/bin
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/mivisionx/lib
  runvx /opt/rocm/mivisionx/samples/gdf/canny.gdf 
  

Release Notes

Known issues

• Package install requires OpenCV v3.4.0 to execute AMD OpenCV extensions
• ROCm 3.0 and above has known to slow down OpenCL kernels.
• OpenCL from ROCm 3.3 and beyond has - cl_version.h: CL_TARGET_OPENCL_VERSION is not defined warning

Tested configurations

• Windows 10
• Linux: Ubuntu - 16.04 / 18.04 & CentOS - 7.5 / 7.6
• ROCm: rocm-dkms - 3.3.0-19
• rocm-cmake - github master:ac45c6e
• MIOpenGEMM - 1.1.5
• MIOpen - 2.5.0
• Protobuf - V3.12.0
• OpenCV - 3.4.0
• RPP - 0.5
• Dependencies for all the above packages
• MIVisionX Setup Script - V1.8.4

Latest Release

Docker Image

Docker Image: docker pull kiritigowda/ubuntu-18.04:tagname

• new component added to the level
• existing component from the previous level

Build Level	MIVisionX Dependencies	Modules	Libraries and Executables	Docker Tag
Level_1	cmake gcc g++	amd_openvx	libopenvx.so - OpenVX™ Lib - CPU libvxu.so - OpenVX™ immediate node Lib - CPU runvx - OpenVX™ Graph Executor - CPU with Display OFF
Level_2	ROCm OpenCL +Level 1	amd_openvx amd_openvx_extensions utilities	libopenvx.so - OpenVX™ Lib - CPU/GPU libvxu.so - OpenVX™ immediate node Lib - CPU/GPU libvx_loomsl.so - Loom 360 Stitch Lib loom_shell - 360 Stitch App runcl - OpenCL™ program debug App runvx - OpenVX™ Graph Executor - Display OFF
Level_3	OpenCV FFMPEG +Level 2	amd_openvx amd_openvx_extensions utilities	libopenvx.so - OpenVX™ Lib libvxu.so - OpenVX™ immediate node Lib libvx_loomsl.so - Loom 360 Stitch Lib loom_shell - 360 Stitch App runcl - OpenCL™ program debug App libvx_amd_media.so - OpenVX™ Media Extension libvx_opencv.so - OpenVX™ OpenCV InterOp Extension mv_compile - Neural Net Model Compile runvx - OpenVX™ Graph Executor - Display ON
Level_4	MIOpenGEMM MIOpen ProtoBuf +Level 3	amd_openvx amd_openvx_extensions apps utilities	libopenvx.so - OpenVX™ Lib libvxu.so - OpenVX™ immediate node Lib libvx_loomsl.so - Loom 360 Stitch Lib loom_shell - 360 Stitch App libvx_amd_media.so - OpenVX™ Media Extension libvx_opencv.so - OpenVX™ OpenCV InterOp Extension mv_compile - Neural Net Model Compile runcl - OpenCL™ program debug App runvx - OpenVX™ Graph Executor - Display ON libvx_nn.so - OpenVX™ Neural Net Extension inference_server_app - Cloud Inference App
Level_5	AMD_RPP RALI deps +Level 4	amd_openvx amd_openvx_extensions apps rali utilities	libopenvx.so - OpenVX™ Lib libvxu.so - OpenVX™ immediate node Lib libvx_loomsl.so - Loom 360 Stitch Lib loom_shell - 360 Stitch App libvx_amd_media.so - OpenVX™ Media Extension libvx_opencv.so - OpenVX™ OpenCV InterOp Extension mv_compile - Neural Net Model Compile runcl - OpenCL™ program debug App runvx - OpenVX™ Graph Executor - Display ON libvx_nn.so - OpenVX™ Neural Net Extension inference_server_app - Cloud Inference App libvx_rpp.so - OpenVX™ RPP Extension librali.so - Radeon Augmentation Library rali_pybind.so - RALI Pybind Lib