This page provides instructions to build, install and use the AIMET software in docker. Please follow the instructions in the order provided, unless specified otherwise.
- Requirements
- Get the code
- Setup the environment
- Build code and install
- Set package and library paths
- Usage examples and documentation
- Docker information
The AIMET package requires the following host platform setup:
- 64-bit Intel x86-compatible processor
- Linux Ubuntu: 22.04 LTS
- bash command shell
- For GPU variants:
- Nvidia GPU card (Compute capability 5.2 or later)
- nvidia-docker - Installation instructions: https://github.com/NVIDIA/nvidia-docker
To use the GPU accelerated training modules an Nvidia CUDA enabled GPU with a minimum Nvidia driver version of 455+ is required. Using the latest driver is always recommended, especially if using a newer GPU. Both CUDA and cuDNN (the more advanced CUDA interface) enabled GPUs are supported.
Recommended host system hardware requirements:
- Intel i7 multicore CPU w/hyperthreading
- 16+ GB RAM
- 500GB+ SSD hard drive
- For GPU variants:
- GPU: Nvidia GeForce GTX 1080 or Tesla V100
While these are not minimum requirements, they are recommended for good performance when training large networks.
To obtain the code, first define a workspace and follow these instructions:
WORKSPACE="<absolute_path_to_workspace>"
mkdir $WORKSPACE && cd $WORKSPACE
git clone https://github.com/quic/aimet.git
Clone the google test repo as follows:
cd aimet
mkdir -p ./ThirdParty/googletest
pushd ./ThirdParty/googletest
git clone https://github.com/google/googletest.git -b release-1.12.1 googletest-release-1.12.1
popd
In order to build and run AIMET code, several dependencies are required (such as python, cmake, tensorflow, pytorch, onnx, etc). Docker files and Docker images with all prerequisites and dependencies are available for each AIMET variant. Following are the available development options:
- Use the appropriate pre-built Docker image using the instructions here. This is the recommended option.
- Build the docker image locally and launch a launch container docker using the instructions here.
- Install the dependencies on your machine and setup your environment using the appropriate Dockerfile as a guide.
Set the common environment variables as follows:
source $WORKSPACE/aimet/packaging/envsetup.sh
Follow these instructions to build the AIMET code:
NOTE: If you are inside the docker, set
WORKSPACE="<absolute_path_to_workspace>"
again.
cd $WORKSPACE/aimet
mkdir build && cd build
# Run cmake (be sure to set the flags in the below command depending on your variant)
# To build for GPU, use -DENABLE_CUDA=ON. To build for CPU, use -DENABLE_CUDA=OFF.
# To include torch, use -DENABLE_TORCH=ON. To exclude torch, use -DENABLE_TORCH=OFF.
# To include tensorflow, use -DENABLE_TENSORFLOW=ON. To exclude tensorflow, use -DENABLE_TENSORFLOW=OFF.
# To include onnx, use -DENABLE_ONNX=ON. To exclude onnx, use -DENABLE_ONNX=OFF.
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DENABLE_CUDA=ON -DENABLE_TORCH=ON -DENABLE_TENSORFLOW=ON -DENABLE_ONNX=ON
make -j8
After a successful build, install the package using the following instructions:
cd $WORKSPACE/aimet/build
make install
Once the installation step is complete, the AIMET package is created at $WORKSPACE/aimet/build/staging/universal/lib/
.
Setup the package and library paths as follows:
export PYTHONPATH=$WORKSPACE/aimet/build/staging/universal/lib/python:$PYTHONPATH
At this point, we are all set to use AIMET!
The following steps would generate AIMET documentation including the user guide, examples and API documentation at $WORKSPACE/aimet/build/staging/universal/Docs
:
cd $WORKSPACE/aimet/build
make doc
To begin navigating the documentation, open the page $WORKSPACE/aimet/build/staging/universal/Docs/user_guide/index.html
on any browser.
Run the following commands to generate installable python wheel packages:
cd $WORKSPACE/aimet/build
make packageaimet
Code may optionally be developed inside a development docker container. This section describes how to build a docker image and launch a container using the provided Dockerfiles.
Set the <variant_string>
to ONE of the following depending on your desired variant
- For the PyTorch 2.1 GPU variant, use
"torch-gpu"
- For the PyTorch 2.1 CPU variant, use
"torch-cpu"
- For the PyTorch 1.13 GPU variant, use
"torch-gpu-pt113"
- For the PyTorch 1.13 CPU variant, use
"torch-cpu-pt113"
- For the TensorFlow GPU variant, use
"tf-gpu"
- For the TensorFlow CPU variant, use
"tf-cpu"
- For the ONNX GPU variant, use
"onnx-gpu"
- For the ONNX CPU variant, use
"onnx-cpu"
export AIMET_VARIANT=<variant_string>
Follow these instructions to use one of the pre-built docker images:
WORKSPACE="<absolute_path_to_workspace>"
docker_image_name="artifacts.codelinaro.org/codelinaro-aimet/aimet-dev:latest.${AIMET_VARIANT}"
docker_container_name="aimet-dev-<any_name>"
NOTE: Feel free to modify the
docker_container_name
as needed.
Follow these instructions ONLY if you want to build the docker image locally. If not, skip to the next section.
WORKSPACE="<absolute_path_to_workspace>"
docker_image_name="aimet-dev-docker:<any_tag>"
docker_container_name="aimet-dev-<any_name>"
docker build -t ${docker_image_name} -f $WORKSPACE/aimet/Jenkins/Dockerfile.${AIMET_VARIANT} .
NOTE: Feel free to modify the
docker_image_name
anddocker_container_name
as needed.
Ensure that a docker named $docker_container_name
is not already running; otherwise remove the existing container and then start a new container as follows:
docker ps -a | grep ${docker_container_name} && docker kill ${docker_container_name}
docker run --rm -it -u $(id -u ${USER}):$(id -g ${USER}) \
-v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro \
-v ${HOME}:${HOME} -v ${WORKSPACE}:${WORKSPACE} \
-v "/local/mnt/workspace":"/local/mnt/workspace" \
--entrypoint /bin/bash -w ${WORKSPACE} --hostname ${docker_container_name} ${docker_image_name}
NOTE
- Feel free to modify the above
docker run
command based on the environment and filesystem on your host machine. - If nvidia-docker 2.0 is installed, then add
--gpus all
to thedocker run
commands in order to enable GPU access inside the docker container. - If nvidia-docker 1.0 is installed, then replace
docker run
withnvidia-docker run
in order to enable GPU access inside the docker container. - Port forwarding needs to be done in order to run the Visualization APIs from docker container. This can be achieved by running the docker container as follows:
port_id="<any-port-number>"
docker run -p ${port_id}:${port_id} --rm -it -u $(id -u ${USER}):$(id -g ${USER}) \
-v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro \
-v ${HOME}:${HOME} -v ${WORKSPACE}:${WORKSPACE} \
-v "/local/mnt/workspace":"/local/mnt/workspace" \
--entrypoint /bin/bash -w ${WORKSPACE} --hostname ${docker_container_name} ${docker_image_name}
The development docker may also be built and launched in interactive mode using the provided script as follows:
NOTE: Add the '-l' option to use the pre-built docker image.
cd aimet
./buildntest.sh -e AIMET_VARIANT -i [-l]
If additional directories need to be mounted, use -m
option with list of targeted directories separated by space surrounded by double quotes ""
cd aimet
./buildntest.sh -e AIMET_VARIANT -i -m "sample_dir_1 sample_dir2" [-l]
To help construct user-specific docker commands, the dry-run option (-n
) can be used with the above script which prints out the equivalent docker command(s):
cd aimet
./buildntest.sh -e AIMET_VARIANT -i -n [-l]
# OR
./buildntest.sh -e AIMET_VARIANT -i -n -m "sample_dir_1 sample_dir2" [-l]