diff --git a/docker/README.md b/docker/README.md index f29b2ac3e..cc3c5a94b 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,6 +1,6 @@ # Docker guide -Docker guide for IntelĀ® Media Transport Library +Docker guide for IntelĀ® Media Transport Library. ## 1. Build Docker image @@ -11,7 +11,9 @@ docker build -t mtl:latest -f ubuntu.dockerfile ./ Refer to below build command if you are in a proxy env. ```bash -docker build -t mtl:latest -f ubuntu.dockerfile --build-arg HTTP_PROXY=http://proxy.xxx.com:xxx --build-arg HTTPS_PROXY=https://proxy.xxx.com:xxx ./ +http_proxy=http://proxy.xxx.com:xxx +https_proxy=https://proxy.xxx.com:xxx +docker build -t mtl:latest -f ubuntu.dockerfile --build-arg HTTP_PROXY=$http_proxy --build-arg HTTPS_PROXY=$https_proxy ./ ``` ## 2. DPDK NIC PMD and env setup on host @@ -24,19 +26,85 @@ The sample usage provided below is enabled with specific privileged settings suc ### 3.1 Run the docker container -The argument `/dev/vfio/` enables the Docker instance to access the VFIO device. - -The arguments `/dev/null, /tmp/kahawai_lcore.lock, and --ipc=host` and touch `/tmp/kahawai_lcore.lock` command are used for managing shared memory within IMTL, primarily for lcore management across multiple IMTL docker containers. +#### 3.1.1 Run multiple docker container with SHM requirement ```bash touch /tmp/kahawai_lcore.lock -docker run --privileged -it -v /dev/vfio/:/dev/vfio/ -v /dev/null:/dev/null -v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock --ipc=host mtl:latest +docker run -it \ + --device /dev/vfio \ + --cap-add SYS_NICE \ + --cap-add IPC_LOCK \ + --cap-add NET_ADMIN \ + --cap-add SYS_TIME \ + --cap-add NET_RAW \ + -v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock \ + -v /dev/null:/dev/null \ + --ipc=host \ + mtl:latest ``` +Explanation of Docker arguments: + +| Argument | Description | +| --- | --- | +| `--device /dev/vfio` | Access the VFIO device | +| `--cap-add SYS_NICE` | For set_mempolicy | +| `--cap-add IPC_LOCK` | For DMA mapping | +| `--cap-add NET_ADMIN` | Optional, for kernel NIC configuration | +| `--cap-add SYS_TIME` | Optional, for systime adjustment | +| `--cap-add NET_RAW` | Optional, for AF_XDP socket | +| `-v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock` | For multiple instances lcore management | +| `-v /dev/null:/dev/null` | For multiple instances lcore management | +| `--ipc=host` | For multiple instances lcore management | + +#### 3.1.2 Run single docker container + If you confirm that all IMTL processes will run within a single Docker container, you can disregard the settings related to shared memory. Simply execute the following command: ```bash -docker run --privileged -it -v /dev/vfio/:/dev/vfio/ mtl:latest +docker run -it \ + --device /dev/vfio \ + --cap-add SYS_NICE \ + --cap-add IPC_LOCK \ + --cap-add NET_ADMIN \ + --cap-add SYS_TIME \ + --cap-add NET_RAW \ + mtl:latest +``` + +#### 3.1.3 Specify NIC devices for container + +If you only need to pass specific NICs to the container, you can use the following command to list the IOMMU group: + +```bash +../script/nicctl.sh list all + +ID PCI BDF Driver NUMA IOMMU IF Name +0 0000:4b:01.0 vfio-pci 0 311 * +1 0000:4b:01.1 vfio-pci 0 312 * +``` + +Then, you can specify the IOMMU group IDs to the `--device` argument: + +```bash +docker run -it \ + --device /dev/vfio/vfio \ + --device /dev/vfio/311 \ + --device /dev/vfio/312 \ + --cap-add SYS_NICE \ + --cap-add IPC_LOCK \ + mtl:latest +``` + +### 3.1.4 Run with docker-compose + +Edit the `docker-compose.yml` file to specify the configuration. + +Run the service: + +```bash +docker-compose run imtl +# docker compose run imtl ``` ### 3.2 Switch to the root user inside a Docker container diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml new file mode 100644 index 000000000..82293c64c --- /dev/null +++ b/docker/docker-compose.yml @@ -0,0 +1,17 @@ +version: '3' + +services: + imtl: + image: mtl:latest + devices: + - "/dev/vfio:/dev/vfio" # or add /dev/vfio/vfio and /dev/vfio/ + ipc: host + volumes: + - "/tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock" + - "/dev/null:/dev/null" + cap_add: + - SYS_NICE + - IPC_LOCK + - NET_ADMIN + - SYS_TIME + - NET_RAW \ No newline at end of file