Skip to content

Commit

Permalink
Update documentation for Docker support
Browse files Browse the repository at this point in the history
  • Loading branch information
jakki-amd committed Nov 29, 2024
1 parent 8f35524 commit f5ce2ec
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 0 deletions.
1 change: 1 addition & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ Creates a docker image with `torchserve` and `torch-model-archiver` installed fr
./build_image.sh -bt dev -g [-cv cu121|cu118] -cpp
```

- For ROCm support (*experimental*), refer to [this documentation](../docs/hardware_support/amd_support.md).

## Start a container with a TorchServe image

Expand Down
14 changes: 14 additions & 0 deletions docs/hardware_support/amd_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,20 @@ If you have 8 accelerators but only want TorchServe to see the last four of them
> ⚠️ Setting both `CUDA_VISIBLE_DEVICES` and `HIP_VISIBLE_DEVICES` may cause unintended behaviour and should be avoided.
> Doing so may cause an exception in the future.

## Docker

**In Development**

`Dockerfile.rocm` provides preliminary ROCm support for TorchServe.

Building and running `dev-image`:

```bash
docker build --file docker/Dockerfile.rocm --target dev-image -t torch-serve-dev-image-rocm --build-arg USE_ROCM_VERSION=rocm62 --build-arg BUILD_FROM_SRC=true .
docker run -it --rm --device=/dev/kfd --device=/dev/dri torch-serve-dev-image-rocm bash
```

## Example Usage

After installing TorchServe with the required dependencies for ROCm you should be ready to serve your model.
Expand Down

0 comments on commit f5ce2ec

Please sign in to comment.