Skip to content

Commit

Permalink
chore: fix devcontainer GPU memory bug
Browse files Browse the repository at this point in the history
This commit ensures the model's folder is correctly mounted into
the dev container. By default, the local container is mounted under the
`/workspace` path when using a dev container. Because of this, the fp16
variant is not used, which causes the memory on smaller cards to be
insufficient.
  • Loading branch information
rickstaa committed Mar 14, 2024
1 parent 1d1e127 commit 75e48d1
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions runner/.devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
{
"name": "ai-runner",

// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
// Image to use for the dev container. More info: https://containers.dev/guide/dockerfile.
"build": {
"dockerfile": "../Dockerfile"
},
Expand All @@ -27,8 +27,13 @@
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
8000
]
],

// Use 'mounts' to make a list of local folders available inside the container.
"mounts": [
"source=${localWorkspaceFolder}/models,target=/models,type=bind"
]

// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": ""

Expand Down

0 comments on commit 75e48d1

Please sign in to comment.