You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the robustness issues is if a container crashes in a model server, which can happen if bad input or an issue happens with VRAM overallocation, etc. It is possible to set a container to auto restart as part of the docker compose spec, which would be good. Should also probably send out a log message at that point
The text was updated successfully, but these errors were encountered:
One of the robustness issues is if a container crashes in a model server, which can happen if bad input or an issue happens with VRAM overallocation, etc. It is possible to set a container to auto restart as part of the docker compose spec, which would be good. Should also probably send out a log message at that point
The text was updated successfully, but these errors were encountered: