Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shutting down the server with ctrl+c hanging with docker image #5818

Open
manickavela29 opened this issue Aug 2, 2024 · 0 comments
Open
Labels

Comments

@manickavela29
Copy link

Running seldon-sore in Docker image, but when I do ctrl+c,
it is hanging as in the below code :


^C[2024-08-02 14:04:09 +0000] [1] [INFO] Handling signal: int
[2024-08-02 14:04:09 +0000] [107] [INFO] Worker exiting (pid: 107)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/multiprocess/util.py", line 357, in _exit_function
    p.join()
  File "/usr/local/lib/python3.8/dist-packages/multiprocess/process.py", line 147, in join
    assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
[2024-08-02 14:04:09 +0000] [84] [INFO] Handling signal: term
[2024-08-02 14:04:09 +0000] [106] [INFO] Worker exiting (pid: 106)
[2024-08-02 14:04:09 +0000] [84] [INFO] Shutting down: Master
[2024-08-02 14:04:10 +0000] [1] [INFO] Shutting down: Master

have to manually terminate the container everytime to get out of this situation

To reproduce

Docker file

FROM nvidia/cuda:12.0.0-cudnn8-devel-ubuntu20.04

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y --no-install-recommends python3 python3-pip && \
    rm -rf /var/lib/apt/lists/*

# nvidia-container-toolkit-base

RUN ln -s /usr/bin/python3 /usr/bin/python

WORKDIR /app

# Install python packagess
COPY requirements.txt requirements.txt
RUN python3 -m pip install -r requirements.txt

RUN python3 -m pip install prometheus-client==0.20.0 pydantic==2.8.2
RUN pip install flash-attn --no-build-isolation
# Copy source code
COPY . .

# Port for GRPC
EXPOSE 8081
# Port for REST
EXPOSE 8080

ENV MODEL_NAME <Project py fail and class name>
ENV SERVICE_TYPE MODEL
ENV API_TYPE REST
ENV PERSISTENCE 0
ENV HF_HOME /app/.cache/
ENV USE_MULTIPROCESS_PACKAGE true
ENV MODEL_PATH /app/models/<project py file>

# Changing folder to default user
RUN chown -R 8888 /app
RUN mkdir -p /app/.cache
RUN chown -R 8888 /app/.cache
RUN mkdir -p /app/model
RUN chown -R 8888 /app/model

CMD exec seldon-core-microservice $MODEL_NAME --service-type $SERVICE_TYPE

Functionally server is up and running sucessfully,
everything is good,
only ctrl+c it is happening

Model details

Running Laama 3 8b (but irrelevant in my case)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant