-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathstop_workers.sh
12 lines (12 loc) · 1.62 KB
/
stop_workers.sh
1
2
3
4
5
6
7
8
9
10
11
12
# Script to stop the workers gracefully. This will
# ask the workers of the servers where it is executed
# to stop after the current task is finished.
# Workers use python rq library. Sending a SIGTERM o SIGHUP to the rq queues process will exit after the current task finishes. The rq is not process with id 1, so we cannot do ```docker kill```. The script enters enter each container send the signal to rq process. It first list all processes with "ps aux". The rq process we want to send the signal to, will be the one with smallest id of all with the exception of pid 1 (which is bash).
# So, if there are four processes with pids 1, 6, 13 and 14. We need to do ```kill 6``` from inside the docker container.
# Note we are doing ```kill 6```, instead ```kill -s 9 6```
# (rq is exucted via bash, like ```bash -c "rq worker queue_name"```. The reason why we do this instead of directly executing ```rq worker``` is because we need bash to remove zombies generated by some complicated plugins.)
# The following command will send signal SIGHUP to the rq process of worker number 6.
# docker exec -ti codex-backend-full_worker_no_vt_6 bash -c 'kill $(ps xao pid | sort |head -n 2 | tail -n 1 | sed "s/ //g" | tr -d "\n")'
# The following command will send signal SIGHUP to the rq process of each worker:
docker ps --format "{{.Names}}" | grep worker | xargs -I '{}' docker exec -t {} bash -c 'echo going to kill $(ps xao pid | sort |head -n 2 | tail -n 1 | sed "s/ //g" | tr -d "\n") from {}'
docker ps --format "{{.Names}}" | grep worker | xargs -I '{}' docker exec -t {} bash -c 'kill $(ps xao pid | sort |head -n 2 | tail -n 1 | sed "s/ //g" | tr -d "\n")'