Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Fully-featured server stop functionality #138

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[Feature] Fully-featured server stop functionality #138

wants to merge 1 commit into from

Conversation

3manifold
Copy link
Contributor

@3manifold 3manifold commented Nov 12, 2024 •
edited
Loading

Description

This feature implements the server stop functionality during a KeyboardInterrupt in different scenarios.

Although pipeline_manager.stop() seems to be invoked in s2s_pipeline.py during a KeyboardInterrupt, it is observed that in practice stop functionality is unresponsive. Ideally, stop functionality should work in both of the following scenarios (currently not the case):

  • Scenario 1:
    • start server -> stop server
  • Scenario 2:
    • start server -> start client -> stop server

The present approach benefits the application in several aspects:

Tested for for both --device cpu and cuda. Fixes #139 and fixes #140 .

Logs

The logs below demonstrate the output of the server for both scenario 1 and scenario 2 after the fix was applied.

Scenario 1

Click to show log 
python3 s2s_pipeline.py --recv_host 0.0.0.0 --send_host 0.0.0.0 --device cpu --log_level debug 
/home/i00680185/.virtualenvs/voicefixer_main/lib/python3.10/site-packages/df/io.py:9: UserWarning: `torchaudio.backend.common.AudioMetaData` has been moved to `torchaudio.AudioMetaData`. Please update the import path.
...
...
... 12:15:00,827 - utils.thread_manager - DEBUG - Thread 139630012253760 has started. Target: Thread-1 (run), type: <class 'connections.socket_receiver.SocketReceiver'>
... 12:15:00,828 - utils.thread_manager - DEBUG - Thread 139630003861056 has started. Target: Thread-2 (run), type: <class 'connections.socket_sender.SocketSender'>
... 12:15:00,829 - connections.socket_sender - INFO - Sender waiting to be connected...
... 12:15:00,829 - utils.thread_manager - DEBUG - Thread 139629874308672 has started. Target: Thread-3 (run), type: <class 'VAD.vad_handler.VADHandler'>
... 12:15:00,829 - connections.socket_receiver - INFO - Receiver waiting to be connected...
... 12:15:00,830 - utils.thread_manager - DEBUG - Thread 139629865915968 has started. Target: Thread-4 (run), type: <class 'STT.whisper_stt_handler.WhisperSTTHandler'>
... 12:15:00,830 - utils.thread_manager - DEBUG - Thread 139629481412160 has started. Target: Thread-5 (run), type: <class 'LLM.language_model.LanguageModelHandler'>
... 12:15:00,831 - utils.thread_manager - DEBUG - Thread 139629473019456 has started. Target: Thread-6 (run), type: <class 'TTS.parler_handler.ParlerTTSHandler'>
... 12:15:00,831 - utils.thread_manager - DEBUG - Thread 139630012253760 attempt to join. Target: Thread-1 (run)
Stopping server..
Server closed.
... 12:15:44,945 - utils.thread_manager - DEBUG - Server stop was invoked
... 12:15:44,945 - connections.socket_receiver - DEBUG - Receiver is in stopping process. Sending END message to the next component to initiate server termination..
... 12:15:44,945 - connections.socket_sender - DEBUG - SocketSender: shutdown socket
... 12:15:44,945 - baseHandler - DEBUG - VADHandler: Received END message, stopping thread
... 12:15:44,945 - connections.socket_sender - DEBUG - SocketSender received exception: [Errno 22] Invalid argument. Possibly the sever is in termination process..
... 12:15:44,946 - connections.socket_sender - INFO - Sender closed
... 12:15:44,946 - baseHandler - DEBUG - VADHandler: Stopping pipeline component..
... 12:15:44,946 - baseHandler - DEBUG - WhisperSTTHandler: Stopping pipeline component..
... 12:15:44,946 - baseHandler - DEBUG - LanguageModelHandler: Stopping pipeline component..
... 12:15:44,946 - baseHandler - DEBUG - ParlerTTSHandler: Stopping pipeline component..
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139630012253760 attempt to join. Target: Thread-1 (run)
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139630012253760 has joined. Target: Thread-1 (run)
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139630003861056 attempt to join. Target: Thread-2 (run)
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139630003861056 has joined. Target: Thread-2 (run)
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139629874308672 attempt to join. Target: Thread-3 (run)
... 12:15:44,946 - baseHandler - DEBUG - WhisperSTTHandler: Received END message, stopping thread
... 12:15:44,946 - utils.thread_manager - DEBUG - Thread 139629874308672 has joined. Target: Thread-3 (run)
... 12:15:44,947 - baseHandler - DEBUG - LanguageModelHandler: Received END message, stopping thread
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629865915968 attempt to join. Target: Thread-4 (run)
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629865915968 has joined. Target: Thread-4 (run)
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629481412160 attempt to join. Target: Thread-5 (run)
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629481412160 has joined. Target: Thread-5 (run)
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629473019456 attempt to join. Target: Thread-6 (run)
... 12:15:44,947 - baseHandler - DEBUG - ParlerTTSHandler: Received END message, stopping thread
... 12:15:44,947 - utils.thread_manager - DEBUG - Thread 139629473019456 has joined. Target: Thread-6 (run)

Process finished with exit code 0

Scenario 2

Click to show log 
python3 s2s_pipeline.py --recv_host 0.0.0.0 --send_host 0.0.0.0 --log_level debug 
...
...
... 11:59:26,387 - utils.thread_manager - DEBUG - Thread 140596932494912 has started. Target: Thread-1 (run), type: <class 'connections.socket_receiver.SocketReceiver'>
... 11:59:26,389 - utils.thread_manager - DEBUG - Thread 140596924102208 has started. Target: Thread-2 (run), type: <class 'connections.socket_sender.SocketSender'>
... 11:59:26,390 - connections.socket_sender - INFO - Sender waiting to be connected...
... 11:59:26,390 - connections.socket_receiver - INFO - Receiver waiting to be connected...
... 11:59:26,393 - utils.thread_manager - DEBUG - Thread 140596915709504 has started. Target: Thread-3 (run), type: <class 'VAD.vad_handler.VADHandler'>
... 11:59:26,395 - utils.thread_manager - DEBUG - Thread 140596907316800 has started. Target: Thread-4 (run), type: <class 'STT.whisper_stt_handler.WhisperSTTHandler'>
... 11:59:26,398 - utils.thread_manager - DEBUG - Thread 140596778821184 has started. Target: Thread-5 (run), type: <class 'LLM.language_model.LanguageModelHandler'>
... 11:59:26,400 - utils.thread_manager - DEBUG - Thread 140596770428480 has started. Target: Thread-6 (run), type: <class 'TTS.parler_handler.ParlerTTSHandler'>
... 11:59:26,401 - utils.thread_manager - DEBUG - Thread 140596932494912 attempt to join. Target: Thread-1 (run)
... 11:59:47,059 - connections.socket_receiver - INFO - Receiver connected
... 11:59:47,060 - connections.socket_sender - INFO - Sender connected
... 11:59:51,812 - VAD.vad_handler - DEBUG - VAD: end of speech detected
... 11:59:51,815 - VAD.vad_handler - DEBUG - Stop listening
... 11:59:51,816 - baseHandler - DEBUG - VADHandler:  0.004 s
... 11:59:51,816 - STT.whisper_stt_handler - DEBUG - infering whisper...
... 11:59:51,878 - STT.whisper_stt_handler - DEBUG - finished whisper inference
... 11:59:51,912 - STT.whisper_stt_handler - DEBUG - Language Code Whisper: a
... 11:59:51,912 - baseHandler - DEBUG - WhisperSTTHandler:  0.096 s
... 11:59:51,915 - LLM.language_model - DEBUG - infering language model...
USER: � fakeiron�
...
...
... 12:00:12,585 - baseHandler - DEBUG - LanguageModelHandler:  20.673 s
ASSISTANT: I'm glad you like it!
... 12:00:16,100 - baseHandler - DEBUG - LanguageModelHandler:  3.515 s
... 12:00:18,021 - parler_tts.modeling_parler_tts - WARNING - `prompt_attention_mask` is specified but `attention_mask` is not. A full `attention_mask` will be created. Make sure this is the intended behaviour.
... 12:00:21,834 - baseHandler - DEBUG - LanguageModelHandler:  5.733 s
... 12:00:38,372 - utils.thread_manager - DEBUG - Server stop was invoked
... 12:00:38,372 - connections.socket_receiver - DEBUG - Receiver is in stopping process. Sending END message to the next component to initiate server termination..
... 12:00:38,373 - connections.socket_sender - DEBUG - SocketSender: shutdown socket
... 12:00:38,373 - baseHandler - DEBUG - VADHandler: Received END message, stopping thread
... 12:00:38,373 - baseHandler - DEBUG - VADHandler: Stopping pipeline component..
... 12:00:38,374 - baseHandler - DEBUG - WhisperSTTHandler: Stopping pipeline component..
... 12:00:38,375 - baseHandler - DEBUG - LanguageModelHandler: Stopping pipeline component..
... 12:00:38,375 - baseHandler - DEBUG - ParlerTTSHandler: Stopping pipeline component..
... 12:00:38,375 - utils.thread_manager - DEBUG - Thread 140596932494912 attempt to join. Target: Thread-1 (run)
... 12:00:38,376 - utils.thread_manager - DEBUG - Thread 140596932494912 has joined. Target: Thread-1 (run)
... 12:00:38,376 - baseHandler - DEBUG - WhisperSTTHandler: Received END message, stopping thread
... 12:00:38,376 - utils.thread_manager - DEBUG - Thread 140596924102208 attempt to join. Target: Thread-2 (run)
... 12:00:38,377 - baseHandler - DEBUG - LanguageModelHandler: Received END message, stopping thread
Stopping server..
... 12:00:38,428 - connections.socket_receiver - INFO - Receiver closed
... 12:03:09,340 - baseHandler - DEBUG - ParlerTTSHandler:  176.754 s
... 12:03:09,342 - connections.socket_sender - INFO - Sender closed
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596924102208 has joined. Target: Thread-2 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596915709504 attempt to join. Target: Thread-3 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596915709504 has joined. Target: Thread-3 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596907316800 attempt to join. Target: Thread-4 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596907316800 has joined. Target: Thread-4 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596778821184 attempt to join. Target: Thread-5 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596778821184 has joined. Target: Thread-5 (run)
... 12:03:09,343 - utils.thread_manager - DEBUG - Thread 140596770428480 attempt to join. Target: Thread-6 (run)
Server closed.
... 12:06:42,062 - baseHandler - DEBUG - ParlerTTSHandler:  212.720 s
... 12:06:42,063 - utils.thread_manager - DEBUG - Thread 140596770428480 has joined. Target: Thread-6 (run)

Process finished with exit code 0

This was referenced Nov 12, 2024
Open
Open
@3manifold 3manifold marked this pull request as ready for review November 12, 2024 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@3manifold