My Demo POC is completed everything is working as expected but not able to run same on server in docker container

[nik0811]

I am using speech to text and text to speech service of deepgram using python sdk, everything work as expected when running on localhost but the time when i am deploying on server inside docker, speech to text stoped working, i thought might be the issue of docker so i tried running on server itself there also i am facing same issue, Don't know what's the issue, ran in debug mode too deepgram api reeives bytes but unable to transcribe, I need to take my poc to the client but i am stuck. Also I raised the concern on dicord but haven't got any response from them, what to do i don't know now, the only solution i am seeing is to change the service provider but that also will force me to do the rework on stt and tts side, which i am avoiding, curently i am using free trial but we will be purchasing plan soon if everything will work as expected.

[jkroll-deepgram]

Hi @nik0811, what Deepgram errors are you seeing? Do you have any differences in networking, API keys, request volume, etc between Docker and your localhost? Can you provide Deepgram request IDs from any failed requests?

[nik0811]

@jkroll-deepgram here is the issue where i haven't got any response back from deepgram while running on server, where as on localhost i am seeing the transcription.

Sending KeepAlive...
2024-10-15 22:19:32,158 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:32,158 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:32,158 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:32,158 - NOTICE - keep_alive succeeded
2024-10-15 22:19:34,193 - INFO - Processed 10 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:35,032 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:35,032 - NOTICE - keep_alive succeeded
2024-10-15 22:19:35,800 - INFO - Voice command detected
2024-10-15 22:19:35,801 - INFO - Sent data of type vad to WebSocket
Sending KeepAlive...
2024-10-15 22:19:37,190 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:37,190 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:37,190 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:37,190 - NOTICE - keep_alive succeeded
2024-10-15 22:19:37,517 - INFO - Processed 20 seconds of audio so far
2024-10-15 22:19:39,139 - INFO - Voice command detected
2024-10-15 22:19:39,139 - INFO - Sent data of type vad to WebSocket
Sending KeepAlive...
2024-10-15 22:19:40,032 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:40,032 - NOTICE - keep_alive succeeded
2024-10-15 22:19:40,876 - INFO - Processed 30 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:42,195 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:42,195 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:42,195 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:42,195 - NOTICE - keep_alive succeeded
2024-10-15 22:19:44,216 - INFO - Processed 40 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:45,034 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:45,035 - NOTICE - keep_alive succeeded
Sending KeepAlive...
2024-10-15 22:19:47,223 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:47,224 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:47,224 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:47,224 - NOTICE - keep_alive succeeded
2024-10-15 22:19:47,555 - INFO - Processed 50 seconds of audio so far
Text data received
2024-10-15 22:19:48,239 - DEBUG - Text data received
AsyncListenWebSocketClient._process_text ENTER
2024-10-15 22:19:48,239 - DEBUG - AsyncListenWebSocketClient._process_text ENTER
Text data received
2024-10-15 22:19:48,239 - DEBUG - Text data received
response_type: Metadata, data: {'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'c4bf9f35-0464-4912-8ed6-010866b2cc99', 'sha256': '22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b', 'created': '2024-10-15T22:19:30.018Z', 'duration': 0.0, 'channels': 0}
2024-10-15 22:19:48,239 - DEBUG - response_type: Metadata, data: {'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'c4bf9f35-0464-4912-8ed6-010866b2cc99', 'sha256': '22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b', 'created': '2024-10-15T22:19:30.018Z', 'duration': 0.0, 'channels': 0}
MetadataResponse: {
    "type": "Metadata",
    "transaction_key": "deprecated",
    "request_id": "c4bf9f35-0464-4912-8ed6-010866b2cc99",
    "sha256": "22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b",
    "created": "2024-10-15T22:19:30.018Z",
    "duration": 0.0,
    "channels": 0
}
2024-10-15 22:19:48,239 - VERBOSE - MetadataResponse: {
    "type": "Metadata",
    "transaction_key": "deprecated",
    "request_id": "c4bf9f35-0464-4912-8ed6-010866b2cc99",
    "sha256": "22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b",
    "created": "2024-10-15T22:19:30.018Z",
    "duration": 0.0,
    "channels": 0
}
AsyncListenWebSocketClient._emit ENTER

[nik0811]

from my frontend i am sending the data via sendAudioMessage(event.data)

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
        const mimeType = MediaRecorder.isTypeSupported('audio/webm') ? 'audio/webm' : 'audio/mp4';

        closeChatConnection();
        const token = localStorage.getItem('access_token') || '';
        initializeAudioWebSocket(token);
        initializeVadWebSocket(token);

        const audioContext = new AudioContext();
        const source = audioContext.createMediaStreamSource(stream);
        const processor = audioContext.createScriptProcessor(4096, 1, 1);

        source.connect(processor);
        processor.connect(audioContext.destination);

        processor.onaudioprocess = (e) => {
          const inputData = e.inputBuffer.getChannelData(0);
          const pcmData = new Int16Array(inputData.length);
          for (let i = 0; i < inputData.length; i++) {
            pcmData[i] = Math.max(-1, Math.min(1, inputData[i])) * 0x7FFF;
          }
          const blob = new Blob([pcmData.buffer], { type: 'application/octet-stream' });
          // Only send VAD message if the blob size is greater than zero
          if (blob.size > 0) {
            sendVadMessage(blob);
          }
        };

        mediaRecorderRef.current = new MediaRecorder(stream, { mimeType });
        mediaRecorderRef.current.ondataavailable = (event) => {
          if (event.data.size > 0) {
            sendAudioMessage(event.data);
          }
        };

@jkroll-deepgram

[jkroll-deepgram]

@nik0811, for the request ID in your log (c4bf9f35-0464-4912-8ed6-010866b2cc99), I see on our backend that you sent Deepgram invalid/corrupt audio data. I recommend debugging your audio source and transmission in the containerized environment.

[nik0811]

@nik0811, for the request ID in your log (c4bf9f35-0464-4912-8ed6-010866b2cc99), I see on our backend that you sent Deepgram invalid/corrupt audio data. I recommend debugging your audio source and transmission in the containerized environment.

Thanks, at least i got the clue, i will have a look, today and will update you with the result accordingly.

[nik0811]

It started working but now facing the issue where i am not able to receive complete audio, it is getting closed before the entire response arrives. I am using the below config,

options = SpeakWSOptions(
            model="aura-luna-en",
            encoding="linear16",
            sample_rate=16000,
        )

and also in speech to text the utterance end is not working properly here i am using,

        encoding="linear16",
        channels=1,
        sample_rate=16000,
        model="nova-2",
        language="en-US",
        smart_format=True,
        punctuate=True,
        interim_results=True,
        utterance_end_ms=2000,
        vad_events=True,
        endpointing=300

@jkroll-deepgram