Slow Audio Output AND MPEG_READER: Not find an effective frame, line:639. #38
Comments
|
freez on detect "wake_word" |
The announcement media_player state has been merged into ESPHome 2024.5.0, so there is no need of including adf-mods anymore.
I just merged a new config parameter for the media_player into dev-next. It lets you configure the audio settings used for announcements. Now, the media_player will try to detect the announcement audio settings if the new config is not set. This consumes some extra time though. Here is an example: media_player:
- platform: adf_pipeline
id: adf_media_player
name: s3-dev_media_player
internal: false
keep_pipeline_alive: false
announcement_audio:
sample_rate: 16000
bits_per_sample: 16
num_channels: 1
pipeline:
- self
- resampler
- adf_i2s_out
currently, only mp3 files are supported. There is a PR in the queue though which will add a couple of other audio codecs.
can you try different radio stations and see if there is a pattern which do work and which don't? |
this error occurs here: ( announcement_audio: )
all radio links are working, checked separately. Deutschlandfunk [MP3 128k]Вести ФМ (Vesti FM) [MP3 192k]``` [15:21:15][D][media_player:061]: 'media_player' - Setting [15:21:15][D][media_player:068]: Media URL: http://icecast.vgtrk.cdnvideo.ru/vestifm_mp3_192kbps [15:21:15][D][esp_audio_sources:058]: Set new uri: http://icecast.vgtrk.cdnvideo.ru/vestifm_mp3_192kbps [15:21:15][D][adf_media_player:047]: Got control call in state IDLE [15:21:15][D][esp_adf_pipeline:060]: Starting request, current state UNINITIALIZED [15:21:15][D][esp_adf_pipeline:437]: [MediaPlayer] Pipeline changed from UNINITIALIZED to INITIALIZING. (REQ: 0) [15:21:15][I][adf_media_player:167]: got new pipeline state: 1, while in MP state IDLE [15:21:15][I][adf_media_player:226]: current mp state: IDLE [15:21:15][I][adf_media_player:227]: anouncement: false [15:21:15][I][adf_media_player:228]: play_intent: false [15:21:15][I][adf_media_player:229]: current_uri_: yes [15:21:15][D][esp-idf:000]: I (4435841) MP3_DECODER: MP3 init[15:21:15][D][esp-idf:000]: I (4435843) I2S: DMA Malloc info, datalen=blocksize=512, dma_buf_count=4 [15:21:15][D][i2s_audio:072]: Installing driver : yes [15:21:15][D][esp-idf:000]: I (4435854) AUDIO_PIPELINE: link el->rb, el:0x3d82023c, tag:decoder, rb:0x3d821774 [15:21:15][D][esp-idf:000]: I (4435856) AUDIO_PIPELINE: link el->rb, el:0x3d8203d8, tag:resampler, rb:0x3d8227b4 [15:21:15][D][esp_adf_pipeline:496]: Setting up event listener. [15:21:15][D][esp-idf:000]: I (4435905) AUDIO_ELEMENT: [http-0x3d820084] Element task created [15:21:15][D][adf_audio_element:108]: Preparing [decoder]... [15:21:15][D][esp-idf:000]: I (4435909) AUDIO_ELEMENT: [decoder-0x3d82023c] Element task created [15:21:15][D][adf_audio_element:108]: Preparing [resampler]... [15:21:15][D][esp-idf:000]: I (4435922) AUDIO_ELEMENT: [resampler-0x3d8203d8] Element task created [15:21:15][D][adf_audio_element:108]: Preparing [i2s_out]... [15:21:15][D][adf_audio_element:165]: Resuming [http]... [15:21:15][I][adf_media_player:167]: got new pipeline state: 11, while in MP state PLAYING [15:21:15][D][esp-idf:000]: W (4435983) AUDIO_ELEMENT: [http] Element has not create when AUDIO_ELEMENT_TERMINATE [15:21:15][D][esp-idf:000]: W (4435985) AUDIO_ELEMENT: [decoder] Element has not create when AUDIO_ELEMENT_TERMINATE [15:21:15][D][esp-idf:000]: W (4435987) AUDIO_ELEMENT: [resampler] Element has not create when AUDIO_ELEMENT_TERMINATE [15:21:15][D][esp-idf:000]: W (4435988) AUDIO_ELEMENT: [i2s_out] Element has not create when AUDIO_ELEMENT_TERMINATE [15:21:15][D][esp-idf:000]: I (4435990) I2S: DMA queue destroyed [15:21:15][D][esp_adf_pipeline:437]: [MediaPlayer] Pipeline changed from DESTROYING to UNINITIALIZED. (REQ: 4) |
please try adding a refresh parameter: - source:
type: git
url: https://github.com/gnumpi/esphome_audio
ref: dev-next
components: [ adf_pipeline, i2s_audio ]
refresh: 0s |
|
the sound is fine now. the problem with freezing remains: If you turn on the music in the player after that everything is ok, but the voice assistant doesn't respond ( pulse green light still ). |
|
first command [16:46:29][D][voice_assistant:563]: Event Type: 6 - doesn't happen after the second command |
|
yes, I just found a bug which prevents setting the MP_ANNOUNCE define correctly. I am about to submit a fix, just waiting for some tests to finish |
done! Could you try again? |
cool, I'm already compiling, testing and will report back. |
|
the response speed became slow, but the freeze error disappeared my code |
|
If I see it correctly in your logs, then you are using a TTS voice with 24000 sampling rate, you should set this value in the announcement_audio settings. |
yes, but it worked in the previous step. |
because of the bug I just fixed, the new announcement_audio settings were not used. But they are now ... you can also leave announcement_audio unset, then it should detected the audio format automatically, but this takes a while which can be problematic if you request playing an audio stream via voice command. |
|
Got it, thanks. It's working now. No error, continuing the test. |
|
if you give commands quickly, it happens again. |
|
I've also noticed this error: |
That's strange that none of the radio streams is working for you. I also have some issues with 192kbp mp3 streams which fail to decode every now and then. With 128kbp I don't have any issues. The http client somehow detects the mp3 stream sometimes as an aac stream, I don't see why. Also I don't know if the decoder even uses this information, unfortunately these codecs are closed source. Debugging the adf-sdk exceeds the time I want to spend for a private project like this, so I won't do it at the moment. |
oh, thanks for reporting. It seems it tries to free some components which haven't been setup yet when stopping directly after starting the creation process. I will look into that. |
Do you use logging via Wifi? I think this error is not related to the media player itself. |
can you send me a link to the 128kbs radio that works for you?
I don't know what you mean. |
|
Oh I get it, yes all logs go through wifi, not through UART or wire |
sure, e.g these never make any problems on any of my devices:
while e.g. this refuses to work every now and then: while copying I realised that this is also the only https stream, this could also be the reason |
|
All your links work through programs like VLC or AIMP, but not through the media player.
|
|
I also came across this: https://github.com/sehraf/esphome-components UDP streaming , I think you can convert everything on the fly via ffmpeg to mp3 |
|
yes sure, you can also simply use https://music-assistant.io/ to stream radio stations and Spotify to the adf media_player. This works great. But it would be nice to know whats going on and to support all streams natively. But you are right, there are workarounds available so let's solve other issues first ;) |
I ran through “music assistant”, with customization (forced mp3 in the player settings) and the radio works without problems continuing the test |
|
Hello everyone, Can anyone help me? The output of the assistant is low and when I play an mp3 file or a radio stream, which also uses mp3, the sound output is very slow. if i set the announcement_audio settings, then the audio output for the assistant is better but for mp3s it remains slow. here is my used script and log. Voice Assistant log: Playing a mp3 log: thank you very much!!!! |
|
what kind of hardware are you using? |
|
I am using as Hardware |
|
In duplex mode the channel format for the input and output needs to be the same as well, unfortunately. Can't you use dedicated clock pins for both components, not running in duplex mode saves you from many troubles. |
if you had time to deal with this problem: sequential dictation of commands with a small pause causes the processing pipeline to freeze. |
are you using the dev-next branch? |
|
yes my code is below, I just upgraded to 24.5.5 when I saw the mention of ring buffer hoping it would solve something, but no. |
|
the stopping point is here: LED blinks green |
|
I apologize, but this is a new error: Video.2024-06-09.14-31-35.mp4 |
ah ok, I understand that |
So I am tested the another code, thats work better. Thanks. |
|
(I think) after updating to the latest esphome version, I have this slow audio playback problem again. Relevant config: |
|
I was having the same problem with the slow voice assistant audio. I made the sample rate change to 24000 and tested but was still getting the same result. Out of curiosity I doubled the bits per sample to 32 and the voice assistant is back to normal. I don't use mine to play music or anything so I can't comment on the affects of that. |
|
Hi, yesterday I did play around with those parameters and also tried your suggestion then. When using 24000/32 I get high pitched audio the first time, second time no audio at all. When using: I get low pitched audio the first time, second time and after it is correct. So now it only sounds low-pitched the very fist time after booting. |
|
if you remove the announcement_audio section completely from you media_player config, the right settings should get auto detected. It takes slightly longer though... |
|
When using: It works just fine, so it must be something in the latest dev-next changes. Removing the: Does not help. |
|
thx, thats really helpful information. I will check. |
|
@HA-TB303, can you try setting your bits_per_sample explicitly in the pipeline settings: adf_pipeline:
- platform: i2s_audio
type: audio_out
id: adf_i2s_out
i2s_audio_id: i2s_out
i2s_dout_pin: GPIO17
bits_per_sample: 16bit
- platform: i2s_audio
type: audio_in
id: adf_i2s_in
i2s_audio_id: i2s_in
i2s_din_pin: GPIO15
pdm: false
channel: right
sample_rate: 16000
bits_per_sample: 32bitand please share some logs, if it doesn't help |
|
@mstamp20, can you try changing the order of the resampler and i2s-output in your media player config: and set bits_per_sample to 16bit media_player:
- platform: adf_pipeline
id: adf_media_player
name: s3-dev_media_player
keep_pipeline_alive: false
internal: false
announcement_audio:
sample_rate: 24000
bits_per_sample: 16
num_channels: 1
pipeline:
- self
- resampler
- adf_i2s_out |
It now sounds fine from the first time. Config: |
I tried this and it went back to slow motion for the first response and then worked fine the rest of the time. I did a few different tests as well. I went through all the variations without resampler and all with resampler. Here are the results I had: The above audio came back slow motion. The above was a little faster but still slow motion. The above was slightly better than the previous, still low and slow. The above played perfectly normal the entire time! Removing announcement audio and adding resampler played slow on the first response and normal the rest of the time. This was extremely slow the first response and slow the rest of the time. This was slow on the first response and normal the rest of the time. This was fast the first response and then slow the rest of the time. This was extremely fast the first response and then normal the rest of the time. I hope this helps! |
|
thx, first of all I need to find out why it behaves differently the first time compared to the other times, this should definitely not be the case. thx for pointing at this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi I'm using your latest code:
HW: Max98357 + INMP441 + esp32s3
esphome: 24.5.0
my script, maybe I'm missing something:
log
log
The text was updated successfully, but these errors were encountered: