Modelization based on frequencies mapping

[alexbelgium]

Hi, I'm a contributor to BirdNET-Pi and was quite interested by your models on bats. I'm also developing an addon for Battybirdnet-pi for homeassistant here : https://github.com/alexbelgium/hassio-addons/tree/master/battybirdnet-pi

I wanted to see if I could merge your model on the main Birdnet-pi fork, but for that i would need a direct tflite model on the 0-15khz range. As I'm a bit of a tinkerer, I wondered if it was not possible to translate main frequencies into the 0-15khz needed for the model.

In particular, have you tried in your creation of the model :

• Map by bands, for example : 15-40khz to 0-5khz ; 40-60khz to 5-10khz ; 60-120khz to 10-15khz which allows to keep the information of initial frequency but decreases the quality of information
• Segment the total audio signal in 15khz bands, discard the first one (0-15khz), estimate the 2 bands with most signal, and overlay them (for example, overlap 15-30khz and 30-45khz to make a fingerprint of the calls). This provides the same quality of signal as the initial file, while losing however the average frequency as differentiator. However, due to the overlap of the segments, the model could re-create this differentiator information from the average frequency in the 0-15khz band ?

Of course then the wav would not be representative of the initial sound, however if this is donc in a systematic manner shouldn't it allow to train a model to detect those species in files that were treated in the same manner ?

I've tried to do it on the Bavaria sample files that you have in the Readme but I'm not sure there is enough files (of enough quality) to be able to test the hypothesis.

[rdz-oss]

Hi @alexbelgium ,

I have not tried that particular approach.

• I believe that Patrick from BirdNet-Pi and some people tried to do a mapping to the 0-15kHz range which worked more or less lossless but took several seconds too long to be used on a real-time stream on the Pi4. Maybe that has changed with a Pi 5. The script was in the and some discussion were here somewhere I believe: Bat detection with BirdNet-Pi mcguirepr89/BirdNET-Pi#300
• The model for bats is derived with a modified fork of the BirdNET-Analyzer . That system is designed to fine-tune for specific birds in your area. It uses a thin layer of a classifier behind the BirdNET ANN. The output of the ANN is used as is and it is 'filtered' or mapped by a linear classifier.
• For bats, it is modified and more transfer learning than fine tuning. The Analyzer approach remains the same, however, i.e. the BirdNET ANN and a classifier for bats is needed.
• You can check the analyze code in the BattyBirdNET-Analyzer to see how that works in detail: https://github.com/rdz-oss/BattyBirdNET-Analyzer/blob/main/analyze.py#L214
• The .tflite files are here: https://github.com/rdz-oss/BattyBirdNET-Analyzer/tree/main/checkpoints/bats/v1.0

Hope that helps,
Richard

[alexbelgium]

Hi Richard,

Thanks very much, I will look at the resources you have linked to and try it in my area !

Best regards,
Alexandre