Batch Processing with BirdNET-Pi

[mcguirepr89]

Original comment/issue: kahst/BirdNET#24 (comment)
Hi, Thomas! @tdelattre

Maybe we should continue this discussion from BirdNETPi's GitHub?

Good idea, I always have needlessly long responses :)

Our goal is to batch-process with BirdNET a huge number of sound files recorded on automatic recorders. The idea is to do that on a server to make use of its computing power, and in particular to run multiple threads at once to accelerate the process. If BirdNET-Pi could do that, that would be wonderful!

BirdNET-Pi can certainly do that for you, and the system does utilize threading (though we've not seen drastically better results).
BirdNET-Pi, however, does a lot more, and may be a little bloated for what your needs are.

BirdNET-Pi does the following (I will situate your needs in bold):

• 24/7 recording
• 24/7 Analysis of those recordings
  • Batch Process a ton of files
• Analysis creates MariaDB database entries into a detections table
• It can also POST those detections (audio and spectrogram visualizations) to BirdWeather.com
• From the analysis data, audio extractions of the detected regions are created, organized by time, species common name, and species binomial scientific name. Those extraction lengths can be adjusted from 3 seconds up to 60 seconds to provide audio context. Likewise, spectrograms are created for each audio extraction.
• All data is made available through a web interface served with Caddy. PHP provides dynamic database views.

Take a look through my example site to see everything listed above https://birdnetpi.pmcgui.xyz

With all of that said, it seems like a full installation of the BirdNET-Pi would be overkill for your needs. OR you may decide that it is very useful to you to have the detections automatically extracted and organized and placed into a database.

If it does seem like overkill, my suggestion is a simple installation script and utilizing the socketed server/analysis system BirdNET-Pi utilizes (more on that below). Then a loop can feed the audio files to the server socket and your analysis will run through quickly.

One thing I'm unclear about is if there are differences in the results provided by BirdNET and BirdNET-Pi? I.e. will the detections and identifications be the same or does the choice of TFLite (for example) in BirdNET-Pi affect the results, compared to BirdNET?

This is an important point. BirdNET-Lite (BirdNET-Pi) will yield different results for three primary reasons:

1. The BirdNET-Lite model is over 6 times the size of the standard BirdNET model (+6,000 species compared to +900 species, respectively).
2. The standard BirdNET analysis model utilizes 2018's eBird grid data for post-processing the prediction confidence scores, while BirdNET-Lite does not utilize any eBird grid data. This means that the standard model leads to less false positives, but also leads to more false negatives. If your goal is to survey common, North American species, the standard model will likely yield less false positives and lower confidence scores for rare species. If, however, your goal is to survey all species for an area (not just North America), you will want to use the BirdNET-Lite model.
3. The third reason is alluded to in the reasons above, but the standard BirdNET model focuses primarily on North American species, while the BirdNET-Lite model is suited to handle nearly any region with similar success (though I've not traveled the world to test, there are BirdNET-Pi users in 11 countries, and 3 continents, and we're all seeing great results).

Truly your aim should guide you to the model, but for what it may be worth, I have focused development for BirdNET-Pi around the BirdNET-Lite model because it is more efficient and the model itself recognizes more species. Also, I would be more upset about a false negative (missing a great bird call) than a false positive (the bird call ending up being a car or a kid).

To elaborate a bit on how BirdNET-Pi's analysis differs from the standard BirdNET project and the BirdNET-Lite project:
For context:

• The standard BirdNET analyze.py script builds the model once, loads the labels once, and then processes all of the files it finds in a particular directory.
  • Pros: This can process all of the files in a directory, better results for common North American species
  • Cons: Uses the standard BirdNET model (less species), uses old eBird grid data
• The standard BirdNET-Lite analyze.py script builds the model once, loads the labels once, and then processes the file you provide it.
  • Pros: Uses TFLite model, so more efficient and more species
  • Cons: Processes one file at a time

What BirdNET-Pi does:

• The analyze.py script has been split in two to create a server socket service to which the analyze.py script sends its data. The server.py service builds the model once, loads the labels once, and waits for data.
  • Pros: Accepts a continuous feed of files, uses TFLite, more species
  • Cons: systemd hates handling sockets as services, so service restarts are tedious

With all of that said, this community is great and knows their stuff, so I'm hoping (and confident) that a call for answers here will lead you to exactly the tool you need (or I'll build it for us 😄)

Hi, Frank! @DD4WH, I would bet anything you've done a lot of batch processing with BirdNET (both versions) -- any suggestions for our new friend Thomas @tdelattre? I would bet there is a fork somewhere for just this need (am I the only one who names their fork differently????)

@tdelattre, let me know your thoughts on what I've laid out above and cross your fingers that this great community will point you in the right direction (or perhaps you will see the appeal of BirdNET-Pi and will ditch your digital recorders for Pis 😄)

[oliverk71]

I like the idea of running birdnet-pi on a server where users can upload recordings and get back the species recognized. And what about a desktop application which can be fed with recordings? Because many birders may have huge amounts of data which not necessarily needs to be uploaded if it can be done on the local computer. Will maybe end up in a whole linux distro for birders: Birdnet Linux :)

[tdelattre]

Agreed on all points :-)

[DD4WH]

We are working on providing an R script for batch processing of audio recordings based on the BirdNET-Lite classifier and for Windows OS.
This is still work in progress and will also contain R code for basic graphical analysis too.
Hopefully we will be able to provide more information on this in one or two months.

[tdelattre]

That's cool @DD4WH ! Do you plan on making an R package out of it? Why would it be windows-specific?
Off the top of my head, I would imagine that it would be less efficient in processing a lot of files (because R isn't that fast and because most servers use Linux). But the integration with R post-processing / graphical analysis would be cool. Some of the things made by the seewave package are really handy.

[tdelattre]

Hi Patrick @mcguirepr89 !
Sorry for the delay, I had a lot to manage with covid-school-kids stuff.
Thanks for your detailed answer.
Indeed BirdNET-Pi does a lot more than what we need right now, but the other functions seem really useful and may come in handy down the line. (by the way, your example site is down today, it was online last week)

If it does seem like overkill, my suggestion is a simple installation script and utilizing the socketed server/analysis system BirdNET-Pi utilizes (more on that below). Then a loop can feed the audio files to the server socket and your analysis will run through quickly.

That would be perfect! Actually, a full installation doesn't bother me (unless you think of some reason it would prevent batch processing). The main things would be 1/ be able to install BirdNET-Pi on a non-Pi OS like ubuntu and 2/have a loop that feeds the audio files to the server socket. Unfortunately, I'm not savvy enough to do either so if anybody in the community could do that, it would be wonderful! I'm sure I wouldn't be the only one to benefit, especially in the scientific and pro naturalist communities which rely a lot on ARUs.

This is an important point. BirdNET-Lite (BirdNET-Pi) will yield different results for three primary reasons:

From what you say it seems obvious that the BirdNET-Lite model is the one to choose (especially since we'll work on European species). Thanks for the info!

[mcguirepr89]

Hi, Thomas @tdelattre

The main things would be 1/ be able to install BirdNET-Pi on a non-Pi OS like ubuntu and 2/have a loop that feeds the audio files to the server socket. Unfortunately, I'm not savvy enough to do either so if anybody in the community could do that, it would be wonderful! I'm sure I wouldn't be the only one to benefit, especially in the scientific and pro naturalist communities which rely a lot on ARUs.

I'll do this :)

I think there is a lot of value in being able to port BirdNET-Pi to any (Debian-based or at least systemd-based) Linux distribution. I will keep you posted on this. I'm juggling a few changes right now, so I will let you know when there is something worth testing.

@oliverk71

I like the idea of running birdnet-pi on a server where users can upload recordings and get back the species recognized. And what about a desktop application which can be fed with recordings? Because many birders may have huge amounts of data which not necessarily needs to be uploaded if it can be done on the local computer. Will maybe end up in a whole linux distro for birders: Birdnet Linux :)

Regarding an API for processing:
I do have a user working on putting together an upload feature. The main issue, however, is that TFLite can only handle 1 chunk of audio at a time (no matter how many threads are handling the process), so this feature disrupts the standard 24/7 analysis that needs to be running. With that said, this is not an issue when the system is not tasked with live monitoring and is quite doable -- one would simply need to upload a file to the server socket which could be handled via a reverse proxy on the Caddy server to add TLS and allow secure remote access to the socket.

I will keep you all posted on this work -- please let me know if you think of other functional needs and I will see what I can do.

My very best,
Patrick

[tdelattre]

Hi Patrick @mcguirepr89

I'll do this :)
I think there is a lot of value in being able to port BirdNET-Pi to any (Debian-based or at least systemd-based) Linux distribution. I will keep you posted on this. I'm juggling a few changes right now, so I will let you know when there is something worth testing.

That's wonderful! I stand ready to crash-test it as soon as you need alpha/beta testers!

I like the idea of running birdnet-pi on a server where users can upload recordings and get back the species recognized. And what about a desktop application which can be fed with recordings? Because many birders may have huge amounts of data which not necessarily needs to be uploaded if it can be done on the local computer. Will maybe end up in a whole linux distro for birders: Birdnet Linux :)

Yes, I think that could be really useful. I think public research institutions, in particular, could set up such a service so that everyone with an automated recording unit (farmers, teachers...) could get a list of species.

I'm always amazed to see people like you being ready to lend their skills and time to everyone's benefit. I guess I should be amazed that everyone else doesn't do the same, but that's the world as it is. Anyway, thanks again!

Thomas

[mcguirepr89]

Hello! @tdelattre

I apologize for the relative silence recently -- I've moved to another state and that move lived up to the notorious reputation moving has earned.

I wanted to let you know about an exciting new tool that will be available in the next several weeks:
The new BirdNET model supports batch processing -- while I know I offered to put together a batch processor for BirdNET-Lite, it would certainly be in your best interest to wait just a bit longer for the new model's release at which point the batch processing will be available to you outside of any full-system installation like BirdNET-Pi. In addition, the model is the newest that is available and will receive regular updates.

Wishing you an abundance of patience while waiting for this exciting new BirdNET release,
Patrick

[tdelattre]

Hi Patrick, Thanks for the heads-up.

Hello! @tdelattre <https://github.com/tdelattre> I apologize for the relative silence recently -- I've moved to another state and that move lived up to the notorious reputation moving has earned.

I know, right. If only we had a script for that ^^

I wanted to let you know about an exciting new tool that will be available in the next several weeks: The new BirdNET model supports batch processing -- while I know I offered to put together a batch processor for BirdNET-Lite, it would certainly be in your best interest to wait just a bit longer for the new model's release at which point the batch processing will be available to you outside of any full-system installation like BirdNET-Pi. In addition, the model is the newest that is available and will receive regular updates.

That's super cool! Is there a place where I can follow the progress on this project? (is it still developed by Stefan Kahl?) By "outside of any full-system installation like BirdNET-Pi" you mean that it will be available as an app? Wishing you an abundance of patience while waiting for this exciting new

BirdNET release, Patrick

Thanks a lot, Thomas

…

— Reply to this email directly, view it on GitHub <#155 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD6TNLQLMQ5KBIF3CMEVKFTVB4EBBANCNFSM5OUZNYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>