BirdNET-Analyzer and new models online!

[nilspupils]

Hi everybody,
just wanted to let you know that Stefan Kahl @kahst released BirdNET-Analyzer with the new models. A big big "thank you" for making this awesome piece of software availeable to the public!
As it seems @mcguirepr89, @DD4WH and @CaiusX were also involved in this! I would love to know some more details about it.
Very exciting!
Check it out here:
https://github.com/kahst/BirdNET-Analyzer

Best,
Nils

Edit:
And obviously (but totally missed by me) the one and only @mcguirepr89 is already on it!
https://github.com/mcguirepr89/BirdNET-Analyzer-Pi

Edit2: Please consider supporting the development of BirdNET and BirdNET-Pi by sponsoring or donating!

[mcguirepr89]

Hi, @nilspupils and everyone!

BirdNET-Pi on the new model is still very much in development, though if you felt like taking a peek at what it does at this point, you're more than welcome.

The installation one-liner is:

curl -s https://raw.githubusercontent.com/mcguirepr89/BirdNET-Analyzer-Pi/main/install_birdnetpi.sh | bash

At this point, the core services are there:

• Recording service
• Analysis service
• Extraction
• Streamlit
• (a new) Weather service
• and Caddy to view files during testing

There is an uninstall.sh script meant to be an easy way to uninstall things during testing to test the installer.

There are a few bash aliases that are included -- you can view those in the file bash_aliases. These are meant to make development in the terminal a bit easier for folks.

I certainly am not requesting folks test the installation yet, but I am also not going to stop anyone who is interested in trying it out at this stage.

If you do decide to play around with it, let me know what you think. Be aware that the new model has far fewer European species than the current BirdNET-Pi 6K model.

Happy birding!

[drahdiwaberl]

Be aware that the new model has far fewer European species than the current BirdNET-Pi 6K model.

might the text in the settings of birdnet-pi be misleading then?

Currently it only supports about 3,500 species worldwide, meaning for some regions (North America, Europe, Australia) it will usually outperform the BirdNET-Lite model, but for other regions it will be worse.

had it activated the first night after installation as i interpreted the sentence as suggesting this model being better ('outperforming') for Europe - but just got some detections that definitly aren't common round here..

[ehpersonal38]

Patrick's comment is a year old, that was model v2.1 which had about 1000 less species than the current model has (the model has a total of 3300). That being said I don't know the exact species coverage for the new models, that text was only added after anecdotal reports from several of our European users that the new model was working well there. But if it's not for you, feel free to change back to the old model until the V2.4 6K model gets released.

[nilspupils]

@mcguirepr89: I totally missed that you are already working on it. That is great news! How does it compare to the old model?

[mcguirepr89]

How does it compare to the old model?

The new model and its toolset are quite impressive:

• The model has a ton of meta data that the analyze.py script uses to make the results more accurate -- the analysis script accepts an argument ("--sf_thresh") that adjusts the "Minimum species occurrence frequency threshold for location filter." This seems tantamount to post-processing with eBird grid data, as it essentially decides whether a species should be considered for the classification based on its recently known presence.
• The segments.py tool is able to fully replace the extract_new_birdsounds.sh script that I put together for BirdNET-Pi. This amazing little tool can be run after the analysis to extract the detections just like they are handled on BirdNET-Pi (though extract_new_birdsounds.sh accomplishes this using sox and bash where segments.py is just python). Its only "shortcoming" compared to extract_new_birdsounds.sh is that you are not able to set the audio format, so everything stays in the original recording format (WAVE for now).
• There is a tool called "species.py" that is able to generate custom species lists by simply passing it geo-coordinates. This can be helpful when you want a custom species list and do not wish to provide Latitude and Longitude during analysis. BirdNET-Pi will still use Lat and Lon, but users will be able to use the species.py tool to generate local species lists should they find a need for that.
• There are a "server.py" and "client.py" that basically break up the analysis the way we chopped it up for BirdNET-Pi. "server.py" will build the model and load the labels and expose the analysis over a port, then "client.py" can send the server the audio for analysis. The server can be configured to store analyses it performs while also sending JSON formatted responses with a user-defined number of prediction results returned, allowing for multiple species predictions per audio sample. (Development on this new repo currently just uses the analyze.py script, but will eventually migrate over to using the server.py and client.py as in the current BirdNET-Pi architecture.)
• Lastly, there is a tool with which I know not what to do! I'll need to quote the repo on this, since I don't know what it means just yet, but the "embeddings.py" script can be used "to extract feature embeddings instead of class predictions. Result file will contain timestamps and lists of float values representing the embedding for a particular 3-second segment. Embeddings can be used for clustering or similarity analysis." So that's cool . . . 😐
• Currently, the new model is about half the size of the 6K model we use for BirdNET-Pi, but it will continue to grow and will eventually be 6K in size, but much more powerful thanks to the metadata and the above-mentioned amazing toolset. This means European users interested in rare species might find the new model does not yet meet their needs, but I'm confident that it soon will.

As of now, I am not allowed to accept any sponsorship donations to aid in developing BirdNET-Pi around the new model. I am still allowed to accept sponsorship and donations for BirdNET-Pi built on the 6K model, however. I really hope this will change in the future so that BirdNET-Pi can continue to develop alongside the current model without hitting the same resource constraints with which I have become increasingly familiar. Cross your fingers that HaikuBox eases up on the restrictions set forth in the license to allow folks who wish to support BirdNET-Pi the freedom to do so.

My best!

PS I sent you an email to which I will kindly ask you reply :)

[nilspupils]

@mcguirepr89 : Thanks for the details! I will check it out when i have the time. Email sent!

[dbrianhendrix]

I saw that several days ago. And it's incredibly faster than previous method. I have two Audiomoths, one in front yard and one in back yard, that record the morning bird chorus from 5:30am to 8:00am CST. The recordings are in 60 second segments. I review them with Audacity, and any new "birds" that I "see" in the spectrograms, I export as a WAV file. I have a "Quick Action" (macOS) created that uses a bash script I wrote so that I just have to right click on the WAV file, select "BirdNET-AnalyzeWAV", and it starts a Docker implementation of BirdNET-Analyze, analyzes the file, and then outputs a CSV file. The script then takes that CSV file and opens it for me in Numbers which I can manipulate as I see fit like any other Numbers / Excel file. Using this process with previous BirdNET could take 20-30 seconds where now I'm seeing 3-5 second analysis time!

[mcguirepr89]

Yes, the new model hands down outperforms for all North American species

[AvnerCohen]

Thanks for the help @mcguirepr89 and @ehpersonal38 This is very encouraging for NA species, I am hoping to see something similar for this area, mainly WP birds, with some african and asian species.

That being said, I was unsuccessful with the install so far.
I am actually not quite sure what do expect, so maybe I'll start with this:
The caddyserver config on the Analyzer repo is very different (https://github.com/mcguirepr89/BirdNET-Analyzer-Pi/blob/main/install_birdnetpi.sh#L137)
So I am wondering if I should even expect to see the full setup (including the PHP ui that provides statistics, recordings and species lists)
Or is the *Analyzer repo not yet ready for that and still require manual analysis of the recordings ?

I am able to run the ./analyze.py manually, but I am looking for the "whole package" and more of an automatic analysis of recordings at this point.

[mcguirepr89]

The BirdNET-Analyzer-Pi repo is still very rudimentary, but does have all of "Core" analysis features:

• Recording Service
• Analysis Service
• Extraction Service
• Weather Service (new)
• Streamlit Analysis (the "Species Stats" page)

Though there is no other web interface, just the files and the Streamlit analysis. As mentioned, you'll want to manually adjust your Caddyfile using the Caddyfile in my comment as a template to get BirdNET-Analyzer-Pi data to be served alongside your BirdNET-Pi data.

It may be too much trouble to have them function in tandem :/

[AvnerCohen]

Thanks. I understand that, so will not be looking to run the two in parallel.
I simply want to test run the new model. Things are still not working, I'll try to troubleshoot that or will start from scratch.
My understand is that running this https://github.com/mcguirepr89/BirdNET-Analyzer-Pi#setup-raspberry-pi-raspios-arm64-lite-for-testing is basically all I need to setup the full website again. Is that correct?

[mcguirepr89]

That is partially correct -- let's open an issue there to discuss this.

I'll see about running a test installation soon to see if I encounter any errors.

For the issue on the BirdNET-Analyzer-Pi repo, after installing with the curl command, let me know what you see when you go to http://<THE_RASPBERRY_PI_HOSTNAME>.local and http://<THE_RASPBERRY_PI_HOSTNAME>.local/stats

My understand is that running this https://github.com/mcguirepr89/BirdNET-Analyzer-Pi#setup-raspberry-pi-raspios-arm64-lite-for-testing is basically all I need to setup the full website again. Is that correct?

Again, there are only the core services for this repo and a minimal file serving web browser -- the Streamlit data visualization is available as well, but no other web interface, so no this will not setup the full website again. It will, however, record, analyze and extract the data while offering Streamlit data visualization at http://<THE_RASPBERRY_PI_HOSTNAME>.local/stats after install.

We'll troubleshoot further on that repo if you'd like ~~

My best,
Patrick

[tomwhite101]

Great work Patrick. Here's hoping that the various restrictions are lifted and the new model can cover the rest of the world too!

[jds3d]

Is the BirdNET-Analyzer a replacement system for BirdNET-Pi but with the updated model? I tried installing it on the same system using "curl -s https://raw.githubusercontent.com/mcguirepr89/BirdNET-Analyzer-Pi/main/install_birdnetpi.sh | bash" and then went to the http://birdnetpi1.local/ website to check it out but all I see now is the file system. I'd really love to try out the new model, as I get a ton of false positives that Merlin can easily identify. For example, today it heard a Blackpoll Warbler that was correctly identified as a Northern Cardinal by Merlin.

Or alternatively, is it only a command line analyzer at this point in time and I should wait for you to finish the work?

[mcguirepr89]

@nilspupils !!!!

Login:

Detections:

User profile management (username, password, Google Auth 2FA):

Manage Browser/Session Cookies:

Set user avatar image:

Edit the BirdNET-Analyzer Config File (config.py):

API TOKENS!!!!!! Super easy to issue and set permissions:

APIs are awesome and easy to extend!!!!

Coming soon

[rangerdavid]

Thanks for the update, and hey, there's always time to become a park ranger!

Okay, I'm a little confused by the different versions, I thought there was one that was now using an updated detection algorithm, "Analyzer" or some-such. Is that now part of the main BirdNET-Pi fork/project?

And yes, the above screenshot is pretty close to what we would need. For a "Kiosk Mode" I think the interface could be 90% a scrolling list of detection in the last 24 hours (so it isn't too long) that someone with a touchscreen could scroll down through. Also, 10% a fixed box or header on the screen where the site name and daily and annual species detection totals (like: Tulelake Wildlife Refuge Discovery Marsh: 12 unique species today, 210 species this year) would always display. And of course, no other buttons allowing the user to leave the screen without a keyboard - like, "Esc" or other key would return to a normal page. So I can lock the RPi in a cabinet below a touchscreen digital exhibit in the visitor center, and access it with a keyboard as needed. Otherwise, it just runs like an appliance 24/7.

Thoughts?

[Svardsten53]

How about implementing this request on a hidden page since it is a special request for visitors?

I myself have been on the same track and worked a bit with making an alternative start page that instead shows the spectrogram view.

[ehpersonal38]

@rangerdavid

Okay, I'm a little confused by the different versions, I thought there was one that was now using an updated detection algorithm, "Analyzer" or some-such. Is that now part of the main BirdNET-Pi fork/project?

There's a separate project called BirdNET-Analyzer, it's basically the main repo where the BirdNET model gets updated and released to the public. Patrick (the creator of BirdNET-Pi) started working on a new version of BirdNET-Pi using the BirdNET-Analyzer model, but he's shelved it.

Also, I've done some work on the "kiosk mode" and pushed it to the main repo, so you're able to check it out if you have a working BirdNET-Pi installation by going to Tools -> System Controls -> Update. After that's finished, you can access kiosk mode by manually going to the URL http://birdnetpi.local/views.php?view=Kiosk, replacing birdnetpi.local with your respective installation URL or IP address.

I've done quite a bit of optimizations such as:

• removing all the buttons users could click
• adding a "scroll back to top" button in the bottom right of the page
• having the detection list auto-refresh every minute
• changing the detection time to a more friendly relative time, so "17:39:24" may be converted into "1 minute ago," for example.

If you're have set your Flickr API key in Tools->Settings you'll also get bird images next to each detection which are clickable to open a larger view of the image.

So, check that out if you're able to, and let me know any feedback you have. 😄

[rangerdavid]

Boy-howdy, that was fast!!! I'm going to update and check this out. If/when we have it installed, I'll post photos!

MANY THANKS!!!!

[jflyons]

Love to see the photos of your installation, and even better, visit Tulelake NWR and see it in action :) (We're nearly neighbors!)

[ehpersonal38]

Hi,

Over the past few days I've done quite a bit of work in implementing the new model into BirdNET-Pi. I realize that there has been a lot of hesitancy surrounding its implementation, mainly in regard to its poor coverage of worldwide species, so for now the old BirdNET-Lite 6K model will still be the default, with the new BirdNET-Analyzer 3k model being available on an explicit opt-in basis in the BirdNET-Pi settings. Hopefully this means that for users in regions where the new model excels, it will be available for use, and for everyone else, nothing will change.

But, I need testers! So if you're willing to try out the new model, this is how you can:

EDIT: The new model is now part of the main branch, meaning as long as you're updated via Tools -> System Controls -> Update, you will be able to access the new model from Tools -> Settings.

outdated steps

1. On your terminal, run the following update command: `bash ~/BirdNET-Pi/scripts/update_birdnet.sh -b new_model`. Wait for a minute or two while it runs. 2. On your BirdNET-Pi web interface, navigate to Tools -> Settings -> Model, and select the new model option under "Select a ModeL" ([the second option](https://i.imgur.com/p4Ps95w.png)). 3. Click "Update Settings" at the bottom, wait for it to do its thing, then check "View Log" at the top of the page to make sure nothing has catastrophically broken. 4. Please then navigate back to Tools -> Settings -> Model, and verify that your model settings have in fact saved and not been cleared or reset back to the defaults or any other bug.

If you encounter any errors or failures or successes along the way please let me know in the discussion dedicated to this new model branch: #699

Hopefully with all your help I can get this pushed to the main branch of this repo in the coming days.
🐤

[DD4WH]

@ehpersonal38
That sounds really great !
I will try that and report back, maybe you would like to open a fresh and new discussion for testing of that particular branch ?

[ehpersonal38]

I've gone ahead and created a new discussion post, so all future discussions should be posted here: #699

[Punkley]

I can't help but not the new model is up to 2.3 now
https://github.com/kahst/BirdNET-Analyzer

Can we get it upgraded :)

[ehpersonal38]

I will get this added right now! Thanks kahst for maintaining BirdNET!

[Svardsten53]

I would be very grateful if, as last time, you could change the Swedish species list so that each species name begins with a capital letter. I assume that there will be a new species list with the updated model and there the species names are misspelled due to the fact that the list is apparently based on data from eBird.
I have reported this error to eBird, but the hope that it will change is perhaps not so great.

Thank you!

[ehpersonal38]

@Svardsten53 Sure, I've just done a push which should hopefully make the changes as requested. Let me know if something doesn't work right and I'll take another look.

[Svardsten53]

Thanks! It's bed time here now and I'll try an update tomorrow. My friend Tore has already updated and I guess he will check this,

[Svardsten53]

Please see my comment here: 0e262ae

[Punkley]

6k currently tells us we have penguins visiting our home :), ill be interested in seeing what v2.4 does (id be happy to test)

[kahst]

Yeah, V2.4 will be very different from the "old" 6K model. Yet, with more species come more false positives :)

[jds3d]

Shouldn't penguins be ruled out completely via lat/lon?

…

On Mon, Apr 24, 2023, 10:10 AM Stefan Kahl ***@***.***> wrote: Yeah, V2.4 will be very different from the "old" 6K model. Yet, with more species come more false positives :) — Reply to this email directly, view it on GitHub <#184 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTIB3G6QWILP2MGDUXCC6DXC2CVDANCNFSM5TNSZYDA> . You are receiving this because you commented.Message ID: ***@***.***>

[kahst]

With V2.4 they will be, because there, we use the metadata filter as binary mask to exclude unlikely species. In the old version, we used a hybrid approach that combined audio and metadata and made a prediction based on both, so if the audio detector says "yes" with high enough confidence, it will show up as a detection independent from the metadata.

[nkwsy]

So on v2.2 & v2.3 I get terrible results. Nearly all my results are geese being called Tree Pipits or coscoroba swan's. I am located in Chicago, I am feeling like the GPS coordinates are not really working correctly.

Below is what I see in the log.
13:16:56---BirdNET-Pi/birdnet/bin/python3 BirdNET-Pi/scripts/analyze.py --i BirdSongs/StreamData/2023-04-26-birdnet-RTSP_2-11:39:57.wav --o BirdSongs/StreamData/2023-04-26-birdnet-RTSP_2-11:39:57.wav.csv --lat 42.XX --lon -87.XX --week 17 --overlap 0.0 --sensitivity 1.00 --min_conf 0.6 --birdweather_id "IN_USE"

2 things, Is there an issue w/ the GPS and species lists for v2.x? How can I have the program analyze all stream data?

[Datore7]

Hi, did an update now to latest model, but something is not working

[Datore7]

After last update, all looking good!

[timsterc]

So cool continuing to see the enthusiasm for this project - and all the continued updates!

A couple requests - from a BirdWeather perspective.

1 - When the 2.4 model is deployed, if you're able to add to this code, it would be awesome as I'm trying to make sure to keep track of which model was used for the detection. It's already a bit out of date as the V2.3 got updated but the algorithm text being posted to BW still reflects 2.2.

                                        if model == "BirdNET_GLOBAL_3K_V2.3_Model_FP16":
                                            post_algorithm = "\"algorithm\": " + "\"2p2\"" + ","
                                        else:
                                            post_algorithm = "\"algorithm\": " + "\"alpha\"" + ","

2 - On the BirdWeather ID Request form, instead of station name, it would be wonderful if you could update the form to ask for them to provide City/State/Country - as that's how I prefer to name the stations. A fair bit of the time it seems like Lat/Lon wind up being their ISP, and if they don't provide their proper location, it turns into a fair bit of back/forth to get a station setup.

Thanks!
-Tim

[ehpersonal38]

Done!

[timsterc]

Lightning fast - Thank you!!!

[Punkley]

oooo 2.4 is out with 6000 birds,

any chance of getting it pushed out :)

[Datore7]

@ehpersonal38 I forgot to change som lines in labels_sv.txt, it's correct now.
labels_nm.zip

I still have the zip with the fixed Uppercase for cs, fi, no, pl and sk, I upload that zip file if you want to use it.

[ehpersonal38]

Thanks!

I'm not sure if we should do it for other languages unless someone requests it. I think it will cause problems for people with existing installations by creating two versions of each species, the lowercase version and the capitalized version.

[Datore7]

@ehpersonal38 It is true, (I see it is mixed in the local names, some have uppercase and some lowercase initial letter. )
Anyway, just let me know and I'll fix it in later versions.

[DD4WH]

Thanks @Datore7!

@DD4WH I will check with you since you were involved in this discussion a while back, do you have anything against the new v2.4 model becoming the BirdNET-Pi default?

First of all, thanks a lot for all your hard work in improving BirdNET-Pi and issueing updates constantly!

No, I have no objections against any new model becoming the default, as long as the oldest BirdNET-Lite model remains an option to choose in the Settings ;-).

The model v2.4 is so new (from June 2023), that nobody really knows about its performance. But if it is set as the default, there will surely be more replies on it signalling whether it makes reasonable detections or not.

[nilspupils]

Hey! Greetings to all!
@mcguirepr89: Still in love with BirdNET-Pi, all of it exists only because of you! I hope you are doing fine!
@ehpersonal38: Thanks so much for your tireless work! You really keep everything alive.
@DD4WH: Awesome to still have you on board. I hope you can use BirdNET-Pi for research, too?
Apart from that, i am using the new model and it works very well over here. I had 1500 detections only for yesterday in my garden setup. The ones i checked were spot on.
Best regards,
Nils