Dealing with overwhelmingly common birds

[avit]

I am surrounded by magpies (pica hudsonia). I'm detecting a lovely variety of other species but the number of magpie calls is well over 100 to 1. These are very chatty birds.

I'm thinking of ideas for how to improve the UX for this situation. What I'd like to address is:

• Latest detections get overwritten almost immediately by magpies
• Today's detections is an endless stream of magpies
• Integration notifications are flooded with magpies
• Charts get squashed down by magpies

I could probably just add them to ignored species, but I also don't want to skew the data by pretending we don't have those birds here. I'm also interested in when they come and go, despite being so common.

I'm wondering if it would make sense to select certain species as "common" (or else automatically flag them based on relative numbers) so they can be handled differently:

Combine repeat detections together

If the previous detection is the same species, only show the latest one (or best one).

Show the number of repeat occurrences with a time range from first to last.

This should be possible even without a "common" flag, but for less common birds it seems better to not hide any of the recordings, making it possible to review them all.

Only show 1 of each species in Latest Detections

This is similar to combining repeat detections, but I think showing [magpie, robin, collared dove] is more helpful than [magpie, robin, magpie]. In other words, 5 latest unique species.

Exclude common species from notifications

The "infrequent" option for notifications is a good start, but if a detection shows up a few times a day then it stops notifying. I would still like to notify for all birds, but only ignore the most common ones.

Option to hide common species from charts

The proportions get lost when the first bar is 100x longer than the rest. The default could be to exclude common species from the graphic, with an option to show them for completeness.

[mcguirepr89]

Hello @avit -- there is a lot to go through here, but I did want to bring to your attention the resampling that is possible in "Species Stats" -- resampling will allow you to keep the abundant magpie data while resampling it down to hourly occurrences. That's all from me for now -- there is a lot more for me to unpack here, but I wanted you to know my initial thought

[ehpersonal38]

I've just pushed a commit to main that allows you to use the "NOT" operator in Today's Detections search feature. For example, you can search:
NOT magpie and it will show you what you requested.

Maybe this feature can be built out to be site-wide, i.e. to exclude a species from showing on all the UI stuff but still leave it in the database.

I do also want to mention that you may find "Recordings" more helpful than "Today's Detections," because it basically groups all the detections into their distinct species. It also allows you to sort by date, # of occurences, and confidence so you can find just the detections you're interested in.

Good suggestion, hopefully this can be seen through in the nearish future. 🐤

[avit]

Thanks @ehpersonal38, that NOT query is a nice addition! Works great for now.

[ehpersonal38]

Okay I've done a little more work regarding this - you can test it out with:

1. git checkout -f ui_exclusions
2. Tools -> System Controls -> Update
3. Tools -> Excluded Species List (UI Only)

Then any species you add to the list will be excluded from Overview and Today's Detections. Anyone wishing to contribute to this branch, be my guest! (i.e. for adding this to notifications, too)

[soumilgurjar]

Thanks a lot for your efforts @ehpersonal38 ! I was wondering if there is any possibility to ignore a certain set of species from the notifications, but still continue to the detected and added to the database. At the moment, there is a possibility to completely exclude a set of species from being detected or to get notified about infrequent species or first detection of each species. If some finer level of control for notifications is possible, thatwould be fantastic.

For example, I'd like to notified every time a Tawny Owl is heard, and since it is not an infrequent species for my location, I cannot choose the infrequent birds notification option. On the other hand, I do not wish to be notified every time a European Robin is detected (extremely common), but I'd still like the detection to be recorded and added to the database.

[ehpersonal38]

Hi @soumilgurjar. Sure, I've gone ahead and added some code to the repo which will hopefully accomplish what you were asking. You can try it out now by updating your system (Tools -> System Controls -> Update), and then navigating to Tools -> Settings, where you'll see a new Notifications configuration option called "ONLY notify for these species." Try it out and let me know if that works for you!

[soumilgurjar]

Appreciate the prompt update! For my particular case, I'd like the Notification configuration option to "Notify all species EXCEPT these species (comma separated common names)". Apologies if I didn't mention this clearly enough earlier, but I am certain your update will already be quite useful for others.

[ehpersonal38]

@soumilgurjar Ah, I see! Okay I've just made a quick edit which will change the functionality from "include these species" to "exclude these species." You can update your BirdNET-Pi again, and hopefully you'll find the edits will meet your needs! :)