Spectral Selection : Event Boxing

[paulpeyret]

Hi @Mattk70,

Thanks for all the updates, it looks wonderful!
I am now considering doing manual annotations (AI assisted) inside Chirpity.
Doing it in Chirpity seems faster than doing it in Audacity.
The only thing that I would miss is the spectral selection and export the boxed sound events (time & frequency bounding boxes).
I know Chirpity and Birdnet models don't do object detection and are doing only the time segmentation of sound events in the spectrogram, however I think it would be very useful to add the event boxing capability inside Chirpity to allow for more precise annotation capabilities.
I would be pleased to see this new feature in Chirpity. 😊

[Mattk70]

HI @paulpeyret, thanks for the suggestion and comments.

I know you can do spectral selection in Audacity and Raven, I presume you use this to highlight a segment of a call or song?

Implementation would be difficult. I rely on the Wavesurfer library for the spectrograms and its region plugin for regions. This doesn't support boxing. I would have to rip out the regions plugin and re-implement my own - or adapt the regions plugin. I think this would be weeks or months of work. Unless there's huge demand for such a feature I'll be honest and say up front, I don't think this is going to happen. I'll leave the issue open to see if there is significant interest, but tag it as an enhancement/won't fix for now

[paulpeyret]

Thanks @Mattk70 for the feedback.

"I know you can do spectral selection in Audacity and Raven, I presume you use this to highlight a segment of a call or song?"

Yes that is useful in bioacoustics to know the frequency bounding of the call when you want to extract some acoustic features out of the songs/calls.
Boxing annotations are also used to train and evaluate models that are doing object detection from spectrograms (like the one of NBM project: https://gitlab.com/nbm.challenge/nbm-nocturnal-bird-migration).
Finally, spectral selection can be useful when listening to the calls. Some sound editing software apply high pass and low pass filter before playback to reduce noise allowing to play the call only in the selected frequency range on the spectrogram (=denoising before playback).

Though, I understand that it is not a quick and easy change.
Thanks for creating the issue.

[Mattk70]

Hey @paulpeyret - thanks for the explanation.

One thing I was thinking to do was enable a tooltip when you hover over the spectrogram that gives the frequency at the mouse pointer. With regards to time, if you hover over a selection you have created, the duration of the selection of the selection will be displayed. Selections created by Chirpity show the time/time-of-day within the file (because all these selections are 3 seconds long).

Finally, spectral selection can be useful when listening to the calls. Some sound editing software apply high pass and low pass filter before playback to reduce noise allowing to play the call only in the selected frequency range on the spectrogram (=denoising before playback).

You can do this in Chirpity by using the high pass / low shelf filters too. You'll find them in the settings.

Let me know how the object detection model comes along - I'd be really interested to hear if you have good results with this technique!!