It would be helpful if we could designate a RoI for tracking

[olinesn]

Hello,

I'm writing to suggest a feature where the user could fence in the region (in pixels) that sleap-track would run on. This would be analogous to cropping the video to a smaller video.

The use case for me is that sometimes, reflections get picked up as skeletons during inference (see below).

Thanks!
Stefan

[talmo]

Agreed, this would be a great feature!

Sub-tasks:

1. What kind of ROIs should we allow?
   • Arbitrary polygons?
   • One or many?
2. UX for defining ROIs
3. Data structures to store them
4. What part of the inference pipeline do they get applied to?
   • How do we handle instances that are partially overlapping?
   • After pose or after tracking? Both?
5. How do we handle new videos?
   • Apply ROI manually to all new videos?
   • Automated ROI detection? (New model type?)

Related: #1488 #1388, #1391

[olinesn]

re: 1, I like the way that idtracker does it, you just click out some points and accept a drawn ROI, and you can always use text to define or import the same set of XY points