Video Preprocessing: Specify Region of Interest

[roomrys]

Preface & Problem
A good amount of SLEAP users have videos which contain stationary subsections within the frame corresponding to unique instances. For example, consider a zoom video where each frame in the video is split into four quadrants - one quadrant per person (one quadrant per unique instance). Although the user knows that the identities in these stationary subsections should never change/switch, SLEAP will sometime switch the identities of the instances in these quadrants.

Current Hack
At the moment our recommendation for cases like this would be to crop the video panels out and import them as separate MP4s into SLEAP. You can do this pretty quickly using an online tool like https://clideo.com/crop-video or https://invideo.io/. If you need more automation or if you have longer videos, you might want to consider using ffmpeg (https://video.stackexchange.com/questions/4563/how-can-i-crop-a-video-with-ffmpeg), but it'll require you to manually specify the coordinates of each box. What some folks have done is to actually use SLEAP to detect the four corners of the bounding boxes as individual instances and use those coordinates for cropping.

Proposed Solution
As a future feature, SLEAP can have the option to specify regions of interest (ROI) where unique instances reside in. The UX might look something like this:

1. From menu, select "Video Preprocessing" → "Specify ROIs"
2. Select bounding boxes (optionally naming them)
3. In the backend we either add a metadata attribute to the Video class which allows us to create "virtual" video crops for each ROI which are treated independently, or have an option to automatically crop → render to individual MP4s → re-import them into SLEAP (possibly with some basic handling of Instance re-importing for instances that overlap with the ROI).