Prediction Labels (Labels2)

'Prediction' Labels (Labels2)

We would like to be able to load in a set of labels and compare/display side-by-side with current manual labels. This second set of labels could be predictions generated by a tracker, but could also be eg an older set of manual labels.

From the Labeler's POV, this is just an abstract second of labels; they do not need to arise as predictions from an automated tracker, so I will just call them 'labels2' as opposed to eg 'labelsPred'.

Save format

• When generated by a tracker, we will store prediction labels in the movie directories, as preferably 1) mat-files with names <vidname>
• .trk files should contain
  • pTrk [npts x d x nfrm x ntrx], bare numeric array of labels. The third dimension of this array needs to correspond precisely with frames of the corresponding movie. Use NaN for untracked pts/frames/targets.
  • pTrkTS [npts x nfrm x ntrx]. timestamps for pTrk
  • pTrkTag [npts x nfrm x ntrx] cell. Tags corresponding to pTrk
  • pTrkiPt [npts]. Optional, indices labeling first dim of .pTrk and .pTrkTS. Sometimes you don't track all the points, only a subset; if so, this tells us which subset you tracked.
• Alternatively, labels on arbitrary sets of movies/frames can be stored in a MAT-file containing a variable 'tblP' with fields: movieNameFull, movieNameShort, frame, p (a row vector of labelpositions), pTS. I guess we might also need an optional variable 'tblPpIpt' in analogy with pTrkiPt above. Hmm.

Labeler property

The original labels are in .labeledpos; we can add .labels2 with precisely the same dimensions (both the "outer" cell array as well as "inner" arrays).

Labeler load/import

For instance:

• Labeler/importLabels2Tbl(lObj,tbl); % tbl is a table with fields as above
• Labeler/importLabels2Trk(lObj); % goes through .movieFilesAll, looking for .trk files; loads mat files and sets .labels2 etc.
• Labeler/clearLabels2(lObj);

Display of labels2

Labeler will handle display/view of labels2. Note, the display of the original labels .labeledpos is handled by LabelCore, not Labeler. This is because display of the original labels is intimately related to the act/process of labeling. In contrast, labels2 are just passive labels, for display only.

Labeler will display labels2 only when they are present; labeler will update this display as user navigates movies/frames. Plot cosmetics will be specified in the pref.yaml -- maybe under "Label2PointsPlot:".

Generation of .trk files

This can be done "offline" (independent of the Labeler) of course, eg for larger projects or cluster runs.

Within the Labeler, we can add some utility methods to LabelTracker for generation of the .trk files. The CPRLabelTracker properties named .trkP* are relevant to tracking results. It looks like most of these are probably applicable to any tracker, except .trkPFull. I could move these props into LabelTracker (base class) and then add LabelTracker/saveTrkFiles. This method will take the .trkP* properties and generate .trk files. Then, in the UI we can add a "Save trk files" item under the Track> menu.

Use of the utility methods would still be optional, as any concrete tracker might want its own particular tracking-results-state and/or want to generate .trks in its own way.