What exactly is being done in the build_targets functions in loss.py?

[mlds2020]

I have watched multiple lectures explaining the yolo algorithm, but I found something in your code which doesn't seem to be discussed at all...

If I understand correctly, it seems like you are generating a total of 9 (3 in 3 loops) anchor boxes for each ground-truth object/bbox in the training batch, after that you seem to compare the w/h-ratio of the ground-truth bbox and the 9 anchors/priors, and whichever are smaller than the "anchor_t" threshold/hyperparameter are discarded. And later on, in the compute_loss function, you seem to calculate the IOU between the predicted bbox and those anchor bboxes which were generated in the build_targets function to later use for the loss calculation.

This is very confusing to me - as from the lectures I understood that we discard all predicted bboxes with an objectness score beneath a certain threshold (i.e. 0.4), and then we compare those left-over bboxes with the ground-truth bboxes directly by just taking the predicted box with the highest IOU as the "final" prediction of our model to then calculate our final loss...

But obviously that doesn't seem to be the case. Is there anywhere any piece of documentation which describes what exactly is being done during the training process of this yolov3 implementation? For instance, I have never heard of the 'anchor_t': (1, 2.0, 8.0), # anchor-multiple threshold hyperparameter, and googling doesn't help either...

Please excuse my ignorance - I have been learning about object-detection algorithms for a little over a week now.

[glenn-jocher]

The YOLO output grids have about 20,000 points that are each capable of detecting an object. build_targets() is simply identifying which grid points should match up with which targets (labels), if any.

anchor_t is the anchor multiple threshold parameter. A value of 4.0 means that anchors will match to any nearby target <4x larger and >4x smaller that it in width and height.

The papers you've read likely correspond to the older, less performant darknet versions of yolov3, which is why you've not seen this before.

[mlds2020]

Thanks a lot for your time and effort!

1. Is the assumption still made that for each grid cell (for each of the three detection kernels) there is max. 1 object to be detected?
2. So build_targets() identifies for every target-bbox its similar (scale- & size-wise) anchors, and that for every detection kernel?
3. And in the end we take the grid-cell where the centroid of our target-object lies, and just look at the anchor boxes identified in build_targets()?
4. And by applying this filter we reduce the scope from 25,200 predictions (given an input network size of 640, and 3x3 anchors) by about ~2-3 orders of magnitude when calculation the IOU?
5. build_targets() returns the values tcls, tbox, indices, anch do I understand correctly what they represent?
   for each detection kernel:

• indices: [0]: index of the image in the batch of the target-object in question, [1]: index of one of the 3 anchors for that detection kernel which is similar, [2]: grid index of the target-bbox on y-axis, [3]: grid index of the target-bbox on x-axis

• anch: width & height of the anchor from indices[1]

• tcls: class of the target-object

• tbox: absolute position of the target-bbox, normalized

Much respect for your extraordinary effort and dedication to all of this!