-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix frame buffer bug where predecessors were not checked properly #155
Conversation
I'm going to go in and fix the test case for this so that the tests would have actually caught this bug... |
Nice catch! Can we generally describe the influence of this fix on the division metrics calculated up to this point? Are old metrics too optimistic/pessimistic? |
@bentaculum The old version incorrectly penalized some (most?) correct divisions when frame_buffer > 0. Specifically, it checked if the successors had the same ID, not if the successors were matched, so if your prediction and candidate nodes had different IDs, it would never count an offset division as correct. For linking only experiments with GT detections, it probably won't change anything, but for experiments with predicted detections, performance should improve for division metrics when frame buffer > 0. |
@cmalinmayor @msschwartz21 took a look at this PR and if I'm understanding correctly, the bug was that when checking successors, we were directly using the graph node IDs rather than converting them via the mapper. And the test was passing because our |
@DragaDoncila that's correct! I'll fix the test case in the next week or so. It's an easy fix but life has been busy 😂 |
@msschwartz21 no rush at all, just making sure I understand the change! |
* Change node ids on divisions graphs to be unique * Update how mapper for division graphs is returned so that direction of gt to pred can be flipped
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! Thanks for editing the tests @msschwartz21
Proposed Change
@bentaculum I also saw issues with the frame buffer not updating when I expected it to. Upon closer inspection, there was a bug in how we were checking the successors. This PR fixes that bug. Sorry!
Previous discussion of possible frame buffer issue #100 (comment)
Types of Changes
What types of changes does your code introduce? Put an x in the boxes that apply.
Which topics does your change affect? Put an x in the boxes that apply.
TrackingGraph
,run_metrics
,cli
, etc.)Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.