Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

retrieval map=1 #16

Open
appleleaves opened this issue Nov 19, 2017 · 2 comments
Open

retrieval map=1 #16

appleleaves opened this issue Nov 19, 2017 · 2 comments

Comments

@appleleaves
Copy link

I made a mistake, setting and the descriptors a same value, and get an mAP=1 in the retrieval task, using Python code.
I wonder why?
It must be a big bug!

@lenck
Copy link
Contributor

lenck commented Nov 20, 2017

Sorry, I don't understand exactly what is the problem. Yes, if you use the dataset or benchmark incorrectly (e.g. in some way employing ground truth by setting equal descriptors for example), of course you can get a higher performance. For the sake of simplicity the provided data is not randomised so it is not robust against these sort of mistakes.

@appleleaves
Copy link
Author

what I mean is that, when you work with the retrieval task, and set all the descriptors to ones. Therefore, the descriptors are useless, because all the descriptors are all the same, no matter if they are similar or not.

Then I check test the performance of these useless descriptors. The retrieval mAP is 1.0, while the matching and verification task mAP are 0.

Therefore, I believe there are mistakes in your retrieval code, but I could not fix it.

I do not need to employ ground truth. And I think it is a bug needed to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants