Fix re-rankings payload schema for NIMs

[dyastremsky]

During the inputs modularization work, the payload for benchmarking re-ranking NIMs was broken. Fix the payload schema.

After changes:

[nv-hwoo]

Thanks for the quick fix! 🚀

[nv-braf]

This looks fine. I like that there is unit testing for this, but I'm curious as to who is the "owner" of the formatting for this? How did you discover it was broken (as the unit test would not have failed)?
Is this something that should be tested live (L0)?

[dyastremsky]

This looks fine. I like that there is unit testing for this, but I'm curious as to who is the "owner" of the formatting for this? How did you discover it was broken (as the unit test would not have failed)? Is this something that should be tested live (L0)?

We had a customer report this as broken. We 100% do need testing for this.

In regards to CI: TPA-509 tracks adding a HuggingFace TEI CI test, which had dependency issues and was deprioritized. The testing you're describing is something that we have outlined as essential for GA: adding CI testing for customers (e.g. CI for NIMs). We do not currently test NIMs in CI.

This unit test breaking should have been identified and noticed sooner. We were racing to get the modularization out, which required a lot of unit testing updates and this failure got lost among them. I was testing against HuggingFace TEI at that point and had not realized that the ranking NIM folks were using a pinned version of GAP for testing, which is why they didn't see or report this failure immediately.