[Feature Request] Improve performance of sort by _id

[dvjyothsna]

Is your feature request related to a problem? Please describe

We have a usecase where we rely on sort by _id to paginate and _id is the only field we can deterministically sort on. One of the workarounds is to copy the _id field into a doc value field. But it doesn't work for our usecase since ther user trying to paginate has readonly access. Another use case is where users can use custom ids and sort on the custom ids. Given the use cases, Is there a plan to optimize sort by _id?

Describe the solution you'd like

Optimize search performance for sort by _id

Related component

Search:Performance

Describe alternatives you've considered

No response

Additional context

No response

[dvjyothsna]

@msfroh

[dvjyothsna]

@saratvemulapalli

[msfroh]

@dvjyothsna -- can you please clarify what the problem is that needs solving?

From what I can see, we do support sorting, but we output this deprecation message:

OpenSearch/server/src/main/java/org/opensearch/index/mapper/IdFieldMapper.java

Line 84 in 09bacee

"Loading the fielddata on the _id field is deprecated and will be removed in future versions. "

One of the workarounds is to copy the _id field into a doc value field. But it doesn't work for our usecase since ther user trying to paginate has readonly access.

The efficient way to support sorting on a field is to enable doc values on that field. I don't think we would ever enable doc values on the _id field by default, since it would increase the size of the index for all users, whether they want to sort on _id or not. We could add a mapping flag to store a doc value for the _id field, but that wouldn't help your readonly user (and would be functionally equivalent to copying the _id field into another field with doc values enabled).