[BUG] Inconsistency for create index between REST and CreateIndex

[amitgalitz]

Describe the bug

I am encountering inconsistencies between the rest API for create index and creating an index through the client in a plugin. When given the same mapping through the rest API create Index call and the client I get different responses. Specifically when using client.admin().indices().create(createIndexRequest… it fails if mappings doesn't have _doc in it with:

org.opensearch.index.mapper.MapperParsingException: Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [passage_text : {type=text}] [passage_embedding : {type=rank_features}] [id : {type=text}]

Related component

Indexing

To Reproduce

When I try to create a template with this mapping through REST API it works:

{
    "my-nlp-index": {
        "mappings": {
            "properties": {
                "id": {
                    "type": "text"
                },
                "passage_embedding": {
                    "type": "rank_features"
                },
                "passage_text": {
                    "type": "text"
                }
            }
        }
    }
}

however when I provide same mapping to:

CreateIndexRequest createIndexRequest = new CreateIndexRequest(indexName).source(configurationsBytes, XContentType.JSON);
            client.admin().indices().create(createIndexRequest, ActionListener.wrap(acknowledgedResponse -> { ...

I get this exception:

org.opensearch.index.mapper.MapperParsingException: Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [passage_text : {type=text}] [passage_embedding : {type=rank_features}] [id : {type=text}]

when I try to create a mapping with the _doc field below _mappings through rest

API it fails with:
{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "The mapping definition cannot be nested under a type"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "The mapping definition cannot be nested under a type"
    },
    "status": 400
}

however it succeeds with client.admin().indices().create(createIndexRequest…

Expected behavior

_doc shouldn't be required in 2.12 with client.admin().indices().create()

Additional Details

No response

[dblock]

Do you have a quick repro, is it reproduced in the previously released version(s)?

[vikasvb90]

Reason of inconsistency is due to the validation being present in rest layer and not in transport layer. Rest layer also wraps mapping properties with _doc here once it has validated that mappings object shouldn't have _doc field. Since, transport layer is now expecting _doc wrapped around properties of mappings, client call to create index fails if it doesn't have _doc.

In core, transport actions which rely on mappings, should be made backward compatible to wrap mapping properties with _doc if it isn't already present. These change(s) would be at cluster manager layer.

[andrross]

[Triage - attendees 1 2 3 4 5 6]
@amitgalitz Thanks for filing.

[sandeshkr419]

@amitgalitz Did your PR address this issue? If yes, can we close this issue or are there any pending items left?