[BUG] Highlight behavior not working as expected for fields with periods (.) as part of their values

[Edds390]

Describe the bug

Documents that follow a pattern such as:"Test. DevPage_1" does not return the full field in the highlights block returned, despite default settings being applied.
As per https://opensearch.org/docs/latest/search-plugins/searching-data/highlight/#:~:text=If%20number_of_fragments%20is%20set%20to,Default%20is%205.

The size of a highlighted fragment, specified as the number of characters. If number_of_fragments is set to 0, fragment_size is ignored. Default is 100.

The maximum number of returned fragments. If number_of_fragments is set to 0, OpenSearch returns the highlighted contents of the entire field. Default is 5.

In this case, "Test. DevPage_1" satisfies the default 100 character fragment size, yet the highlight is only returning "DevPage_1".

Related component

Search

To Reproduce

1. Index a document with title "Test. DevPage_1"

2. Search the document with highlights enabled:

GET /index/_search
{
  "query": {
    "match": {
      "title": "DevPage_1"
    }
  },
  "highlight": {
    "encoder": "default",
    "fields": {
      "title": { }
    }
  },
  "_source": ["title"]
}

This returns a response with the full title omitted.

{
  "took": 67,
  "timed_out": false,
  ...
  "hits": {
    "total": {
      "value": 36,
      "relation": "eq"
    },
    "hits": [
      {
        "_index": "index",
         ...
        "_source": {
          "title": "Test. DevPage_1"
        },
        "highlight": {
          "title": [
            "<em>DevPage_1</em>"
          ]
        }
      },

Expected behavior

Expected the full field to be returned in the highlight i.e. "Test. DevPage_1"

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: OS X
• Version 2.11

Additional context
Add any other context about the problem here.

[dblock]

Want to try to turn this into a (failing) YAML REST test?

[rishabhmaurya]

@Edds390 The default highlighter is unified whereas fragmenter is only applicable when type is plain. If you set it to plain it will work -

{
  "query": {
    "match_phrase": {
      "title": "DevPage_1"
    }
  },
  "highlight": {
    "fields": {
      "title": {
        "type": "plain"
        }

    }
  },
  "_source": ["title"]
}