[Security Solution] [Bug] Index Values are not available in dropdown under New Index Enter for Knowledge Base. (#199610)

[e40pud]

Summary

This PR fixes the issue described here #199610 (comment)

In short, we do not show all indices with semantic_text fields. We only show those which have semantic_text fields named content.

Testing notes

Add the index with the semantic_text field by running this command in dev tools:

PUT test_index
{
  "mappings": {
  "properties": {
    "attachment": {
      "properties": {
        "content": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "content_length": {
          "type": "long"
        },
        "content_type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "format": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "language": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "content2": {
      "type": "semantic_text",
      "inference_id": "elastic-security-ai-assistant-elser2"
    }
    }
}
}

You should see the test_index index in the Knowledge Base > New > Index > Index (dropdown).

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[stephmilovic]

I think you have too much mock data here that is superfluous. By changing mockData to the following, the tests still prove the fix:

const mockFieldCaps = {
  indices: [
    'gtr-1',
    'gtr-with-bug',
    'gtr-with-semantic-1',
    'metrics-endpoint.metadata_current_default',
    'search-elastic-security-docs',
  ],
  fields: {
    content: {
      semantic_text: {
        indices: ['gtr-with-semantic-1'],
      },
    },
    ai_embeddings: {
      semantic_text: {
        indices: ['gtr-with-bug', 'search-elastic-security-docs'],
      },
    },
    semantic_text: {
      semantic_text: {
        indices: ['search-elastic-security-docs'],
      },
    },
  },
};

[stephmilovic]

Ah, webstorm is so slow. I see this messes up the type. Feel free to disregard. But maybe we can cut down on those indices

[e40pud]

Yeah, I just copied mappings from my local kibana to make it as close as possible to real setup :-) Will simplify it though.

[stephmilovic]

LGTM! Thanks for the fix

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: a9b0d77

Metrics [docs]

✅ unchanged

History

• 💚 Build #250804 succeeded 62957ba

cc @e40pud

[kibanamachine]

Starting backport for target branches: 8.16, 8.x

https://github.com/elastic/kibana/actions/runs/11896662204

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.16
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[muskangulati-qasource]

Hi @MadameSheema,

We have tested this ticket on the latest 8.16.1 BC2 and found the issue is now fixed.

Please find below the testing details:

Build details

VERSION: 8.16.1
BUILD: 79724
COMMIT: c8b46e87c4d61de4fe046ce5ea0a0b68aad5acf9

Screenshots

• The index is created successfully
  

• The same index is visible under the drop down
  

• The index is used and entry is created successfully
  

Thanks!!