[Security Solution] [Bug] Index Values are not available in dropdown under New Index Enter for Knowledge Base.

[arvindersingh-qasource]

Describe the bug
Index Values are not available in dropdown under New Index Enter for Knowledge Base.

Kibana/Elasticsearch Stack version

VERSION: 8.16.0 BC5
BUILD: 79644
COMMIT: a8a07dfc586d78b8f4b7997b00e126363d68c043

Pre Conditions

1. Kibana v8.16.0 build be must be available
2. AI Connector must be configured.
3. Bulk Alerts must be present on Kibana.
4. Entity Score must be enabled.
5. Knowledge Base must be setup.

Steps

1. Navigate to Management -> AI Assistant - Knowledge Base
2. Click on ➕ New button and select Index option.
3. For Index field, Observe that Index Values are not available in dropdown.

Expected Result
Index Values should be available in dropdown under New Index Enter for Knowledge Base.

Screen Shot

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[arvindersingh-qasource]

@muskangulati-qasource Please review this ticket.

Thanks.

[muskangulati-qasource]

Reviewed and assigned to @MadameSheema

[jamesspi]

This is expected, as we fixed a bug that ensures only indices with semantic_text fields are shown in this list.

Here's how you can test this:

• Enable and initialize the knowledge base from the AI Assistant settings in stack management (You've already done this)
• Run this command in dev tools:

PUT kb-pdfs
{
  "mappings": {
  "properties": {
    "attachment": {
      "properties": {
        "content": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "content_length": {
          "type": "long"
        },
        "content_type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "format": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "language": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "content": {
      "type": "semantic_text",
      "inference_id": "elastic-security-ai-assistant-elser2"
    }
    }
}
}

That creates an index with a mapping that should enable it to show up the the indices drop down.

[e40pud]

There is a valid bug related to the description. At the moment we consider indices with the semantic_text field type only when the field is named content. Any other indices with semantic_text fields won't appear in the dropdown menu.

To see the issue, run this command in dev tools:

PUT kb-pdfs-2
{
  "mappings": {
  "properties": {
    "attachment": {
      "properties": {
        "content": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "content_length": {
          "type": "long"
        },
        "content_type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "format": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "language": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "content2": {
      "type": "semantic_text",
      "inference_id": "elastic-security-ai-assistant-elser2"
    }
    }
}
}

You won't see the index kb-pdfs-2 in the dropdown menu since the semantic_text field has name content2.

cc @MadameSheema

[e40pud]

@MadameSheema this bug was fixed and merged into 8.16.1, 8.17.0 and 9.0.0 and ready for QA love

[MadameSheema]

@muskangulati-qasource @arvindersingh-qasource can you please validate the issue in latest 8.16.1 BC? Thanks!

[muskangulati-qasource]

Hi @MadameSheema,

We have tested this ticket on the latest 8.16.1 BC2 and found the issue is now fixed.

Please find below the testing details:

Build details

VERSION: 8.16.1
BUILD: 79724
COMMIT: c8b46e87c4d61de4fe046ce5ea0a0b68aad5acf9

Screenshots

• The index is created successfully
  

• The same index is visible under the drop down
  

• The index is used and entry is created successfully
  

Hence, we are closing this issue and marking it as 'QA validated'.

Thanks!!