[BUG] Incomplete Saved Object Query From DataSource Selector

[derek-ho]

Describe the bug

The data source selector query is wrong/incomplete. This leads to missing fields and inability to filter datasources.

To Reproduce

• Visit Data source specific page. We see the API call made is bulk get call. Returned saved object is like the following:

{
    "id": "fa529a40-438a-11ef-9a4f-372f52117890",
    "type": "data-source",
    "namespaces": [],
    "updated_at": "2024-07-16T15:49:25.603Z",
    "version": "WzE3MSwxXQ==",
    "attributes": {
        "title": "test-serverless",
        "description": "",
        "endpoint": "https://0n67y3fdzp6j6zicktqk.us-west-2.aoss.amazonaws.com",
        "auth": {
            "type": "sigv4",
            "credentials": {
                "region": "us-west-2",
                "accessKey": "AgV487B55do0I4c74uQNy3fGNsbOh9kyOrSqJiHsfMW1bdQApAACABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREF3eFM4bnRiWEpta3VyaFNnYVJSOUlkczVPbnNPSlJGTE16WXdGb0lnT3RGOXlhUGRsL0xrUWtBdmxjQ092bDRpUT09AAhlbmRwb2ludAA5aHR0cHM6Ly8wbjY3eTNmZHpwNmo2emlja3Rxay51cy13ZXN0LTIuYW9zcy5hbWF6b25hd3MuY29tAAEACGNoYW5nZW1lABxjaGFuZ2VtZQAAAIAAAAAMsdxJBBQEc7edymcpADA0mOWN+ImJmPGUeu6qEQuTxKPDNL2v9q5B++FriO/2K8k6aRiD6ca+lPyiwEpEVw8CAAAQAKjaojNpXpFnBD3J5sYBRawqBouKqA98gpwMePsKOjLziq/PtW+KKQuKvWYqVRa+Mf////8AAAABAAAAAAAAAAAAAAABAAAAFCO8UVt5RsDdJ/IDX5FUu6S7GmzNSFT4FM36/RIvn1eJUNMZnQBoMGYCMQDZ61b6+458DxIRjN1VPPWqYJn7fzohTIfB3Vg9pvSU9q6/3vHMsc5V3yg1BCl5CLQCMQDm8QCfRRHisbYeajIaGKLBqog7iUws2K5k5m0wN6aB57IOEnKGBake/Kp4fxSPAFs=",
                "secretKey": "AgV41IksJhLhLfymyoRYDWxaEHxAA+S6aVdCLcxriFDSx+cApAACABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFvMEpvOWR0Z0tndExuODR0aWI3c3ptKzhhTitpM1IvWGJ2dVR2ZzlYNXo4SFJOWmV1RmxEMHNLL0tHOStPUkhWUT09AAhlbmRwb2ludAA5aHR0cHM6Ly8wbjY3eTNmZHpwNmo2emlja3Rxay51cy13ZXN0LTIuYW9zcy5hbWF6b25hd3MuY29tAAEACGNoYW5nZW1lABxjaGFuZ2VtZQAAAIAAAAAMu10eB8DsbiqFsynLADD6/eNacwmMgeXix7Hn88ABnbwpxTz4DFK1iEFTeN73jQNYMBFwerYk8teYQqsj/toCAAAQADoGXFMRYVJNhgWx0hhiEzL2KbGYdmaKyxoYTK+7h0W5wAYDKxpQ07h/LmmzD6SLb/////8AAAABAAAAAAAAAAAAAAABAAAAKLeK3FUzCrFPuHt/i84O/MAuJANoWUTP/eaN2ZZyXwWQPfyCv8uE4tOIE5ciyXjPEfKU2FT+v2mOAGcwZQIwbM4Ji7lFD8AnamvQbUtmmPPyen3t193NarRfLRm9YqMuF2CppvkHOeq13iBvJDavAjEAySgs1ntJJihDDQBKc16d5b2tR4rb1rJxAEMh6kpZVwt3Iqtwgvf832rMFWquZdC0",
                "service": "aoss"
            }
        },
        "dataSourceVersion": "",
        "dataSourceEngineType": "OpenSearch Serverless",
        "installedPlugins": []
    },
    "references": [],
    "migrationVersion": {
        "data-source": "2.4.0"
    }
}

The query sent by the data source selector is filtering on

fields=id&fields=title&fields=auth.type&fields=dataSourceVersion&fields=installedPlugins&per_page=10000&type=data-source

Which leads to the same saved object to be returned as

{
    "type": "data-source",
    "id": "fa529a40-438a-11ef-9a4f-372f52117890",
    "attributes": {
        "installedPlugins": [],
        "auth": {
            "type": "sigv4"
        },
        "title": "test-serverless",
        "dataSourceVersion": ""
    },
    "references": [],
    "migrationVersion": {
        "data-source": "2.4.0"
    },
    "updated_at": "2024-07-16T15:49:25.603Z",
    "version": "WzE3MSwxXQ==",
    "score": 0
}

Thus, AOSS data sources cannot be filtered out from the data source selector. Since this is a common requirement for plugins, the fields necessary to filter out AOSS should be returned.

Expected behavior

ds.attributes.auth.credentials?.service

Should be returned, so AOSS can be filtered out
OpenSearch Version
Please list the version of OpenSearch being used.

Dashboards Version
Please list the version of OpenSearch Dashboards being used.

Plugins

Please list all plugins currently enabled.

Screenshots

If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: [e.g. iOS]
• Browser and version [e.g. 22]

Additional context

Add any other context about the problem here.

[BionIT]

@ZilongX could you help with this? I think the ask is to add dataSourceEngineType to the response so that plugins can filter based on service type

[BionIT]

@derek-ho do you want to contribute?

[dblock]

[Catch All Triage - 1, 2, 3]