[8.6] [Fleet] Add check for legacy managed_by field in datastream map…

…pings (elastic#149504) (elastic#149583)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[Fleet] Add check for legacy managed_by field in datastream mappings
(elastic#149504)](elastic#149504)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Cristina
Amico","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-01-26T08:15:14Z","message":"[Fleet]
Add check for legacy managed_by field in datastream mappings
(elastic#149504)\n\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/147605\r\n\r\n###
Description\r\nClusters that exist since pre-8.0 might have data streams
that have\r\n`managed_by: ingest_manager` in their` _meta` properties,
instead of the\r\ncurrent value `managed_by: fleet`.\r\nHowever, with
the merge of\r\nhttps://github.com/elastic/pull/143300, the data
streams view\r\nfilters out any data streams that don't have
`managed_by: fleet`\r\nmetadata, so when updating from 7.x to 8.6.x no
data streams are\r\nreturned.\r\n\r\n### Solution\r\nIt was decided to
simply add an additional check in the data stream\r\nhandler to allow
for \"legacy\" metadata, and to avoid doing migrations\r\nthat can be
dangerous for the users data.\r\n\r\n\r\n### Testing\r\nI'm looking for
a way to reliably test this locally - I've only managed\r\nto reproduce
it on
cloud","sha":"49ff27e2ff109ef28a8640e056db1cda645986ca","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v8.7.0","v8.6.1"],"number":149504,"url":"https://github.com/elastic/kibana/pull/149504","mergeCommit":{"message":"[Fleet]
Add check for legacy managed_by field in datastream mappings
(elastic#149504)\n\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/147605\r\n\r\n###
Description\r\nClusters that exist since pre-8.0 might have data streams
that have\r\n`managed_by: ingest_manager` in their` _meta` properties,
instead of the\r\ncurrent value `managed_by: fleet`.\r\nHowever, with
the merge of\r\nhttps://github.com/elastic/pull/143300, the data
streams view\r\nfilters out any data streams that don't have
`managed_by: fleet`\r\nmetadata, so when updating from 7.x to 8.6.x no
data streams are\r\nreturned.\r\n\r\n### Solution\r\nIt was decided to
simply add an additional check in the data stream\r\nhandler to allow
for \"legacy\" metadata, and to avoid doing migrations\r\nthat can be
dangerous for the users data.\r\n\r\n\r\n### Testing\r\nI'm looking for
a way to reliably test this locally - I've only managed\r\nto reproduce
it on
cloud","sha":"49ff27e2ff109ef28a8640e056db1cda645986ca"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/149504","number":149504,"mergeCommit":{"message":"[Fleet]
Add check for legacy managed_by field in datastream mappings
(elastic#149504)\n\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/147605\r\n\r\n###
Description\r\nClusters that exist since pre-8.0 might have data streams
that have\r\n`managed_by: ingest_manager` in their` _meta` properties,
instead of the\r\ncurrent value `managed_by: fleet`.\r\nHowever, with
the merge of\r\nhttps://github.com/elastic/pull/143300, the data
streams view\r\nfilters out any data streams that don't have
`managed_by: fleet`\r\nmetadata, so when updating from 7.x to 8.6.x no
data streams are\r\nreturned.\r\n\r\n### Solution\r\nIt was decided to
simply add an additional check in the data stream\r\nhandler to allow
for \"legacy\" metadata, and to avoid doing migrations\r\nthat can be
dangerous for the users data.\r\n\r\n\r\n### Testing\r\nI'm looking for
a way to reliably test this locally - I've only managed\r\nto reproduce
it on
cloud","sha":"49ff27e2ff109ef28a8640e056db1cda645986ca"}},{"branch":"8.6","label":"v8.6.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

x-pack/plugins/fleet/server/routes/data_streams/handlers.ts

@@ -16,6 +16,8 @@ import { defaultFleetErrorHandler } from '../../errors';
 import { getDataStreamsQueryMetadata } from './get_data_streams_query_metadata';
 
 const DATA_STREAM_INDEX_PATTERN = 'logs-*-*,metrics-*-*,traces-*-*,synthetics-*-*';
+const MANAGED_BY = 'fleet';
+const LEGACY_MANAGED_BY = 'ingest-manager';
 
 interface ESDataStreamInfo {
   name: string;
@@ -59,8 +61,11 @@ export const getListHandler: RequestHandler = async (context, request, response)
       getPackageSavedObjects(savedObjects.client),
     ]);
 
+    // managed_by property 'ingest-manager' added to allow for legacy data streams to be displayed
+    // See https://github.com/elastic/elastic-agent/issues/654
+
     const filteredDataStreamsInfo = dataStreamsInfo.filter(
-      (ds) => ds?._meta?.managed_by === 'fleet'
+      (ds) => ds?._meta?.managed_by === MANAGED_BY || ds?._meta?.managed_by === LEGACY_MANAGED_BY
     );
 
     const dataStreamsInfoByName = keyBy<ESDataStreamInfo>(filteredDataStreamsInfo, 'name');
@@ -121,6 +126,7 @@ export const getListHandler: RequestHandler = async (context, request, response)
     // Query additional information for each data stream
     const dataStreamPromises = dataStreamNames.map(async (dataStreamName) => {
       const dataStream = dataStreams[dataStreamName];
+
       const dataStreamResponse: DataStream = {
         index: dataStreamName,
         dataset: '',