[Security Solution] Rule find requests failing due to migration error

[xcrzx]

Summary

Requests to /kbn/internal/alerting/rules/_find fail when called with the fields param. This occurs when documents created under the first rule model version are read after another rule model version is added. Migration fails as a result of the attempt to migrate a partial document due to the fields parameter in the find API.

Steps to Reproduce

1. Create rules under the first rule model version.
2. Introduce the second rule model version. Here's the second model version introduced by ResponseOps: https://github.com/elastic/kibana/pull/190019/files#diff-12c3ecfb676c8d71cfbd3ca6e4ce29223604dfa58d35a3a108ee6399da34f8da
3. Attempt a find API call with specified fields (e.g., this call used on the rules management table).
4. Observe 500 Server Error due to failed migration

Root Cause

• The fields parameter in the find API limits document fields read from ES:

  kibana/packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/apis/find.ts

  Line 188 in 343e43d

  _source: includedFields(allowedTypes, fields),

• Migration helpers expect a full document, leading to failure if partial data is provided:

  kibana/packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/apis/helpers/migration.ts

  Line 71 in 343e43d

  return this.migrator.migrateDocument(doc, { allowDowngrade: true }) as SavedObject<T>;

• The error message is: Failed to migrate document to the latest version.: [name]: expected value of type [string] but got [undefined]

Proposed Solution

• To mitigate the problem short-term we can remove the fields param from the find request
• Consider either disabling migration on partial reads or enforcing a full read before migration.
• Another potential approach is modifying the find API to include all required fields when performing migrations.

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[xcrzx]

Lowering the impact as the error is only reproducible during down migrations, not up.

Will be fixed on the Core side: #198655