[8.11] [ML] Fix Anomaly charts embeddable fails to load if partition …

…value is empty string (#167827) (#168273)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[ML] Fix Anomaly charts embeddable fails to load if partition value
is empty string
(#167827)](#167827)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Quynh Nguyen
(Quinn)","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-06T16:49:32Z","message":"[ML]
Fix Anomaly charts embeddable fails to load if partition value is empty
string (#167827)\n\n## Summary\r\n\r\nFix
#165847. This PR
fixes\r\nAnomaly charts embeddable failing to load if partition value is
an empty\r\nstring. This bug is caused by invalid kuery string when a
field value is\r\nempty.\r\n\r\nPreviously, embeddable input was
receiving the following kql query\r\n`{exampleFieldName}:` if the value
is empty. The fix in this PR makes it\r\nso it's
`{exampleFieldName}:\"\"`. So a new
function\r\ngetEscapedKueryForSelectionInfluencer is added for this
specific\r\nscenario.\r\n\r\nMaking a new function here instead of
updating\r\n`escapeKueryForFieldValuePair` because all the other usages
of that\r\nfunction are related to opening a URL so it requires a
special escape.\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/43350163/f09b207f-1c4c-42a5-aaea-dad50512676c\r\n\r\n\r\n\r\n\r\n###
Steps to reproduce:\r\n\r\n- In a data view, create a new runtime field.
For this runtime field,\r\nset to `keyword` type, and use following
script to mimic having an empty\r\nfield for certain docs.\r\n```\r\ndef
source = doc['ampersand&name'].value;\r\nif (source == \"AWE\")
{\r\n\temit(\"\");\r\n}\r\nelse
{\r\n\temit(doc['ampersand&name'].value);\r\n}\r\n```\r\n- Create an
Anomaly detection using that data view, using the runtime\r\nfield as a
split field or partition field.\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"ecc9d9f267e6f60509805bfdc01e8eed2858a586","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix",":ml","Feature:Anomaly
Detection","v8.11.0","v8.12.0"],"number":167827,"url":"https://github.com/elastic/kibana/pull/167827","mergeCommit":{"message":"[ML]
Fix Anomaly charts embeddable fails to load if partition value is empty
string (#167827)\n\n## Summary\r\n\r\nFix
#165847. This PR
fixes\r\nAnomaly charts embeddable failing to load if partition value is
an empty\r\nstring. This bug is caused by invalid kuery string when a
field value is\r\nempty.\r\n\r\nPreviously, embeddable input was
receiving the following kql query\r\n`{exampleFieldName}:` if the value
is empty. The fix in this PR makes it\r\nso it's
`{exampleFieldName}:\"\"`. So a new
function\r\ngetEscapedKueryForSelectionInfluencer is added for this
specific\r\nscenario.\r\n\r\nMaking a new function here instead of
updating\r\n`escapeKueryForFieldValuePair` because all the other usages
of that\r\nfunction are related to opening a URL so it requires a
special escape.\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/43350163/f09b207f-1c4c-42a5-aaea-dad50512676c\r\n\r\n\r\n\r\n\r\n###
Steps to reproduce:\r\n\r\n- In a data view, create a new runtime field.
For this runtime field,\r\nset to `keyword` type, and use following
script to mimic having an empty\r\nfield for certain docs.\r\n```\r\ndef
source = doc['ampersand&name'].value;\r\nif (source == \"AWE\")
{\r\n\temit(\"\");\r\n}\r\nelse
{\r\n\temit(doc['ampersand&name'].value);\r\n}\r\n```\r\n- Create an
Anomaly detection using that data view, using the runtime\r\nfield as a
split field or partition field.\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"ecc9d9f267e6f60509805bfdc01e8eed2858a586"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/167827","number":167827,"mergeCommit":{"message":"[ML]
Fix Anomaly charts embeddable fails to load if partition value is empty
string (#167827)\n\n## Summary\r\n\r\nFix
#165847. This PR
fixes\r\nAnomaly charts embeddable failing to load if partition value is
an empty\r\nstring. This bug is caused by invalid kuery string when a
field value is\r\nempty.\r\n\r\nPreviously, embeddable input was
receiving the following kql query\r\n`{exampleFieldName}:` if the value
is empty. The fix in this PR makes it\r\nso it's
`{exampleFieldName}:\"\"`. So a new
function\r\ngetEscapedKueryForSelectionInfluencer is added for this
specific\r\nscenario.\r\n\r\nMaking a new function here instead of
updating\r\n`escapeKueryForFieldValuePair` because all the other usages
of that\r\nfunction are related to opening a URL so it requires a
special escape.\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/43350163/f09b207f-1c4c-42a5-aaea-dad50512676c\r\n\r\n\r\n\r\n\r\n###
Steps to reproduce:\r\n\r\n- In a data view, create a new runtime field.
For this runtime field,\r\nset to `keyword` type, and use following
script to mimic having an empty\r\nfield for certain docs.\r\n```\r\ndef
source = doc['ampersand&name'].value;\r\nif (source == \"AWE\")
{\r\n\temit(\"\");\r\n}\r\nelse
{\r\n\temit(doc['ampersand&name'].value);\r\n}\r\n```\r\n- Create an
Anomaly detection using that data view, using the runtime\r\nfield as a
split field or partition field.\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"ecc9d9f267e6f60509805bfdc01e8eed2858a586"}}]}]
BACKPORT-->

Co-authored-by: Quynh Nguyen (Quinn) <[email protected]>

x-pack/plugins/ml/public/application/explorer/anomaly_context_menu.tsx

@@ -40,7 +40,7 @@ import { JobId } from '../../../common/types/anomaly_detection_jobs';
 import { getDefaultExplorerChartsPanelTitle } from '../../embeddables/anomaly_charts/anomaly_charts_embeddable';
 import { MAX_ANOMALY_CHARTS_ALLOWED } from '../../embeddables/anomaly_charts/anomaly_charts_initializer';
 import { useAnomalyExplorerContext } from './anomaly_explorer_context';
-import { escapeKueryForFieldValuePair } from '../util/string_utils';
+import { escapeKueryForEmbeddableFieldValuePair } from '../util/string_utils';
 import { useCasesModal } from '../contexts/kibana/use_cases_modal';
 import { DEFAULT_MAX_SERIES_TO_PLOT } from '../services/anomaly_explorer_charts_service';
 import {
@@ -154,7 +154,7 @@ export const AnomalyContextMenu: FC<AnomalyContextMenuProps> = ({
       const influencers = selectionInfluencers ?? [];
       const config = getDefaultEmbeddablePanelConfig(jobIds, queryString);
       const queryFromSelectedCells = influencers
-        .map((s) => escapeKueryForFieldValuePair(s.fieldName, s.fieldValue))
+        .map((s) => escapeKueryForEmbeddableFieldValuePair(s.fieldName, s.fieldValue))
         .join(' or ');
 
       // When adding anomaly charts to Dashboard, we want to respect the Dashboard's time range

x-pack/plugins/ml/public/application/util/string_utils.test.ts

@@ -14,6 +14,7 @@ import {
   toLocaleString,
   mlEscape,
   escapeForElasticsearchQuery,
+  escapeKueryForEmbeddableFieldValuePair,
 } from './string_utils';
 
 describe('ML - string utils', () => {
@@ -160,4 +161,13 @@ describe('ML - string utils', () => {
       expect(escapeForElasticsearchQuery('foo/bar')).toBe('foo\\/bar');
     });
   });
+  describe('escapeKueryForEmbeddableFieldValuePair', () => {
+    test('should return correct escaping of kuery values', () => {
+      expect(escapeKueryForEmbeddableFieldValuePair('fieldName', '')).toBe('fieldName:""');
+      expect(escapeKueryForEmbeddableFieldValuePair('', 'fieldValue')).toBe('"":fieldValue');
+      expect(escapeKueryForEmbeddableFieldValuePair('@#specialCharsName%', '<>:;[})')).toBe(
+        '@#specialCharsName%:\\<\\>\\:;[}\\)'
+      );
+    });
+  });
 });

x-pack/plugins/ml/public/application/util/string_utils.ts

@@ -140,6 +140,26 @@ export function escapeKueryForFieldValuePair(
   return `${escapeKuery(name)}:${escapeKuery(value.toString())}`;
 }
 
+const replaceEmptyStringWithQuotation = (s: string) => (s === '' ? '""' : s);
+
+/**
+ *
+ * Helper function to returns escaped combined field name and value
+ * which also replaces empty str with " to ensure compatability with kql queries
+ * @param name fieldName of selection
+ * @param value fieldValue of selection
+ * @returns {string} escaped `name:value` compatible with embeddable input
+ */
+export function escapeKueryForEmbeddableFieldValuePair(
+  name: string,
+  value: string | number | boolean | undefined
+): string {
+  if (!isDefined(name) || !isDefined(value)) return '';
+  return `${replaceEmptyStringWithQuotation(escapeKuery(name))}:${replaceEmptyStringWithQuotation(
+    escapeKuery(value.toString())
+  )}`;
+}
+
 export function calculateTextWidth(txt: string | number, isNumber: boolean) {
   txt = isNumber && typeof txt === 'number' ? d3.format(',')(txt) : txt;