Skip to content

Commit

Permalink
[8.x] [Response Ops][Maintenance Window] Fix Maintenance Window Wildc…
Browse files Browse the repository at this point in the history
…ard Scoped Queries (#194777) (#197928)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Response Ops][Maintenance Window] Fix Maintenance Window Wildcard
Scoped Queries (#194777)](#194777)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Jiawei
Wu","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-26T09:47:29Z","message":"[Response
Ops][Maintenance Window] Fix Maintenance Window Wildcard Scoped Queries
(#194777)\n\n## Summary\r\n\r\nIssue:
https://github.com/elastic/sdh-kibana/issues/4923\r\n\r\nFixes
maintenance window scoped query using wildcards by injecting
the\r\n`analyze_wildcard` property to the DSL used to determine which
alerts\r\nshould be associated with the maintenance window.\r\n\r\nAlso
fixes the update route to correctly take into account the
user's\r\n`allowLeadingWildcard` flag. It was implemented for the create
route but\r\nnot the update route.\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/194763\r\n\r\n### To
test:\r\n1. Install sample
data:\r\n\r\n![image](https://github.com/user-attachments/assets/4be72fc8-e4ab-47a3-b5db-48f97b1827ae)\r\n\r\n2.
Create a maintenance window with the following scoped query:
\r\n\r\n![image](https://github.com/user-attachments/assets/e2d37fd0-b957-4e76-bea3-8d954651c557)\r\n\r\n3.
Create a ES query rule and trigger
actions:\r\n\r\n![image](https://github.com/user-attachments/assets/551f5145-9ab7-48c4-a48e-e674b4f0509a)\r\n\r\n4.
Assert the `maintenance_window_id` on the 4 alerts are
set\r\n\r\n![image](https://github.com/user-attachments/assets/7ace95d3-d992-4305-a564-cf3004c9ae9e)\r\n\r\n\r\n###
Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios)\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"7ad937db574603e53aeebe69d591554801cf857b","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:ResponseOps","v9.0.0","backport:prev-major","v8.16.0","v8.17.0"],"title":"[Response
Ops][Maintenance Window] Fix Maintenance Window Wildcard Scoped
Queries","number":194777,"url":"https://github.com/elastic/kibana/pull/194777","mergeCommit":{"message":"[Response
Ops][Maintenance Window] Fix Maintenance Window Wildcard Scoped Queries
(#194777)\n\n## Summary\r\n\r\nIssue:
https://github.com/elastic/sdh-kibana/issues/4923\r\n\r\nFixes
maintenance window scoped query using wildcards by injecting
the\r\n`analyze_wildcard` property to the DSL used to determine which
alerts\r\nshould be associated with the maintenance window.\r\n\r\nAlso
fixes the update route to correctly take into account the
user's\r\n`allowLeadingWildcard` flag. It was implemented for the create
route but\r\nnot the update route.\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/194763\r\n\r\n### To
test:\r\n1. Install sample
data:\r\n\r\n![image](https://github.com/user-attachments/assets/4be72fc8-e4ab-47a3-b5db-48f97b1827ae)\r\n\r\n2.
Create a maintenance window with the following scoped query:
\r\n\r\n![image](https://github.com/user-attachments/assets/e2d37fd0-b957-4e76-bea3-8d954651c557)\r\n\r\n3.
Create a ES query rule and trigger
actions:\r\n\r\n![image](https://github.com/user-attachments/assets/551f5145-9ab7-48c4-a48e-e674b4f0509a)\r\n\r\n4.
Assert the `maintenance_window_id` on the 4 alerts are
set\r\n\r\n![image](https://github.com/user-attachments/assets/7ace95d3-d992-4305-a564-cf3004c9ae9e)\r\n\r\n\r\n###
Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios)\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"7ad937db574603e53aeebe69d591554801cf857b"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/194777","number":194777,"mergeCommit":{"message":"[Response
Ops][Maintenance Window] Fix Maintenance Window Wildcard Scoped Queries
(#194777)\n\n## Summary\r\n\r\nIssue:
https://github.com/elastic/sdh-kibana/issues/4923\r\n\r\nFixes
maintenance window scoped query using wildcards by injecting
the\r\n`analyze_wildcard` property to the DSL used to determine which
alerts\r\nshould be associated with the maintenance window.\r\n\r\nAlso
fixes the update route to correctly take into account the
user's\r\n`allowLeadingWildcard` flag. It was implemented for the create
route but\r\nnot the update route.\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/194763\r\n\r\n### To
test:\r\n1. Install sample
data:\r\n\r\n![image](https://github.com/user-attachments/assets/4be72fc8-e4ab-47a3-b5db-48f97b1827ae)\r\n\r\n2.
Create a maintenance window with the following scoped query:
\r\n\r\n![image](https://github.com/user-attachments/assets/e2d37fd0-b957-4e76-bea3-8d954651c557)\r\n\r\n3.
Create a ES query rule and trigger
actions:\r\n\r\n![image](https://github.com/user-attachments/assets/551f5145-9ab7-48c4-a48e-e674b4f0509a)\r\n\r\n4.
Assert the `maintenance_window_id` on the 4 alerts are
set\r\n\r\n![image](https://github.com/user-attachments/assets/7ace95d3-d992-4305-a564-cf3004c9ae9e)\r\n\r\n\r\n###
Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios)\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"7ad937db574603e53aeebe69d591554801cf857b"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Jiawei Wu <[email protected]>
  • Loading branch information
kibanamachine and JiaweiWu authored Oct 26, 2024
1 parent f308bda commit 75e4b69
Show file tree
Hide file tree
Showing 6 changed files with 306 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ import {
import { SummarizedAlertsChunk, ScopedQueryAlerts } from '../..';
import { FormatAlert } from '../../types';
import { expandFlattenedAlert } from './format_alert';
import { injectAnalyzeWildcard } from './inject_analyze_wildcard';

const MAX_ALERT_DOCS_TO_RETURN = 100;
enum AlertTypes {
Expand Down Expand Up @@ -310,9 +311,14 @@ export const getQueryByScopedQueries = ({
return;
}

const scopedQueryFilter = generateAlertsFilterDSL({
query: scopedQuery as AlertsFilter['query'],
})[0] as { bool: BoolQuery };
const scopedQueryFilter = generateAlertsFilterDSL(
{
query: scopedQuery as AlertsFilter['query'],
},
{
analyzeWildcard: true,
}
)[0] as { bool: BoolQuery };

aggs[id] = {
filter: {
Expand All @@ -324,6 +330,7 @@ export const getQueryByScopedQueries = ({
aggs: {
alertId: {
top_hits: {
size: MAX_ALERT_DOCS_TO_RETURN,
_source: {
includes: [ALERT_UUID],
},
Expand All @@ -340,11 +347,19 @@ export const getQueryByScopedQueries = ({
};
};

const generateAlertsFilterDSL = (alertsFilter: AlertsFilter): QueryDslQueryContainer[] => {
const generateAlertsFilterDSL = (
alertsFilter: AlertsFilter,
options?: { analyzeWildcard?: boolean }
): QueryDslQueryContainer[] => {
const filter: QueryDslQueryContainer[] = [];
const { analyzeWildcard = false } = options || {};

if (alertsFilter.query) {
filter.push(JSON.parse(alertsFilter.query.dsl!));
const parsedQuery = JSON.parse(alertsFilter.query.dsl!);
if (analyzeWildcard) {
injectAnalyzeWildcard(parsedQuery);
}
filter.push(parsedQuery);
}
if (alertsFilter.timeframe) {
filter.push(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { injectAnalyzeWildcard } from './inject_analyze_wildcard';

const getQuery = (query?: string) => {
return {
bool: {
must: [],
filter: [
{
bool: {
filter: [
{
bool: {
should: [
{
query_string: {
fields: ['kibana.alert.instance.id'],
query: query || '*elastic*',
},
},
],
minimum_should_match: 1,
},
},
{
bool: {
should: [
{
match: {
'kibana.alert.action_group': 'test',
},
},
],
minimum_should_match: 1,
},
},
],
},
},
],
should: [],
must_not: [
{
match_phrase: {
_id: 'assdasdasd',
},
},
],
},
};
};
describe('injectAnalyzeWildcard', () => {
test('should inject analyze_wildcard field', () => {
const query = getQuery();
injectAnalyzeWildcard(query);
expect(query).toMatchInlineSnapshot(`
Object {
"bool": Object {
"filter": Array [
Object {
"bool": Object {
"filter": Array [
Object {
"bool": Object {
"minimum_should_match": 1,
"should": Array [
Object {
"query_string": Object {
"analyze_wildcard": true,
"fields": Array [
"kibana.alert.instance.id",
],
"query": "*elastic*",
},
},
],
},
},
Object {
"bool": Object {
"minimum_should_match": 1,
"should": Array [
Object {
"match": Object {
"kibana.alert.action_group": "test",
},
},
],
},
},
],
},
},
],
"must": Array [],
"must_not": Array [
Object {
"match_phrase": Object {
"_id": "assdasdasd",
},
},
],
"should": Array [],
},
}
`);
});

test('should not inject analyze_wildcard if the query does not contain *', () => {
const query = getQuery('test');
injectAnalyzeWildcard(query);
expect(query).toMatchInlineSnapshot(`
Object {
"bool": Object {
"filter": Array [
Object {
"bool": Object {
"filter": Array [
Object {
"bool": Object {
"minimum_should_match": 1,
"should": Array [
Object {
"query_string": Object {
"fields": Array [
"kibana.alert.instance.id",
],
"query": "test",
},
},
],
},
},
Object {
"bool": Object {
"minimum_should_match": 1,
"should": Array [
Object {
"match": Object {
"kibana.alert.action_group": "test",
},
},
],
},
},
],
},
},
],
"must": Array [],
"must_not": Array [
Object {
"match_phrase": Object {
"_id": "assdasdasd",
},
},
],
"should": Array [],
},
}
`);
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';

export const injectAnalyzeWildcard = (query: QueryDslQueryContainer): void => {
if (!query) {
return;
}

if (Array.isArray(query)) {
return query.forEach((child) => injectAnalyzeWildcard(child));
}

if (typeof query === 'object') {
Object.entries(query).forEach(([key, value]) => {
if (key !== 'query_string') {
return injectAnalyzeWildcard(value);
}

if (typeof value.query === 'string' && value.query.includes('*')) {
value.analyze_wildcard = true;
}
});
}
};
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,21 @@ describe('MaintenanceWindowClient - update', () => {
eventEndTime: '2023-03-05T01:00:00.000Z',
})
);

expect(uiSettings.get).toHaveBeenCalledTimes(3);
expect(uiSettings.get.mock.calls).toMatchInlineSnapshot(`
Array [
Array [
"query:allowLeadingWildcards",
],
Array [
"query:queryString:options",
],
Array [
"courier:ignoreFilterIfFieldNotInIndex",
],
]
`);
});

it('should not regenerate all events if rrule and duration did not change', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import Boom from '@hapi/boom';
import { buildEsQuery, Filter } from '@kbn/es-query';
import type { MaintenanceWindowClientContext } from '../../../../../common';
import { getScopedQueryErrorMessage } from '../../../../../common';
import { getEsQueryConfig } from '../../../../lib/get_es_query_config';
import type { MaintenanceWindow } from '../../types';
import {
generateMaintenanceWindowEvents,
Expand Down Expand Up @@ -45,9 +46,10 @@ async function updateWithOCC(
context: MaintenanceWindowClientContext,
params: UpdateMaintenanceWindowParams
): Promise<MaintenanceWindow> {
const { savedObjectsClient, getModificationMetadata, logger } = context;
const { savedObjectsClient, getModificationMetadata, logger, uiSettings } = context;
const { id, data } = params;
const { title, enabled, duration, rRule, categoryIds, scopedQuery } = data;
const esQueryConfig = await getEsQueryConfig(uiSettings);

try {
updateMaintenanceWindowParamsSchema.validate(params);
Expand All @@ -62,7 +64,8 @@ async function updateWithOCC(
buildEsQuery(
undefined,
[{ query: scopedQuery.kql, language: 'kuery' }],
scopedQuery.filters as Filter[]
scopedQuery.filters as Filter[],
esQueryConfig
)
);
scopedQueryWithGeneratedValue = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -245,5 +245,72 @@ export default function maintenanceWindowScopedQueryTests({ getService }: FtrPro
retry,
});
});

it('should associate alerts when scoped query contains wildcards', async () => {
await createMaintenanceWindow({
supertest,
objectRemover,
overwrites: {
scoped_query: {
kql: 'kibana.alert.rule.name: *test*',
filters: [],
},
category_ids: ['management'],
},
});

// Create action and rule
const action = await await createAction({
supertest,
objectRemover,
});

const { body: rule } = await supertestWithoutAuth
.post(`${getUrlPrefix(Spaces.space1.id)}/api/alerting/rule`)
.set('kbn-xsrf', 'foo')
.send(
getTestRuleData({
name: 'rule-test-rule',
rule_type_id: 'test.always-firing-alert-as-data',
schedule: { interval: '24h' },
tags: ['test'],
throttle: undefined,
notify_when: 'onActiveAlert',
params: {
index: alertAsDataIndex,
reference: 'test',
},
actions: [
{
id: action.id,
group: 'default',
params: {},
},
{
id: action.id,
group: 'recovered',
params: {},
},
],
})
)
.expect(200);

objectRemover.add(Spaces.space1.id, rule.id, 'rule', 'alerting');

// Run the first time - active
await getRuleEvents({
id: rule.id,
activeInstance: 2,
retry,
getService,
});

await expectNoActionsFired({
id: rule.id,
supertest,
retry,
});
});
});
}

0 comments on commit 75e4b69

Please sign in to comment.