Skip to content

Commit

Permalink
[8.x] [ResponseOps][Cases] Miscount of total numbers of alerts in tel…
Browse files Browse the repository at this point in the history 
…emetry (#196112) (#197976)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ResponseOps][Cases] Miscount of total numbers of alerts in telemetry
(#196112)](#196112)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Georgiana-Andreea
Onoleață","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-28T11:39:32Z","message":"[ResponseOps][Cases]
Miscount of total numbers of alerts in telemetry (#196112)\n\nCloses
https://github.com/elastic/kibana/issues/177208\r\n\r\n##
Summary\r\n\r\nProblem: \r\n- the metrics collected in telemetry for
alerts don't count the total\r\nnumber of alerts on a case
correctly.\r\n\r\nSolution: \r\n- added new aggregation function:
getUniqueAlertCommentsCountQuery,\r\nwhich is now responsible for
defining the cardinality aggregation for\r\ncounting unique alert
comments by alertId.\r\n- in the aggs section of the
savedObjectsClient.find, the new\r\ncardinality aggregation query was
added\r\n- the total number of alerts is updated to be the result
extracted from\r\nthe new aggregation\r\n\r\nExample:
\r\n\r\n![Screenshot 2024-10-22 at 15
20\r\n40](https://github.com/user-attachments/assets/c418c82e-2e35-4c7f-969d-7f4f25bdbc9d)\r\n\r\n\r\n-
in the telemetry object, we have the following info: \r\n<img
width=\"331\" alt=\"Screenshot 2024-10-22 at 15 21
40\"\r\nsrc=\"https://github.com/user-attachments/assets/6419e72d-84b4-4068-a741-6e32c6e966f7\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Antonio
<[email protected]>","sha":"73c22a50fda298537f28bd000731b0584503e7ae","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:ResponseOps","v9.0.0","backport:prev-minor"],"title":"[ResponseOps][Cases]
Miscount of total numbers of alerts in
telemetry","number":196112,"url":"https://github.com/elastic/kibana/pull/196112","mergeCommit":{"message":"[ResponseOps][Cases]
Miscount of total numbers of alerts in telemetry (#196112)\n\nCloses
https://github.com/elastic/kibana/issues/177208\r\n\r\n##
Summary\r\n\r\nProblem: \r\n- the metrics collected in telemetry for
alerts don't count the total\r\nnumber of alerts on a case
correctly.\r\n\r\nSolution: \r\n- added new aggregation function:
getUniqueAlertCommentsCountQuery,\r\nwhich is now responsible for
defining the cardinality aggregation for\r\ncounting unique alert
comments by alertId.\r\n- in the aggs section of the
savedObjectsClient.find, the new\r\ncardinality aggregation query was
added\r\n- the total number of alerts is updated to be the result
extracted from\r\nthe new aggregation\r\n\r\nExample:
\r\n\r\n![Screenshot 2024-10-22 at 15
20\r\n40](https://github.com/user-attachments/assets/c418c82e-2e35-4c7f-969d-7f4f25bdbc9d)\r\n\r\n\r\n-
in the telemetry object, we have the following info: \r\n<img
width=\"331\" alt=\"Screenshot 2024-10-22 at 15 21
40\"\r\nsrc=\"https://github.com/user-attachments/assets/6419e72d-84b4-4068-a741-6e32c6e966f7\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Antonio
<[email protected]>","sha":"73c22a50fda298537f28bd000731b0584503e7ae"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196112","number":196112,"mergeCommit":{"message":"[ResponseOps][Cases]
Miscount of total numbers of alerts in telemetry (#196112)\n\nCloses
https://github.com/elastic/kibana/issues/177208\r\n\r\n##
Summary\r\n\r\nProblem: \r\n- the metrics collected in telemetry for
alerts don't count the total\r\nnumber of alerts on a case
correctly.\r\n\r\nSolution: \r\n- added new aggregation function:
getUniqueAlertCommentsCountQuery,\r\nwhich is now responsible for
defining the cardinality aggregation for\r\ncounting unique alert
comments by alertId.\r\n- in the aggs section of the
savedObjectsClient.find, the new\r\ncardinality aggregation query was
added\r\n- the total number of alerts is updated to be the result
extracted from\r\nthe new aggregation\r\n\r\nExample:
\r\n\r\n![Screenshot 2024-10-22 at 15
20\r\n40](https://github.com/user-attachments/assets/c418c82e-2e35-4c7f-969d-7f4f25bdbc9d)\r\n\r\n\r\n-
in the telemetry object, we have the following info: \r\n<img
width=\"331\" alt=\"Screenshot 2024-10-22 at 15 21
40\"\r\nsrc=\"https://github.com/user-attachments/assets/6419e72d-84b4-4068-a741-6e32c6e966f7\">\r\n\r\n---------\r\n\r\nCo-authored-by:
Antonio
<[email protected]>","sha":"73c22a50fda298537f28bd000731b0584503e7ae"}}]}]
BACKPORT-->

Co-authored-by: Georgiana-Andreea Onoleață <[email protected]>
  • Loading branch information
@kibanamachine @georgianaonoleata1904
kibanamachine and georgianaonoleata1904 authored Oct 28, 2024
1 parent 051014c commit f4cee5a
Show file tree
Hide file tree
Showing 6 changed files with 438 additions and 14 deletions.
40 changes: 33 additions & 7 deletions x-pack/plugins/cases/server/telemetry/queries/alerts.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,20 @@ describe('alerts', () => {
const telemetrySavedObjectsClient = new TelemetrySavedObjectsClient(savedObjectsClient);

savedObjectsClient.find.mockResolvedValue({
total: 5,
total: 3,
saved_objects: [],
per_page: 1,
page: 1,
aggregations: {
counts: {
buckets: [
{ doc_count: 1, key: 1 },
{ doc_count: 2, key: 2 },
{ doc_count: 3, key: 3 },
{ topAlertsPerBucket: { value: 12 } },
{ topAlertsPerBucket: { value: 5 } },
{ topAlertsPerBucket: { value: 3 } },
],
},
references: { cases: { max: { value: 1 } } },
uniqueAlertCommentsCount: { value: 5 },
},
});

Expand All @@ -42,12 +43,13 @@ describe('alerts', () => {
savedObjectsClient: telemetrySavedObjectsClient,
logger,
});

expect(res).toEqual({
all: {
total: 5,
daily: 3,
weekly: 2,
monthly: 1,
weekly: 5,
monthly: 12,
maxOnACase: 1,
},
});
Expand Down Expand Up @@ -76,6 +78,13 @@ describe('alerts', () => {
},
],
},
aggregations: {
topAlertsPerBucket: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
},
references: {
aggregations: {
Expand All @@ -85,10 +94,22 @@ describe('alerts', () => {
terms: {
field: 'cases-comments.references.id',
},
aggregations: {
reverse: {
reverse_nested: {},
aggregations: {
topAlerts: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
},
},
},
max: {
max_bucket: {
buckets_path: 'ids._count',
buckets_path: 'ids>reverse.topAlerts',
},
},
},
Expand All @@ -103,6 +124,11 @@ describe('alerts', () => {
path: 'cases-comments.references',
},
},
uniqueAlertCommentsCount: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
filter: {
arguments: [
Expand Down
7 changes: 2 additions & 5 deletions x-pack/plugins/cases/server/telemetry/queries/alerts.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,14 @@
* 2.0.
*/

import { CASE_COMMENT_SAVED_OBJECT } from '../../../common/constants';
import type { CasesTelemetry, CollectTelemetryDataParams } from '../types';
import { getCountsAndMaxData, getOnlyAlertsCommentsFilter } from './utils';
import { getCountsAndMaxAlertsData } from './utils';

export const getAlertsTelemetryData = async ({
savedObjectsClient,
}: CollectTelemetryDataParams): Promise<CasesTelemetry['comments']> => {
const res = await getCountsAndMaxData({
const res = await getCountsAndMaxAlertsData({
savedObjectsClient,
savedObjectType: CASE_COMMENT_SAVED_OBJECT,
filter: getOnlyAlertsCommentsFilter(),
});

return res;
Expand Down
228 changes: 228 additions & 0 deletions x-pack/plugins/cases/server/telemetry/queries/utils.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,12 @@ import type {
import {
findValueInBuckets,
getAggregationsBuckets,
getAlertsCountsFromBuckets,
getAttachmentsFrameworkStats,
getBucketFromAggregation,
getConnectorsCardinalityAggregationQuery,
getCountsAggregationQuery,
getCountsAndMaxAlertsData,
getCountsAndMaxData,
getCountsFromBuckets,
getCustomFieldsTelemetry,
Expand All @@ -28,6 +30,7 @@ import {
getOnlyConnectorsFilter,
getReferencesAggregationQuery,
getSolutionValues,
getUniqueAlertCommentsCountQuery,
} from './utils';
import { TelemetrySavedObjectsClient } from '../telemetry_saved_objects_client';

Expand Down Expand Up @@ -994,6 +997,63 @@ describe('utils', () => {
});
});

describe('getAlertsCountsFromBuckets', () => {
it('returns the correct counts', () => {
const buckets = [
{ topAlertsPerBucket: { value: 12 } },
{ topAlertsPerBucket: { value: 5 } },
{ topAlertsPerBucket: { value: 3 } },
];

expect(getAlertsCountsFromBuckets(buckets)).toEqual({
daily: 3,
weekly: 5,
monthly: 12,
});
});

it('returns zero counts when the bucket does not have the topAlertsPerBucket field', () => {
const buckets = [{}];
// @ts-expect-error
expect(getAlertsCountsFromBuckets(buckets)).toEqual({
daily: 0,
weekly: 0,
monthly: 0,
});
});

it('returns zero counts when the bucket is undefined', () => {
// @ts-expect-error
expect(getAlertsCountsFromBuckets(undefined)).toEqual({
daily: 0,
weekly: 0,
monthly: 0,
});
});

it('returns zero counts when the topAlertsPerBucket field is missing in some buckets', () => {
const buckets = [{ doc_count: 1, key: 1, topAlertsPerBucket: { value: 5 } }, {}, {}];
// @ts-expect-error
expect(getAlertsCountsFromBuckets(buckets)).toEqual({
daily: 0,
weekly: 0,
monthly: 5,
});
});
});

describe('getUniqueAlertCommentsCountQuery', () => {
it('returns the correct query', () => {
expect(getUniqueAlertCommentsCountQuery()).toEqual({
uniqueAlertCommentsCount: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
});
});
});

describe('getCountsAndMaxData', () => {
const savedObjectsClient = savedObjectsRepositoryMock.create();
savedObjectsClient.find.mockResolvedValue({
Expand Down Expand Up @@ -1125,6 +1185,174 @@ describe('utils', () => {
});
});

describe('getCountsAndMaxAlertsData', () => {
const savedObjectsClient = savedObjectsRepositoryMock.create();
savedObjectsClient.find.mockResolvedValue({
total: 3,
saved_objects: [],
per_page: 1,
page: 1,
aggregations: {
counts: {
buckets: [
{ doc_count: 1, key: 1, topAlertsPerBucket: { value: 5 } },
{ doc_count: 2, key: 2, topAlertsPerBucket: { value: 3 } },
{ doc_count: 3, key: 3, topAlertsPerBucket: { value: 1 } },
],
},
references: { cases: { max: { value: 1 } } },
uniqueAlertCommentsCount: { value: 5 },
},
});

beforeEach(() => {
jest.clearAllMocks();
});

it('returns the correct counts and max data', async () => {
const telemetrySavedObjectsClient = new TelemetrySavedObjectsClient(savedObjectsClient);

const res = await getCountsAndMaxAlertsData({
savedObjectsClient: telemetrySavedObjectsClient,
});
expect(res).toEqual({
all: {
total: 5,
daily: 1,
weekly: 3,
monthly: 5,
maxOnACase: 1,
},
});
});

it('returns zero data if the response aggregation is not as expected', async () => {
const telemetrySavedObjectsClient = new TelemetrySavedObjectsClient(savedObjectsClient);
savedObjectsClient.find.mockResolvedValue({
total: 5,
saved_objects: [],
per_page: 1,
page: 1,
});

const res = await getCountsAndMaxAlertsData({
savedObjectsClient: telemetrySavedObjectsClient,
});
expect(res).toEqual({
all: {
total: 0,
daily: 0,
weekly: 0,
monthly: 0,
maxOnACase: 0,
},
});
});

it('should call find with correct arguments', async () => {
const telemetrySavedObjectsClient = new TelemetrySavedObjectsClient(savedObjectsClient);

await getCountsAndMaxAlertsData({
savedObjectsClient: telemetrySavedObjectsClient,
});

expect(savedObjectsClient.find).toBeCalledWith({
aggs: {
counts: {
date_range: {
field: 'cases-comments.attributes.created_at',
format: 'dd/MM/YYYY',
ranges: [
{
from: 'now-1d',
to: 'now',
},
{
from: 'now-1w',
to: 'now',
},
{
from: 'now-1M',
to: 'now',
},
],
},
aggregations: {
topAlertsPerBucket: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
},
references: {
aggregations: {
cases: {
aggregations: {
ids: {
terms: {
field: 'cases-comments.references.id',
},
aggregations: {
reverse: {
reverse_nested: {},
aggregations: {
topAlerts: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
},
},
},
max: {
max_bucket: {
buckets_path: 'ids>reverse.topAlerts',
},
},
},
filter: {
term: {
'cases-comments.references.type': 'cases',
},
},
},
},
nested: {
path: 'cases-comments.references',
},
},
uniqueAlertCommentsCount: {
cardinality: {
field: 'cases-comments.attributes.alertId',
},
},
},
filter: {
arguments: [
{
isQuoted: false,
type: 'literal',
value: 'cases-comments.attributes.type',
},
{
isQuoted: false,
type: 'literal',
value: 'alert',
},
],
function: 'is',
type: 'function',
},
page: 0,
perPage: 0,
type: 'cases-comments',
namespaces: ['*'],
});
});
});

describe('getBucketFromAggregation', () => {
it('returns the buckets', () => {
expect(
Expand Down
Loading

0 comments on commit f4cee5a

Please sign in to comment.