Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(investigation): add usage collector #197659

Merged
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@
"security",
"observability",
"licensing",
"ruleRegistry"
"ruleRegistry",
"usageCollection"
],
"requiredBundles": [
"esql",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { ElasticsearchClientMock, savedObjectsRepositoryMock } from '@kbn/core/server/mocks';
import { CollectorFetchContext } from '@kbn/usage-collection-plugin/server';
import { fetcher } from './fetcher';

let savedObjectClient: ReturnType<typeof savedObjectsRepositoryMock.create>;

let closeMock: jest.Mock;
let esClient: ElasticsearchClientMock;

describe('Investigation usage collector fetcher', () => {
beforeEach(() => {
savedObjectClient = savedObjectsRepositoryMock.create();
closeMock = jest.fn();
});

it('without any existing investigation', async () => {
savedObjectClient.createPointInTimeFinder.mockReturnValue({
find: async function* find() {
return {
[Symbol.asyncIterator]: async () => {},
next: () => {},
};
},
close: closeMock,
});

const results = await fetcher({
soClient: savedObjectClient,
esClient,
} as CollectorFetchContext);

expect(closeMock).toHaveBeenCalled();
expect(results.investigation).toMatchInlineSnapshot(`
Object {
"by_origin": Object {
"alert": 0,
"blank": 0,
},
"by_status": Object {
"active": 0,
"cancelled": 0,
"mitigated": 0,
"resolved": 0,
"triage": 0,
},
"items": Object {
"avg": 0,
"max": 0,
"min": 0,
"p90": 0,
"p95": 0,
},
"notes": Object {
"avg": 0,
"max": 0,
"min": 0,
"p90": 0,
"p95": 0,
},
"total": 0,
}
`);
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { CollectorFetchContext } from '@kbn/usage-collection-plugin/server';
import { StoredInvestigation } from '../../models/investigation';
import { SO_INVESTIGATION_TYPE } from '../../saved_objects/investigation';
import { computeMetrics } from './helpers/metrics';
import { Usage } from './type';

export const fetcher = async (context: CollectorFetchContext) => {
const finder = context.soClient.createPointInTimeFinder<StoredInvestigation>({
type: SO_INVESTIGATION_TYPE,
perPage: 1000,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be potentially very memory heavy task, if we let's say store lot of data as part of investigation, i imagine storing images, screenshots, lens embeddables attributes JSON, LLM output, all of that data is stored in items right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's possible indeed - I can reduce the batch size to something smaller, like 10?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the status part could be fetched as an aggregation, the length of notes and items arrays is tricky to get since that isn't mapped.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah let's reduce the perPage for now. i can't think of anything else. since we can't use fields as well. If we really want to optimise, we can save length of items and notes as separate fields and get those. that way items and notes we won't have to fetch. but i guess it might be overkill.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed with 89bc326


let usage: Usage['investigation'] = {
total: 0,
by_status: {
triage: 0,
active: 0,
mitigated: 0,
resolved: 0,
cancelled: 0,
},
by_origin: {
alert: 0,
blank: 0,
},
items: {
avg: 0,
p90: 0,
p95: 0,
max: 0,
min: 0,
},
notes: {
avg: 0,
p90: 0,
p95: 0,
max: 0,
min: 0,
},
};

const items: number[] = [];
const notes: number[] = [];

for await (const response of finder.find()) {
usage = response.saved_objects.reduce((acc, so) => {
items.push(so.attributes.items.length);
notes.push(so.attributes.notes.length);

return {
...acc,
total: acc.total + 1,
by_status: {
...acc.by_status,
...(so.attributes.status === 'triage' && { triage: acc.by_status.triage + 1 }),
...(so.attributes.status === 'active' && { active: acc.by_status.active + 1 }),
...(so.attributes.status === 'mitigated' && { mitigated: acc.by_status.mitigated + 1 }),
...(so.attributes.status === 'resolved' && { resolved: acc.by_status.resolved + 1 }),
...(so.attributes.status === 'cancelled' && { cancelled: acc.by_status.cancelled + 1 }),
},
by_origin: {
...acc.by_origin,
...(so.attributes.origin.type === 'alert' && { alert: acc.by_origin.alert + 1 }),
...(so.attributes.origin.type === 'blank' && { blank: acc.by_origin.blank + 1 }),
},
};
}, usage);
}

usage.items = computeMetrics(items.sort());
usage.notes = computeMetrics(notes.sort());

await finder.close();

return {
investigation: usage,
};
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { computeMetrics } from './metrics';

describe('ComputeMetrics', () => {
it('computes the metrics correctly', async () => {
expect(computeMetrics([])).toMatchInlineSnapshot(`
Object {
"avg": 0,
"max": 0,
"min": 0,
"p90": 0,
"p95": 0,
}
`);
expect(computeMetrics([10, 10, 100])).toMatchInlineSnapshot(`
Object {
"avg": 40,
"max": 100,
"min": 10,
"p90": 100,
"p95": 100,
}
`);

const arr = Array.from({ length: 100 }, (_, i) => i);
expect(computeMetrics(arr)).toMatchInlineSnapshot(`
Object {
"avg": 49.5,
"max": 99,
"min": 0,
"p90": 90,
"p95": 95,
}
`);
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { sum } from 'lodash';

export function computeMetrics(arr: number[]) {
if (arr.length === 0) {
return {
avg: 0,
p90: 0,
p95: 0,
max: 0,
min: 0,
};
}

const total = sum(arr);
const r90 = (90 / 100) * (arr.length - 1) + 1;
const r95 = (95 / 100) * (arr.length - 1) + 1;

return {
avg: total / arr.length,
p90: arr[Math.floor(r90)],
p95: arr[Math.floor(r95)],
max: arr[arr.length - 1],
min: arr[0],
};
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { UsageCollectionSetup } from '@kbn/usage-collection-plugin/server';
import { fetcher } from './fetcher';
import type { Usage } from './type';

export function registerUsageCollector(usageCollection?: UsageCollectionSetup): void {
if (!usageCollection) {
return;
}

const usageCollector = usageCollection.makeUsageCollector<Usage>({
type: 'investigation',
schema: {
investigation: {
total: {
type: 'long',
_meta: {
description: 'The total number of investigations in the cluster',
},
},
by_status: {
triage: {
type: 'long',
_meta: {
description: 'The number of investigations in triage status in the cluster',
},
},
active: {
type: 'long',
_meta: { description: 'The number of investigations in active status in the cluster' },
},
mitigated: {
type: 'long',
_meta: {
description: 'The number of investigations in mitigated status in the cluster',
},
},
resolved: {
type: 'long',
_meta: {
description: 'The number of investigations in resolved status in the cluster',
},
},
cancelled: {
type: 'long',
_meta: {
description: 'The number of investigations in cancelled status in the cluster',
},
},
},
by_origin: {
alert: {
type: 'long',
_meta: {
description: 'The number of investigations created from alerts in the cluster',
},
},
blank: {
type: 'long',
_meta: {
description: 'The number of investigations created from scratch in the cluster',
},
},
},
items: {
avg: {
type: 'long',
_meta: {
description: 'The average number of items across all investigations in the cluster',
},
},
p90: {
type: 'long',
_meta: {
description:
'The 90th percentile of the number of items across all investigations in the cluster',
},
},
p95: {
type: 'long',
_meta: {
description:
'The 95th percentile of the number of items across all investigations in the cluster',
},
},
max: {
type: 'long',
_meta: {
description: 'The maximum number of items across all investigations in the cluster',
},
},
min: {
type: 'long',
_meta: {
description: 'The minimum number of items across all investigations in the cluster',
},
},
},
notes: {
avg: {
type: 'long',
_meta: {
description: 'The average number of notes across all investigations in the cluster',
},
},
p90: {
type: 'long',
_meta: {
description:
'The 90th percentile of the number of notes across all investigations in the cluster',
},
},
p95: {
type: 'long',
_meta: {
description:
'The 95th percentile of the number of notes across all investigations in the cluster',
},
},
max: {
type: 'long',
_meta: {
description: 'The maximum number of notes across all investigations in the cluster',
},
},
min: {
type: 'long',
_meta: {
description: 'The minimum number of notes across all investigations in the cluster',
},
},
},
},
},
isReady: () => true,
fetch: fetcher,
});

// register usage collector
usageCollection.registerCollector(usageCollector);
}
Loading