Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cases] Case action: Time window #171754

Merged
merged 56 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0e1bfb0
Register the case action
cnasikas Oct 9, 2023
4777f9c
Register the cases oracle
cnasikas Oct 9, 2023
65a88fa
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Oct 9, 2023
eeb35a6
Calculate the hash of the record ID
cnasikas Oct 9, 2023
649dea9
Merge branch 'register_case_action' of github.com:cnasikas/kibana int…
cnasikas Oct 10, 2023
4491476
Get oracle record
cnasikas Oct 10, 2023
f4a81a1
Rename folder
cnasikas Oct 11, 2023
5822d73
Sort grouping definition
cnasikas Oct 11, 2023
df77537
Increase counter
cnasikas Oct 11, 2023
9955239
Change grouping to record
cnasikas Oct 12, 2023
58bc3d6
Make the rule ID optional in the key
cnasikas Oct 12, 2023
581819a
Better types
cnasikas Oct 13, 2023
449a1b7
Add version when updating
cnasikas Oct 13, 2023
c210a0c
Improve types
cnasikas Oct 13, 2023
c802637
Fix tests
cnasikas Oct 13, 2023
b249032
Merge branch 'case_action' into register_case_action
cnasikas Oct 16, 2023
c120c97
Add model version and improve mapping
cnasikas Oct 16, 2023
043a9fe
Fix tests
cnasikas Oct 16, 2023
a9db13f
Fix mapping test
cnasikas Oct 16, 2023
32af9e3
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Oct 16, 2023
90acabf
Merge branch 'case_action' into register_case_action
cnasikas Oct 18, 2023
b7605b7
Merge branch 'register_case_action' of github.com:cnasikas/kibana int…
cnasikas Oct 18, 2023
7b27008
Define connector params initial schema
cnasikas Oct 17, 2023
13fd013
Bulk get records
cnasikas Oct 17, 2023
69a8778
Group alerts and bulk get oracle records
cnasikas Oct 17, 2023
e5c73a3
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Oct 18, 2023
1ba6117
Bulk create records
cnasikas Oct 18, 2023
bf8a32d
Merge branch 'ca_part_2' of github.com:cnasikas/kibana into ca_part_2
cnasikas Oct 18, 2023
c6982a5
Add TODOs
cnasikas Oct 19, 2023
f184a08
Move bulkGetOrCreateOracleRecords logic to the connector
cnasikas Oct 20, 2023
40f865e
Generate case ids
cnasikas Oct 20, 2023
f70bf1e
Get service with a factory
cnasikas Oct 21, 2023
04805ac
Fix docs
cnasikas Oct 21, 2023
a3d47e6
Merge branch 'main' into saf_get_instance
cnasikas Oct 21, 2023
3988644
Merge branch 'saf_get_instance' into ca_part_2
cnasikas Oct 21, 2023
af1b8ae
Pass the cases client to the case connector
cnasikas Oct 21, 2023
4c6f5ae
Attach alerts to a case
cnasikas Oct 23, 2023
a7f1ba1
Merge branch 'case_action' into register_case_action
cnasikas Oct 23, 2023
a283b93
Merge branch 'register_case_action' into ca_part_2
cnasikas Oct 23, 2023
8fd2a84
Merge branch 'case_action' into ca_part_2
cnasikas Nov 8, 2023
13012eb
Merge branch 'case_action' into ca_part_2
cnasikas Nov 8, 2023
c8e82e5
Bulk create non existing cases
cnasikas Nov 8, 2023
1ead049
Improve the case request
cnasikas Nov 10, 2023
b31c167
Small improvements
cnasikas Nov 10, 2023
2621c7d
Add time window to schema
cnasikas Nov 10, 2023
ab42f16
Merge branch 'case_action' into ca_time_window
cnasikas Nov 15, 2023
ad14c3f
Merge branch 'case_action' into ca_time_window
cnasikas Nov 22, 2023
5d340aa
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Nov 22, 2023
b20ef53
Upsert oracle records
cnasikas Nov 23, 2023
b3806ef
Increase the counter when the time window has passed
cnasikas Nov 29, 2023
27209ac
Merge branch 'case_action' into ca_time_window
cnasikas Nov 29, 2023
346735b
Merge branch 'ca_time_window' of github.com:cnasikas/kibana into ca_t…
cnasikas Nov 29, 2023
7e5989e
Add more tests
cnasikas Nov 29, 2023
b5fecef
Merge branch 'case_action' into ca_time_window
cnasikas Dec 8, 2023
81ab220
Fix bug with Regex
cnasikas Dec 8, 2023
d4e1e8c
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Dec 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions packages/kbn-check-mappings-update-cli/current_fields.json
Original file line number Diff line number Diff line change
Expand Up @@ -966,5 +966,14 @@
"kuery",
"serviceEnvironmentFilterEnabled",
"serviceNameFilterEnabled"
],
"cases-oracle": [
"cases",
"cases.id",
"counter",
"createdAt",
"rules",
"rules.id",
"updatedAt"
]
}
189 changes: 172 additions & 17 deletions x-pack/plugins/cases/server/connectors/cases/cases_connector.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
* 2.0.
*/

import dateMath from '@kbn/datemath';
import moment from 'moment';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we already use moment in cases? Is there no alternative? It's a deprecated 72.1kb (gzipped) library :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! Good point. Unfortunately, the @kbn/datemath library which is developed and maintained by Kibana uses moment behind the scenes. I need @kbn/datemath to allow users to set the time window as 7d etc. This is a test file. I wanted to mock the datemath to make the test predicable. Do you have in mind any alternatives?

import { actionsConfigMock } from '@kbn/actions-plugin/server/actions_config.mock';
import { actionsMock } from '@kbn/actions-plugin/server/mocks';
import { loggingSystemMock } from '@kbn/core-logging-server-mocks';
Expand All @@ -19,9 +21,11 @@ import type { Cases } from '../../../common';

jest.mock('./cases_oracle_service');
jest.mock('./cases_service');
jest.mock('@kbn/datemath');

const CasesOracleServiceMock = CasesOracleService as jest.Mock;
const CasesServiceMock = CasesService as jest.Mock;
const dateMathMock = dateMath as jest.Mocked<typeof dateMath>;

describe('CasesConnector', () => {
const services = actionsMock.createServices();
Expand Down Expand Up @@ -53,7 +57,9 @@ describe('CasesConnector', () => {
tags: ['rule', 'test'],
ruleUrl: 'https://example.com/rules/rule-test-id',
};

const owner = 'cases';
const timeWindow = '7d';

const groupedAlertsWithOracleKey = [
{
Expand Down Expand Up @@ -91,8 +97,8 @@ describe('CasesConnector', () => {
cases: [],
rules: [],
grouping: groupedAlertsWithOracleKey[1].grouping,
createdAt: '2023-10-10T10:23:42.769Z',
updatedAt: '2023-10-10T10:23:42.769Z',
createdAt: '2023-10-12T10:23:42.769Z',
updatedAt: '2023-10-12T10:23:42.769Z',
},
{
id: groupedAlertsWithOracleKey[2].oracleKey,
Expand All @@ -114,6 +120,7 @@ describe('CasesConnector', () => {
const mockGetRecordId = jest.fn();
const mockBulkGetRecords = jest.fn();
const mockBulkCreateRecord = jest.fn();
const mockBulkUpdateRecord = jest.fn();
const mockGetCaseId = jest.fn();

const getCasesClient = jest.fn();
Expand Down Expand Up @@ -141,6 +148,7 @@ describe('CasesConnector', () => {
version: 'so-version-2',
},
]),
bulkUpdateRecord: mockBulkUpdateRecord.mockResolvedValue([]),
};
});

Expand Down Expand Up @@ -168,12 +176,14 @@ describe('CasesConnector', () => {
services,
},
});

dateMathMock.parse.mockImplementation(() => moment('2023-10-09T10:23:42.769Z'));
});

describe('run', () => {
describe('Oracle records', () => {
it('generates the oracle keys correctly with grouping by one field', async () => {
await connector.run({ alerts, groupingBy: ['host.name'], owner, rule });
await connector.run({ alerts, groupingBy: ['host.name'], owner, rule, timeWindow });

expect(mockGetRecordId).toHaveBeenCalledTimes(2);

Expand All @@ -193,7 +203,7 @@ describe('CasesConnector', () => {
});

it('generates the oracle keys correct with grouping by multiple fields', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockGetRecordId).toHaveBeenCalledTimes(3);

Expand All @@ -208,7 +218,7 @@ describe('CasesConnector', () => {
});

it('gets the oracle records correctly', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockBulkGetRecords).toHaveBeenCalledWith([
groupedAlertsWithOracleKey[0].oracleKey,
Expand All @@ -218,7 +228,7 @@ describe('CasesConnector', () => {
});

it('created the non found oracle records correctly', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockBulkCreateRecord).toHaveBeenCalledWith([
{
Expand All @@ -235,15 +245,109 @@ describe('CasesConnector', () => {
it('does not create oracle records if there are no 404 errors', async () => {
mockBulkGetRecords.mockResolvedValue([oracleRecords[0]]);

await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockBulkCreateRecord).not.toHaveBeenCalled();
});

it('does not create oracle records if there are other errors than 404', async () => {
mockBulkGetRecords.mockResolvedValue([
{
id: groupedAlertsWithOracleKey[2].oracleKey,
type: CASE_ORACLE_SAVED_OBJECT,
message: 'Conflict',
statusCode: 409,
error: 'Conflict',
},
]);

await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

/**
* TODO: Change it to: expect(mockBulkCreateRecord).not.toHaveBeenCalled();
*/
expect(mockBulkCreateRecord).toHaveBeenCalledWith([]);
});

it('does not increase the counter if the time window has not passed', async () => {
mockBulkGetRecords.mockResolvedValue([oracleRecords[0]]);
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockBulkUpdateRecord).not.toHaveBeenCalled();
});

it('updates the counter correctly if the time window has passed', async () => {
dateMathMock.parse.mockImplementation(() => moment('2023-11-10T10:23:42.769Z'));
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockBulkUpdateRecord).toHaveBeenCalledWith([
{ payload: { counter: 2 }, recordId: 'so-oracle-record-0', version: 'so-version-0' },
{ payload: { counter: 2 }, recordId: 'so-oracle-record-1', version: 'so-version-1' },
]);
});

it('run correctly with all records: valid, counter increased, counter did not increased, created', async () => {
dateMathMock.parse.mockImplementation(() => moment('2023-10-11T10:23:42.769Z'));
mockBulkCreateRecord.mockResolvedValue([
{
...oracleRecords[0],
id: groupedAlertsWithOracleKey[2].oracleKey,
grouping: groupedAlertsWithOracleKey[2].grouping,
version: 'so-version-2',
},
// Returning errors to verify that the code does not return them
{
id: 'test-id',
type: CASE_ORACLE_SAVED_OBJECT,
message: 'Conflict',
statusCode: 409,
error: 'Conflict',
},
]);

mockBulkUpdateRecord.mockResolvedValue([
{ ...oracleRecords[0], counter: 2 },
// Returning errors to verify that the code does not return them
{
id: 'test-id',
type: CASE_ORACLE_SAVED_OBJECT,
message: 'Conflict',
statusCode: 409,
error: 'Conflict',
},
]);

await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

// 1. Get all records
expect(mockBulkGetRecords).toHaveBeenCalledWith([
groupedAlertsWithOracleKey[0].oracleKey,
groupedAlertsWithOracleKey[1].oracleKey,
groupedAlertsWithOracleKey[2].oracleKey,
]);

// 2. Create the non found records
expect(mockBulkCreateRecord).toHaveBeenCalledWith([
{
recordId: groupedAlertsWithOracleKey[2].oracleKey,
payload: {
cases: [],
grouping: groupedAlertsWithOracleKey[2].grouping,
rules: [],
},
},
]);

// 3. Update the counter for the records where the time window has passed
expect(mockBulkUpdateRecord).toHaveBeenCalledWith([
{ payload: { counter: 2 }, recordId: 'so-oracle-record-0', version: 'so-version-0' },
]);
});
});

describe('Cases', () => {
it('generates the case ids correctly', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockGetCaseId).toHaveBeenCalledTimes(3);

Expand All @@ -258,8 +362,54 @@ describe('CasesConnector', () => {
}
});

it('generates the case ids correctly when the time window has passed', async () => {
dateMathMock.parse.mockImplementation(() => moment('2023-10-11T10:23:42.769Z'));

mockBulkUpdateRecord.mockResolvedValue([{ ...oracleRecords[0], counter: 2 }]);

await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(mockGetCaseId).toBeCalledTimes(3);

/**
* Oracle record index: 1
* Should not update the counter
*/
expect(mockGetCaseId).nthCalledWith(1, {
counter: 1,
grouping: { 'dest.ip': '0.0.0.1', 'host.name': 'B' },
owner: 'cases',
ruleId: 'rule-test-id',
spaceId: 'default',
});

/**
* Oracle record index: 3
* Not found. Created.
*/
expect(mockGetCaseId).nthCalledWith(2, {
counter: 1,
grouping: { 'dest.ip': '0.0.0.3', 'host.name': 'B' },
owner: 'cases',
ruleId: 'rule-test-id',
spaceId: 'default',
});

/**
* Oracle record index: 0
* Should update the counter
*/
expect(mockGetCaseId).nthCalledWith(3, {
counter: 2,
grouping: { 'dest.ip': '0.0.0.1', 'host.name': 'A' },
owner: 'cases',
ruleId: 'rule-test-id',
spaceId: 'default',
});
});

it('gets the cases correctly', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(casesClientMock.cases.bulkGet).toHaveBeenCalledWith({
ids: ['mock-id-1', 'mock-id-2', 'mock-id-3'],
Expand All @@ -286,7 +436,7 @@ describe('CasesConnector', () => {
],
});

await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(casesClientMock.cases.bulkCreate).toHaveBeenCalledWith({
cases: [
Expand Down Expand Up @@ -323,15 +473,15 @@ describe('CasesConnector', () => {
],
});

await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(casesClientMock.cases.bulkCreate).not.toHaveBeenCalled();
});
});

describe('Alerts', () => {
it('attach the alerts to the correct cases correctly', async () => {
await connector.run({ alerts, groupingBy, owner, rule });
await connector.run({ alerts, groupingBy, owner, rule, timeWindow });

expect(casesClientMock.attachments.bulkCreate).toHaveBeenCalledTimes(3);

Expand Down Expand Up @@ -397,6 +547,11 @@ describe('CasesConnector', () => {
});
});

/**
* In this testing group we test
* only the functionality that differs
* from the testing with grouping
*/
describe('Without grouping', () => {
beforeEach(() => {
jest.clearAllMocks();
Expand Down Expand Up @@ -440,7 +595,7 @@ describe('CasesConnector', () => {

describe('Oracle records', () => {
it('generates the oracle keys correctly with no grouping', async () => {
await connector.run({ alerts, groupingBy: [], owner, rule });
await connector.run({ alerts, groupingBy: [], owner, rule, timeWindow });

expect(mockGetRecordId).toHaveBeenCalledTimes(1);

Expand All @@ -453,15 +608,15 @@ describe('CasesConnector', () => {
});

it('gets the oracle records correctly', async () => {
await connector.run({ alerts, groupingBy: [], owner, rule });
await connector.run({ alerts, groupingBy: [], owner, rule, timeWindow });

expect(mockBulkGetRecords).toHaveBeenCalledWith(['so-oracle-record-0']);
});
});

describe('Cases', () => {
it('generates the case ids correctly', async () => {
await connector.run({ alerts, groupingBy: [], owner, rule });
await connector.run({ alerts, groupingBy: [], owner, rule, timeWindow });

expect(mockGetCaseId).toHaveBeenCalledTimes(1);

Expand All @@ -475,7 +630,7 @@ describe('CasesConnector', () => {
});

it('gets the cases correctly', async () => {
await connector.run({ alerts, groupingBy: [], owner, rule });
await connector.run({ alerts, groupingBy: [], owner, rule, timeWindow });

expect(casesClientMock.cases.bulkGet).toHaveBeenCalledWith({
ids: ['mock-id-1'],
Expand All @@ -485,7 +640,7 @@ describe('CasesConnector', () => {

describe('Alerts', () => {
it('attach all alerts to the same case when the grouping is not defined', async () => {
await connector.run({ alerts, groupingBy: [], owner, rule });
await connector.run({ alerts, groupingBy: [], owner, rule, timeWindow });
expect(casesClientMock.attachments.bulkCreate).toHaveBeenCalledTimes(1);

expect(casesClientMock.attachments.bulkCreate).nthCalledWith(1, {
Expand Down
Loading
Loading