Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Risk engine initialisation, update from legacy risk engine workflow and status change #162400

Merged
merged 54 commits into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
34a040e
Risk engine API start
nkhristinin Jul 12, 2023
62b04b6
Add saved objects and status
nkhristinin Jul 24, 2023
ef9d3e5
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jul 24, 2023
9f2e3d9
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Jul 24, 2023
347f4ff
Error handling
nkhristinin Jul 27, 2023
ad59803
Return who last updated
nkhristinin Jul 27, 2023
8423d5c
Add risk_score_update_panel
nkhristinin Jul 28, 2023
fdc0084
Risk update panel
nkhristinin Jul 28, 2023
05f28c5
Add risk score update panels
nkhristinin Jul 28, 2023
e7079c9
Delete old transforms
nkhristinin Jul 28, 2023
4ffaa99
Add mapping for SO
nkhristinin Jul 28, 2023
1b74331
fix name
nkhristinin Jul 28, 2023
8518cbb
Fix types
nkhristinin Jul 28, 2023
27e39c2
type
nkhristinin Jul 28, 2023
b4c212d
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jul 28, 2023
805d243
add tests
nkhristinin Jul 28, 2023
34fa0ed
Wrong rebase
nkhristinin Jul 28, 2023
2cce865
clean
nkhristinin Jul 28, 2023
189e941
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jul 28, 2023
6d7c016
Merge branch 'main' into enable-risk-score
kibanamachine Jul 30, 2023
b99c3e1
Typos and PR fixes
nkhristinin Jul 31, 2023
b4f5955
add api tets
nkhristinin Jul 31, 2023
a835c61
fix unit tests
nkhristinin Jul 31, 2023
e1ed39e
Try to enable feature in cypress tests
nkhristinin Jul 31, 2023
e6dcad4
udpate mappings
nkhristinin Jul 31, 2023
e31d85b
Merge branch 'main' into enable-risk-score
kibanamachine Jul 31, 2023
8373a58
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jul 31, 2023
e72b7d4
Refactoring
nkhristinin Aug 1, 2023
38e23bc
Fix jest tests
nkhristinin Aug 1, 2023
9a53d21
Merge branch 'main' into enable-risk-score
kibanamachine Aug 1, 2023
450871a
Try to fix types
nkhristinin Aug 1, 2023
1527f2a
Merge branch 'main' into enable-risk-score
kibanamachine Aug 1, 2023
b47ec84
Merge branch 'main' into enable-risk-score
kibanamachine Aug 2, 2023
5dffa4d
Change enable risk button
nkhristinin Aug 2, 2023
ca8329b
Add cy tests
nkhristinin Aug 2, 2023
7aad6cb
PR fixes
nkhristinin Aug 2, 2023
dac1f34
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Aug 2, 2023
6265ac3
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Aug 2, 2023
7fd2016
Merge branch 'main' into enable-risk-score
kibanamachine Aug 3, 2023
b31ab04
Remove last updated_by
nkhristinin Aug 3, 2023
a77a838
fix hest integration tests
nkhristinin Aug 3, 2023
ba83b7b
Fix cypress tetss
nkhristinin Aug 3, 2023
9742878
Fix jest tests
nkhristinin Aug 3, 2023
c77a82e
fix cypress tests
nkhristinin Aug 3, 2023
96b6ddc
Access for license and serverless
nkhristinin Aug 3, 2023
3b1bb9f
Chaange approach for link with license and capabilities
nkhristinin Aug 3, 2023
f8cf032
Fix links
nkhristinin Aug 3, 2023
7237b5f
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Aug 3, 2023
7df7d22
fix ts problems
nkhristinin Aug 3, 2023
ddbe593
Merge branch 'main' into enable-risk-score
rylnd Aug 3, 2023
1bc3314
Simplify logic for showing EA management page
rylnd Aug 3, 2023
d9b03b0
Fix tesxt
nkhristinin Aug 4, 2023
b8b638e
Hide update panel for serverless
nkhristinin Aug 4, 2023
b44c3b6
Merge branch 'main' into enable-risk-score
kibanamachine Aug 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/kbn-check-mappings-update-cli/current_mappings.json
Original file line number Diff line number Diff line change
Expand Up @@ -2938,6 +2938,16 @@
}
}
},
"risk-engine-configuration": {
"properties": {
"enabled": {
"type": "boolean"
},
"last_updated_by": {
"type": "keyword"
}
}
},
"infrastructure-ui-source": {
"dynamic": false,
"properties": {}
Expand Down
26 changes: 25 additions & 1 deletion packages/kbn-test/src/kbn_client/kbn_client_saved_objects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@

import { chunk } from 'lodash';
import type { ToolingLog } from '@kbn/tooling-log';
import type { SavedObjectsBulkDeleteResponse } from '@kbn/core-saved-objects-api-server';
import type {
SavedObjectsBulkDeleteResponse,
SavedObjectsFindResponse,
} from '@kbn/core-saved-objects-api-server';

import { KbnClientRequester, uriencode } from './kbn_client_requester';

Expand All @@ -30,6 +33,11 @@ interface SavedObjectResponse<Attributes extends Record<string, any>> {
version?: string;
}

interface GetFindOptions {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
interface GetFindOptions {
interface FindOptions {

type: string;
space?: string;
}

interface GetOptions {
type: string;
id: string;
Expand Down Expand Up @@ -152,6 +160,22 @@ export class KbnClientSavedObjects {
return data;
}

/**
* Find saved objects
*/
public async find<Attributes extends Record<string, any>>(options: GetFindOptions) {
this.log.debug('Find saved objects: %j', options);

const { data } = await this.requester.request<SavedObjectsFindResponse<Attributes>>({
description: 'find saved objects',
path: options.space
? uriencode`/s/${options.space}/internal/ftr/kbn_client_so/_find?type=${options.type}`
: uriencode`/internal/ftr/kbn_client_so/_find?type=${options.type}`,
method: 'GET',
});
return data;
}

/**
* Create a saved object
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ describe('checking migration metadata changes on all registered SO types', () =>
"osquery-pack-asset": "b14101d3172c4b60eb5404696881ce5275c84152",
"osquery-saved-query": "44f1161e165defe3f9b6ad643c68c542a765fcdb",
"query": "8db5d48c62d75681d80d82a42b5642f60d068202",
"risk-engine-configuration": "4dbbc5fd0d1bacc4d76e9c63dfb7887fb6d57079",
"rules-settings": "892a2918ebaeba809a612b8d97cec0b07c800b5f",
"sample-data-telemetry": "37441b12f5b0159c2d6d5138a494c9f440e950b5",
"search": "8d5184dd5b986d57250b6ffd9ae48a1925e4c7a3",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ const previouslyRegisteredTypes = [
'search-telemetry',
'security-rule',
'security-solution-signals-migration',
'risk-engine-configuration',
'server',
'siem-detection-engine-rule-actions',
'siem-detection-engine-rule-execution-info',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ describe('split .kibana index into multiple system indices', () => {
"osquery-pack-asset",
"osquery-saved-query",
"query",
"risk-engine-configuration",
"rules-settings",
"sample-data-telemetry",
"search-session",
Expand Down
6 changes: 6 additions & 0 deletions x-pack/plugins/security_solution/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,12 @@ export const RISK_SCORE_CREATE_STORED_SCRIPT = `${INTERNAL_RISK_SCORE_URL}/store
export const RISK_SCORE_DELETE_STORED_SCRIPT = `${INTERNAL_RISK_SCORE_URL}/stored_scripts/delete`;
export const RISK_SCORE_PREVIEW_URL = `${INTERNAL_RISK_SCORE_URL}/preview`;

export const RISK_ENGINE_URL = `${INTERNAL_RISK_SCORE_URL}/engine`;
export const RISK_ENGINE_STATUS_URL = `${RISK_ENGINE_URL}/status`;
export const RISK_ENGINE_INIT_URL = `${RISK_ENGINE_URL}/init`;
export const RISK_ENGINE_ENABLE_URL = `${RISK_ENGINE_URL}/enable`;
export const RISK_ENGINE_DISABLE_URL = `${RISK_ENGINE_URL}/disable`;

/**
* Public Risk Score routes
*/
Expand Down
14 changes: 14 additions & 0 deletions x-pack/plugins/security_solution/common/risk_engine/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,17 @@ export enum RiskScoreEntity {
host = 'host',
user = 'user',
}

export enum RiskEngineStatus {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I tend to pluralize enums to distinguish them from other types:

Suggested change
export enum RiskEngineStatus {
export enum RiskEngineStatuses {

NOT_INSTALLED = 'NOT_INSTALLED',
DISABLED = 'DISABLED',
ENABLED = 'ENABLED',
}

export interface InitRiskEngineResult {
legacyRiskEngineDisabled: boolean;
riskEngineResourcesInstalled: boolean;
riskEngineConfigurationCreated: boolean;
riskEngineEnabled: boolean;
errors: string[];
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,71 +24,75 @@ import { createRule } from '../../tasks/api_calls/rules';
import { updateDateRangeInLocalDatePickers } from '../../tasks/date_picker';
import { fillLocalSearchBar, submitLocalSearch } from '../../tasks/search_bar';

describe('Entity analytics management page', () => {
before(() => {
cleanKibana();
cy.task('esArchiverLoad', 'all_users');
});

beforeEach(() => {
login();
visitWithoutDateRange(ALERTS_URL);
createRule(getNewRule({ query: 'user.name:* or host.name:*', risk_score: 70 }));
visit(ENTITY_ANALYTICS_MANAGEMENT_URL);
});

after(() => {
cy.task('esArchiverUnload', 'all_users');
});

it('renders page as expected', () => {
cy.get(PAGE_TITLE).should('have.text', 'Entity Risk Score');
});

describe('Risk preview', () => {
it('risk scores reacts on change in datepicker', () => {
const START_DATE = 'Jan 18, 2019 @ 20:33:29.186';
const END_DATE = 'Jan 19, 2019 @ 20:33:29.186';

cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);

updateDateRangeInLocalDatePickers(LOCAL_QUERY_BAR_SELECTOR, START_DATE, END_DATE);

cy.get(HOST_RISK_PREVIEW_TABLE).contains('No items found');
cy.get(USER_RISK_PREVIEW_TABLE).contains('No items found');
describe(
'Entity analytics management page',
{ env: { ftrConfig: { enableExperimental: ['riskScoringRoutesEnabled'] } } },
() => {
before(() => {
cleanKibana();
cy.task('esArchiverLoad', 'all_users');
});

it('risk scores reacts on change in search bar query', () => {
cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);
beforeEach(() => {
login();
visitWithoutDateRange(ALERTS_URL);
createRule(getNewRule({ query: 'user.name:* or host.name:*', risk_score: 70 }));
visit(ENTITY_ANALYTICS_MANAGEMENT_URL);
});

fillLocalSearchBar('host.name: "test-host1"');
submitLocalSearch(LOCAL_QUERY_BAR_SELECTOR);
after(() => {
cy.task('esArchiverUnload', 'all_users');
});

cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 1);
cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).contains('test-host1');
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 1);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).contains('test1');
it('renders page as expected', () => {
cy.get(PAGE_TITLE).should('have.text', 'Entity Risk Score');
});

it('show error panel if API returns error and then try to refetch data', () => {
cy.intercept('POST', '/internal/risk_score/preview', {
statusCode: 500,
describe('Risk preview', () => {
it('risk scores reacts on change in datepicker', () => {
const START_DATE = 'Jan 18, 2019 @ 20:33:29.186';
const END_DATE = 'Jan 19, 2019 @ 20:33:29.186';

cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);

updateDateRangeInLocalDatePickers(LOCAL_QUERY_BAR_SELECTOR, START_DATE, END_DATE);

cy.get(HOST_RISK_PREVIEW_TABLE).contains('No items found');
cy.get(USER_RISK_PREVIEW_TABLE).contains('No items found');
});

cy.get(RISK_PREVIEW_ERROR).contains('Preview failed');
it('risk scores reacts on change in search bar query', () => {
cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 5);

cy.intercept('POST', '/internal/risk_score/preview', {
statusCode: 200,
body: {
scores: { host: [], user: [] },
},
fillLocalSearchBar('host.name: "test-host1"');
submitLocalSearch(LOCAL_QUERY_BAR_SELECTOR);

cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).should('have.length', 1);
cy.get(HOST_RISK_PREVIEW_TABLE_ROWS).contains('test-host1');
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).should('have.length', 1);
cy.get(USER_RISK_PREVIEW_TABLE_ROWS).contains('test1');
});

cy.get(RISK_PREVIEW_ERROR_BUTTON).click();
it('show error panel if API returns error and then try to refetch data', () => {
cy.intercept('POST', '/internal/risk_score/preview', {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Wrapt these intercepts into a task method making clear what you are doing or just add a simple comment about the stubbing you are doing :)

statusCode: 500,
});

cy.get(RISK_PREVIEW_ERROR).contains('Preview failed');

cy.intercept('POST', '/internal/risk_score/preview', {
statusCode: 200,
body: {
scores: { host: [], user: [] },
},
});

cy.get(RISK_PREVIEW_ERROR).should('not.exist');
cy.get(RISK_PREVIEW_ERROR_BUTTON).click();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Wrap this into a task method.


cy.get(RISK_PREVIEW_ERROR).should('not.exist');
});
});
});
});
}
);
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,22 @@
* 2.0.
*/

import { RISK_SCORE_PREVIEW_URL } from '../../../common/constants';
import {
RISK_ENGINE_STATUS_URL,
RISK_SCORE_PREVIEW_URL,
RISK_ENGINE_ENABLE_URL,
RISK_ENGINE_DISABLE_URL,
RISK_ENGINE_INIT_URL,
} from '../../../common/constants';

import { KibanaServices } from '../../common/lib/kibana';
import type { CalculateScoresResponse } from '../../../server/lib/risk_engine/types';
import type {
CalculateScoresResponse,
GetRiskEngineEnableResponse,
GetRiskEngineStatusResponse,
InitRiskEngineResponse,
GetRiskEngineDisableResponse,
} from '../../../server/lib/risk_engine/types';
import type { RiskScorePreviewRequestSchema } from '../../../common/risk_engine/risk_score_preview/request_schema';

/**
Expand All @@ -27,3 +39,44 @@ export const fetchRiskScorePreview = async ({
signal,
});
};

/**
* Fetches risks engine status
*/
export const fetchRiskEngineStatus = async ({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this with your previous PR, but: can you explain the logic behind the folder naming here? Explore team originally introduced risk_score folders, and I tried to generalize that to risk_engine stuff, but now it seems like we're broadening that to entity_analytics, here? Not a judgement, I'm just trying to understand the intention and keep things consistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My original idea was to introduce the new public folder, for all things related to EA (watchlist, etc)

If you think we should rename/move things please let me know

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds reasonable; thanks for the explanation. Perhaps as we move the client code to pull from the new risk score data, we can move those files/folders under entity_analytics ?

signal,
}: {
signal?: AbortSignal;
}): Promise<GetRiskEngineStatusResponse> => {
return KibanaServices.get().http.fetch<GetRiskEngineStatusResponse>(RISK_ENGINE_STATUS_URL, {
method: 'GET',
signal,
});
};

/**
* Init risk score engine
*/
export const initRiskEngine = async (): Promise<InitRiskEngineResponse> => {
return KibanaServices.get().http.fetch<InitRiskEngineResponse>(RISK_ENGINE_INIT_URL, {
method: 'POST',
});
};

/**
* Enable risk score engine
*/
export const enableRiskEngine = async (): Promise<GetRiskEngineEnableResponse> => {
return KibanaServices.get().http.fetch<GetRiskEngineEnableResponse>(RISK_ENGINE_ENABLE_URL, {
method: 'POST',
});
};

/**
* Disable risk score engine
*/
export const disableRiskEngine = async (): Promise<GetRiskEngineDisableResponse> => {
return KibanaServices.get().http.fetch<GetRiskEngineDisableResponse>(RISK_ENGINE_DISABLE_URL, {
method: 'POST',
});
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { UseMutationOptions } from '@tanstack/react-query';
import { useMutation } from '@tanstack/react-query';
import { disableRiskEngine } from '../api';
import { useInvalidateRiskEngineStatusQuery } from './use_risk_engine_status';
import type {
GetRiskEngineDisableResponse,
EnableDisableRiskEngineResponse,
} from '../../../../server/lib/risk_engine/types';

export const DISABLE_RISK_ENGINE_MUTATION_KEY = ['POST', 'DISABLE_RISK_ENGINE'];
rylnd marked this conversation as resolved.
Show resolved Hide resolved

export const useDisableRiskEngineMutation = (options?: UseMutationOptions<{}>) => {
const invalidateRiskEngineStatusQuery = useInvalidateRiskEngineStatusQuery();

return useMutation<GetRiskEngineDisableResponse, EnableDisableRiskEngineResponse>(
() => disableRiskEngine(),
{
...options,
mutationKey: DISABLE_RISK_ENGINE_MUTATION_KEY,
onSettled: (...args) => {
invalidateRiskEngineStatusQuery();

if (options?.onSettled) {
options.onSettled(...args);
}
},
}
);
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { UseMutationOptions } from '@tanstack/react-query';
import { useMutation } from '@tanstack/react-query';
import { enableRiskEngine } from '../api';
import { useInvalidateRiskEngineStatusQuery } from './use_risk_engine_status';
import type {
GetRiskEngineEnableResponse,
EnableDisableRiskEngineResponse,
} from '../../../../server/lib/risk_engine/types';
export const ENABLE_RISK_ENGINE_MUTATION_KEY = ['POST', 'ENABLE_RISK_ENGINE'];

export const useEnableRiskEngineMutation = (options?: UseMutationOptions<{}>) => {
const invalidateRiskEngineStatusQuery = useInvalidateRiskEngineStatusQuery();

return useMutation<GetRiskEngineEnableResponse, EnableDisableRiskEngineResponse>(
() => enableRiskEngine(),
{
...options,
mutationKey: ENABLE_RISK_ENGINE_MUTATION_KEY,
onSettled: (...args) => {
invalidateRiskEngineStatusQuery();

if (options?.onSettled) {
options.onSettled(...args);
}
},
}
);
};
Loading