Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Entity Analytics] New API endpoint to cleanup the risk engine installation and data #191843

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
bb7ad1f
Adding initial changes for new API endpoint to cleanup the risk engin…
abhishekbhatia1710 Aug 30, 2024
efc4edd
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 2, 2024
2bb6c02
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 3, 2024
5b00e14
Adding test cases for the changes
abhishekbhatia1710 Sep 3, 2024
05d1c84
Merge branch 'ea-66-api-delete-risk-engine-installation-data' of http…
abhishekbhatia1710 Sep 3, 2024
cabaa19
Adding a test case to check for multiple errors while tearing down th…
abhishekbhatia1710 Sep 4, 2024
759e067
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 4, 2024
685a2c9
Shortening the variable name for the cleanup URL
abhishekbhatia1710 Sep 4, 2024
0433147
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 5, 2024
a8b4d04
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 5, 2024
ba553f1
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 5, 2024
cb25970
Adding openAPI spec for the new API endpoint and changes to error and…
abhishekbhatia1710 Sep 5, 2024
5cc0dff
Using the correct variable name
abhishekbhatia1710 Sep 5, 2024
2bf4316
Adding the missing export
abhishekbhatia1710 Sep 6, 2024
eeab638
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 6, 2024
8fcf60e
Adding a test case to return 400 if the API is called multiple times
abhishekbhatia1710 Sep 12, 2024
9a7437a
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 12, 2024
e6a3f20
Adding this file as a re-commit with single quotes
abhishekbhatia1710 Sep 12, 2024
98e4749
Merge branch 'ea-66-api-delete-risk-engine-installation-data' of http…
abhishekbhatia1710 Sep 12, 2024
45de858
Reverting all double quotes to single quotes
abhishekbhatia1710 Sep 12, 2024
785a611
Adding a file generated by openapi:generate
abhishekbhatia1710 Sep 12, 2024
2655f7d
Modifying old unit tests to support changes to the new tearDown function
abhishekbhatia1710 Sep 16, 2024
8867fc6
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 16, 2024
302f887
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 16, 2024
2eb2bc1
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 17, 2024
c588877
Modified test case to access the private method for mocking
abhishekbhatia1710 Sep 17, 2024
da90fd2
Renaming 'risk_engine_cleanup' to 'cleanup_successful'
abhishekbhatia1710 Sep 17, 2024
92e19ce
Removing the condition to check for enabled state of risk engine befo…
abhishekbhatia1710 Sep 18, 2024
fe0f3f5
Addressing review comments : Move to Public URL space and lowercase
abhishekbhatia1710 Sep 19, 2024
30fc92c
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 19, 2024
1ed2078
Reverting the risk_engine_data_client.ts to original version
abhishekbhatia1710 Sep 19, 2024
303ca7d
Adding files after running openapi-:bundle
abhishekbhatia1710 Sep 19, 2024
30742af
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 19, 2024
d263a5e
Adding E2E integration test for the new endpoint
abhishekbhatia1710 Sep 22, 2024
ed67951
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 22, 2024
d1d76fb
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 23, 2024
578dedb
Adding openapi:bundle changes (although unrelated to this PRs changes)
abhishekbhatia1710 Sep 23, 2024
f49de67
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 23, 2024
6f422ab
Openapi:bundle changes
abhishekbhatia1710 Sep 23, 2024
b9e5d7e
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 24, 2024
8cb4de7
Changes after 'make api-docs && make api-docs-staging'
abhishekbhatia1710 Sep 24, 2024
3a7473b
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 24, 2024
a449040
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 24, 2024
46e81b3
Changes to integration test since the key is_max_amount_of_risk_engin…
abhishekbhatia1710 Sep 24, 2024
f9e7ee6
Merge branch 'main' of https://github.com/abhishekbhatia1710/kibana i…
abhishekbhatia1710 Sep 25, 2024
b647467
Adding changes for openapi:bundle
abhishekbhatia1710 Sep 25, 2024
7c10552
Merge branch 'main' into ea-66-api-delete-risk-engine-installation-data
abhishekbhatia1710 Sep 25, 2024
498d5bb
Changes after make api-docs and make api-docs-staging
abhishekbhatia1710 Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions oas_docs/output/kibana.serverless.staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15392,6 +15392,39 @@ paths:
tags:
- Security Timeline API
- access:securitySolution
/api/risk_score/engine/dangerously_delete_data:
delete:
description: >-
Cleaning up the the Risk Engine by removing the indices, mapping and
transforms
operationId: CleanUpRiskEngine
responses:
'200':
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
type: object
properties:
cleanup_successful:
type: boolean
description: Successful response
'400':
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
$ref: >-
#/components/schemas/Security_Entity_Analytics_API_TaskManagerUnavailableResponse
description: Task manager is unavailable
default:
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
$ref: >-
#/components/schemas/Security_Entity_Analytics_API_CleanUpRiskEngineErrorResponse
description: Unexpected error
summary: Cleanup the Risk Engine
tags:
- Security Entity Analytics API
/api/risk_score/engine/schedule_now:
post:
description: >-
Expand Down Expand Up @@ -29750,6 +29783,27 @@ components:
required:
- id_value
- id_field
Security_Entity_Analytics_API_CleanUpRiskEngineErrorResponse:
type: object
properties:
cleanup_successful:
example: false
type: boolean
errors:
items:
type: object
properties:
error:
type: string
seq:
type: integer
required:
- seq
- error
type: array
required:
- cleanup_successful
- errors
Security_Entity_Analytics_API_CreateAssetCriticalityRecord:
allOf:
- $ref: >-
Expand Down
54 changes: 54 additions & 0 deletions oas_docs/output/kibana.staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18822,6 +18822,39 @@ paths:
tags:
- Security Timeline API
- access:securitySolution
/api/risk_score/engine/dangerously_delete_data:
delete:
description: >-
Cleaning up the the Risk Engine by removing the indices, mapping and
transforms
operationId: CleanUpRiskEngine
responses:
'200':
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
type: object
properties:
cleanup_successful:
type: boolean
description: Successful response
'400':
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
$ref: >-
#/components/schemas/Security_Entity_Analytics_API_TaskManagerUnavailableResponse
description: Task manager is unavailable
default:
content:
application/json; Elastic-Api-Version=2023-10-31:
schema:
$ref: >-
#/components/schemas/Security_Entity_Analytics_API_CleanUpRiskEngineErrorResponse
description: Unexpected error
summary: Cleanup the Risk Engine
tags:
- Security Entity Analytics API
/api/risk_score/engine/schedule_now:
post:
description: >-
Expand Down Expand Up @@ -37759,6 +37792,27 @@ components:
required:
- id_value
- id_field
Security_Entity_Analytics_API_CleanUpRiskEngineErrorResponse:
type: object
properties:
cleanup_successful:
example: false
type: boolean
errors:
items:
type: object
properties:
error:
type: string
seq:
type: integer
required:
- seq
- error
type: array
required:
- cleanup_successful
- errors
Security_Entity_Analytics_API_CreateAssetCriticalityRecord:
allOf:
- $ref: >-
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

/*
* NOTICE: Do not edit this file manually.
* This file is automatically generated by the OpenAPI Generator, @kbn/openapi-generator.
*
* info:
* title: Risk Scoring API
* version: 1
*/

import { z } from '@kbn/zod';

export type CleanUpRiskEngineErrorResponse = z.infer<typeof CleanUpRiskEngineErrorResponse>;
export const CleanUpRiskEngineErrorResponse = z.object({
cleanup_successful: z.boolean(),
errors: z.array(
z.object({
seq: z.number().int(),
error: z.string(),
})
),
});

export type CleanUpRiskEngineResponse = z.infer<typeof CleanUpRiskEngineResponse>;
export const CleanUpRiskEngineResponse = z.object({
cleanup_successful: z.boolean().optional(),
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
openapi: 3.0.0
info:
version: '1'
title: Risk Scoring API
description: These APIs allow the consumer to manage Entity Risk Scores within Entity Analytics.
paths:
/api/risk_score/engine/dangerously_delete_data:
delete:
x-labels: [ess, serverless]
x-codegen-enabled: true
operationId: CleanUpRiskEngine
summary: Cleanup the Risk Engine
description: Cleaning up the the Risk Engine by removing the indices, mapping and transforms
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
cleanup_successful:
type: boolean
'400':
description: Task manager is unavailable
content:
application/json:
schema:
$ref: '../common/common.schema.yaml#/components/schemas/TaskManagerUnavailableResponse'
default:
description: Unexpected error
content:
application/json:
schema:
$ref: '#/components/schemas/CleanUpRiskEngineErrorResponse'

components:
schemas:
CleanUpRiskEngineErrorResponse:
type: object
required:
- cleanup_successful
- errors
properties:
cleanup_successful:
type: boolean
example: false
errors:
type: array
items:
type: object
required:
- seq
- error
properties:
seq:
type: integer
error:
type: string

Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ export * from './calculation_route.gen';
export * from './preview_route.gen';
export * from './entity_calculation_route.gen';
export * from './get_risk_engine_privileges.gen';
export * from './engine_cleanup_route.gen';
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,7 @@ import type {
ListEntitiesRequestQueryInput,
ListEntitiesResponse,
} from './entity_analytics/entity_store/entities/list_entities.gen';
import type { CleanUpRiskEngineResponse } from './entity_analytics/risk_engine/engine_cleanup_route.gen';
import type { DisableRiskEngineResponse } from './entity_analytics/risk_engine/engine_disable_route.gen';
import type { EnableRiskEngineResponse } from './entity_analytics/risk_engine/engine_enable_route.gen';
import type { InitRiskEngineResponse } from './entity_analytics/risk_engine/engine_init_route.gen';
Expand Down Expand Up @@ -540,6 +541,21 @@ If asset criticality records already exist for the specified entities, those rec
})
.catch(catchAxiosErrorFormatAndThrow);
}
/**
* Cleaning up the the Risk Engine by removing the indices, mapping and transforms
*/
async cleanUpRiskEngine() {
this.log.info(`${new Date().toISOString()} Calling API CleanUpRiskEngine`);
return this.kbnClient
.request<CleanUpRiskEngineResponse>({
path: '/api/risk_score/engine/dangerously_delete_data',
headers: {
[ELASTIC_HTTP_VERSION_HEADER]: '1',
},
method: 'DELETE',
})
.catch(catchAxiosErrorFormatAndThrow);
}
async createAlertsIndex() {
this.log.info(`${new Date().toISOString()} Calling API CreateAlertsIndex`);
return this.kbnClient
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export const RISK_ENGINE_SETTINGS_URL = `${RISK_ENGINE_URL}/settings` as const;
// Public Risk Score routes
export const PUBLIC_RISK_ENGINE_URL = `${PUBLIC_RISK_SCORE_URL}/engine` as const;
export const RISK_ENGINE_SCHEDULE_NOW_URL = `${RISK_ENGINE_URL}/schedule_now` as const;
export const RISK_ENGINE_CLEANUP_URL = `${PUBLIC_RISK_ENGINE_URL}/dangerously_delete_data` as const;

type ClusterPrivilege = 'manage_index_templates' | 'manage_transform';
export const RISK_ENGINE_REQUIRED_ES_CLUSTER_PRIVILEGES = [
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
openapi: 3.0.3
info:
description: ''
title: Security Entity Analytics API (Elastic Cloud and self-hosted)
version: '1'
servers:
- url: http://{kibana_host}:{port}
variables:
kibana_host:
default: localhost
port:
default: '5601'
paths:
/api/risk_score/engine/dangerously_delete_data:
delete:
description: >-
Cleaning up the the Risk Engine by removing the indices, mapping and
transforms
operationId: CleanUpRiskEngine
responses:
'200':
content:
application/json:
schema:
type: object
properties:
cleanup_successful:
type: boolean
description: Successful response
'400':
content:
application/json:
schema:
$ref: '#/components/schemas/TaskManagerUnavailableResponse'
description: Task manager is unavailable
default:
content:
application/json:
schema:
$ref: '#/components/schemas/CleanUpRiskEngineErrorResponse'
description: Unexpected error
summary: Cleanup the Risk Engine
tags:
- Security Entity Analytics API
components:
schemas:
CleanUpRiskEngineErrorResponse:
type: object
properties:
cleanup_successful:
example: false
type: boolean
errors:
items:
type: object
properties:
error:
type: string
seq:
type: integer
required:
- seq
- error
type: array
required:
- cleanup_successful
- errors
TaskManagerUnavailableResponse:
description: Task manager is unavailable
type: object
properties:
message:
type: string
status_code:
minimum: 400
type: integer
required:
- status_code
- message
securitySchemes:
BasicAuth:
scheme: basic
type: http
security:
- BasicAuth: []
tags:
- description: ''
name: Security Entity Analytics API
Loading