Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Adds tests for coverage overview page #168058

Merged
merged 52 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
fffc5f5
adds initial tests
dplumlee Oct 5, 2023
c5063f4
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Oct 30, 2023
146e173
fixes cypress merge
dplumlee Oct 31, 2023
2c1b76f
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 1, 2023
053909a
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 2, 2023
0058dc6
adds remaining cypress tests
dplumlee Nov 6, 2023
d456e10
removes duplicated ftr tests
dplumlee Nov 7, 2023
7f102aa
removes unnecessary util
dplumlee Nov 7, 2023
d845cbe
adds error cases
dplumlee Nov 7, 2023
4ecf9a9
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 7, 2023
85008c5
moves ftr tests into new folder structure
dplumlee Nov 7, 2023
1bcf6c0
add config file
dplumlee Nov 7, 2023
4c58bcd
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 7, 2023
e9f8e15
adds serverless config file
dplumlee Nov 7, 2023
e585ede
fixes types
dplumlee Nov 7, 2023
eeb1d23
adds back rule test
dplumlee Nov 8, 2023
7539e30
adds scripts
dplumlee Nov 9, 2023
ab56cd7
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 13, 2023
a0f1cc4
refactors some utils
dplumlee Nov 15, 2023
7067309
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 15, 2023
51f4bbd
addresses cypress comments
dplumlee Nov 20, 2023
b64e55a
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 20, 2023
d8b2620
switches package.json for flaky test runner
dplumlee Nov 21, 2023
b48dc43
resets package.json
dplumlee Nov 21, 2023
72aca4d
resets package.json
dplumlee Nov 21, 2023
ad2d829
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 21, 2023
a0343bd
removes non-security test
dplumlee Nov 21, 2023
aea805b
fixes ftr tests
dplumlee Nov 21, 2023
f248349
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 29, 2023
e2dc4b3
fixes rest of merge conflicts
dplumlee Nov 29, 2023
e66c647
fix import path updates
dplumlee Nov 29, 2023
a240bc9
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 1, 2023
a37bbc3
fixes non-security rule test
dplumlee Dec 1, 2023
a06fb5e
fixes import bug
dplumlee Dec 1, 2023
2172170
add alerting api service to ess config
dplumlee Dec 1, 2023
2c79610
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 5, 2023
6d471f6
addresses comments
dplumlee Dec 6, 2023
089ee4b
adds back ess non security rule util
dplumlee Dec 6, 2023
f24d4f8
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 13, 2023
879594d
merge fixes
dplumlee Dec 13, 2023
00b5f5f
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 13, 2023
e57b430
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 18, 2023
72d4166
changes script
dplumlee Dec 20, 2023
76b4469
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 3, 2024
2cfc275
modifies mitre mock data and generation script
dplumlee Jan 4, 2024
4b13726
fix unused code
dplumlee Jan 4, 2024
a8b6d95
splits ftr test tags
dplumlee Jan 4, 2024
e18f573
fixes script conflict
dplumlee Jan 4, 2024
19282d8
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 5, 2024
6cd40c3
addresses comments
dplumlee Jan 5, 2024
4442f7c
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 8, 2024
636311c
merge conflicts
dplumlee Jan 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .buildkite/ftr_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -460,10 +460,7 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_license/rule_management/configs/serverless.config.ts
dplumlee marked this conversation as resolved.
Show resolved Hide resolved
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_license/rule_management/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts/configs/ess.config.ts

dplumlee marked this conversation as resolved.
Show resolved Hide resolved




Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Status: `in progress`. The current test plan matches `Milestone 1 - MVP` of the

- **Rule Source**: The filter type defining rule type, current options are `prebuilt`(from elastic prebuilt rules package) and `custom`(created by user)

-**Initial filter state**: The filters present on initial page load. Rule activity will be set to `enabled`, rule source will be set to `prebuilt` and `custom` simultaneously.
- **Initial filter state**: The filters present on initial page load. Rule activity will be set to `enabled`, rule source will be set to `prebuilt` and `custom` simultaneously.

-**Dashboard containing the rule data**: The normal render of the coverage overview dashboard. Any returned rule data mapped correctly to the tile layout of all the MITRE data in a colored grid
- **Dashboard containing the rule data**: The normal render of the coverage overview dashboard. Any returned rule data mapped correctly to the tile layout of all the MITRE data in a colored grid

### Assumptions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@
import type { UseQueryOptions } from '@tanstack/react-query';
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { useCallback } from 'react';
import * as i18n from '../../../rule_management_ui/pages/coverage_overview/translations';
import type { CoverageOverviewFilter } from '../../../../../common/api/detection_engine';
import { RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL } from '../../../../../common/api/detection_engine';
import { fetchCoverageOverview } from '../api';
import { buildCoverageOverviewDashboardModel } from '../../logic/coverage_overview/build_coverage_overview_dashboard_model';
import type { CoverageOverviewDashboard } from '../../model/coverage_overview/dashboard';
import { DEFAULT_QUERY_OPTIONS } from './constants';
import { useAppToasts } from '../../../../common/hooks/use_app_toasts';

const COVERAGE_OVERVIEW_QUERY_KEY = ['POST', RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL];

Expand All @@ -29,6 +31,8 @@ export const useFetchCoverageOverviewQuery = (
filter: CoverageOverviewFilter = {},
options?: UseQueryOptions<CoverageOverviewDashboard>
) => {
const { addError } = useAppToasts();

return useQuery<CoverageOverviewDashboard>(
[...COVERAGE_OVERVIEW_QUERY_KEY, filter],
async ({ signal }) => {
Expand All @@ -39,6 +43,11 @@ export const useFetchCoverageOverviewQuery = (
{
...DEFAULT_QUERY_OPTIONS,
...options,
onError: (error) => {
addError(error, {
title: i18n.COVERAGE_OVERVIEW_FETCH_ERROR_TITLE,
});
},
}
);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ const RuleSourceFilterComponent = ({
`}
>
<EuiPopover
id="ruleActivityFilterPopover"
id="ruleSourceFilterPopover"
button={button}
isOpen={isPopoverOpen}
closePopover={closePopover}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,10 @@ export const CoverageOverviewDashboardInformation = i18n.translate(
"Your current coverage of MITRE ATT&CK\u00AE tactics and techniques, based on installed rules. Click a cell to view and enable a technique's rules. Rules must be mapped to the MITRE ATT&CK\u00AE framework to be displayed.",
}
);

export const COVERAGE_OVERVIEW_FETCH_ERROR_TITLE = i18n.translate(
'xpack.securitySolution.coverageOverviewDashboard.fetchErrorTitle',
{
defaultMessage: 'Failed to fetch coverage overview data',
}
);
dplumlee marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
deleteAllRules,
getSimpleRule,
installPrebuiltRulesAndTimelines,
installPrebuiltRules,
} from '../../utils';
import { createNonSecurityRule } from '../../utils/create_non_security_rule';

Expand Down Expand Up @@ -471,6 +472,119 @@ export default ({ getService }: FtrProviderContext): void => {
},
});
});

it('returns response filtered by prebuilt rules', async () => {
await createPrebuiltRuleAssetSavedObjects(es, [
createRuleAssetSavedObject({
rule_id: 'prebuilt-rule-1',
threat: generateThreatArray(1),
}),
]);
const {
results: { created },
} = await installPrebuiltRules(es, supertest);
const expectedRule = created[0];

await createRule(supertest, log, {
...getSimpleRule('rule-1'),
threat: generateThreatArray(2),
});

const { body } = await supertest
.post(RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '1')
.send({
filter: {
source: ['prebuilt'],
},
})
.expect(200);

expect(body).to.eql({
coverage: {
T001: [expectedRule.id],
TA001: [expectedRule.id],
'T001.001': [expectedRule.id],
},
unmapped_rule_ids: [],
rules_data: {
[expectedRule.id]: {
activity: 'disabled',
name: 'Query with a rule id',
},
},
});
});

it('returns all rules if both custom and prebuilt filters are specified in the request', async () => {
await createPrebuiltRuleAssetSavedObjects(es, [
createRuleAssetSavedObject({
rule_id: 'prebuilt-rule-1',
threat: generateThreatArray(1),
}),
]);
const {
results: { created },
} = await installPrebuiltRules(es, supertest);
const expectedPrebuiltRule = created[0];

const expectedCustomRule = await createRule(supertest, log, {
...getSimpleRule('rule-1'),
threat: generateThreatArray(2),
});

const { body } = await supertest
.post(RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '1')
.send({
filter: {
source: ['prebuilt', 'custom'],
},
})
.expect(200);

expect(body).to.eql({
coverage: {
T001: [expectedPrebuiltRule.id],
TA001: [expectedPrebuiltRule.id],
'T001.001': [expectedPrebuiltRule.id],
T002: [expectedCustomRule.id],
TA002: [expectedCustomRule.id],
'T002.002': [expectedCustomRule.id],
},
unmapped_rule_ids: [],
rules_data: {
[expectedPrebuiltRule.id]: {
activity: 'disabled',
name: 'Query with a rule id',
},
[expectedCustomRule.id]: {
activity: 'disabled',
name: 'Simple Rule Query',
},
},
});
});
});
});

describe('error cases', async () => {
it('throws error when request body is not valid', async () => {
const { body } = await supertest
.post(RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '1')
.send({ filter: { source: ['give me all the rules'] } })
.expect(400);

expect(body).to.eql({
statusCode: 400,
error: 'Bad Request',
message:
'[request body]: Invalid value "give me all the rules" supplied to "filter,source"',
});
});
});
});
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { createTestConfig } from './config.base';

export default createTestConfig({
license: 'basic',
ssl: true,
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { FtrConfigProviderContext } from '@kbn/test';

export default async function ({ readConfigFile }: FtrConfigProviderContext) {
const functionalConfig = await readConfigFile(
require.resolve('../../../../../config/ess/config.base.basic')
);

return {
...functionalConfig.getAll(),
testFiles: [require.resolve('..')],
junit: {
reportName: 'Detection Engine ESS/ Rule management API Integration Tests',
},
};
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { createTestConfig } from '../../../../../config/serverless/config.base';

export default createTestConfig({
testFiles: [require.resolve('..')],
junit: {
reportName: 'Detection Engine Serverless/ Rule management API Integration Tests',
},
});
Loading