Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Create Entity Store index #175025

Merged

Conversation

machadoum
Copy link
Member

@machadoum machadoum commented Jan 17, 2024

This PR is going to be merged to the entity-store-poc feature branch; it won't impact the main branch.

Summary

  • Create entity_store/init route that creates the Entity Store index.
  • Create FTR tests.

Out of scope

  • User fields are out of scope.
  • API privileges are out of scope.

How to test it?

  • Call API
 KIBANA_URL="http://localhost:5601"
 USER_PASS="{USER}:{PASSWORD}"

curl "$KIBANA_URL/internal/entity_store/init" \
  -H 'kbn-xsrf:bleh' \
  --user "$USER_PASS"\
  -X 'POST' \
  -H 'elastic-api-version: 1'
  • Open the console and check if the index .entities.entities-default exists

Run tests

serverless
yarn run initialize-server:ea:default entity_store serverless
yarn run run-tests:ea:default entity_store serverless serverlessEnv

ess
yarn run initialize-server:ea:default entity_store ess
yarn run run-tests:ea:default entity_store ess essEnv

Checklist

@machadoum machadoum self-assigned this Jan 22, 2024
@machadoum machadoum added Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Theme: entity_analytics Feature:Entity Analytics Security Solution Entity Analytics features Team:Entity Analytics Security Entity Analytics Team labels Jan 22, 2024
@machadoum machadoum marked this pull request as ready for review January 22, 2024 15:21
@machadoum machadoum requested review from a team as code owners January 22, 2024 15:21
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftr_configs.yml

@machadoum
Copy link
Member Author

@elasticmachine merge upstream

Copy link
Contributor

@oatkiller oatkiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good!

@oatkiller
Copy link
Contributor

I think you need a change to the request_context mock:

diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts
index cb0b48b8e3f..4f436bfcf2a 100644
--- a/x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts
+++ b/x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts
@@ -116,6 +116,7 @@ const createSecuritySolutionRequestContextMock = (
   const kibanaRequest = requestMock.create();
 
   return {
+    getEntityStoreDataClient: jest.fn(() => { throw new Error('Not implemented') }),
     core,
     getServerBasePath: jest.fn(() => ''),
     getEndpointAuthz: jest.fn(async () =>

Copy link
Contributor

@hop-dev hop-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work 🚀 A few more general questions

logger: this.options.logger,
options: {
index: getEntityStoreIndex(this.options.namespace),
mappings: mappingFromFieldMap(entityStoreFieldMap, 'strict'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a pattern used in other places, but I'm interested. What is the benefit of the field map instead of just storing as a pure elastic mapping?

I have looked at our used of mappingFromFieldMap and I couldn't see where the field map is used anywhere else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just keep it for consistency. Maybe @nkhristinin or @rylnd can answer this question.

@machadoum
Copy link
Member Author

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

kibana-ci commented Jan 23, 2024

💔 Build Failed

Failed CI Steps

Metrics [docs]

‼️ ERROR: no builds found for mergeBase sha [3b401a2]

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @machadoum

@machadoum machadoum merged commit 1a7f155 into elastic:security/feature/entity-store-poc Jan 23, 2024
36 of 37 checks passed
hop-dev pushed a commit to hop-dev/kibana that referenced this pull request Apr 22, 2024
**This PR is going to be merged to the
[entity-store-poc](https://github.com/elastic/kibana/tree/security/feature/entity-store-poc)
feature branch; it won't impact the main branch.**

## Summary

* Create `entity_store/init` route that creates the Entity Store index.
* Create FTR tests.
### Out of scope
  * User fields are out of scope.
  * API privileges are out of scope.

### How to test it?
* Call API
```
 KIBANA_URL="http://localhost:5601"
 USER_PASS="{USER}:{PASSWORD}"

curl "$KIBANA_URL/internal/entity_store/init" \
  -H 'kbn-xsrf:bleh' \
  --user "$USER_PASS"\
  -X 'POST' \
  -H 'elastic-api-version: 1'
```
* Open the console and check if the index `.entities.entities-default`
exists

#### Run tests
**serverless**
`yarn run initialize-server:ea:default entity_store serverless`
`yarn run run-tests:ea:default entity_store serverless serverlessEnv`

**ess**
`yarn run initialize-server:ea:default entity_store ess`
`yarn run run-tests:ea:default entity_store ess essEnv`

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Entity Analytics Security Solution Entity Analytics features Team:Entity Analytics Security Entity Analytics Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Theme: entity_analytics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants