Skip to content

Commit

Permalink
[SecuritySolutions] Create Entity Store 'entities/list' API (elastic#…
Browse files Browse the repository at this point in the history
…192806)

This PR introduces the following API routes for listing Entity Store
"entities":

<meta charset="utf-8"><b style="font-weight:normal;"
id="docs-internal-guid-9410c5d7-7fff-e873-6830-887939a306fb"><div
dir="ltr" style="margin-left:-0.75pt;" align="left">
List Entities | GET /api/entity_store/entities/list
-- | --
</div></b>

The PR includes the following:
 - The OpenAPI schemas for the route
 - The actual Kibana side endpoint
 - Add searchEntities function to the `EntityStoreDataClient`
 

### How to test

1. Add some host/user data
* Easiest is to use
[elastic/security-data-generator](https://github.com/elastic/security-documents-generator)
2. Make sure to add `entityStoreEnabled` under
`xpack.securitySolution.enableExperimental` in your `kibana.dev.yml`
3. In kibana dev tools or your terminal, call the `INIT` route for
either `user` or `host`.
4. You should now see 2 transforms in kibana. Make sure to re-trigger
them if needed so they process the documents.
5. Call the new API, and it should return entities 



Implements elastic/security-team#10517

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
machadoum and kibanamachine authored Sep 19, 2024
1 parent 0781e51 commit 27f5da4
Show file tree
Hide file tree
Showing 29 changed files with 1,864 additions and 53 deletions.
1 change: 1 addition & 0 deletions .buildkite/ftr_security_serverless_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/risk_engine/trial_license_complete_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/risk_engine/basic_license_essentials_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/entity_store/trial_license_complete_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/exception_lists_items/trial_license_complete_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/lists_items/trial_license_complete_tier/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/explore/hosts/trial_license_complete_tier/configs/serverless.config.ts
Expand Down
1 change: 1 addition & 0 deletions .buildkite/ftr_security_stateful_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/detections_response/user_roles/trial_license_complete_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/risk_engine/trial_license_complete_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/risk_engine/basic_license_essentials_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/entity_analytics/entity_store/trial_license_complete_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/exception_lists_items/trial_license_complete_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/lists_items/trial_license_complete_tier/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/explore/hosts/trial_license_complete_tier/configs/ess.config.ts
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,9 @@ export const EngineDescriptor = z.object({
status: EngineStatus.optional(),
filter: z.string().optional(),
});

export type InspectQuery = z.infer<typeof InspectQuery>;
export const InspectQuery = z.object({
response: z.array(z.string()),
dsl: z.array(z.string()),
});
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ info:
paths: {}
components:
schemas:

EntityType:
type: string
enum:
Expand All @@ -31,7 +30,21 @@ components:
- installing
- started
- stopped

IndexPattern:
type: string


InspectQuery:
type: object
properties:
response:
type: array
items:
type: string
dsl:
type: array
items:
type: string
required:
- dsl
- response
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

/*
* NOTICE: Do not edit this file manually.
* This file is automatically generated by the OpenAPI Generator, @kbn/openapi-generator.
*
* info:
* title: Common Entities Schemas
* version: 1
*/

import { z } from '@kbn/zod';

export type UserEntity = z.infer<typeof UserEntity>;
export const UserEntity = z.object({
user: z
.object({
full_name: z.array(z.string()).optional(),
domain: z.array(z.string()).optional(),
roles: z.array(z.string()).optional(),
name: z.string(),
id: z.array(z.string()).optional(),
email: z.array(z.string()).optional(),
hash: z.array(z.string()).optional(),
})
.optional(),
entity: z
.object({
lastSeenTimestamp: z.string().datetime(),
schemaVersion: z.string(),
definitionVersion: z.string(),
displayName: z.string(),
identityFields: z.array(z.string()),
id: z.string(),
type: z.literal('node'),
firstSeenTimestamp: z.string().datetime(),
definitionId: z.string(),
})
.optional(),
});

export type HostEntity = z.infer<typeof HostEntity>;
export const HostEntity = z.object({
host: z
.object({
hostname: z.array(z.string()).optional(),
domain: z.array(z.string()).optional(),
ip: z.array(z.string()).optional(),
name: z.string(),
id: z.array(z.string()).optional(),
type: z.array(z.string()).optional(),
mac: z.array(z.string()).optional(),
architecture: z.array(z.string()).optional(),
})
.optional(),
entity: z
.object({
lastSeenTimestamp: z.string().datetime(),
schemaVersion: z.string(),
definitionVersion: z.string(),
displayName: z.string(),
identityFields: z.array(z.string()),
id: z.string(),
type: z.literal('node'),
firstSeenTimestamp: z.string().datetime(),
definitionId: z.string(),
})
.optional(),
});

export type Entity = z.infer<typeof Entity>;
export const Entity = z.union([UserEntity, HostEntity]);
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
openapi: 3.0.0
info:
title: Common Entities Schemas
description: Common Entities schemas for the Entity Store
version: '1'
paths: {}
components:
schemas:
UserEntity:
type: object
properties:
user:
type: object
properties:
full_name:
type: array
items:
type: string
domain:
type: array
items:
type: string
roles:
type: array
items:
type: string
name:
type: string
id:
type: array
items:
type: string
email:
type: array
items:
type: string
hash:
type: array
items:
type: string
required:
- name
entity:
type: object
properties:
lastSeenTimestamp:
type: string
format: date-time
schemaVersion:
type: string
definitionVersion:
type: string
displayName:
type: string
identityFields:
type: array
items:
type: string
id:
type: string
type:
type: string
enum:
- node
firstSeenTimestamp:
type: string
format: date-time
definitionId:
type: string
required:
- lastSeenTimestamp
- schemaVersion
- definitionVersion
- displayName
- identityFields
- id
- type
- firstSeenTimestamp
- definitionId
HostEntity:
type: object
properties:
host:
type: object
properties:
hostname:
type: array
items:
type: string
domain:
type: array
items:
type: string
ip:
type: array
items:
type: string
name:
type: string
id:
type: array
items:
type: string
type:
type: array
items:
type: string
mac:
type: array
items:
type: string
architecture:
type: array
items:
type: string
required:
- name
entity:
type: object
properties:
lastSeenTimestamp:
type: string
format: date-time
schemaVersion:
type: string
definitionVersion:
type: string
displayName:
type: string
identityFields:
type: array
items:
type: string
id:
type: string
type:
type: string
enum:
- node
firstSeenTimestamp:
type: string
format: date-time
definitionId:
type: string
required:
- lastSeenTimestamp
- schemaVersion
- definitionVersion
- displayName
- identityFields
- id
- type
- firstSeenTimestamp
- definitionId

Entity:
oneOf:
- $ref: '#/components/schemas/UserEntity'
- $ref: '#/components/schemas/HostEntity'
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

/*
* NOTICE: Do not edit this file manually.
* This file is automatically generated by the OpenAPI Generator, @kbn/openapi-generator.
*
* info:
* title: Entities List Schema
* version: 2023-10-31
*/

import { z } from '@kbn/zod';
import { ArrayFromString } from '@kbn/zod-helpers';

import { EntityType, InspectQuery } from '../common.gen';
import { Entity } from './common.gen';

export type ListEntitiesRequestQuery = z.infer<typeof ListEntitiesRequestQuery>;
export const ListEntitiesRequestQuery = z.object({
sort_field: z.string().optional(),
sort_order: z.enum(['asc', 'desc']).optional(),
page: z.coerce.number().int().min(1).optional(),
per_page: z.coerce.number().int().min(1).max(10000).optional(),
/**
* An ES query to filter by.
*/
filterQuery: z.string().optional(),
entities_types: ArrayFromString(EntityType),
});
export type ListEntitiesRequestQueryInput = z.input<typeof ListEntitiesRequestQuery>;

export type ListEntitiesResponse = z.infer<typeof ListEntitiesResponse>;
export const ListEntitiesResponse = z.object({
records: z.array(Entity),
page: z.number().int().min(1),
per_page: z.number().int().min(1).max(1000),
total: z.number().int().min(0),
inspect: InspectQuery.optional(),
});
Loading

0 comments on commit 27f5da4

Please sign in to comment.