Skip to content

Commit

Permalink
[Security Solution] OpenAPI docs bundler (elastic#171526)
Browse files Browse the repository at this point in the history
**Addresses:** elastic/security-team#7981

## Summary

This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.

## Details

Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.

The bundler addresses the following issues

- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)

and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.

### Related changes

- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.

## How to test?

There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.

To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below

```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```

It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.

## Open issues

- [x] Circular references (implemented in
elastic@dfdf0a5)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](elastic@0ae7ad5)
allows to mix OpenAPI `3.0` and `3.1` specs.

## Improvements

- [ ] Flexible configuration
- [ ] CLI support?

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
maximpn and kibanamachine authored Nov 29, 2023
1 parent 27f9a00 commit e3d95e9
Show file tree
Hide file tree
Showing 109 changed files with 3,590 additions and 23 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,7 @@ x-pack/plugins/observability @elastic/obs-ux-management-team
x-pack/plugins/observability_shared @elastic/observability-ui
x-pack/test/security_api_integration/plugins/oidc_provider @elastic/kibana-security
test/common/plugins/otel_metrics @elastic/obs-ux-infra_services-team
packages/kbn-openapi-bundler @elastic/security-detection-rule-management
packages/kbn-openapi-generator @elastic/security-detection-rule-management
packages/kbn-optimizer @elastic/kibana-operations
packages/kbn-optimizer-webpack-helpers @elastic/kibana-operations
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -1244,6 +1244,7 @@
"@kbn/managed-vscode-config-cli": "link:packages/kbn-managed-vscode-config-cli",
"@kbn/management-storybook-config": "link:packages/kbn-management/storybook/config",
"@kbn/mock-idp-plugin": "link:packages/kbn-mock-idp-plugin",
"@kbn/openapi-bundler": "link:packages/kbn-openapi-bundler",
"@kbn/openapi-generator": "link:packages/kbn-openapi-generator",
"@kbn/optimizer": "link:packages/kbn-optimizer",
"@kbn/optimizer-webpack-helpers": "link:packages/kbn-optimizer-webpack-helpers",
Expand Down
Loading

0 comments on commit e3d95e9

Please sign in to comment.