Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sustainable Kibana Architecture: Categorise straightforward packages #199630

Merged
merged 62 commits into from
Nov 22, 2024

Conversation

gsoldevila
Copy link
Contributor

@gsoldevila gsoldevila commented Nov 11, 2024

Summary

This PR is part of the Kibana Sustainable Architecture effort.

The goal is to start categorising Kibana packages into generic platform (group: "platform") vs solution-specific.

group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'

Uncategorised modules are considered to be group: 'common', visibility: 'shared' by default.

We want to prevent code from solution A to depend on code from solution B.
Thus, the rules are pretty simple:

  • Modules can only depend on:
    • Modules in the same group
    • OR modules with 'shared' visibility
  • Modules in 'observability', 'security', 'search' groups are mandatorily visibility: "private".

Long term, the goal is to re-organise packages into dedicated folders, e.g.:

x-pack/platform/plugins/private
x-pack/observability/packages

For this first wave, we have categorised packages that seem "straightforward":

  • Any packages that have:
    • at least one dependant module
    • all dependants belong to the same group
  • Categorise all Core packages:
    • @kbn/core-...-internal => platform/private
    • everything else => platform/shared
  • Categorise as platform/shared those packages that:
    • Have at least one dependant in the platform group.
    • Don't have any devOnly: true dependants.

What we ask from you, as CODEOWNERS of the package manifests, is that you confirm that the categorisation is correct:

  • group: "platform", visibility: "private" if it's a package that should only be used from platform code, not from any solution code. It will be loaded systematically in all serverless flavors, but solution plugins and packages won't be able to import from it.
  • group: "platform", visibility: "shared" if it's a package that can be consumed by both platform and solutions code. It will be loaded systematically in all serverless flavors, and anybody can import / use code from it.
  • group: "observability" | "security" | "search", visibility: "private" if it's a package that is intented to be used exclusively from a given solution. It won't be accessible nor loaded from other solutions nor platform code.

Please refer to #kibana-sustainable-architecture for any related questions.

@gsoldevila gsoldevila added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Nov 11, 2024
@gsoldevila gsoldevila changed the title Sustainable Kibana Architecture: Categorise platform/private packages Sustainable Kibana Architecture: Categorise straightforward packages Nov 11, 2024
@gsoldevila gsoldevila marked this pull request as ready for review November 12, 2024 11:37
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@gsoldevila gsoldevila requested review from a team as code owners November 12, 2024 11:37
@gsoldevila gsoldevila requested a review from rylnd November 12, 2024 11:37
Copy link
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the visibility property @gsoldevila ??

The ES|QL editor is shared to solutions teams too. So private seems wrong or am I interpreting this wrongly?

@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Nov 12, 2024
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@gsoldevila gsoldevila force-pushed the kbn-team-1284-categorise-packages branch from e18baab to 68d97f5 Compare November 21, 2024 12:04
@gsoldevila gsoldevila requested a review from a team as a code owner November 21, 2024 12:04
@elasticmachine
Copy link
Contributor

elasticmachine commented Nov 21, 2024

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/core-http-server-utils - 2 +2
@kbn/repo-source-classifier 6 7 +1
total +3
Unknown metric groups

API count

id before after diff
@kbn/core-http-server-utils - 6 +6
@kbn/repo-source-classifier 6 14 +8
total +14

History

Copy link
Contributor

@tomsonpl tomsonpl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Osquery / Defend Workflows LGTM 👍

@gsoldevila gsoldevila merged commit b24fdf5 into elastic:main Nov 22, 2024
58 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11970029914

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [Spaces UI] Role Editor Flyout Should Match in Roles Mgmt (#198182)
- Surface Kibana security route deprecations in Upgrade Assistant (#199656)
- Change spaces button loading design (#197922)

Manual backport

To create the backport manually run:

node scripts/backport --pr 199630

Questions ?

Please refer to the Backport tool documentation

@gsoldevila
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

gsoldevila added a commit to gsoldevila/kibana that referenced this pull request Nov 22, 2024
elastic#199630)

## Summary

This PR is part of the Kibana Sustainable Architecture effort.

The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.

```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.

We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:

* Modules can only depend on:
  * Modules in the same group
  * OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.

Long term, the goal is to re-organise packages into dedicated folders,
e.g.:

```
x-pack/platform/plugins/private
x-pack/observability/packages
```

For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
  * at least one dependant module
  * all dependants belong to the same group
* Categorise all Core packages:
  * `@kbn/core-...-internal` => _platform/private_
  * everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
  * Have at least one dependant in the _platform_ group.
  * Don't have any `devOnly: true` dependants.

### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:

* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.

Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit b24fdf5)

# Conflicts:
#	.github/CODEOWNERS
#	packages/kbn-flot-charts/kibana.jsonc
#	packages/kbn-item-buffer/kibana.jsonc
#	packages/kbn-mapbox-gl/kibana.jsonc
#	packages/kbn-typed-react-router-config/kibana.jsonc
#	x-pack/packages/maps/vector_tile_utils/kibana.jsonc
#	x-pack/packages/security/ui_components/kibana.jsonc
#	x-pack/plugins/security/tsconfig.json
#	x-pack/plugins/spaces/tsconfig.json
gsoldevila added a commit that referenced this pull request Nov 22, 2024
…ackages (#199630) (#201340)

# Backport

This will backport the following commits from `main` to `8.x`:
- [Sustainable Kibana Architecture: Categorise `straightforward`
packages (#199630)](#199630)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gerard
Soldevila","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-22T09:33:25Z","message":"Sustainable
Kibana Architecture: Categorise `straightforward` packages
(#199630)\n\n## Summary\r\n\r\nThis PR is part of the Kibana Sustainable
Architecture effort.\r\n\r\nThe goal is to start categorising Kibana
packages into _generic\r\nplatform_ (`group: \"platform\"`) vs
_solution-specific_.\r\n\r\n```\r\ngroup?: 'search' | 'security' |
'observability' | 'platform'\r\nvisibility?: 'private' |
'shared'\r\n```\r\nUncategorised modules are considered to be `group:
'common', visibility:\r\n'shared'` by default.\r\n\r\nWe want to prevent
code from solution A to depend on code from solution\r\nB.\r\nThus, the
rules are pretty simple:\r\n\r\n* Modules can only depend on:\r\n *
Modules in the same group\r\n * OR modules with 'shared' visibility\r\n*
Modules in `'observability', 'security', 'search'` groups
are\r\nmandatorily `visibility: \"private\"`.\r\n\r\nLong term, the goal
is to re-organise packages into dedicated
folders,\r\ne.g.:\r\n\r\n```\r\nx-pack/platform/plugins/private\r\nx-pack/observability/packages\r\n```\r\n\r\nFor
this first wave, we have categorised packages that
seem\r\n\"straightforward\":\r\n* Any packages that have:\r\n * at least
one dependant module\r\n * all dependants belong to the same group\r\n*
Categorise all Core packages:\r\n * `@kbn/core-...-internal` =>
_platform/private_\r\n * everything else => _platform/shared_\r\n*
Categorise as _platform/shared_ those packages that:\r\n * Have at least
one dependant in the _platform_ group.\r\n * Don't have any `devOnly:
true` dependants.\r\n\r\n### What we ask from you, as CODEOWNERS of the
_package manifests_, is\r\nthat you confirm that the categorisation is
correct:\r\n\r\n* `group: \"platform\", visibility: \"private\"` if it's
a package that\r\nshould only be used from platform code, not from any
solution code. It\r\nwill be loaded systematically in all serverless
flavors, but solution\r\nplugins and packages won't be able to `import`
from it.\r\n* `group: \"platform\", visibility: \"shared\"` if it's a
package that can\r\nbe consumed by both platform and solutions code. It
will be loaded\r\nsystematically in all serverless flavors, and anybody
can import / use\r\ncode from it.\r\n* `group: \"observability\" |
\"security\" | \"search\", visibility:\r\n\"private\"` if it's a package
that is intented to be used exclusively\r\nfrom a given solution. It
won't be accessible nor loaded from other\r\nsolutions nor platform
code.\r\n\r\nPlease refer
to\r\n[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)\r\nfor
any related questions.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"b24fdf5d3f6b7454a4edcedb8141b82f571e1d74","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","release_note:skip","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services"],"number":199630,"url":"https://github.com/elastic/kibana/pull/199630","mergeCommit":{"message":"Sustainable
Kibana Architecture: Categorise `straightforward` packages
(#199630)\n\n## Summary\r\n\r\nThis PR is part of the Kibana Sustainable
Architecture effort.\r\n\r\nThe goal is to start categorising Kibana
packages into _generic\r\nplatform_ (`group: \"platform\"`) vs
_solution-specific_.\r\n\r\n```\r\ngroup?: 'search' | 'security' |
'observability' | 'platform'\r\nvisibility?: 'private' |
'shared'\r\n```\r\nUncategorised modules are considered to be `group:
'common', visibility:\r\n'shared'` by default.\r\n\r\nWe want to prevent
code from solution A to depend on code from solution\r\nB.\r\nThus, the
rules are pretty simple:\r\n\r\n* Modules can only depend on:\r\n *
Modules in the same group\r\n * OR modules with 'shared' visibility\r\n*
Modules in `'observability', 'security', 'search'` groups
are\r\nmandatorily `visibility: \"private\"`.\r\n\r\nLong term, the goal
is to re-organise packages into dedicated
folders,\r\ne.g.:\r\n\r\n```\r\nx-pack/platform/plugins/private\r\nx-pack/observability/packages\r\n```\r\n\r\nFor
this first wave, we have categorised packages that
seem\r\n\"straightforward\":\r\n* Any packages that have:\r\n * at least
one dependant module\r\n * all dependants belong to the same group\r\n*
Categorise all Core packages:\r\n * `@kbn/core-...-internal` =>
_platform/private_\r\n * everything else => _platform/shared_\r\n*
Categorise as _platform/shared_ those packages that:\r\n * Have at least
one dependant in the _platform_ group.\r\n * Don't have any `devOnly:
true` dependants.\r\n\r\n### What we ask from you, as CODEOWNERS of the
_package manifests_, is\r\nthat you confirm that the categorisation is
correct:\r\n\r\n* `group: \"platform\", visibility: \"private\"` if it's
a package that\r\nshould only be used from platform code, not from any
solution code. It\r\nwill be loaded systematically in all serverless
flavors, but solution\r\nplugins and packages won't be able to `import`
from it.\r\n* `group: \"platform\", visibility: \"shared\"` if it's a
package that can\r\nbe consumed by both platform and solutions code. It
will be loaded\r\nsystematically in all serverless flavors, and anybody
can import / use\r\ncode from it.\r\n* `group: \"observability\" |
\"security\" | \"search\", visibility:\r\n\"private\"` if it's a package
that is intented to be used exclusively\r\nfrom a given solution. It
won't be accessible nor loaded from other\r\nsolutions nor platform
code.\r\n\r\nPlease refer
to\r\n[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)\r\nfor
any related questions.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"b24fdf5d3f6b7454a4edcedb8141b82f571e1d74"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199630","number":199630,"mergeCommit":{"message":"Sustainable
Kibana Architecture: Categorise `straightforward` packages
(#199630)\n\n## Summary\r\n\r\nThis PR is part of the Kibana Sustainable
Architecture effort.\r\n\r\nThe goal is to start categorising Kibana
packages into _generic\r\nplatform_ (`group: \"platform\"`) vs
_solution-specific_.\r\n\r\n```\r\ngroup?: 'search' | 'security' |
'observability' | 'platform'\r\nvisibility?: 'private' |
'shared'\r\n```\r\nUncategorised modules are considered to be `group:
'common', visibility:\r\n'shared'` by default.\r\n\r\nWe want to prevent
code from solution A to depend on code from solution\r\nB.\r\nThus, the
rules are pretty simple:\r\n\r\n* Modules can only depend on:\r\n *
Modules in the same group\r\n * OR modules with 'shared' visibility\r\n*
Modules in `'observability', 'security', 'search'` groups
are\r\nmandatorily `visibility: \"private\"`.\r\n\r\nLong term, the goal
is to re-organise packages into dedicated
folders,\r\ne.g.:\r\n\r\n```\r\nx-pack/platform/plugins/private\r\nx-pack/observability/packages\r\n```\r\n\r\nFor
this first wave, we have categorised packages that
seem\r\n\"straightforward\":\r\n* Any packages that have:\r\n * at least
one dependant module\r\n * all dependants belong to the same group\r\n*
Categorise all Core packages:\r\n * `@kbn/core-...-internal` =>
_platform/private_\r\n * everything else => _platform/shared_\r\n*
Categorise as _platform/shared_ those packages that:\r\n * Have at least
one dependant in the _platform_ group.\r\n * Don't have any `devOnly:
true` dependants.\r\n\r\n### What we ask from you, as CODEOWNERS of the
_package manifests_, is\r\nthat you confirm that the categorisation is
correct:\r\n\r\n* `group: \"platform\", visibility: \"private\"` if it's
a package that\r\nshould only be used from platform code, not from any
solution code. It\r\nwill be loaded systematically in all serverless
flavors, but solution\r\nplugins and packages won't be able to `import`
from it.\r\n* `group: \"platform\", visibility: \"shared\"` if it's a
package that can\r\nbe consumed by both platform and solutions code. It
will be loaded\r\nsystematically in all serverless flavors, and anybody
can import / use\r\ncode from it.\r\n* `group: \"observability\" |
\"security\" | \"search\", visibility:\r\n\"private\"` if it's a package
that is intented to be used exclusively\r\nfrom a given solution. It
won't be accessible nor loaded from other\r\nsolutions nor platform
code.\r\n\r\nPlease refer
to\r\n[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)\r\nfor
any related questions.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"b24fdf5d3f6b7454a4edcedb8141b82f571e1d74"}}]}]
BACKPORT-->
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Nov 26, 2024
elastic#199630)

## Summary

This PR is part of the Kibana Sustainable Architecture effort.

The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.

```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.

We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:

* Modules can only depend on:
  * Modules in the same group
  * OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.

Long term, the goal is to re-organise packages into dedicated folders,
e.g.:

```
x-pack/platform/plugins/private
x-pack/observability/packages
```

For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
  * at least one dependant module
  * all dependants belong to the same group
* Categorise all Core packages:
  * `@kbn/core-...-internal` => _platform/private_
  * everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
  * Have at least one dependant in the _platform_ group.
  * Don't have any `devOnly: true` dependants.

### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:

* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.

Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
…us-common (elastic#200524)

## Summary

- Exposes the interfaces that define the format of the response of the
`/api/status` endpoint.
- Moves them from `@kbn/core-status-common-internal` to
`@kbn/core-status-common`.
- Removes the former package, as it no longer contains anything.
- Fixes some of the illegal dependencies uncovered by
elastic#199630.

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
elastic#199630)

## Summary

This PR is part of the Kibana Sustainable Architecture effort.

The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.

```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.

We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:

* Modules can only depend on:
  * Modules in the same group
  * OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.

Long term, the goal is to re-organise packages into dedicated folders,
e.g.:

```
x-pack/platform/plugins/private
x-pack/observability/packages
```

For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
  * at least one dependant module
  * all dependants belong to the same group
* Categorise all Core packages:
  * `@kbn/core-...-internal` => _platform/private_
  * everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
  * Have at least one dependant in the _platform_ group.
  * Don't have any `devOnly: true` dependants.

### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:

* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.

Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.