Create stable API for accessing safe Elasticsearch config values in plugins

[joshdover]

Blocked by #117496

In #117496, we added the ability for plugins to read CA fingerprints from the Elasticsearch configuration, a feature we plan to support for the long-term to allow plugins to integrate directly with the new security-on-by-default feature.

In order to minimize scope for the 8.0 release, this change was added to the existing legacy.globalConfig$() API. We should move this and any other necessary ES config values to a stable, undeprecated API directly on the Elasticsearch service.

It should still be accessible on the PluginInitializerContext interface so that it can be accessed in the same way as all other configuration values.

Scope:

• Identify config values in use that are in legacy & should be available on a stable API
• Determine whether they definitely need to be on PluginInitializerContext or whether providing on the ES API is sufficient
• Remove legacy config

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

It should still be accessible on the PluginInitializerContext interface so that it can be accessed in the same way as all other configuration values.

@joshdover any reason to want this exposed via the initializer context instead of an API from the elasticsearch service's setup contract?

[joshdover]

It should still be accessible on the PluginInitializerContext interface so that it can be accessed in the same way as all other configuration values.

This was my only reason: consistency with other config values. Though tbh, I've always felt that this constructor being separate from setup was really unnecessary and just added complexity rather than made anything simpler. Maybe we should move away from PluginInitializerContext entirely?

[pgayvallet]

This was my only reason: consistency with other config values.

Ok, thanks, just wanted to confirm using a core service's contract wouldn't be an issue for you.

FWIW, the config exposed via PluginInitializerContext was only meant to expose a plugin's specific config properties (and then we added this ugly legacyConfig workaround). If a plugin was to share parts of its config to some consumers, it would do so with a setup and/or start contract API. I think following the same logic here would make sense, therefor exposing this via the elasticsearch setup contract.

Identify config values in use that are in legacy & should be available on a stable API

Note that in addition to the legacyConfig we're exposing to plugins

kibana/src/core/server/plugins/types.ts

Lines 374 to 375 in 1061835

	export type SharedGlobalConfig = RecursiveReadonly<{
	elasticsearch: Pick<ElasticsearchConfigType, typeof SharedGlobalConfigKeys.elasticsearch[number]>;

We're also exposing the whole config via ElasticsearchServiceSetup.legacy.config$:

kibana/src/core/server/elasticsearch/elasticsearch_service.ts

Lines 94 to 96 in d2bd7f8

	legacy: {
	config$: this.config$,
	},

which is (at least) used by the console proxy:

kibana/src/plugins/console/server/plugin.ts

Line 50 in f6a9afe

this.esLegacyConfigService.setup(elasticsearch.legacy.config$);

Not sure if this one should be considered as part of the scope of this issue?

[joshdover]

We're also exposing the whole config via ElasticsearchServiceSetup.legacy.config$:

Ah interesting, didn't realize we had that as well. FWIW our main use case for this has been eliminated and we no longer plan to need to read any ES config values from the Fleet UI side of things. I could see this changing in the future, but no plans currently.

Other than Console, I'm not sure what the requirements are for other plugins for config values. We definitely should remove legacy anything though and have a long-term option.

[TinaHeiligers]

I'm enabling Jira sync to get this issue on the team's roadmap. We're long past the platform migration initiative and should be removing anything legacy-related.

[richkuz]

Enterprise Search has a use case for fetching the external Elasticsearch URL: #77464 (comment)

[lukeelmers]

@richkuz Based on Alex's comment (#77464 (comment)), it sounds like simply exposing the ES URL that Kibana is configured with isn't going to solve the problem for Enterprise Search since y'all need the external URL.

Would you mind opening a separate issue for that use case (exposing external ES URL to Kibana plugins?). It would be a blocker for #77464 and for the Enterprise Search use case.

[richkuz]

👍 Submitted #139405