Migrate core domain services to packages: Dependency analysis on remaining server-side services

[lukeelmers]

Part of #134112

After doing the initial work of moving server-side service types to packages, let's stop and come up with a plan for migrating the remaining services. The obvious next step will be to tackle the HTTP service, as it's used by all remaining plugins. We’ll need to handle the isolation issues between http/elasticsearch/context. Once this part is done, the next services should be way easier.

Remaining services:

• Capabilities (registers routes)
• Context (bad isolation with HTTP, cross imports of types between the two)
• CoreApp (registers routes)
• CoreUsageData
• Deprecations (registers routes)
• Elasticsearch (bad isolation with HTTP, cross imports of types between the two)
• Http / externalUrl / CSP / HttpResources (main issue here)
• I18n (registers routes)
• Metrics
• Plugins (depends on CoreSetup/Start, this one will be fun too as it needs to be top level)
• Preboot
• Rendering
• SavedObjects (depends on ES->HTTP)
• Status (registers routes)
• UiSettings (depends on SO->ES->HTTP)
• HttpResources (depends on http)

The task here is to do a dependency analysis and come up with a clear plan for migrating the remaining items.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

The next step (and the main difficulty) will be the splitting / ordering of the http, context and elasticsearch domains

Regarding http and context:

- Both domains import mocks from the other one

E.g

kibana/src/core/server/http/http_service.test.ts

Line 20 in a969d42

import { contextServiceMock } from '../context/context_service.mock';

and

kibana/src/core/server/context/container/context.test.ts

Line 11 in a02c00b

import { httpServerMock } from '../../http/http_server.mocks';

- Both domains import types from the other one

e.g

kibana/src/core/server/context/container/context.ts

Line 13 in 9b75efd

import type { RequestHandler, RequestHandlerContext } from '../..';

and

kibana/src/core/server/http/types.ts

Line 9 in a02c00b

import { IContextProvider, IContextContainer } from '../context';

- The http service depends on the context service

kibana/src/core/server/http/http_service.ts

Lines 43 to 45 in 9b75efd

	export interface PrebootDeps {
	context: InternalContextPreboot;
	}

In theory, given the http service has a direct dependency on the context service, the dependencies should only be in that order, and no context type should depends on http types.

In practice, the some types of the two domains are tightly coupled around the 'request handler' context. The http domain exposes the RequestHandlerContext type, and the context domain exposes the IContextProvider types.

In short, we have a terrible isolation between those two services, and at the moment I don't see a clear path toward splitting them.

[pgayvallet]

This is actually a bit worse than that. I actually missed that IContextProvider explicitly depends on RequestHandlerContext

kibana/src/core/server/context/container/context.ts

Lines 28 to 29 in 9b75efd

	export type IContextProvider<
	Context extends RequestHandlerContext,

And RequestHandlerContext has a core property that is of type CoreRequestHandlerContext

and this type is the whole context exposed to plugins via the router handler context:

kibana/src/core/server/index.ts

Lines 522 to 539 in 9398abc

	export interface CoreRequestHandlerContext {
	savedObjects: {
	client: SavedObjectsClientContract;
	typeRegistry: ISavedObjectTypeRegistry;
	getClient: (options?: SavedObjectsClientProviderOptions) => SavedObjectsClientContract;
	getExporter: (client: SavedObjectsClientContract) => ISavedObjectsExporter;
	getImporter: (client: SavedObjectsClientContract) => ISavedObjectsImporter;
	};
	elasticsearch: {
	client: IScopedClusterClient;
	};
	uiSettings: {
	client: IUiSettingsClient;
	};
	deprecations: {
	client: DeprecationsClient;
	};
	}

Meaning that the context and http domains actually depends on types from other domains, most of them for which the domain's service actually depends on the http types:

• http types depends on other types
• other service implementation depends on http service implementation

At this point, I don't really see any other option that starting by creating the public type packages for all the domains that are referenced from CoreRequestHandlerContext, and have a 'reversed dependency' between the public types packages (where http will for example depends on savedObjects) and the internal implementation (where savedObjects will depends on http)...

An alternative would be to stop this per-domain public type separation, and to have a single public type package for core browser/shared/server areas...

[pgayvallet]

I created #135703 to track the migration of the server-side http domain

[pgayvallet]

There are 4 services remaining on the server-side:

• httpResources
• rendering
• plugins
• coreApp

Dependency analysis

Only listing the unmigrated domains dependencies:

httpResources

Depends on rendering

kibana/src/core/server/http_resources/http_resources_service.ts

Line 22 in b8340dd

import { InternalRenderingServicePreboot, InternalRenderingServiceSetup } from '../rendering';

rendering

Depends on plugins

kibana/src/core/server/rendering/rendering_service.tsx

Line 19 in 15270a5

import type { UiPlugins } from '../plugins';

plugins

No dependencies, except the core lifecycle contracts, both public and internal, that aren't migrated yet.

coreApp

Depends on httpResources

kibana/src/core/server/core_app/core_app.ts

Line 23 in 885e80a

import { HttpResources, HttpResourcesServiceToolkit } from '../http_resources';

Depends on plugins

kibana/src/core/server/core_app/core_app.ts

Line 26 in 885e80a

import { UiPlugins } from '../plugins';

Topological ordering

plugins -> rendering -> httpResources -> coreApp

Will create the remaining issues and update #134112 accordingly.

Keeping this issue open just in case, but we should likely be able to close it right now.

[pgayvallet]

Okay, it would have been too easy, it's not actually as trivial as I described it...

The plugins service depends on the internal core lifecycle contracts

kibana/src/core/server/plugins/plugins_service.ts

Line 32 in e08e4b4

import { InternalCorePreboot, InternalCoreSetup, InternalCoreStart } from '../internal_types';

which, and I missed it, have references to services that are higher in the dependency chain (rendering and httpResources)

kibana/src/core/server/internal_types.ts

Lines 77 to 96 in d697729

	export interface InternalCoreSetup {
	analytics: AnalyticsServiceSetup;
	capabilities: CapabilitiesSetup;
	context: InternalContextSetup;
	docLinks: DocLinksServiceSetup;
	http: InternalHttpServiceSetup;
	elasticsearch: InternalElasticsearchServiceSetup;
	executionContext: InternalExecutionContextSetup;
	i18n: I18nServiceSetup;
	savedObjects: InternalSavedObjectsServiceSetup;
	status: InternalStatusServiceSetup;
	uiSettings: InternalUiSettingsServiceSetup;
	environment: InternalEnvironmentServiceSetup;
	rendering: InternalRenderingServiceSetup;
	httpResources: InternalHttpResourcesSetup;
	logging: InternalLoggingServiceSetup;
	metrics: InternalMetricsServiceSetup;
	deprecations: InternalDeprecationsServiceSetup;
	coreUsageData: InternalCoreUsageDataSetup;
	}

Which introduces a cyclic dependency via these internal types....

[pgayvallet]

I think we should consider that plugins should / will always be at the highest level in the dependency tree (a leaf), and try to work around with that statement.

We will probably have to introduce a -base package for the plugin domains containing the types that are used by other domains, which should break the cyclic dependency. Gonna take a look at that.

[TinaHeiligers]

@pgayvallet can you remember why the Jira issue for this is marked as blocked?

[pgayvallet]

IIRC it was more marked as 'awaiting' than blocked.

But I think we should be good closing this issue now, WDYT?