Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search][ES3] Enable Inference Management UI in ES3 #200109

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b58d99f
Fix index management link issue for both stacks
Samiul-TheSoccerFan Nov 14, 2024
b16064a
Adding FTR tests for inference management ui
Samiul-TheSoccerFan Nov 14, 2024
aec1441
Updating index management link to support for ES3
Samiul-TheSoccerFan Nov 18, 2024
9f9841c
Updating tests
Samiul-TheSoccerFan Nov 18, 2024
a244eee
Fix formatting issue with yml and json file
Samiul-TheSoccerFan Nov 18, 2024
9679312
Adding useCallBack to remove reinitialization
Samiul-TheSoccerFan Nov 18, 2024
4a7ec94
Update navigation config for navigation ftr tests
Samiul-TheSoccerFan Nov 20, 2024
86d38d9
update tests for preconfigured and other scenarios
Samiul-TheSoccerFan Nov 20, 2024
d6d570c
Adding locator for plugins ES3 link support
Samiul-TheSoccerFan Nov 20, 2024
690f6db
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Nov 20, 2024
a33a1c0
Adding locator support for inference management in enterprise search
Samiul-TheSoccerFan Nov 20, 2024
43c794a
Fix nav tests due to update plugin link
Samiul-TheSoccerFan Nov 20, 2024
1ab1d82
Update api documentation link to create inference endpoint
Samiul-TheSoccerFan Nov 20, 2024
83da4ab
Remove redundant tests and added type import
Samiul-TheSoccerFan Nov 20, 2024
4361cb7
fixing i18n issues
Samiul-TheSoccerFan Nov 20, 2024
41d3620
fixing formatting issue in tsconfig file
Samiul-TheSoccerFan Nov 20, 2024
4b757e1
Fixing semantic_text field tests
Samiul-TheSoccerFan Nov 21, 2024
7cd2d91
Fix navigation issue with FTR tests
Samiul-TheSoccerFan Nov 21, 2024
505d488
Removing empty state from search inference plugin
Samiul-TheSoccerFan Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/serverless.es.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ data_visualizer.resultLinks.fileBeat.enabled: false
xpack.searchPlayground.ui.enabled: true

# Search InferenceEndpoints
xpack.searchInferenceEndpoints.ui.enabled: false
xpack.searchInferenceEndpoints.ui.enabled: true
TattdCodeMonkey marked this conversation as resolved.
Show resolved Hide resolved

# Search Notebooks
xpack.search.notebooks.catalog.url: https://elastic-enterprise-search.s3.us-east-2.amazonaws.com/serverless/catalog.json
Expand Down
3 changes: 3 additions & 0 deletions packages/kbn-doc-links/src/get_doc_links.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1001,5 +1001,8 @@ export const getDocLinks = ({ kibanaBranch, buildFlavor }: GetDocLinkOptions): D
context: `${KIBANA_DOCS}playground-context.html`,
hiddenFields: `${KIBANA_DOCS}playground-query.html#playground-hidden-fields`,
},
inferenceManagement: {
inferenceAPIDocumentation: `${ELASTIC_WEBSITE_URL}docs/api/doc/elasticsearch/operation/operation-inference-put`,
},
});
};
3 changes: 3 additions & 0 deletions packages/kbn-doc-links/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -676,6 +676,9 @@ export interface DocLinks {
readonly context: string;
readonly hiddenFields: string;
};
readonly inferenceManagement: {
readonly inferenceAPIDocumentation: string;
};
}

export type BuildFlavor = 'serverless' | 'traditional';
2 changes: 1 addition & 1 deletion x-pack/plugins/enterprise_search/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ export const SEARCH_RELEVANCE_PLUGIN = {
DESCRIPTION: i18n.translate('xpack.enterpriseSearch.inferenceEndpoints.description', {
defaultMessage: 'Manage your inference endpoints for semantic search and AI use cases.',
}),
URL: '/app/enterprise_search/relevance',
URL: '/app/elasticsearch/relevance',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 🙇 Thanks for aligning the stack & serverless URLs

LOGO: 'logoEnterpriseSearch',
SUPPORT_URL: 'https://discuss.elastic.co/c/enterprise-search/',
};
Expand Down
3 changes: 3 additions & 0 deletions x-pack/plugins/enterprise_search/common/locators/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,17 @@
*/

import type { SharePluginSetup } from '@kbn/share-plugin/public';
import { SerializableRecord } from '@kbn/utility-types';

import {
CreateIndexLocatorDefinition,
type CreateIndexLocatorParams,
} from './create_index_locator';
import { SearchInferenceEndpointLocatorDefinition } from './inference_locator';
import { PlaygroundLocatorDefinition, type PlaygroundLocatorParams } from './playground_locator';

export function registerLocators(share: SharePluginSetup) {
share.url.locators.create<CreateIndexLocatorParams>(new CreateIndexLocatorDefinition());
share.url.locators.create<PlaygroundLocatorParams>(new PlaygroundLocatorDefinition());
share.url.locators.create<SerializableRecord>(new SearchInferenceEndpointLocatorDefinition());
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { LocatorDefinition } from '@kbn/share-plugin/common';
import type { SharePluginSetup } from '@kbn/share-plugin/public';
import type { SerializableRecord } from '@kbn/utility-types';

import { SEARCH_RELEVANCE_PLUGIN } from '../constants';

export function registerLocators(share: SharePluginSetup) {
share.url.locators.create<SerializableRecord>(new SearchInferenceEndpointLocatorDefinition());
}

export class SearchInferenceEndpointLocatorDefinition
implements LocatorDefinition<SerializableRecord>
{
public readonly getLocation = async () => {
return {
app: SEARCH_RELEVANCE_PLUGIN.ID,
path: '/inference_endpoints',
state: {},
};
};

public readonly id = 'SEARCH_INFERENCE_ENDPOINTS';
}
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ const baseNavItems = [
items: [
{
'data-test-subj': 'searchSideNav-InferenceEndpoints',
href: '/app/enterprise_search/relevance/inference_endpoints',
href: '/app/elasticsearch/relevance/inference_endpoints',
id: 'inference_endpoints',
items: undefined,
name: 'Inference Endpoints',
Expand Down Expand Up @@ -205,7 +205,7 @@ const mockNavLinks = [
{
id: 'searchInferenceEndpoints:inferenceEndpoints',
title: 'Inference Endpoints',
url: '/app/enterprise_search/relevance/inference_endpoints',
url: '/app/elasticsearch/relevance/inference_endpoints',
},
{
id: 'appSearch:engines',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -727,6 +727,11 @@ describe('<IndexDetailsPage />', () => {
isActive: true,
hasAtLeast: jest.fn((type) => true),
};
const INFERENCE_LOCATOR = 'SEARCH_INFERENCE_ENDPOINTS';
const createMockLocator = (id: string) => ({
useUrl: jest.fn().mockReturnValue('https://redirect.me/to/inference_endpoints'),
});
const mockInferenceManagementLocator = createMockLocator(INFERENCE_LOCATOR);
beforeEach(async () => {
httpRequestsMockHelpers.setInferenceModels({
data: [
Expand All @@ -750,7 +755,9 @@ describe('<IndexDetailsPage />', () => {
docLinks: {
links: {
ml: '',
enterpriseSearch: '',
inferenceManagement: {
inferenceAPIDocumentation: 'https://abc.com/inference-api-create',
},
},
},
core: {
Expand Down Expand Up @@ -819,6 +826,20 @@ describe('<IndexDetailsPage />', () => {
},
},
},
share: {
url: {
locators: {
get: jest.fn((id) => {
switch (id) {
case INFERENCE_LOCATOR:
return mockInferenceManagementLocator;
default:
throw new Error(`Unknown locator id: ${id}`);
}
}),
},
},
},
},
},
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ import { InferenceAPIConfigResponse } from '@kbn/ml-trained-models-utils';

const createInferenceEndpointMock = jest.fn();
const mockDispatch = jest.fn();
const INFERENCE_LOCATOR = 'SEARCH_INFERENCE_ENDPOINTS';
const createMockLocator = (id: string) => ({
useUrl: jest.fn().mockReturnValue('https://redirect.me/to/inference_endpoints'),
});
const mockInferenceManagementLocator = createMockLocator(INFERENCE_LOCATOR);

jest.mock('../../../public/application/app_context', () => ({
useAppContext: jest.fn().mockReturnValue({
Expand All @@ -33,8 +38,8 @@ jest.mock('../../../public/application/app_context', () => ({
},
docLinks: {
links: {
enterpriseSearch: {
inferenceApiCreate: 'https://abc.com/inference-api-create',
inferenceManagement: {
inferenceAPIDocumentation: 'https://abc.com/inference-api-create',
},
},
},
Expand All @@ -47,6 +52,20 @@ jest.mock('../../../public/application/app_context', () => ({
},
},
},
share: {
url: {
locators: {
get: jest.fn((id) => {
switch (id) {
case INFERENCE_LOCATOR:
return mockInferenceManagementLocator;
default:
throw new Error(`Unknown locator id: ${id}`);
}
}),
},
},
},
},
}),
}));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,15 @@ const SelectInferenceIdContent: React.FC<SelectInferenceIdContentProps> = ({
value,
}) => {
const {
core: { application, http },
core: { application },
docLinks,
plugins: { ml },
plugins: { ml, share },
} = useAppContext();
const config = getFieldConfig('inference_id');

const inferenceEndpointsPageLink = `${http.basePath.get()}/app/enterprise_search/relevance/inference_endpoints`;
const inferenceEndpointsPageLink = share?.url.locators
Copy link
Contributor

@mattkime mattkime Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm new to this code - mind posting a screenshot that shows the UI affected by this? Also, can you remind me why this conditional is needed and which environments lack the enterprise_search app? I can assume, but you might know better than me and it would be nice to have it directly mentioned in the pr. Actually, the PR description is vague - what was broken and how was it fixed? If this is mentioned in an issue simply linking to it would be fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mattkime , Thank you for bringing this up. We are releasing inference management UI in Serverless (PR title fixed) and hit many small issues that needed fixing.

This particular code change required as Manage Inference Endpoint in Popover links(please see attached screenshot) were hardcoded and will always redirect to enterprise search -> inference endpoints. However, for Serverless, we want them to redirect to Serverless -> inference endpoints.

This is a bit odd as both enterprise-search and serverless render the same plugin into two different URLs. From 81.8, this will change however we need a fix till then.

Hope that helps.

Screenshot 2024-11-21 at 11 49 34 AM

.get('SEARCH_INFERENCE_ENDPOINTS')
?.useUrl({});

const [isInferenceFlyoutVisible, setIsInferenceFlyoutVisible] = useState<boolean>(false);
const [availableTrainedModels, setAvailableTrainedModels] = useState<
Expand Down Expand Up @@ -224,24 +226,28 @@ const SelectInferenceIdContent: React.FC<SelectInferenceIdContentProps> = ({
panelPaddingSize="m"
closePopover={() => setIsInferencePopoverVisible(!isInferencePopoverVisible)}
>
<EuiContextMenuPanel>
<EuiContextMenuItem
key="manageInferenceEndpointButton"
icon="gear"
size="s"
data-test-subj="manageInferenceEndpointButton"
onClick={async () => {
application.navigateToUrl(inferenceEndpointsPageLink);
}}
>
{i18n.translate(
'xpack.idxMgmt.mappingsEditor.parameters.inferenceId.popover.manageInferenceEndpointButton',
{
defaultMessage: 'Manage Inference Endpoints',
}
)}
</EuiContextMenuItem>
</EuiContextMenuPanel>
{inferenceEndpointsPageLink && (
<EuiContextMenuPanel>
<EuiContextMenuItem
key="manageInferenceEndpointButton"
icon="gear"
size="s"
data-test-subj="manageInferenceEndpointButton"
href={inferenceEndpointsPageLink}
onClick={(e) => {
e.preventDefault();
application.navigateToUrl(inferenceEndpointsPageLink);
}}
>
{i18n.translate(
'xpack.idxMgmt.mappingsEditor.parameters.inferenceId.popover.manageInferenceEndpointButton',
{
defaultMessage: 'Manage Inference Endpoints',
}
)}
</EuiContextMenuItem>
</EuiContextMenuPanel>
)}
<EuiHorizontalRule margin="none" />
<EuiPanel color="transparent" paddingSize="s">
<EuiTitle size="xxxs">
Expand Down Expand Up @@ -292,7 +298,7 @@ const SelectInferenceIdContent: React.FC<SelectInferenceIdContentProps> = ({
<EuiHorizontalRule margin="none" />
<EuiContextMenuItem icon={<EuiIcon type="help" color="primary" />} size="s">
<EuiLink
href={docLinks.links.enterpriseSearch.inferenceApiCreate}
href={docLinks.links.inferenceManagement.inferenceAPIDocumentation}
target="_blank"
data-test-subj="learn-how-to-create-inference-endpoints"
>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class InferenceEndpointsDocLinks {
constructor() {}

setDocLinks(newDocLinks: DocLinks) {
this.createInferenceEndpoint = newDocLinks.enterpriseSearch.inferenceApiCreate;
this.createInferenceEndpoint = newDocLinks.inferenceManagement.inferenceAPIDocumentation;
this.semanticSearchElser = newDocLinks.enterpriseSearch.elser;
this.semanticSearchE5 = newDocLinks.enterpriseSearch.e5Model;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,49 +26,6 @@ export const MANAGE_INFERENCE_ENDPOINTS_LABEL = i18n.translate(
}
);

export const CREATE_FIRST_INFERENCE_ENDPOINT_DESCRIPTION = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.createFirstInferenceEndpointDescription',
{
defaultMessage:
"Inference endpoints enable you to perform inference tasks using NLP models provided by third-party services or Elastic's built-in models like ELSER and E5. Set up tasks such as text embedding, completions, reranking, and more by using the Create Inference API.",
}
);

export const START_WITH_PREPARED_ENDPOINTS_LABEL = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.startWithPreparedEndpointsLabel',
{
defaultMessage: 'Learn more about built-in NLP models:',
}
);

export const ELSER_TITLE = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.elserTitle',
{
defaultMessage: 'ELSER',
}
);

export const LEARN_HOW_TO_CREATE_INFERENCE_ENDPOINTS_LINK = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.learnHowToCreateInferenceEndpoints',
{
defaultMessage: 'Learn how to create inference endpoints',
}
);

export const SEMANTIC_SEARCH_WITH_ELSER_LINK = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.semanticSearchWithElser',
{
defaultMessage: 'Semantic search with ELSER',
}
);

export const SEMANTIC_SEARCH_WITH_E5_LINK = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.semanticSearchWithE5',
{
defaultMessage: 'Semantic search with E5 Multilingual',
}
);

export const VIEW_YOUR_MODELS_LINK = i18n.translate(
'xpack.searchInferenceEndpoints.viewYourModels',
{
Expand All @@ -83,25 +40,6 @@ export const API_DOCUMENTATION_LINK = i18n.translate(
}
);

export const ELSER_DESCRIPTION = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.elserDescription',
{
defaultMessage: "ELSER is Elastic's sparse vector NLP model for semantic search in English.",
}
);

export const E5_TITLE = i18n.translate('xpack.searchInferenceEndpoints.addEmptyPrompt.e5Title', {
defaultMessage: 'E5 Multilingual',
});

export const E5_DESCRIPTION = i18n.translate(
'xpack.searchInferenceEndpoints.addEmptyPrompt.e5Description',
{
defaultMessage:
'E5 is a third-party NLP model that enables you to perform multilingual semantic search by using dense vector representations.',
}
);

export const ERROR_TITLE = i18n.translate('xpack.searchInferenceEndpoints.inferenceId.errorTitle', {
defaultMessage: 'Error adding inference endpoint',
});
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/search_inference_endpoints/kibana.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"optionalPlugins": [
"cloud",
"console",
"serverless"
],
"requiredBundles": [
"kibanaReact"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,4 @@ export const DEFAULT_INFERENCE_ENDPOINTS_TABLE_STATE: AllInferenceEndpointsTable
};

export const PIPELINE_URL = 'ingest/ingest_pipelines';
export const SERVERLESS_INDEX_MANAGEMENT_URL = 'index_details';
TattdCodeMonkey marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ interface UseFilterParams {
options: MultiSelectFilterOption[];
renderOption?: (option: MultiSelectFilterOption) => React.ReactNode;
selectedOptionKeys?: string[];
dataTestSubj?: string;
}

export const MultiSelectFilter: React.FC<UseFilterParams> = ({
Expand All @@ -41,6 +42,7 @@ export const MultiSelectFilter: React.FC<UseFilterParams> = ({
options: rawOptions,
selectedOptionKeys = [],
renderOption,
dataTestSubj,
}) => {
const { euiTheme } = useEuiTheme();
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
Expand All @@ -55,7 +57,7 @@ export const MultiSelectFilter: React.FC<UseFilterParams> = ({
);

return (
<EuiFilterGroup>
<EuiFilterGroup data-test-subj={dataTestSubj}>
<EuiPopover
ownFocus
button={
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ export const ServiceProviderFilter: React.FC<Props> = ({ optionKeys, onChange })
options={options}
renderOption={(option) => option.label}
selectedOptionKeys={optionKeys}
dataTestSubj="service-field-endpoints"
/>
);
};
Loading