[Security Solution] [Elastic AI Assistant] Consolidate feature flag registration within assistant plugin server #172509

spong · 2023-12-04T18:25:29Z

Summary

Currently Elastic Assistant feature flag registration is a little convoluted and not as flexible as it needs to be. This issue is for outlining and resolving this behavior.

Current Behavior

Feature flag registration happens within the consuming plugin, and is then injected to the Assistant Client UI via the ElasticAssistantProvider, at which point it is available anywhere within the Assistant UI code via the useAssistantContext hook. Using the isModelEvaluationEnabled feature flag for example:

Here within the Security Solution plugin we define the assistantModelEvaluation experimental feature flag:

kibana/x-pack/plugins/security_solution/common/experimental_features.ts

Lines 87 to 90 in be7f6cf

    
             /** 
        
              * Enables the Assistant Model Evaluation advanced setting and API endpoint, introduced in `8.11.0`. 
        
              */ 
        
             assistantModelEvaluation: false,

It's then injected into the assistant when instantiated:

kibana/x-pack/plugins/security_solution/public/assistant/provider.tsx

Line 71 in e74e5e2

modelEvaluatorEnabled={isModelEvaluationEnabled}

and then used in the UI for toggling the evaluation feature:

kibana/x-pack/packages/kbn-elastic-assistant/impl/assistant/settings/assistant_settings.tsx

Lines 257 to 267 in 413715f

    
           {modelEvaluatorEnabled && ( 
        
             <EuiKeyPadMenuItem 
        
               id={EVALUATION_TAB} 
        
               label={i18n.EVALUATION_MENU_ITEM} 
        
               isSelected={selectedSettingsTab === EVALUATION_TAB} 
        
               onClick={() => setSelectedSettingsTab(EVALUATION_TAB)} 
        
               data-test-subj={`${EVALUATION_TAB}-button`} 
        
             > 
        
               <EuiIcon type="crossClusterReplicationApp" size="l" /> 
        
             </EuiKeyPadMenuItem> 
        
           )}

This means that for each new feature flag, it must be plumbed through the context, injected, and then can only be used within the client UI, which isn't adequate when needing to control server logic.

Proposed Behavior

Move this logic serverside such that when the consuming plugin starts, it can register any feature flags/capabilities it needs, and then expose a 'capabilities' route within the assistant that will then make those values available to the client UI. This can be built off of the new AppContextService being introduced in support of arbitrary tool registration.

The text was updated successfully, but these errors were encountered:

elasticmachine · 2023-12-04T18:25:31Z

Pinging @elastic/security-solution (Team: SecuritySolution)

…ature registration (#174317) ## Summary Resolves #172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: #166269 * Includes OpenAPI docs bundling script as detailed in: #171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]>

…ature registration (elastic#174317) ## Summary Resolves elastic#172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: elastic#166269 * Includes OpenAPI docs bundling script as detailed in: elastic#171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](elastic#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]>

spong added chore Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Security Assistant Security Assistant v8.13.0 labels Dec 4, 2023

spong self-assigned this Dec 4, 2023

spong mentioned this issue Dec 4, 2023

[Security solution] Add feature flag for AI streaming #172505

Merged

spong mentioned this issue Jan 5, 2024

[Security Solution] [Elastic AI Assistant] Adds support for plugin feature registration #174317

Merged

2 tasks

spong closed this as completed in #174317 Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] [Elastic AI Assistant] Consolidate feature flag registration within assistant plugin server #172509

[Security Solution] [Elastic AI Assistant] Consolidate feature flag registration within assistant plugin server #172509

spong commented Dec 4, 2023

elasticmachine commented Dec 4, 2023

[Security Solution] [Elastic AI Assistant] Consolidate feature flag registration within assistant plugin server #172509

[Security Solution] [Elastic AI Assistant] Consolidate feature flag registration within assistant plugin server #172509

Comments

spong commented Dec 4, 2023

Summary

Current Behavior

Proposed Behavior

elasticmachine commented Dec 4, 2023