From c56281ce80d35e616c0e734a8a008eb8af0987fe Mon Sep 17 00:00:00 2001 From: Mykola Harmash Date: Wed, 18 Sep 2024 13:53:23 +0200 Subject: [PATCH] [Onboarding] AWS Service detection re-design for Firehose flow (#192860) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes https://github.com/elastic/kibana/issues/191989 Closes https://github.com/elastic/kibana/issues/190799 Closes https://github.com/elastic/kibana/issues/191731 This change implements the design changes done to improve the AWS service discovery in the new Firehose flow. [Figma](https://www.figma.com/design/CPhMyRNOgo0wsEiaIMZJ14/Onboarding-Quick-Starts?node-id=454-24601&t=Y7saMXwJfMinghMq-1) https://github.com/user-attachments/assets/57e0bbb3-1ace-42df-ae6d-5e34d0fd9368 ### How To Test You going to need an AWS account. You can use a personal one or "Elastic Observability" account which you can access through Okta (type "AWS" in Okta's search and you should see "AWS - Elastic Observability"). In case you decide to use the shared "Elastic Observability" account, make sure it does not already have `Elastic-CloudwatchLogsAndMetricsToFirehose` CloudFormation stack left from the previous tester. Feel free to delete it if it's there. 1. In AWS account, create a few entities that generate logs and put them into a CloudWatch log group (see instructions below for a few services). 1. Generate some logs by accessing the entities that you've created and make sure they appear in CloudWatch (mind that there is a ~1 minute delay). **If you don't see anything in CloudWatch, there is no point in proceeding further, make sure to fix your AWS setup before starting the flow in Kibana.** 1. Go to the serverless Kibana instance deployed from this PR (see the latest `[Deploy Serverless Kibana] ...` comment by ` kibanamachine`) 1. Add Data → Collect and analyze logs → View AWS Collection → Firehose quickstart 1. Open the Firehose flow and create CloudFormation stack using one of the two options. 1. Wait for the stack to finish creating. 1. Generate some some logs by accessing the AWS services you've created. 1. Go back to the Kibana screen, after a minute or so incoming logs should be detected and corresponding AWS service will be appear. ### Example AWS Services Configs **Before creating any resources, make sure you're in the same region (top right corner in AWS Console) you've used while configuring AWS CLI.** #### API Gateway 1. [Create an IAM role](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html#set-up-access-logging-permissions) to grant API Gateway permissions to write into a CloudWatch log groups 1. Copy the ARN of the created role 1. Open "CloudWatch" in AWS and select "Log groups" in the sidebar 1. Create a new log group with default parameters 1. Copy the ARN of the new group 1. Open **API Gateway** in AWS 1. Navigate to "Settings" in the sidebar 1. In the "Logging" section click "Edit" and paste the ARN of the IAM role you created in step 1. Hit "Save changes" 1. Now go back to "APIs" in the sidebar and click "Create API" 1. In "REST API" click "Build" 1. Select "Example API" and click "Create API" 1. Click on "Deploy API" 1. For "Stage" dropdown select "New stage", give it any name and click "Deploy" 1. You will now see "Invoke URL", you can use it later to access this API and generate logs 1. Scroll to "Logs and tracing" section and click "Edit" 1. In the dropdown select "Full request and response logs" 1. Toggle "Custom access logging" 1. Paste the ARN of the CloudWatch log group you've created in step 4. But make sure to not include ":*" symbols at the end. 1. In the log format input paste [this format from our docs](https://www.elastic.co/docs/current/integrations/aws/apigateway#data-streams) and click "Save" ``` {"requestId":"$context.requestId","ip":"$context.identity.sourceIp","caller":"$context.identity.caller","user":"$context.identity.user","requestTime":"$context.requestTime","httpMethod":"$context.httpMethod","resourcePath":"$context.resourcePath","status":"$context.status","protocol":"$context.protocol","responseLength":"$context.responseLength","apiId":"$context.apiId","domainName":"$context.domainName","stage":"$context.stage"} ``` 1. Now when you access this API, you should see logs coming into the CloudWatch group. #### WAF **This sets up WAF for an API Gateway, see above if you don't have one already.** 1. Open WAF in AWS 3. Click "Web ACLs" in the sidebar 4. Click "Create web ACL" 5. Select the region where you've created your API Gateway and give ACL any name 6. In the "Associated AWS resources" section click "Add AWS resources" 7. Select you API Gateway and click "Add" 8. Click "Next" 9. Create some basic rule, for example to block requests that have a specific parameter in the URL 10. Click through the other configuration step leaving everything as is and then finally click "Create web ACL" 11. Select the created ACL and click on the "Logging and metrics" tab 12. Click "Edit" in "Logging" section 13. Click "Create new" in the "Amazon CloudWatch Logs log group" section 14. Create a new log group. **The log group name should start with `aws-waf-logs-`**. 15. Select the new group in the dropdown and click "Save" 16. Now you should have logs generated and saved into the log group when you access your API gateway #### VPC 1. [Create an IAM role](https://docs.aws.amazon.com/vpc/latest/tgw/flow-logs-cwl.html#flow-logs-iam) to write flow logs to CloudWatch log groups. 3. Create and EC2 instance and configure there some HTTP server like Nginx. Using Docker would probably be the fastest way. 4. Create a CloudWatch log group with default parameters 5. Open "VPC" in AWS and select the VPC where you've created the EC2 instance. 6. Click the "Flow logs" tab and click "Create flow logs" 7. In "Maximum aggregation interval" select 1 minute to see logs faster 8. In "Destination log group" select the log group you've created in step 3 9. In "IAM role" select the role you've created in step 1 10. Click "Create flow log" 11. Now when you access your EC2 instance, you should see logs in the CloudWatch log group --- .../public/application/pages/firehose.tsx | 3 +- .../firehose/auto_refresh_callout.tsx | 54 ++++++ .../firehose/create_stack_command_snippet.tsx | 22 ++- .../firehose/create_stack_in_aws_console.tsx | 73 ++++++++ .../firehose/download_template_callout.tsx | 35 ++++ .../quickstart_flows/firehose/index.tsx | 123 ++++++++++++- .../firehose/progress_callout.tsx | 87 ++++++++++ .../use_aws_service_get_started_list.ts | 2 +- .../quickstart_flows/firehose/utils.ts | 40 +++++ .../firehose/visualize_data.tsx | 161 ++++++++++++------ .../shared/accordion_with_icon.tsx | 5 +- .../observability/onboarding/firehose.ts | 12 +- 12 files changed, 537 insertions(+), 80 deletions(-) create mode 100644 x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/auto_refresh_callout.tsx create mode 100644 x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_in_aws_console.tsx create mode 100644 x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/download_template_callout.tsx create mode 100644 x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/progress_callout.tsx diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/pages/firehose.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/pages/firehose.tsx index 638d931997ec3..3f986f080fc3e 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/pages/firehose.tsx +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/pages/firehose.tsx @@ -19,7 +19,7 @@ export const FirehosePage = () => ( headlineCopy={i18n.translate( 'xpack.observability_onboarding.experimentalOnboardingFlow.customHeader.firehose.text', { - defaultMessage: 'Setting up Amazon Data Firehose', + defaultMessage: 'Set up Amazon Data Firehose', } )} captionCopy={i18n.translate( @@ -29,6 +29,7 @@ export const FirehosePage = () => ( 'This installation is tailored for setting up Firehose in your Observability project with minimal configuration.', } )} + isTechnicalPreview={true} /> } > diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/auto_refresh_callout.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/auto_refresh_callout.tsx new file mode 100644 index 0000000000000..e4cd98c37ee1c --- /dev/null +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/auto_refresh_callout.tsx @@ -0,0 +1,54 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0; you may not use this file except in compliance with the Elastic License + * 2.0. + */ + +import React from 'react'; +import { i18n } from '@kbn/i18n'; +import { + EuiFlexGroup, + EuiFlexItem, + EuiIcon, + EuiText, + useEuiTheme, + useGeneratedHtmlId, +} from '@elastic/eui'; +import { css } from '@emotion/react'; +import { HAS_DATA_FETCH_INTERVAL } from './utils'; + +export function AutoRefreshCallout() { + const { euiTheme } = useEuiTheme(); + const messageId = useGeneratedHtmlId(); + + return ( + + + + + +

+ {i18n.translate( + 'xpack.observability_onboarding.firehosePanel.autorefreshCalloutLabel', + { + defaultMessage: 'Auto-refreshing every {intervalSeconds} s', + values: { intervalSeconds: Math.round(HAS_DATA_FETCH_INTERVAL / 1000) }, + } + )} +

+ + + + + ); +} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_command_snippet.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_command_snippet.tsx index 774f02c23a902..dedc05c701e00 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_command_snippet.tsx +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_command_snippet.tsx @@ -5,7 +5,6 @@ * 2.0. */ -import React from 'react'; import { EuiAccordion, EuiCodeBlock, @@ -14,19 +13,20 @@ import { EuiText, useGeneratedHtmlId, } from '@elastic/eui'; -import { FormattedMessage } from '@kbn/i18n-react'; import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n-react'; +import React from 'react'; import { FIREHOSE_CLOUDFORMATION_STACK_NAME, FIREHOSE_LOGS_STREAM_NAME, FIREHOSE_METRICS_STREAM_NAME, } from '../../../../common/aws_firehose'; import { CopyToClipboardButton } from '../shared/copy_to_clipboard_button'; +import { DownloadTemplateCallout } from './download_template_callout'; import { buildCreateStackCommand, buildStackStatusCommand } from './utils'; interface Props { encodedApiKey: string; - onboardingId: string; elasticsearchUrl: string; templateUrl: string; isCopyPrimaryAction: boolean; @@ -57,7 +57,7 @@ export function CreateStackCommandSnippet({

+ +

+ +

@@ -94,7 +98,15 @@ export function CreateStackCommandSnippet({ - + {stackStatusCommand} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_in_aws_console.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_in_aws_console.tsx new file mode 100644 index 0000000000000..3c62632b382ad --- /dev/null +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/create_stack_in_aws_console.tsx @@ -0,0 +1,73 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0; you may not use this file except in compliance with the Elastic License + * 2.0. + */ + +import { EuiButton, EuiSpacer, EuiText } from '@elastic/eui'; +import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n-react'; +import React from 'react'; +import { + FIREHOSE_CLOUDFORMATION_STACK_NAME, + FIREHOSE_LOGS_STREAM_NAME, + FIREHOSE_METRICS_STREAM_NAME, +} from '../../../../common/aws_firehose'; +import { DownloadTemplateCallout } from './download_template_callout'; +import { buildCreateStackAWSConsoleURL } from './utils'; + +interface Props { + encodedApiKey: string; + elasticsearchUrl: string; + templateUrl: string; + isPrimaryAction: boolean; +} + +export function CreateStackInAWSConsole({ + encodedApiKey, + elasticsearchUrl, + templateUrl, + isPrimaryAction, +}: Props) { + const awsConsoleURL = buildCreateStackAWSConsoleURL({ + templateUrl, + stackName: FIREHOSE_CLOUDFORMATION_STACK_NAME, + logsStreamName: FIREHOSE_LOGS_STREAM_NAME, + metricsStreamName: FIREHOSE_METRICS_STREAM_NAME, + elasticsearchUrl, + encodedApiKey, + }); + + return ( + <> + +

+ +

+

+ +

+ + + + + + {i18n.translate( + 'xpack.observability_onboarding.createStackInAWSConsole.createFirehoseStreamInAWSConsoleButtonLabel', + { defaultMessage: 'Create Firehose Stream in AWS' } + )} + + + ); +} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/download_template_callout.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/download_template_callout.tsx new file mode 100644 index 0000000000000..9a6e9b97bde84 --- /dev/null +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/download_template_callout.tsx @@ -0,0 +1,35 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0; you may not use this file except in compliance with the Elastic License + * 2.0. + */ + +import React from 'react'; +import { EuiLink } from '@elastic/eui'; +import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n-react'; +import { FIREHOSE_CLOUDFORMATION_TEMPLATE_URL } from '../../../../common/aws_firehose'; + +export function DownloadTemplateCallout() { + return ( + + {i18n.translate( + 'xpack.observability_onboarding.firehosePanel.downloadCloudFormationTemplateButtonLabel', + { defaultMessage: 'download and modify the CloudFormation template' } + )} + + ), + }} + /> + ); +} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/index.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/index.tsx index 520171c835808..00a5b582ba1e7 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/index.tsx +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/index.tsx @@ -5,23 +5,57 @@ * 2.0. */ -import React, { useState } from 'react'; +import React, { useCallback, useState } from 'react'; +import { i18n } from '@kbn/i18n'; import { + EuiButtonGroup, + EuiLink, EuiPanel, EuiSkeletonRectangle, EuiSkeletonText, EuiSpacer, EuiSteps, EuiStepStatus, + EuiText, } from '@elastic/eui'; import useEvent from 'react-use/lib/useEvent'; +import { FormattedMessage } from '@kbn/i18n-react'; import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher'; import { EmptyPrompt } from '../shared/empty_prompt'; import { CreateStackCommandSnippet } from './create_stack_command_snippet'; import { VisualizeData } from './visualize_data'; +import { CreateStackInAWSConsole } from './create_stack_in_aws_console'; +import { FeedbackButtons } from '../shared/feedback_buttons'; + +enum CreateStackOption { + AWS_CONSOLE_UI = 'createCloudFormationOptionAWSConsoleUI', + AWS_CLI = 'createCloudFormationOptionAWSCLI', +} + +const OPTIONS = [ + { + id: CreateStackOption.AWS_CONSOLE_UI, + label: i18n.translate( + 'xpack.observability_onboarding.firehosePanel.createStackAWSConsoleOptionLabel', + { + defaultMessage: 'Via AWS Console', + } + ), + }, + { + id: CreateStackOption.AWS_CLI, + label: i18n.translate( + 'xpack.observability_onboarding.firehosePanel.createStackAWSCLIOptionLabel', + { defaultMessage: 'Via AWS CLI' } + ), + }, +]; export function FirehosePanel() { const [windowLostFocus, setWindowLostFocus] = useState(false); + const [selectedOptionId, setSelectedOptionId] = useState( + CreateStackOption.AWS_CONSOLE_UI + ); const { data, status, error, refetch } = useFetcher( (callApi) => { return callApi('POST /internal/observability_onboarding/firehose/flow'); @@ -32,6 +66,10 @@ export function FirehosePanel() { useEvent('blur', () => setWindowLostFocus(true), window); + const onOptionChange = useCallback((id: string) => { + setSelectedOptionId(id as CreateStackOption); + }, []); + if (error !== undefined) { return ; } @@ -41,7 +79,45 @@ export function FirehosePanel() { const steps = [ { - title: 'Create a Firehose delivery stream and ingest CloudWatch logs', + title: i18n.translate('xpack.observability_onboarding.firehosePanel.prerequisitesTitle', { + defaultMessage: 'Prerequisites', + }), + children: ( + <> + +

+ +

+

+ + {i18n.translate( + 'xpack.observability_onboarding.firehosePanel.documentationLinkLabel', + { defaultMessage: 'Check the documentation' } + )} + + ), + }} + /> +

+ + + ), + }, + { + title: 'Create a Firehose delivery stream to ingest CloudWatch logs and metrics', children: ( <> {status !== FETCH_STATUS.SUCCESS && ( @@ -52,13 +128,41 @@ export function FirehosePanel() { )} {status === FETCH_STATUS.SUCCESS && data !== undefined && ( - + <> + + + + + {selectedOptionId === CreateStackOption.AWS_CONSOLE_UI && ( + + )} + + {selectedOptionId === CreateStackOption.AWS_CLI && ( + + )} + )} ), @@ -73,6 +177,7 @@ export function FirehosePanel() { return ( + ); } diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/progress_callout.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/progress_callout.tsx new file mode 100644 index 0000000000000..b520515c2635d --- /dev/null +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/progress_callout.tsx @@ -0,0 +1,87 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0; you may not use this file except in compliance with the Elastic License + * 2.0. + */ + +import { EuiFlexGroup, EuiText, EuiIconTip, EuiHorizontalRule } from '@elastic/eui'; +import { i18n } from '@kbn/i18n'; +import React from 'react'; +import { css } from '@emotion/react'; +import { ProgressIndicator } from '../shared/progress_indicator'; + +const SUPPORTED_SERVICES = [ + 'API Gateway', + 'AWS Usage', + 'CloudTrail', + 'DynamoDB', + 'EBS', + 'EC2', + 'ECS', + 'ELB', + 'EMR', + 'Kinesis Data Stream', + 'Lambda', + 'MSK', + 'NAT Gateway', + 'RDS', + 'Route53', + 'S3', + 'SNS', + 'SQS', + 'VPC', + 'VPN', +]; + +export function ProgressCallout() { + return ( + + +

+ {i18n.translate('xpack.observability_onboarding.firehosePanel.waitingForDataTitle', { + defaultMessage: 'Retrieving data from Amazon Data Firehose', + })} +

+ + + + {i18n.translate( + 'xpack.observability_onboarding.progressCallout.strong.allServicesWeCanLabel', + { defaultMessage: 'All services we can detect' } + )} + + +
    + {SUPPORTED_SERVICES.map((service) => ( +
  • {service}
    • + ))} +
  • + {i18n.translate( + 'xpack.observability_onboarding.progressCallout.li.otherLabel', + { + defaultMessage: + 'Other (Unsupported logs will be stored in a generic Firehose index).', + } + )} +
    • +
+ + } + position="top" + type="iInCircle" + /> + + } + isLoading={true} + css={css` + display: inline-block; + `} + /> + ); +} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/use_aws_service_get_started_list.ts b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/use_aws_service_get_started_list.ts index 2aa08f7a6bed9..277c565986d3c 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/use_aws_service_get_started_list.ts +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/use_aws_service_get_started_list.ts @@ -98,7 +98,7 @@ export function useAWSServiceGetStartedList(): AWSServiceGetStartedConfig[] { const generateMetricsDiscoverActionLink = useCallback( (namespace: string, name: string) => ({ - id: `logs-explorer-${namespace}`, + id: `discover-${namespace}`, title: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreDataTitle', { defaultMessage: 'See {name} metrics data in Discover', values: { name }, diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/utils.ts b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/utils.ts index 7fdfe5890830b..0fa28276f7fcc 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/utils.ts +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/utils.ts @@ -5,6 +5,8 @@ * 2.0. */ +export const HAS_DATA_FETCH_INTERVAL = 5000; + export function buildCreateStackCommand({ templateUrl, stackName, @@ -48,3 +50,41 @@ export function buildStackStatusCommand({ stackName }: { stackName: string }) { .replace(/\n/g, ' ') .replace(/\s\s+/g, ' '); } + +export function buildCreateStackAWSConsoleURL({ + templateUrl, + stackName, + logsStreamName, + metricsStreamName, + elasticsearchUrl, + encodedApiKey, +}: { + templateUrl: string; + stackName: string; + logsStreamName: string; + metricsStreamName: string; + elasticsearchUrl: string; + encodedApiKey: string; +}): string { + const url = new URL('https://console.aws.amazon.com'); + const params = new URLSearchParams({ + templateURL: templateUrl, + stackName, + /** + * 'param_' format is enforced by AWS + * but template parameters are in CamelCase + * which triggers the eslint rule. + */ + /* eslint-disable @typescript-eslint/naming-convention */ + param_FirehoseStreamNameForLogs: logsStreamName, + param_FirehoseStreamNameForMetrics: metricsStreamName, + param_ElasticEndpointURL: elasticsearchUrl, + param_ElasticAPIKey: encodedApiKey, + /* eslint-enable @typescript-eslint/naming-convention */ + }); + + url.pathname = '/cloudformation/home'; + url.hash = `/stacks/quickcreate?${params.toString()}`; + + return url.toString(); +} diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/visualize_data.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/visualize_data.tsx index aee32dee4fa95..ca16519eb6cc2 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/visualize_data.tsx +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/firehose/visualize_data.tsx @@ -5,27 +5,35 @@ * 2.0. */ -import { EuiIcon, EuiSpacer, useEuiTheme, useGeneratedHtmlId } from '@elastic/eui'; -import { css } from '@emotion/react'; +import { EuiIcon, EuiSpacer, EuiText, useGeneratedHtmlId } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; -import React from 'react'; +import React, { useEffect, useState } from 'react'; import useInterval from 'react-use/lib/useInterval'; +import { union } from 'lodash'; +import { useKibana } from '@kbn/kibana-react-plugin/public'; +import { FormattedMessage } from '@kbn/i18n-react'; +import { ObservabilityOnboardingAppServices } from '../../..'; import { FIREHOSE_CLOUDFORMATION_STACK_NAME, FIREHOSE_LOGS_STREAM_NAME, + type AWSIndexName, } from '../../../../common/aws_firehose'; import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher'; import { AccordionWithIcon } from '../shared/accordion_with_icon'; import { GetStartedPanel } from '../shared/get_started_panel'; -import { ProgressIndicator } from '../shared/progress_indicator'; import { useAWSServiceGetStartedList } from './use_aws_service_get_started_list'; +import { AutoRefreshCallout } from './auto_refresh_callout'; +import { ProgressCallout } from './progress_callout'; +import { HAS_DATA_FETCH_INTERVAL } from './utils'; -const FETCH_INTERVAL = 2000; const REQUEST_PENDING_STATUS_LIST = [FETCH_STATUS.LOADING, FETCH_STATUS.NOT_INITIATED]; export function VisualizeData() { const accordionId = useGeneratedHtmlId({ prefix: 'accordion' }); - const { euiTheme } = useEuiTheme(); + const [orderedPopulatedAWSLogsIndexList, setOrderedPopulatedAWSLogsIndexList] = useState< + AWSIndexName[] + >([]); + const [shouldShowDataReceivedToast, setShouldShowDataReceivedToast] = useState(true); const { data: populatedAWSLogsIndexList, status, @@ -40,6 +48,49 @@ export function VisualizeData() { }, }); }, []); + const { + services: { notifications }, + } = useKibana(); + + useEffect(() => { + if ( + shouldShowDataReceivedToast && + Array.isArray(populatedAWSLogsIndexList) && + populatedAWSLogsIndexList.length > 0 + ) { + notifications?.toasts.addSuccess( + { + title: i18n.translate( + 'xpack.observability_onboarding.firehosePanel.dataReceivedToastTitle', + { + defaultMessage: 'Your data is on its way', + } + ), + text: i18n.translate( + 'xpack.observability_onboarding.firehosePanel.dataReceivedToastText', + { + defaultMessage: + 'We’ve begun processing your data. In the background, we automatically refresh every few seconds to capture more incoming data.', + } + ), + }, + { + toastLifeTimeMs: 10000, + } + ); + setShouldShowDataReceivedToast(false); + } + + setOrderedPopulatedAWSLogsIndexList((currentList) => + /** + * Using union() to ensure items in the array are unique + * add stay in the insertion order to keep the order of + * the AWS services in the UI. + */ + union(currentList, populatedAWSLogsIndexList) + ); + }, [notifications?.toasts, populatedAWSLogsIndexList, shouldShowDataReceivedToast]); + const awsServiceGetStartedConfigList = useAWSServiceGetStartedList(); useInterval(() => { @@ -48,7 +99,7 @@ export function VisualizeData() { } refetch(); - }, FETCH_INTERVAL); + }, HAS_DATA_FETCH_INTERVAL); if (populatedAWSLogsIndexList === undefined) { return null; @@ -56,58 +107,58 @@ export function VisualizeData() { return ( <> - + +

+ +

+ - + + + {orderedPopulatedAWSLogsIndexList.length === 0 && } + {orderedPopulatedAWSLogsIndexList.length > 0 && } + +
- {awsServiceGetStartedConfigList.map( - ({ id, indexNameList, actionLinks, title, logoURL, previewImage }) => { - const isEnabled = indexNameList.some((indexName) => - populatedAWSLogsIndexList.includes(indexName) - ); - - return ( - } - title={i18n.translate( - 'xpack.observability_onboarding.firehosePanel.awsServiceDataFoundTitle', - { - defaultMessage: '{title}', - values: { title }, - } - )} - extraAction={ - isEnabled ? : null - } - isDisabled={!isEnabled} - css={{ - paddingRight: euiTheme.size.s, - filter: `grayscale(${isEnabled ? 0 : 1})`, - }} - > - - - ); + {orderedPopulatedAWSLogsIndexList.map((indexName, index) => { + const getStartedConfig = awsServiceGetStartedConfigList.find(({ indexNameList }) => + indexNameList.includes(indexName) + ); + + if (!getStartedConfig) { + return null; } - )} + + const { id, actionLinks, title, logoURL, previewImage } = getStartedConfig; + + return ( + } + title={title} + initialIsOpen={true} + borders={ + index === 0 || index === orderedPopulatedAWSLogsIndexList.length - 1 + ? 'none' + : 'horizontal' + } + > + + + ); + })}
); diff --git a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/shared/accordion_with_icon.tsx b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/shared/accordion_with_icon.tsx index 2f18c299bbc56..5ac8f2bb05792 100644 --- a/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/shared/accordion_with_icon.tsx +++ b/x-pack/plugins/observability_solution/observability_onboarding/public/application/quickstart_flows/shared/accordion_with_icon.tsx @@ -15,13 +15,14 @@ import { } from '@elastic/eui'; interface AccordionWithIconProps - extends Omit { + extends Omit { title: string; icon: React.ReactNode; } export const AccordionWithIcon: FunctionComponent = ({ title, icon, + borders = 'horizontal', children, ...rest }) => { @@ -39,7 +40,7 @@ export const AccordionWithIcon: FunctionComponent = ({ } buttonProps={{ paddingSize: 'l' }} - borders="horizontal" + borders={borders} paddingSize="none" >
{children}
diff --git a/x-pack/test_serverless/functional/test_suites/observability/onboarding/firehose.ts b/x-pack/test_serverless/functional/test_suites/observability/onboarding/firehose.ts index 313dd001c4f4a..7380de0dfc2f3 100644 --- a/x-pack/test_serverless/functional/test_suites/observability/onboarding/firehose.ts +++ b/x-pack/test_serverless/functional/test_suites/observability/onboarding/firehose.ts @@ -38,6 +38,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) { }); beforeEach(async () => { + await (await testSubjects.find('createCloudFormationOptionAWSCLI')).click(); await testSubjects.existOrFail('observabilityOnboardingFirehoseCreateStackCommand'); }); @@ -51,10 +52,10 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) { it('starts to monitor for incoming data after user leaves the page', async () => { await browser.execute(`window.dispatchEvent(new Event("blur"))`); - await testSubjects.isDisplayed('observabilityOnboardingAWSServiceList'); + await testSubjects.isDisplayed('observabilityOnboardingFirehoseProgressCallout'); }); - it('highlights an AWS service when data is detected', async () => { + it('shows an AWS service when data is detected', async () => { const DATASET = 'aws.vpcflow'; const AWS_SERVICE_ID = 'vpc-flow'; await testSubjects.clickWhenNotDisabled('observabilityOnboardingCopyToClipboardButton'); @@ -79,11 +80,8 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) { }) ); - // Checking that an AWS service item is enabled after data is detected - await testSubjects - .find(`observabilityOnboardingAWSService-${AWS_SERVICE_ID}`) - .then((el) => el.findByTagName('button')) - .then((el) => el.isEnabled()); + // Checking that an AWS service item is visible after data is detected + await testSubjects.isDisplayed(`observabilityOnboardingAWSService-${AWS_SERVICE_ID}`); }); }); }