Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MDS] Add security analytics, alerting, feature anaywhere in the multiple data source document #7328

Merged
merged 67 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
a0bc430
Add security analy
yujin-emma Jun 6, 2024
3af10a4
Merge branch 'main' into sa
yujin-emma Jun 7, 2024
1ba563b
Apply suggestions from code review
Naarcha-AWS Jun 7, 2024
051c523
Update multi-data-sources.md
yujin-emma Jun 7, 2024
2ad12d1
Apply suggestions from code review
Naarcha-AWS Jun 11, 2024
c2f6003
Update multi-data-sources.md
yujin-emma Jun 11, 2024
e55e2d8
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
71bb0dc
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
de46d92
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
bb605a4
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
6619933
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
d07d702
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
1276cf8
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
59d79c1
Merge branch 'main' into sa
vagimeli Jun 12, 2024
f1abacd
Update _dashboards/management/multi-data-sources.md
yujin-emma Jun 12, 2024
ae853c5
Update multi-data-sources.md
vagimeli Jun 12, 2024
5107bb0
Merge branch 'main' into sa
vagimeli Jun 12, 2024
48fc7a8
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 12, 2024
72f8bf6
Update multi-data-sources.md
vagimeli Jun 12, 2024
531e66b
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
148e15a
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
fdd2837
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
b73357e
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
42bc080
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
0858d58
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
9880fc5
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
7019a5d
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
1297790
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
052715b
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
b98ec37
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
f9cef41
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
b92ba78
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
7434145
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
39c1ab7
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
093472d
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
82b5fc8
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
f488316
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
1dbab05
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
37e6c42
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
6397a03
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
b03f9e4
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
bf52847
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
1b9f61f
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
3000d71
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
fe958db
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
d70c9cd
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
64d1d98
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
c752c26
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
e35fcd7
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
eda8f14
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
d8d608f
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
899719f
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
c079f21
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
c8c6d47
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
dd92084
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
0a0e0d0
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
54a9a1f
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
7161af7
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
85d550d
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
4438b19
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
b966339
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
c327ec8
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
eabaf33
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
e33eb01
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
275aa70
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
161304c
Merge branch 'main' into sa
vagimeli Jun 13, 2024
4c8d581
Update _dashboards/management/multi-data-sources.md
vagimeli Jun 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 39 additions & 10 deletions _dashboards/management/multi-data-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
To create a new data source connection:

1. From the OpenSearch Dashboards main menu, select **Dashboards Management** > **Data sources** > **Create data source connection**.

2. Add the required information to each field to configure the **Connection Details** and **Authentication Method**.

- Under **Connection Details**, enter a title and endpoint URL. For this tutorial, use the URL `https://localhost:9200/`. Entering a description is optional.
Expand All @@ -51,16 +51,16 @@

- After you have entered the appropriate details in all of the required fields, the **Test connection** and **Create data source** buttons become active. You can select **Test connection** to confirm that the connection is valid.

3. Select **Create data source** to save your settings. The connection is created, and the new data source appears in the list on the **Data Sources** main page. The first data source you create is marked as your default.
3. Select **Create data source** to save your settings. The connection is created, and the new data source appears in the list on the **Data Sources** main page. The first data source you create is marked as your default.

4. Edit or update a data source connection.

- On the **Data Sources** main page, select the connection you want to modify. The **Connection Details** window opens.

- To mark the selected data source as the default, select the **Set as default** option.
- To mark the selected data source as the default, select the **Set as default** option.

- To make changes to **Connection Details**, edit one or both of the **Title** and **Description** fields and select **Save changes** in the lower-right corner of the screen. You can also cancel changes here. To change the **Authentication Method**, choose a different authentication method, enter your credentials (if applicable), and then select **Save changes** in the lower-right corner of the screen. The changes are saved.

- When **Username & Password** is the selected authentication method, you can update the password by choosing **Update stored password** next to the **Password** field. In the pop-up window, enter a new password in the first field and then enter it again in the second field to confirm. Select **Update stored password** in the pop-up window. The new password is saved. Select **Test connection** to confirm that the connection is valid.
- When **AWS SigV4** is the selected authentication method, you can update the credentials by selecting **Update stored AWS credential**. In the pop-up window, enter a new access key in the first field and a new secret key in the second field. Select **Update stored AWS credential** in the pop-up window. The new credentials are saved. Select **Test connection** in the upper-right corner of the screen to confirm that the connection is valid.

Expand Down Expand Up @@ -113,7 +113,7 @@
Introduced 2.13
{: .label .label-purple }

A feature flag in your `opensearch_dashboards.yml` file allows you to show or hide authentication methods within the `data_source` plugin. The following example setting, shown in a 10-second demo, hides the authentication method for `AWSSigV4`.
A feature flag in your `opensearch_dashboards.yml` file allows you to show or hide authentication methods within the `data_source` plugin. The following example setting, shown in a 10-second demo, hides the authentication method for `AWSSigV4`.

````
# Set enabled to false to hide the authentication method from multiple data source in OpenSearch Dashboards.
Expand All @@ -134,10 +134,10 @@
Introduced 2.13
{: .label .label-purple }

A feature flag in your `opensearch_dashboards.yml` file allows you to hide the local cluster option within the `data_source` plugin. This option hides the local cluster from the data source dropdown menu and index creation page, which is ideal for environments with or without a local OpenSearch cluster. The following example setting, shown in a 20-second demo, hides the local cluster.
A feature flag in your `opensearch_dashboards.yml` file allows you to hide the local cluster option within the `data_source` plugin. This option hides the local cluster from the data source dropdown menu and index creation page, which is ideal for environments with or without a local OpenSearch cluster. The following example setting, shown in a 20-second demo, hides the local cluster:

````
# hide local cluster in the data source dropdown and index pattern creation page.
# hide local cluster in the data source dropdown and index pattern creation page.
yujin-emma marked this conversation as resolved.
Show resolved Hide resolved
data_source.hideLocalCluster: true
````

Expand All @@ -155,7 +155,7 @@

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/ism_mds1.gif" alt="Multiple data sources in ISM list page"/>

To perform operations on a specific index within a data source, select the individual index from the list. To create a new index, select the **Create Index** button, which opens a form. Fill in the required information and select the **Create** button. The index is created within the selected data source. The following GIF illustrates these steps.
To perform operations on a specific index within a data source, select the individual index from the list. To create a new index, select the **Create Index** button, which opens a form. Fill in the required information and select the **Create** button. The index is created within the selected data source. The following GIF illustrates these steps.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/ism_mds2.gif" alt="Multiple data sources in ISM create page"/>

Expand Down Expand Up @@ -195,11 +195,40 @@

### Search relevance

When the data source feature is enabled, you can navigate to **Search Relevance** under the **OpenSearch Plugins** menu. On the navigation bar on the upper right, you can view all available data sources. To compare search results between indexes from different data sources, first select a data source and an index for **Query 1**, and then select a data source and an index for **Query 2**. Select **Search** to run the queries. The following GIF illustrates these steps.
When the data source feature is enabled, you can navigate to **Search Relevance** under the **OpenSearch Plugins** menu. On the navigation bar on the upper right, you can view all available data sources. To compare search results between indexes from different data sources, first select a data source and an index for **Query 1**, and then select a data source and an index for **Query 2**. Select **Search** to run the queries. The following GIF illustrates these steps.
yujin-emma marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/searchrelevance_mds1.gif" alt="Multiple data sources in Search Relevance plugin"/>

## Next steps

### Security analytics

When the data source feature is enabled, you can navigate to **Security analytics** under the **OpenSearch Plugins** menu. Using detection rules as an example, you can view all connected data sources and select a specific one from the navigation bar on the upper right. By default, the detection rules from the designated default data source are displayed. However, you can select any connected data source to view its corresponding detection rules. The following GIF illustrates these steps.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/mds_sa_detection_rules_view.gif" alt="Multiple data sources in Security analytics list page"/>

To create a new detection rule, select the **Create detection rule** button, which opens a form. Fill in the required information and select the **Create detection rule** button. The detection rule is created within the selected data source. The following GIF illustrates these steps.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/mds_sa_detection_rules_create.gif" alt="Multiple data sources in Security analytics create page"/>

vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Alerting

When the data source feature is enabled, you can navigate to **Alerting** under the **OpenSearch Plugins** menu. Using monitors as an example, you can view all connected data sources and select a specific one from the navigation bar on the upper right. By default, the monitors from the designated default data source are displayed. However, you can select any connected data source to view its corresponding monitors. The following GIF illustrates these steps.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/mds_monitor_view.gif" alt="Multiple data sources in Alerting list page"/>

To create a new monitor, select the **Create monitor** button, which opens a form. Fill in the required information and select the **Create** button. The monitor is created within the selected data source.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

### Feature anywhere
Copy link
Contributor

@vagimeli vagimeli Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yujin-emma Is this a proper name "Feature Anywhere"? If not, what does feature anywhere mean to the user?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a feature allows https://opensearch.org/docs/latest/observing-your-data/ad/dashboards-anomaly-detection/ and https://opensearch.org/docs/latest/observing-your-data/alerting/dashboards-alerting/. It supports multi data source in 2.15. Looks like we do call it feature anywhere in the repo as project name, opensearch-project/OpenSearch-Dashboards#3481, do you have suggestion with a better name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yujin-emma Thank you for those resources. I'll add some context around Feature Anywhere.


When the data source feature is enabled, you can navigate to **Dashboards** under the **OpenSearch Dashboards** menu. Select a dashboard from the list, click the option button of the dashboads, then click on **Alerting** -> **Associated monitors**. Then you can see the associated monitors with the selected dashboard. The following GIF illustrates these steps.

Check failure on line 224 in _dashboards/management/multi-data-sources.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: dashboads. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: dashboads. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_dashboards/management/multi-data-sources.md", "range": {"start": {"line": 224, "column": 184}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/mds_feature_anywhere_view_alerting.gif" alt="Multiple data sources with Feature anywhere associated monitor"/>

To add new alerting monitor for that dashboard, select the option button, which opens a flyout. Fill in the required information and select the **Create monitor** button. The alerting monitor is created within the selected data source.The following GIF illustrates these steps.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/mds_feature_anywhere_create_alerting.gif" alt="Multiple data sources with Feature anywhere add associated monitor"/>
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
## Next steps

After configuring multiple data sources, you can analyze the data from each source. Refer to the following resources for more information:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/dashboards/mds_monitor_view.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading