Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDSaaS: StreamNative: Crawler Integration v1.0.0 #18782

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
7137dad
DDSaaS: StreamNative: Crawler integration without Assets
narendranandaniya-crest Oct 1, 2024
f34e5c9
initial commit
narendranandaniya-crest Oct 7, 2024
462c200
Sorted metadata.csv file
narendranandaniya-crest Oct 7, 2024
c8f5029
updated image path
narendranandaniya-crest Oct 7, 2024
2fcd238
sorted metdata file
narendranandaniya-crest Oct 7, 2024
0a338ca
updated metadata file
narendranandaniya-crest Oct 7, 2024
19eb166
fixed metadata failure
narendranandaniya-crest Oct 7, 2024
f405191
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Oct 8, 2024
9e4ec3a
updated manifest file
narendranandaniya-crest Oct 8, 2024
8da2014
Merge branch 'streamnative-assets-v1.0.0' of https://github.com/bharg…
narendranandaniya-crest Oct 8, 2024
6f70e58
updated readme steps
narendranandaniya-crest Oct 8, 2024
001b23e
added metric check
narendranandaniya-crest Oct 8, 2024
16f0cfb
updated manifest
narendranandaniya-crest Oct 8, 2024
0d4c180
updated readme
narendranandaniya-crest Oct 10, 2024
587acd1
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Oct 10, 2024
a6fa123
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 7, 2024
cd2f120
updated readme & monitor as per review comments
narendranandaniya-crest Nov 7, 2024
7666e7a
updated readme & monitor names
narendranandaniya-crest Nov 7, 2024
64fbe4e
updated asset files as per review
narendranandaniya-crest Nov 12, 2024
e39ece2
updated readme as per review
narendranandaniya-crest Nov 14, 2024
5b321b7
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 15, 2024
462056e
updated readme
narendranandaniya-crest Nov 15, 2024
82ef7d7
updated readme file
narendranandaniya-crest Nov 20, 2024
435e804
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,11 @@ vonage/*.md @DataDog/saas-integration
vonage/manifest.json @DataDog/saas-integrations @DataDog/documentation
vonage/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend

/streamnative/ @DataDog/saas-integrations
/streamnative/*.md @DataDog/saas-integrations @DataDog/documentation
/streamnative/manifest.json @DataDog/saas-integrations @DataDog/documentation
/streamnative/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend

# To keep Security up-to-date with changes to the signing tool.
/datadog_checks_dev/datadog_checks/dev/tooling/signing.py @DataDog/agent-integrations
# As well as the secure downloader.
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/config/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,8 @@ integration/ssh_check:
- ssh_check/**/*
integration/statsd:
- statsd/**/*
integration/streamnative:
- streamnative/**/*
integration/strimzi:
- strimzi/**/*
integration/supervisord:
Expand Down
7 changes: 7 additions & 0 deletions streamnative/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# CHANGELOG - StreamNative

## 1.0.0 / 2024-10-01

***Added***:

* Initial Release
69 changes: 69 additions & 0 deletions streamnative/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# StreamNative

## Overview

[StreamNative][1] provides an enterprise-grade messaging and event streaming platform built on Apache Pulsar. It offers scalable, real-time data streaming solutions with features like multi-tenancy, geo-replication, and seamless integration with cloud services.

The StreamNative integration collects the below types of [metrics][2]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"collects the below" > Collects the following types of metrics

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


1. Health
2. Pulsar Resource
3. Source Connector
4. Sink Connector
5. Kafka Connect

## Setup

### Configuration

#### Get StreamNative credentials

##### Log in to the [StreamNative Cloud Console Account][3].
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
##### Log in to the [StreamNative Cloud Console Account][3].
Log into the [StreamNative Cloud Console Account][3].

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

##### Get the `Organization ID` and `Instance Name`:
1. Click the profile icon and select the **Organizations** option.
2. Select the **Organization** for which data needs to be collected.
3. Obtain the **Instance Name** from the **Select an Instance** dropdown.
##### Get the `Client ID` and `Client Secret`:
1. Click the profile icon and select the **Accounts & Accesses** tab.
2. Select the Service Account for which **Admin** permission is **Enabled**.
3. If the Service Account does not exist, click on **New -> Service Account** to create a new one, and ensure the **Super Admin** option is enabled.
4. Click on the `...` button on the right side of the selected Service Account.
5. Click **Download OAuth2 Key** to get the **Client ID** and **Client Secret**.


#### Add StreamNative credentials
Copy link
Contributor

@apiazza-dd apiazza-dd Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add more clarity here. What do you think of the following:

1. Click the profile icon and select **Organizations**.
2. Choose the **Organization** for which data needs to be collected.
3. From the **Select an Instance** dropdown, note the **Instance Name**.

#### Get the `Client ID` and `Client Secret`:

1. Click the profile icon and navigate to the **Accounts & Accesses** tab.
2. Find the Service Account with **Admin** permissions set to **Enabled**.
   - If no Service Account exists, select **New -> Service Account** to create one, and make sure to enable the **Super Admin** option.
3. On the right side of the chosen Service Account, click the `...` button.
4. Select **Download OAuth2 Key** to obtain the **Client ID** and **Client Secret**.



### Setting Up the Integration Tile

After gathering the necessary details, add the following Stream Native credentials to the integration tile

- **Organization ID**
- **Instance Name**
- **Client ID**
- **Client Secret**

Copy link

@ashah-crest ashah-crest Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @apiazza-dd, we have updated the first half including the steps to get the configuration details.
Regarding the second part from "Setting Up the Integration Tile", earlier on the similar lines we have had an discussion around this over slack with @dpoint01 ( ref ). Can you please have a look. thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashah-crest Thanks, I don't have access to that slack environment, but if @dpoint01 approved, lgtm.


- Organization ID
- Instance Name
- Client ID
- Client Secret


## Data Collected
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved

### Logs

The StreamNative integration does not include any logs.

### Metrics

The StreamNative integration collects and forwards the following metrics to Datadog.

{{< get-metrics-from-git "streamnative" >}}

### Service Checks

The StreamNative integration does not include any service checks.

### Events

The StreamNative integration does not include any events.

## Troubleshooting

Need help? Contact [Datadog support][4].

[1]: https://streamnative.io/
[2]: https://docs.streamnative.io/docs/cloud-metrics-api#metrics-endpoint
[3]: https://console.streamnative.cloud/
[4]: https://docs.datadoghq.com/help/
Loading
Loading