Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDSaaS: StreamNative: Crawler Integration v1.0.0 #18782

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
7137dad
DDSaaS: StreamNative: Crawler integration without Assets
narendranandaniya-crest Oct 1, 2024
f34e5c9
initial commit
narendranandaniya-crest Oct 7, 2024
462c200
Sorted metadata.csv file
narendranandaniya-crest Oct 7, 2024
c8f5029
updated image path
narendranandaniya-crest Oct 7, 2024
2fcd238
sorted metdata file
narendranandaniya-crest Oct 7, 2024
0a338ca
updated metadata file
narendranandaniya-crest Oct 7, 2024
19eb166
fixed metadata failure
narendranandaniya-crest Oct 7, 2024
f405191
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Oct 8, 2024
9e4ec3a
updated manifest file
narendranandaniya-crest Oct 8, 2024
8da2014
Merge branch 'streamnative-assets-v1.0.0' of https://github.com/bharg…
narendranandaniya-crest Oct 8, 2024
6f70e58
updated readme steps
narendranandaniya-crest Oct 8, 2024
001b23e
added metric check
narendranandaniya-crest Oct 8, 2024
16f0cfb
updated manifest
narendranandaniya-crest Oct 8, 2024
0d4c180
updated readme
narendranandaniya-crest Oct 10, 2024
587acd1
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Oct 10, 2024
a6fa123
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 7, 2024
cd2f120
updated readme & monitor as per review comments
narendranandaniya-crest Nov 7, 2024
7666e7a
updated readme & monitor names
narendranandaniya-crest Nov 7, 2024
64fbe4e
updated asset files as per review
narendranandaniya-crest Nov 12, 2024
e39ece2
updated readme as per review
narendranandaniya-crest Nov 14, 2024
5b321b7
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 15, 2024
462056e
updated readme
narendranandaniya-crest Nov 15, 2024
82ef7d7
updated readme file
narendranandaniya-crest Nov 20, 2024
435e804
Merge branch 'master' into streamnative-assets-v1.0.0
narendranandaniya-crest Nov 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,11 @@ vonage/*.md @DataDog/saas-integration
vonage/manifest.json @DataDog/saas-integrations @DataDog/documentation
vonage/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend

/streamnative/ @DataDog/saas-integrations
/streamnative/*.md @DataDog/saas-integrations @DataDog/documentation
/streamnative/manifest.json @DataDog/saas-integrations @DataDog/documentation
/streamnative/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend

# To keep Security up-to-date with changes to the signing tool.
/datadog_checks_dev/datadog_checks/dev/tooling/signing.py @DataDog/agent-integrations
# As well as the secure downloader.
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/config/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,8 @@ integration/ssh_check:
- ssh_check/**/*
integration/statsd:
- statsd/**/*
integration/streamnative:
- streamnative/**/*
integration/strimzi:
- strimzi/**/*
integration/supervisord:
Expand Down
7 changes: 7 additions & 0 deletions streamnative/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# CHANGELOG - StreamNative

## 1.0.0 / 2024-10-01

***Added***:

* Initial Release
73 changes: 73 additions & 0 deletions streamnative/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# StreamNative

## Overview

[StreamNative][1] provides an enterprise-grade messaging and event streaming platform built on Apache Pulsar. It offers scalable, real-time data streaming solutions with features like multi-tenancy, geo-replication, and seamless integration with cloud services.

The StreamNative integration collects below types of [metrics][2]:
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved

1. Health
2. Pulsar Resource
3. Source Connector
4. Sink Connector
5. Kafka Connect

## Setup

### Configuration

#### Get StreamNative Credentials
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved

1. Login to the [StreamNative Cloud Console Account][3].
2. Get the `Organization ID` and `Instance Name`
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved
- Click the profile icon and select the **Organizations** option.
- Select the **Organization** for which data needs to be collected.
- Obtain the **Instance Name** from the **Select an Instance** dropdown.
3. Get the `Client ID` and `Client Secret`
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved
- Click the profile icon and select the **Accounts & Accesses** tab.
- Select the Service Account for which **Admin** permission is **Enabled**.
- If the Service Account does not exist, click on **New -> Service Account** to create a new one, and ensure the **Super Admin** option is enabled.
- Click on `...` button on the right side of the selected Service Account.
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved
- Click **Download OAuth2 Key** to get the **Client ID** and **Client Secret**.


#### Add StreamNative Credentials
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved

- Organization ID
- Instance Name
- Client ID
- Client Secret


## Data Collected
jhgilbert marked this conversation as resolved.
Show resolved Hide resolved

### Logs

The StreamNative integration does not include any logs.

### Metrics

The StreamNative integration collects and forwards the following metrics to Datadog.

1. Health

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@apiazza-dd is this where we should have the table view with all the metric names + descriptions for the public docs?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Metrics section on the public docs should reflect a table view of all the metrics collected that show up on the "Data collected" tab on the tile (from the metadata.csv)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see comment above

2. Pulsar Resource
3. Source Connector
4. Sink Connector
5. Kafka Connect

### Service Checks

The StreamNative integration does not include any service checks.

### Events

The StreamNative integration does not include any events.

## Troubleshooting

Need help? Contact [Datadog support][4].

[1]: https://streamnative.io/
[2]: https://docs.streamnative.io/docs/cloud-metrics-api#metrics-endpoint
[3]: https://console.streamnative.cloud/
[4]: https://docs.datadoghq.com/help/
Loading
Loading