Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refresh intakes documentation #1892

Merged
merged 1 commit into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ In details, the following table denotes the type of events produced by this inte

### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/docs/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/docs/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/docs/xdr/features/detect/sigma) and to leverage the full potential of the collected data.
This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "gke_container_runtime2.json"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The following table lists the data source offered by this integration.

### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/docs/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/docs/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/docs/xdr/features/detect/sigma) and to leverage the full potential of the collected data.
This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "auth-action-changed-login-id-to.json"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ In details, the following table denotes the type of events produced by this inte

### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/docs/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/docs/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/docs/xdr/features/detect/sigma) and to leverage the full potential of the collected data.
This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "test_aaatm.json"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ In details, the following table denotes the type of events produced by this inte

### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/docs/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/docs/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/docs/xdr/features/detect/sigma) and to leverage the full potential of the collected data.
This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "amsi_detected_harmful_content.json"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

## Event Categories
### Event Categories


The following table lists the data source offered by this integration.
Expand All @@ -23,10 +23,9 @@ In details, the following table denotes the type of events produced by this inte



## Event Samples

Find below few samples of events and how they are normalized by Sekoia.io.
### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "test_journal.json"

Expand Down Expand Up @@ -216,7 +215,7 @@ Find below few samples of events and how they are normalized by Sekoia.io.



## Extracted Fields
### Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Expand Down Expand Up @@ -245,3 +244,6 @@ The following table lists the fields that are extracted, normalized under the EC
|`mimecast.siem.virus_found` | `keyword` | The name of the virus found on the email, if applicable. |
|`source.ip` | `ip` | IP address of the source. |



For more information on the Intake Format, please find the code of the Parser, Smart Descriptions, and Supported Events [here](https://github.com/SEKOIA-IO/intake-formats/tree/main/Mimecast/mimecast-email-security).
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@

### Raw Events Samples

In this section, you will find examples of raw logs as generated natively by the source. These examples are provided to help integrators understand the data format before ingestion into Sekoia.io. It is crucial for setting up the correct parsing stages and ensuring that all relevant information is captured.


=== "test_journal"


```json
{
"aggregateId": "vC80NNxvOWKkBPnzSs04FA_1715699686",
"processingId": "PGZfGuxEAu_kE-nGy1sjThBr5EYbm1ZcDKg-vXbRHLA_1715699686",
"accountId": "CDE22A102",
"timestamp": 1715699697146,
"senderEnvelope": "[email protected]",
"recipients": "[email protected]",
"direction": "Inbound",
"type": "journal",
"subtype": null,
"_offset": 105760,
"_partition": 137
}
```



=== "test_process"


```json
{
"aggregateId": "J5JwSy0HNvG7AvCg1sgDvQ_1715708284",
"processingId": "hP5f7mBanAVkWJWfh4vYvca3zOi9I3jROBmH3Z_Kysk_1715708284",
"accountId": "CDE22A102",
"action": "Hld",
"timestamp": 1715708287466,
"senderEnvelope": "[email protected]",
"messageId": "<CAF7=BmDb+6qHo+J5EB9oH+S4ncJOfEMsUYAEirX4MRZRJX+esw@mail.gmail.com>",
"subject": "Moderate",
"holdReason": "Spm",
"totalSizeAttachments": "0",
"numberAttachments": "0",
"attachments": null,
"emailSize": "3466",
"type": "process",
"subtype": "Hld",
"_offset": 105825,
"_partition": 137
}
```



=== "test_process_with_attachment"


```json
{
"processingId": "processingId",
"aggregateId": "aggregateId",
"numberAttachments": "2",
"attachments": "tpsreport.doc",
"subject": "siem_process - email subject line",
"senderEnvelope": "[email protected]",
"messageId": "messageId",
"eventType": "process",
"accountId": "C0A0",
"action": "Allow",
"holdReason": null,
"subType": "Allow",
"totalSizeAttachments": "642",
"timestamp": 1689685338609,
"emailSize": "56422"
}
```



=== "test_receipt"


```json
{
"aggregateId": "J5JwSy0HNvG7AvCg1sgDvQ_1715708284",
"processingId": "hP5f7mBanAVkWJWfh4vYvca3zOi9I3jROBmH3Z_Kysk_1715708284",
"accountId": "CDE22A102",
"timestamp": 1715708286579,
"action": "Acc",
"senderEnvelope": "[email protected]",
"messageId": "<CAF7=BmDb+6qHo+J5EB9oH+S4ncJOfEMsUYAAarX4MRZRJX+esw@mail.gmail.com>",
"subject": "Moderate",
"recipients": "[email protected]",
"senderIp": "209.123.123.123",
"rejectionType": null,
"rejectionCode": null,
"direction": "Inbound",
"numberAttachments": "0",
"senderHeader": "[email protected]",
"rejectionInfo": null,
"tlsVersion": "TLSv1.3",
"tlsCipher": "TLS_AES_256_GCM_SHA384",
"spamInfo": "[]",
"spamProcessingDetail": "{\"spf\":{\"allow\":true,\"info\":\"ALLOW\"},\"dkim\":{\"allow\":true,\"info\":\"ALLOW\"},\"dmarc\":{\"allow\":true,\"info\":\"ALLOW\"}}",
"virusFound": null,
"type": "receipt",
"subtype": "Acc",
"_offset": 105826,
"_partition": 137
}
```



Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ In details, the following table denotes the type of events produced by this inte
| ---- | ------ |
| Kind | `` |
| Category | `authentication`, `configuration`, `file`, `iam`, `session` |
| Type | `access`, `admin`, `connection` |
| Type | `access`, `admin`, `change`, `connection` |




### Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/docs/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/docs/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/docs/xdr/features/detect/sigma) and to leverage the full potential of the collected data.
This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the [built-in detection rules](/xdr/features/detect/rules_catalog) and hunting activities in the [events page](/xdr/features/investigate/events). Understanding these transformations is essential for analysts to create effective detection mechanisms with [custom detection rules](/xdr/features/detect/sigma) and to leverage the full potential of the collected data.

=== "test_admin_sample1.json"

Expand Down Expand Up @@ -727,6 +727,65 @@ This section demonstrates how the raw logs will be transformed by our parsers. I
```


=== "test_suspend_user.json"

```json

{
"message": "{\"kind\":\"admin#reports#activity\",\"id\":{\"time\":\"2024-07-09T14:05:42.528Z\",\"uniqueQualifier\":\"0123456789101112131\",\"applicationName\":\"admin\",\"customerId\":\"C03foh000\"},\"etag\":\"BvGfkzKoKVD0NM7VdXdzkXDD-nHLkyMjheL_9Z5X0\",\"actor\":{\"callerType\":\"USER\",\"email\":\"[email protected]\",\"profileId\":\"102788027662650927386\"},\"ipAddress\":\"1.2.3.4\",\"events\":[{\"type\":\"USER_SETTINGS\",\"name\":\"SUSPEND_USER\",\"parameters\":[{\"name\":\"USER_EMAIL\",\"value\":\"[email protected]\"}]}]}",
"event": {
"action": "SUSPEND_USER",
"category": [
"configuration"
],
"dataset": "admin#reports#activity",
"type": [
"change"
]
},
"@timestamp": "2024-07-09T14:05:42.528000Z",
"cloud": {
"account": {
"id": "C03foh000"
}
},
"google": {
"report": {
"actor": {
"email": "[email protected]"
},
"parameters": {
"name": "USER_EMAIL",
"value": "[email protected]"
}
}
},
"network": {
"application": "admin"
},
"related": {
"ip": [
"1.2.3.4"
],
"user": [
"john.doe"
]
},
"source": {
"address": "1.2.3.4",
"ip": "1.2.3.4"
},
"user": {
"domain": "test.fr",
"email": "[email protected]",
"id": "102788027662650927386",
"name": "john.doe"
}
}

```


=== "test_target_user.json"

```json
Expand Down Expand Up @@ -955,6 +1014,8 @@ The following table lists the fields that are extracted, normalized under the EC
|`google.report.chat.message.id` | `keyword` | Message id |
|`google.report.chat.room.name` | `keyword` | Room name |
|`google.report.meet.code` | `keyword` | Meet code |
|`google.report.parameters.name` | `keyword` | Name of the item associated with the activity |
|`google.report.parameters.value` | `keyword` | Value of the item associated with the activity |
|`google.report.parameters.visibility` | `keyword` | Visibility of the Drive item associated with the activity |
|`google.report.token.app_name` | `keyword` | Token authorization application name |
|`google.report.token.type` | `keyword` | Token type |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -968,6 +968,42 @@ In this section, you will find examples of raw logs as generated natively by the



=== "test_suspend_user"


```json
{
"kind": "admin#reports#activity",
"id": {
"time": "2024-07-09T14:05:42.528Z",
"uniqueQualifier": "0123456789101112131",
"applicationName": "admin",
"customerId": "C03foh000"
},
"etag": "BvGfkzKoKVD0NM7VdXdzkXDD-nHLkyMjheL_9Z5X0",
"actor": {
"callerType": "USER",
"email": "[email protected]",
"profileId": "102788027662650927386"
},
"ipAddress": "1.2.3.4",
"events": [
{
"type": "USER_SETTINGS",
"name": "SUSPEND_USER",
"parameters": [
{
"name": "USER_EMAIL",
"value": "[email protected]"
}
]
}
]
}
```



=== "test_target_user"


Expand Down
Loading
Loading