[8.16] [Cloud Security] Third-party cloud data (#6046) (#6115)

* [ESS] [8.16] Ingest Sysdig Falco data to Elasticsearch * fixes build error * fixes kibana reference error * adds information about 3p workflows to cloud native sec section * fix build error * save my work * Incorporates Nick's review * format tweaks for falco doc * Incorporates Nick's second review * Incorporates Joe's review * Update docs/cloud-native-security/wiz.asciidoc Co-authored-by: Joe Peeples <[email protected]> --------- Co-authored-by: Joe Peeples <[email protected]> (cherry picked from commit dc06264) Co-authored-by: Benjamin Ironside Goldstein <[email protected]>
elastic · Nov 11, 2024 · bc45a61 · bc45a61
1 parent 92c75cc
commit bc45a61
Show file tree

Hide file tree

Showing 13 changed files with 185 additions and 2 deletions.
diff --git a/docs/cloud-native-security/aws-securityhub.asciidoc b/docs/cloud-native-security/aws-securityhub.asciidoc
@@ -0,0 +1,18 @@
+[[ingest-aws-securityhub-data]]
+= Ingest AWS Security Hub data
+
+In order to enrich your {elastic-sec} workflows with third-party cloud security posture data collected by AWS Security Hub:
+
+* Follow the steps to {integrations-docs}/aws/securityhub[set up the AWS Security Hub integration]. 
+
+* Make sure the integration version is at least 2.31.1. 
+
+* Ensure you have `read` privileges for the `security_solution-*.misconfiguration_latest` index.
+
+* While configuring the AWS Security Hub integration, turn on **Collect AWS Security Hub Findings from AWS**. We recommend you also set the **Initial Interval** value to `2160h` (equivalent to 90 days) to ingest existing logs.
+
+image::images/aws-config-finding-logs.png[AWS Security Hub integration settings showing the findings toggle]
+
+After you've completed these steps, AWS Security Hub data will appear on the Misconfigurations tab of the <<cspm-findings-page, Findings>> page. 
+
+Any available findings data will also appear in the entity details flyouts for related <<insights-section, alerts>>. If alerts are present for a user or host that has findings data from AWS Security Hub, the findings will appear on the <<user-details-flyout,users>>, and <<host-details-flyout,hosts>> flyouts. 
diff --git a/docs/cloud-native-security/cloud-native-security-index.asciidoc b/docs/cloud-native-security/cloud-native-security-index.asciidoc
@@ -66,3 +66,8 @@ include::cloud-nat-sec-kubernetes-dashboard.asciidoc[leveloffset=+2]
 
 include::cloud-workload-protection.asciidoc[leveloffset=+1]
 include::environment-variable-capture.asciidoc[leveloffset=+1]
+
+include::ingest-cncf-data.asciidoc[leveloffset=+1]
+include::falco-setup.asciidoc[leveloffset=+2]
+include::aws-securityhub.asciidoc[leveloffset=+2]
+include::wiz.asciidoc[leveloffset=+2]
diff --git a/docs/cloud-native-security/cspm-findings.asciidoc b/docs/cloud-native-security/cspm-findings.asciidoc
@@ -1,7 +1,7 @@
 [[cspm-findings-page]]
 = Findings page
 
-The **Misconfigurations** tab on the Findings page displays the configuration risks identified by the <<cspm,CSPM>> and <<kspm,KSPM>> integrations.
+The **Misconfigurations** tab on the Findings page displays the configuration risks identified by the <<cspm,CSPM>> and <<kspm,KSPM>> integrations, as well as data from <<ingest-third-party-cloud-security-data, third-party integrations>>.
 
 [role="screenshot"]
 image::images/findings-page.png[Findings page]

diff --git a/docs/cloud-native-security/falco-setup.asciidoc b/docs/cloud-native-security/falco-setup.asciidoc
@@ -0,0 +1,115 @@
+[[ingest-falco]]
+= Ingest CNCF Falco data
+
+CNCF Falco is an open-source runtime security tool that detects anomalous activity in Linux hosts, containers, Kubernetes, and cloud environments. You can ingest Falco alerts into {es} to view them on {elastic-sec}'s Alerts page and incorporate them into your security workflows by using Falcosidekick, a proxy forwarder which can send alerts from your Falco deployments to {es}.
+
+First, you'll need to configure {elastic-sec} to receive data from Falco, then you'll need to configure Falco and Falcosidekick to send data to {es}. 
+
+[discrete]
+[[ingest-falco-setup-kibana]]
+== Configure {elastic-sec} to receive Falco data
+
+In {elastic-sec}:
+
+. Click **Add integrations**.
+. Search the Integrations page for `Falco`, then select it.
+. Go to the Falco integration's **Settings** tab. 
+. Click **Install Falco**, then confirm by clicking **Install Falco** again. Installation should take less than a minute. 
+
+{elastic-sec} is now ready to receive data from Falco. The Falco integration page now has an **Assets** tab where you can inspect the newly installed assets that help to ingest Falco data.
+
+Next, to make alerts from Falco appear on {elastic-sec}'s Alerts page:
+
+. Find the **Rules** page in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field].
+. Click **Detection rules (SIEM)**.
+. Search for a rule called `External Alerts`. Install it if necessary, and enable it.
+
+
+[discrete]
+[[ingest-falco-setup-falco]]
+== Configure Falco and Falcosidekick 
+
+You can either:
+
+* <<ingest-falco-setup-falco-vm,Send Falco data to {es} from virtual machines (VMs)>>; or,
+* <<ingest-falco-setup-falco-kubernetes,Send Falco data to {es} from Kubernetes>>.
+
+[discrete]
+[[ingest-falco-setup-falco-vm]]
+=== Configure Falco and Falcosidekick for VMs
+
+Multiple methods for configuring Falco to send data from VMs to {es} are available. This guide uses the https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[Falco sidekick on Docker using environment variables] method. 
+
+[discrete]
+=== Configure Falco for VMs:
+
+. Refer to Falco's documentation to https://falco.org/docs/setup/packages/[install Falco on the Linux VMs you wish to monitor]. 
+. Once Falco is installed, update `/etc/falco/falco.yaml` as follows:
+.. Enable JSON output: `json_output: true`
+.. Enable HTTP output: under `http_output`, for the `url` value, enter the `url:port` where Falcosidekick will listen. For example, if Falcosidekick is running on localhost:
++
+```
+http_output:
+  enabled: true
+  url: "http://0.0.0.0:2801/"
+```
+
+[discrete]
+[[falco-config-falco-for-vms]]
+=== Configure Falcosidekick for VMs:
+
+. Refer to Falcosidekick's documentation to https://github.com/falcosecurity/falcosidekick?tab=readme-ov-file#installation[install Falcosidekick].
+. Use the https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[Falcosidekick on Docker using environment variables] method and set your environment variables as follows:
+.. `ELASTICSEARCH_HOSTPORT`: Your {es} endpoint URL, which can be found under **Connection details** on the upper right of the **Integrations** page in {kib}.
+.. `ELASTICSEARCH_INDEX`: The {es} index where you want to store Falco logs. 
++
+IMPORTANT: Your `ELASTICSEARCH_INDEX` value must match `logs-falco.alerts-*`.
++
+.. `ELASTICSEARCH_SUFFIX`: The frequency with which you want the {es} index suffix to change. Either `daily`, `monthly`, `annually`, or `none`. 
+.. `ELASTICSEARCH_APIKEY`: The recommended way to authenticate to {es}, by providing an {kibana-ref}/api-keys.html[API key]. Note that support for this environment variable starts with Falcosidekick version 2.30. You can access the latest version on Falcosidekick's https://hub.docker.com/r/falcosecurity/falcosidekick[Docker Hub].
+.. `ELASTICSEARCH_USERNAME` and `ELASTICSEARCH_PASSWORD`: The username and password for an account on your {es} instance. Authentication using these environment variables is not supported on {ecloud} Serverless.
+.. `ELASTICSEARCH_MUTUALTLS` and `ELASTICSEARCH_CHECKCERT`: For security reasons, we recommend setting these to `true`.
+
+For example:
+
+```
+docker run -d -p 2801:2801 
+           -e ELASTICSEARCH_HOSTPORT=https://test-falco.es.us-west2.gcp.elastic-cloud.com 
+           -e ELASTICSEARCH_INDEX=logs-falco.alerts-all 
+           -e ELASTICSEARCH_SUFFIX=none 
+           -e ELASTICSEARCH_APIKEY=XXXXXXXXXXXXX 
+           -e ELASTICSEARCH_MUTUALTLS=true 
+           -e ELASTICSEARCH_CHECKCERT=true falcosecurity/falcosidekick
+```
+
+IMPORTANT: The {es} account used to authenticate Falcosidekick only needs sufficient privileges to create and write to new indices. We recommend following the principle of least privilege when provisioning this account.
+
+After installing and configuring Falcosidekick, restart Falco with `sudo systemctl restart falco`. Falcosidekick should start sending alerts to {es}.
+
+
+[discrete]
+[[ingest-falco-setup-falco-kubernetes]]
+== Configure Falco and Falcosidekick for Kubernetes
+
+1. Add the Falco https://github.com/falcosecurity/charts/blob/master/README.md[Helm charts]:
++
+```
+helm repo add falcosecurity https://falcosecurity.github.io/charts
+helm repo update
+```
++
+2. Next, install Falco and Falcosidekick using the `falcosecurity/falco` Helm chart with https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[appropriate values] for each of the `falcosidekick.config.elasticsearch.*` fields:
++
+```
+helm install falco falcosecurity/falco \
+        --set falcosidekick.enabled=true \
+        --set tty=true \
+        --set driver.kind=modern_ebpf \
+        --set collectors.kubernetes.enabled=true \
+        --set falcosidekick.config.elasticsearch.hostport="https://<ES host>" \
+        --set falcosidekick.config.elasticsearch.username="<elastic>" \
+        --set falcosidekick.config.elasticsearch.password="<password>" \
+        --set falcosidekick.config.elasticsearch.index="logs-falco.alerts-all" \
+        --set falcosidekick.config.elasticsearch.suffix="none"
+```
+
diff --git a/docs/cloud-native-security/images/aws-config-finding-logs.png b/docs/cloud-native-security/images/aws-config-finding-logs.png
diff --git a/docs/cloud-native-security/images/falco-external-alerts-rule.gif b/docs/cloud-native-security/images/falco-external-alerts-rule.gif
diff --git a/docs/cloud-native-security/images/falco-kibana-setup.gif b/docs/cloud-native-security/images/falco-kibana-setup.gif
diff --git a/docs/cloud-native-security/images/wiz-config-finding-logs.png b/docs/cloud-native-security/images/wiz-config-finding-logs.png
diff --git a/docs/cloud-native-security/images/wiz-config-vuln-logs.png b/docs/cloud-native-security/images/wiz-config-vuln-logs.png
diff --git a/docs/cloud-native-security/images/wiz-findings.png b/docs/cloud-native-security/images/wiz-findings.png
diff --git a/docs/cloud-native-security/ingest-cncf-data.asciidoc b/docs/cloud-native-security/ingest-cncf-data.asciidoc
@@ -0,0 +1,22 @@
+[[ingest-third-party-cloud-security-data]]
+= Ingest third-party cloud security data
+
+This section describes how to ingest cloud security data from third-party tools into {es}. Once ingested, this data can provide additional context and enrich your {elastic-sec} workflows.
+
+You can ingest both third-party cloud workload protection data and third-party security posture and vulnerability data. 
+
+[discrete]
+== Ingest third-party workload protection data
+
+You can ingest third-party cloud security alerts into {elastic-sec} to view them on the <<alerts-page>> and incorporate them into your triage and threat hunting workflows.
+
+* Learn to <<ingest-falco, ingest alerts from Sysdig Falco>>.
+
+[discrete]
+== Ingest third-party security posture and vulnerability data
+
+You can ingest third-party data into {elastic-sec} to review and investigate it alongside data collected by {elastic-sec}'s native cloud security integrations. Once ingested, cloud security posture and vulnerability data appears on the <<cspm-findings-page,Findings>> page, on the <<cspm-posture-dashboard, Cloud Posture dashboard>>, and in the entity details flyouts for <<insights-section, alerts>>, <<user-details-flyout,users>>, and <<host-details-flyout,hosts>>.
+
+* Learn to <<ingest-aws-securityhub-data, ingest cloud security posture data from AWS Security Hub>>.
+
+* Learn to <<ingest-wiz-data, ingest cloud security posture and vulnerability data from Wiz>>.
diff --git a/docs/cloud-native-security/vuln-management-findings.asciidoc b/docs/cloud-native-security/vuln-management-findings.asciidoc
@@ -1,7 +1,7 @@
 [[vuln-management-findings]]
 = Findings page
 
-The **Vulnerabilities** tab on the Findings page displays the vulnerabilities detected by the <<vuln-management-overview, CNVM integration>>. 
+The **Vulnerabilities** tab on the Findings page displays the vulnerabilities detected by the <<vuln-management-overview, CNVM integration>>, as well as those identified by <<ingest-third-party-cloud-security-data, third-party integrations>>.
 
 image::images/cnvm-findings-page.png[The Vulnerabilities tab of the Findings page]
 

diff --git a/docs/cloud-native-security/wiz.asciidoc b/docs/cloud-native-security/wiz.asciidoc
@@ -0,0 +1,23 @@
+[[ingest-wiz-data]]
+= Ingest Wiz data
+
+In order to enrich your {elastic-sec} workflows with third-party cloud security posture and vulnerability data collected by Wiz:
+
+* Follow the steps to {integrations-docs}/wiz[set up the Wiz integration]. 
+
+* Make sure the integration version is at least 2.0.1. 
+
+* Ensure you have `read` privileges for the following indices: `security_solution-*.misconfiguration_latest`, `security_solution-*.vulnerability_latest`.
+
+* While configuring the Wiz integration, turn on **Cloud Configuration Finding logs** and **Vulnerability logs**. We recommend you also set the **Initial Interval** values for both settings to `2160h` (equivalent to 90 days) to ingest existing logs.
+
+image::images/wiz-config-finding-logs.png[Wiz integration settings showing the findings toggle]
+
+image::images/wiz-config-vuln-logs.png[Wiz integration settings showing the vulnerabilities toggle]
+
+After you've completed these steps, Wiz data will appear on the <<cspm-findings-page, Misconfiguations>> and <<vuln-management-findings, Vulnerabilities>> tabs of the Findings page.
+
+image::images/wiz-findings.png[Wiz data on the Findings page]
+
+Any available findings data will also appear in the entity details flyouts for related <<insights-section, alerts>>. If alerts are present for a user or host that has findings data from Wiz, the findings will appear on the <<user-details-flyout,users>>, and <<host-details-flyout,hosts>> flyouts. 
+