[8.11] CSPM onboarding for Azure (subscription-level) and for GCP (or…

…ganization-level) (#4111) (#4203) * saving work * preliminary draft * fixes build errors * Incorporates new information received today for Azure and GCP * formatting fix for azure * add missing punctuation * minor update * adjust internal ToC * Update docs/cloud-native-security/cspm-get-started-azure.asciidoc Co-authored-by: Orestis Floros <[email protected]> * Update docs/cloud-native-security/cspm-get-started-azure.asciidoc Co-authored-by: Orestis Floros <[email protected]> * Update docs/cloud-native-security/cspm-get-started-azure.asciidoc Co-authored-by: Orestis Floros <[email protected]> * Update docs/cloud-native-security/cspm-get-started-azure.asciidoc Co-authored-by: Orestis Floros <[email protected]> * Update docs/cloud-native-security/cspm-get-started-gcp.asciidoc Co-authored-by: Amir Ben Nun <[email protected]> * Update docs/cloud-native-security/cspm-get-started-gcp.asciidoc Co-authored-by: Amir Ben Nun <[email protected]> * incorporate feedback * incorporates feedback * minor formatting fix * Update docs/cloud-native-security/cspm-get-started-gcp.asciidoc * setup -> set up * incorporates Joe's feedback * typo fix * minor fix --------- Co-authored-by: Orestis Floros <[email protected]> Co-authored-by: Amir Ben Nun <[email protected]> (cherry picked from commit c474501) Co-authored-by: Benjamin Ironside Goldstein <[email protected]>
elastic · Nov 7, 2023 · f2fd678 · f2fd678
1 parent ddbd870
commit f2fd678
Show file tree

Hide file tree

Showing 3 changed files with 212 additions and 32 deletions.
diff --git a/docs/cloud-native-security/cloud-native-security-index.asciidoc b/docs/cloud-native-security/cloud-native-security-index.asciidoc
@@ -40,6 +40,7 @@ include::security-posture-management.asciidoc[leveloffset=+1]
 include::cspm.asciidoc[leveloffset=+1]
 include::cspm-get-started-aws.asciidoc[leveloffset=+2]
 include::cspm-get-started-gcp.asciidoc[leveloffset=+2]
+include::cspm-get-started-azure.asciidoc[leveloffset=+2]
 include::cspm-findings.asciidoc[leveloffset=+2]
 include::cspm-benchmark-rules.asciidoc[leveloffset=+2]
 include::cspm-cloud-posture-dashboard.asciidoc[leveloffset=+2]

diff --git a/docs/cloud-native-security/cspm-get-started-azure.asciidoc b/docs/cloud-native-security/cspm-get-started-azure.asciidoc
@@ -0,0 +1,121 @@
+[[cspm-get-started-azure]]
+= Get started with CSPM for Azure
+
+[discrete]
+[[cspm-overview-azure]]
+== Overview
+
+This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature.
+
+.Requirements
+[sidebar]
+--
+* The CSPM integration is available to all {ecloud} users. On-premise deployments require an https://www.elastic.co/pricing[Enterprise subscription].
+* To view posture data, you need `read` privileges for the following {es} indices:
+** `logs-cloud_security_posture.findings_latest-*`
+** `logs-cloud_security_posture.scores-*`
+** `logs-cloud_security_posture.findings`
+* The user who gives the CSPM integration permissions must be an Azure subscription `admin`.
+--
+
+[discrete]
+[[cspm-setup-azure]]
+== Set up CSPM for Azure
+
+To set up CSPM for Azure, first add the CSPM integration, then enable cloud account access.
+
+
+[discrete]
+[[cspm-add-and-name-integration-azure]]
+=== Add your CSPM integration
+. From the Elastic Security *Get started* page, click *Add integrations*.
+. Search for `CSPM`, then click on the result.
+. Click *Add Cloud Security Posture Management (CSPM)*.
+. Under **Configure integration**, select **Azure**.
+. Give your integration a name that matches the purpose or team of the Azure subscription you want to monitor, for example, `azure-CSPM-1`.
+
+[discrete]
+[[cspm-set-up-cloud-access-section-azure]]
+=== Set up cloud account access
+
+NOTE: To set up CSPM for an Azure subscription, you will need admin privileges for that subscription.
+
+For most users, the simplest option is to use an Azure Resource Manager (ARM) template to automatically provision the necessary resources and permissions in Azure. If you prefer a more hands-on approach or require a specific configuration not supported by the ARM template, you can use one of the manual setup options described below.
+
+[discrete]
+[[cspm-set-up-ARM]]
+== ARM template setup (recommended)
+
+. Under *Setup Access*, select *ARM Template*.
+. Under **Where to add this integration**:
+.. Select **New Hosts**.
+.. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `azure-dev-policy`. Click **Save and continue**. The *ARM Template deployment* window appears.
+.. In a new tab, log in to the Azure portal, then return to {kib} and click **Launch ARM Template**. This will open the ARM template in Azure.
+.. (Optional) Change the `Resource Group Name` parameter. Otherwise the name of the resource group defaults to a timestamp prefixed with `cloudbeat-`.
+.. Copy the `Fleet URL` and `Enrollment Token` that appear in {kib} to the corresponding fields in the ARM Template, then click *Review + create*.
+. Return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data.
+
+[discrete]
+[[cspm-set-up-manual-azure]]
+== Manual setup
+
+For manual setup, there are two authentication methods: using managed identities (recommended), or using environment variables with authentication secrets. The first method requires you to deploy {agent} to a VM in the Azure subscription you want to monitor with CSPM; the second method allows {agent} to be deployed anywhere, including a VM outside Azure or a personal laptop.
+
+[discrete]
+[[cspm-azure-managed-identity-setup]]
+=== Option 1: Managed identity (recommended)
+
+This method involves creating an Azure VM (or using an existing one), giving it read access to the subscription you want to monitor with CSPM, and installing {agent} on it.
+
+. Go to the Azure portal to https://portal.azure.com/#create/Microsoft.VirtualMachine-ARM[create a new Azure VM].
+. Follow the setup process, and make sure you enable **System assigned managed identity** under the **Management** tab.
+. Go to your Azure subscription list and select the subscription you want to monitor with CSPM.
+. Go to **Access control (IAM)**, and select **Add Role Assignment**.
+. Select the `Reader` function role, assign access to **Managed Identity**, then select your VM.
+
+After assigning the role:
+
+. Return to the **Add CSPM** page in {kib}. 
+. Under **Configure integration**, select **Azure**. Under **Setup access**, select **Manual**.
+. Under **Where to add this integration**, select **New hosts**.
+. Click **Save and continue**, then follow the instructions to install {agent} on your Azure VM.
+
+Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data.
+
+[discrete]
+[[cspm-azure-environment-variables-setup]]
+=== Option 2: Environment variables with authentication secrets
+
+Before using this method, you must have set up a https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#get-tenant-and-app-id-values-for-signing-in[Microsoft Entra application and service principal that can access resources]. 
+
+. Go to the **Registered apps** section of https://ms.portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps[Microsoft Entra ID].
+. Click on **New Registration**, name your app and click **Register**.
+. Copy your new app's `Directory (tenant) ID` and `Application (client) ID`.
+. Select **Certificates & secrets**, then select **New client secret**. Copy the new secret.
+. Go to your Azure subscription list and select the subscription you want to monitor with CSPM.
+. Go to **Access control (IAM)** and select **Add Role Assignment**. 
+. Select the `Reader` function role, assign access to **User, group, or service principal**, and select your new app.
+
+On the VM where you plan to install {agent}, create the file `/etc/sysconfig/elastic-agent`. Paste the following content into the new file, substituting the values you copied for the placeholder values:
+
+```
+AZURE_TENANT_ID=<Directory (tenant) ID>
+AZURE_CLIENT_ID=<Application (client) ID>
+AZURE_CLIENT_SECRET=<Secret Value>
+```
+
+After creating the file:
+
+. Return to the **Add CSPM** page in {kib}. 
+. Under **Configure integration**, select **Azure**. Under **Setup access**, select **Manual**.
+. Under **Where to add this integration**, select **New hosts**.
+. Click **Save and continue**, then follow the instructions to install {agent} on your selected host.
+
+If you created `/etc/sysconfig/elastic-agent` after installing {agent}, you might need to restart it with the following commands:
+
+```
+systemctl daemon-reload 
+systemctl restart elastic-agent
+```
+
+Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data.
diff --git a/docs/cloud-native-security/cspm-get-started-gcp.asciidoc b/docs/cloud-native-security/cspm-get-started-gcp.asciidoc
@@ -20,9 +20,9 @@ This page explains how to get started monitoring the security posture of your GC
 
 [discrete]
 [[cspm-setup-gcp]]
-== Set up CSPM for GCP
+== Initial setup
 
-To set up CSPM for GCP, first add the CSPM integration, then enable cloud account access.
+To set up CSPM for GCP, you need to first add the CSPM integration, then enable cloud account access.
 
 
 [discrete]
@@ -31,20 +31,23 @@ To set up CSPM for GCP, first add the CSPM integration, then enable cloud accoun
 . From the Elastic Security *Get started* page, click *Add integrations*.
 . Search for `CSPM`, then click on the result.
 . Click *Add Cloud Security Posture Management (CSPM)*.
+. Under *Configure integration*, select *GCP*, then either *GCP Organization* (recommended) or *Single Account*.
 . Give your integration a name that matches the purpose or team of the GCP account you want to monitor, for example, `dev-gcp-project`.
 
+
 [discrete]
 [[cspm-set-up-cloud-access-section-gcp]]
 === Set up cloud account access
-To setup CSPM for a GCP project, you will need admin privileges for the project.
+
+NOTE: To set up CSPM for a GCP project, you need admin privileges for the project.
 
 For most users, the simplest option is to use a Google Cloud Shell script to automatically provision the necessary resources and permissions in your GCP account. This method, as well as two manual options, are described below.
 
 [discrete]
 [[cspm-set-up-cloudshell]]
-=== Cloud Shell script setup (recommended)
+== Cloud Shell script setup (recommended)
 
-. Under **Setup Access**, select **Google Cloud Shell**.
+. Under **Setup Access**, select **Google Cloud Shell**. Enter your GCP Project ID, and for GCP Organization deployments, your GCP Organization ID.
 . Under **Where to add this integration**:
 .. Select **New Hosts**.
 .. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`.
@@ -55,47 +58,102 @@ For most users, the simplest option is to use a Google Cloud Shell script to aut
 +
 image::images/cspm-cloudshell-trust.png[The cloud shell confirmation popup]
 +
-. In Google Cloud Shell, execute the command you copied earlier. Once it finishes, return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data.
+. In Google Cloud Shell, execute the command you copied. Once it finishes, return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data.
 
 NOTE: During Cloud Shell setup, the CSPM integration adds roles to Google's default service account, which enables custom role creation and attachment of the service account to a compute instance.
 After setup, these roles are removed from the service account. If you attempt to delete the deployment but find the deployment manager lacks necessary permissions, consider adding the missing roles to the service account:
 https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectIamAdmin[Project IAM Admin], https://cloud.google.com/iam/docs/understanding-roles#iam.roleAdmin[Role Administrator].
 
 [discrete]
-[[cspm-set-up-manual-gcp]]
-=== Manual authentication
+[[cspm-set-up-manual-gcp-org]]
+== Manual authentication (GCP organization)
+
+To authenticate manually to monitor a GCP organization, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. 
+
+Use the following commands, after replacing `<SA_NAME>` with the name of your new service account, `<ORG_ID>` with your GCP organization's ID, and `<PROJECT_ID>` with the GCP project ID of the project where you want to provision the compute instance that will run CSPM.
+
+Create a new service account:
+```
+gcloud iam service-accounts create <SA_NAME> \
+    --description="Elastic agent service account for CSPM" \
+    --display-name="Elastic agent service account for CSPM" \
+    --project=<PROJECT_ID>
+```
+
+Assign the necessary roles to the service account:
+```
+gcloud organizations add-iam-policy-binding <ORG_ID> \
+    --member=serviceAccount:<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com \
+    --role=roles/cloudasset.viewer
+
+gcloud organizations add-iam-policy-binding <ORG_ID> \
+    --member=serviceAccount:<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com \
+    --role=roles/browser
+```
+NOTE: The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy.
+
+Download the credentials JSON (first, replace `<KEY_FILE>` with the location where you want to save it):
+```
+gcloud iam service-accounts keys create <KEY_FILE> \
+    --iam-account=<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com
+```
+
+Keep the credentials JSON in a secure location; you will need it later.
+
+Provide credentials to the CSPM integration:
+
+. On the CSPM setup screen under **Setup Access**, select **Manual**.
+. Enter your GCP **Organization ID**. Enter the GCP **Project ID** of the project where you want to provision the compute instance that will run CSPM. 
+. Select **Credentials JSON**, and enter the value you generated earlier.
+. Under **Where to add this integration**, select **New Hosts**.
+. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`.
+. Click **Save and continue**, then follow the instructions to install {agent} in your chosen GCP project.
+
+Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data.
+
+[discrete]
+[[cspm-set-up-manual-gcp-project]]
+== Manual authentication (GCP project)
 
-To authenticate manually, you'll first need to generate credentials for a new GCP service account with the necessary roles, then provide those credentials to the CSPM integration.
+To authenticate manually to monitor an individual GCP project, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. 
 
-Generate GCP credentials:
+Use the following commands, after replacing `<SA_NAME>` with the name of your new service account, and `<PROJECT_ID>` with your GCP project ID.
 
-. Access the GCP console and select your project.
-. Navigate to **IAM & Admin -> Service accounts**.
-. Click **Create Service Account**.
-. Provide an account name.
-. Enable the required roles:
-.. `Cloud Asset Viewer`: Grants read access to cloud asset metadata.
-.. `Browser`: Grants read access to the project hierarchy.
-. Click **Continue**, then click **Done**.
-. Select the new service account from the list.
-. Go to the **KEYS** tab, then click **ADD KEY**.
-. Select **JSON** as the key type, then click **CREATE**.
+Create a new service account:
+```
+gcloud iam service-accounts create <SA_NAME> \
+    --description="Elastic agent service account for CSPM" \
+    --display-name="Elastic agent service account for CSPM" \
+    --project=<PROJECT_ID>
+```
 
-The credentials JSON will download to your local machine. Keep it secure since it provides access to your GCP resources.
+Assign the necessary roles to the service account:
+```
+gcloud projects add-iam-policy-binding <PROJECT_ID> \
+    --member=serviceAccount:<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com \
+    --role=roles/cloudasset.viewer
+
+gcloud projects add-iam-policy-binding <PROJECT_ID> \
+    --member=serviceAccount:<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com \
+    --role=roles/browser
+```
+NOTE: The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy.
+
+Download the credentials JSON (first, replace `<KEY_FILE>` with the location where you want to save it):
+```
+gcloud iam service-accounts keys create <KEY_FILE> \
+    --iam-account=<SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com
+```
+
+Keep the credentials JSON in a secure location; you will need it later.
 
 Provide credentials to the CSPM integration:
 
 . On the CSPM setup screen under **Setup Access**, select **Manual**.
 . Enter your GCP **Project ID**.
-. Select either **Credentials File** or **Credentials JSON**, and enter the credentials information in your selected format.
-. Under **Where to add this integration**:
-.. If you want to monitor a GCP project where you have not yet deployed {agent}:
-... Select **New Hosts**.
-... Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`.
-... Click **Save and continue**, then **Add {agent} to your hosts**. The **Add agent** wizard appears and provides {agent} binaries, which you can download and deploy to a VM in your GCP account.
-.. If you want to monitor a GCP project where you have already deployed {agent}:
-... Select **Existing hosts**.
-... Select an agent policy that applies the GCP project you want to monitor.
-. Click **Save and continue**.
+. Select **Credentials JSON**, and enter the value you generated earlier.
+. Under **Where to add this integration**, select **New Hosts**.
+. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`.
+. Click **Save and continue**, then follow the instructions to install {agent} in your chosen GCP project.
 
 Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data.