diff --git a/docs/configuration-parameters-1830bca.md b/docs/configuration-parameters-1830bca.md index a7e6afd..7c99d1f 100644 --- a/docs/configuration-parameters-1830bca.md +++ b/docs/configuration-parameters-1830bca.md @@ -243,7 +243,7 @@ Integer -Indirectly, this parameter sets the maximum disk size for storing observability data as described in [Service Plans](service-plans-a9d2d1b.md). This parameter has no effect for the `dev` plan. Needs to be between `2` and `10`. Default is `10`. +Indirectly, this parameter sets the maximum disk size for storing observability data as described in [Service Plans](service-plans-a9d2d1b.md). This parameter has no effect for the *dev* plan. Needs to be between `2` and `10`. Default is `10`. @@ -351,7 +351,7 @@ Integer -Specifies the maximum number of provisionable ingest instances, which are scaled automatically based on their overall CPU utilization. Must be between `2` and `10`. Defaults to `2`. This parameter impacts peak throughput and buffering. Scale-out happens when the overall CPU utilization exceeds 80%. Scale-in happens when the overall CPU utilization or configuration parameter decreases. This parameter has no effect on the `dev` plan, which is limited to a single instance. +Specifies the maximum number of provisionable ingest instances, which are scaled automatically based on their overall CPU utilization. Must be between `2` and `10`. Defaults to `2`. This parameter impacts peak throughput and buffering. Scale-out happens when the overall CPU utilization exceeds 80%. Scale-in happens when the overall CPU utilization or configuration parameter decreases. This parameter has no effect on the *dev* plan, which is limited to a single instance. @@ -486,7 +486,7 @@ String -The SAML group to grant administrative access and permissions to modify the security module. Required if `enabled` is set to `true`. Required if `enabled` is set to `true`. +The SAML group to grant administrative access and permissions to modify the security module. Required if `enabled` is set to `true`. Required if *enabled* is set to `true`. @@ -508,7 +508,7 @@ Boolean -Enables IdP-initiated SSO. Required if `enabled` is set to `true`. +Enables IdP-initiated SSO. Required if *enabled* is set to `true`. @@ -532,7 +532,7 @@ String The list of backend\_roles will be read from this attribute during user login. -This field must be set to the corresponding attribute for IdP groups,usually `groups`. Required if `enabled` is set to `true`. +This field must be set to the corresponding attribute for IdP groups,usually `groups`. Required if *enabled* is set to `true`. @@ -554,7 +554,7 @@ URL -The URL to get the SAML IdP metadata from. Required if `enabled` is set to `true`. +The URL to get the SAML IdP metadata from. Required if *enabled* is set to `true`. @@ -578,7 +578,7 @@ String The Entity ID of the SAML IdP. -Open the metadata URL in your browser and copy the full value of the `entityID` field. It is located in the first line of the response. Required if `enabled` is set to `true`. +Open the metadata URL in your browser and copy the full value of the `entityID` field. It is located in the first line of the response. Required if *enabled* is set to `true`. @@ -600,7 +600,7 @@ String -The Entity ID of the service provider. Generally, this parameter is set to the name of your application configured in your IdP. Required if `enabled` is set to `true`. +The Entity ID of the service provider. Generally, this parameter is set to the name of your application configured in your IdP. Required if *enabled* is set to `true`. @@ -666,7 +666,7 @@ String -Key to sign tokens. Provide a `random` key with an `even number (min. length: 32)` of `alphanumeric characters (A-Z, a-z, 0-9)`. A random key is generated if the key isn't provided. +Key to sign tokens. Provide a *random* key with an *even number \(min. length: 32\)* of *alphanumeric characters \(A-Z, a-z, 0-9\)*. A random key is generated if the key isn't provided. diff --git a/docs/create-an-sap-cloud-logging-instance-through-sap-btp-cockpit-3aca7af.md b/docs/create-an-sap-cloud-logging-instance-through-sap-btp-cockpit-3aca7af.md index 5b77572..145ec29 100644 --- a/docs/create-an-sap-cloud-logging-instance-through-sap-btp-cockpit-3aca7af.md +++ b/docs/create-an-sap-cloud-logging-instance-through-sap-btp-cockpit-3aca7af.md @@ -20,15 +20,15 @@ See [Prerequisites](prerequisites-41d8559.md). To create an SAP Cloud Logging instance using the SAP BTP Cockpit, follow these steps: -1. Open the SAP BTP Cockpit and navigate to the `Instances and Subscriptions` page of your subaccount. -2. Click `Create`. +1. Open the SAP BTP Cockpit and navigate to the *Instances and Subscriptions* page of your subaccount. +2. Click *Create*. 3. Configure your Instance: - - Select `cloud-logging service`. + - Select *cloud-logging service*. - Select your preferred service plan \(see [Service Plans](service-plans-a9d2d1b.md)\). - - Set an `Instance Name`. + - Set an *Instance Name*. -4. Configure `Service Configuration Parameters` \(see [Configuration Parameters](configuration-parameters-1830bca.md)\) -5. Review and click `Create`. It takes some time until SAP Cloud Logging is up. +4. Configure *Service Configuration Parameters* \(see [Configuration Parameters](configuration-parameters-1830bca.md)\) +5. Review and click *Create*. It takes some time until SAP Cloud Logging is up. @@ -38,7 +38,7 @@ To create an SAP Cloud Logging instance using the SAP BTP Cockpit, follow these Get a service key to access instance credentials. -1. Select your SAP Cloud Logging instance to open the `Bindings` panel and click `Create`. -2. Enter a name for the binding and click `Create`. -3. Click the three dots next to the newly created binding and select `View` to show the credentials of the service instance. +1. Select your SAP Cloud Logging instance to open the *Bindings* panel and click *Create*. +2. Enter a name for the binding and click *Create*. +3. Click the three dots next to the newly created binding and select *View* to show the credentials of the service instance. diff --git a/docs/create-an-sap-cloud-logging-instance-through-sap-btp-service-operator-f6aa131.md b/docs/create-an-sap-cloud-logging-instance-through-sap-btp-service-operator-f6aa131.md index a7de26b..6aa0ce8 100644 --- a/docs/create-an-sap-cloud-logging-instance-through-sap-btp-service-operator-f6aa131.md +++ b/docs/create-an-sap-cloud-logging-instance-through-sap-btp-service-operator-f6aa131.md @@ -106,7 +106,7 @@ This step results in a `secret` with the name `cls``sap-cloud-logging-integratio ``` - We recommend you enable [credentials rotation](https://github.com/SAP/sap-btp-service-operator/blob/main/README.md#credentials-rotation) for the ServiceBinding, so that the configuration is updated automatically. Assure that the `rotationFrequency` binding parameter reflects a period of more than a day to avoid frequent restarts. + We recommend you enable [credentials rotation](https://github.com/SAP/sap-btp-service-operator/blob/main/README.md#credentials-rotation) for the ServiceBinding, so that the configuration is updated automatically. Assure that the *rotationFrequency* binding parameter reflects a period of more than a day to avoid frequent restarts. The binding creation automatically triggers the creation of a `secret` with the name `sap-cloud-logging` in in the same namespace. diff --git a/docs/index.md b/docs/index.md index 960cf76..fd5003c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,9 +12,10 @@ SAP Cloud Logging service is an instance-based and environment-agnostic observab - [Create an SAP Cloud Logging Instance through SAP BTP CLI](create-an-sap-cloud-logging-instance-through-sap-btp-cli-21eb1bd.md) - [Create an SAP Cloud Logging Instance through SAP BTP Service Operator](create-an-sap-cloud-logging-instance-through-sap-btp-service-operator-f6aa131.md) - [Ingest Observability Data](ingest-observability-data-ba16ff7.md) - - [Ingest via Cloud Foundry Runtime](ingest-via-cloud-foundry-runtime-f5a7c99.md) + - [Ingest from Cloud Foundry Runtime](ingest-from-cloud-foundry-runtime-f5a7c99.md) - [Ingest via Kyma Runtime](ingest-via-kyma-runtime-612c7b9.md) - [Ingest via OpenTelemetry API Endpoint](ingest-via-opentelemetry-api-endpoint-fdc78af.md) + - [Ingest via JSON API Endpoint](ingest-via-json-api-endpoint-3416f8f.md) - [Rotate the Ingestion Root CA Certificate](rotate-the-ingestion-root-ca-certificate-bbcb3e7.md) - [Access and Analyze Observability Data](access-and-analyze-observability-data-dad5b01.md) - [Data Protection and Privacy](data-protection-and-privacy-80e76fd.md) diff --git a/docs/ingest-from-cloud-foundry-runtime-f5a7c99.md b/docs/ingest-from-cloud-foundry-runtime-f5a7c99.md new file mode 100644 index 0000000..eadc0dc --- /dev/null +++ b/docs/ingest-from-cloud-foundry-runtime-f5a7c99.md @@ -0,0 +1,129 @@ + + +# Ingest from Cloud Foundry Runtime + +Shipping logs from applications deployed on SAP BTP Cloud Foundry can be configured in different ways. Even without any specific application logs, you can analyze your applications based on the automatically issued request logs from the Cloud Foundry router. Further, you can benefit from default contents, such as dashboards, index patterns, and retention settings. + + + +> ### Note: +> You can also [Ingest via OpenTelemetry API Endpoint](ingest-via-opentelemetry-api-endpoint-fdc78af.md). + +> ### Caution: +> Ensure that you consider [SAP BTP Security Recommendation BTP-CLS-0002](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations?seclist-index=BTP-CLS-0002). + +All configuration steps can be done either using SAP BTP Cockpit or the Cloud Foundry Command Line Interface. + + + + + +## Procedures + +Shipping logs from applications deployed on SAP BTP Cloud Foundry can be configured using one of the below options: + +- [Bind the Application to the Service Instance](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__bind_the_application), optionally using [Share Service Instance Across Different Spaces](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__share_service_instance_across_different_spaces) to consolidate observability data across spaces within one organization. +- [Bind the Application to a User Provided Service](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__bind_the_application_to_user_provided_service). +- We recommend you use one of the Cloud Foundry open source logging libraries \([Java](https://github.com/SAP/cf-java-logging-support)/[NodeJS](https://github.com/SAP/cf-nodejs-logging-support)\) to configure logging within your application. + + + +### Bind the Application to the Service Instance + +**Bind the Application Using the Command Line Interface** + +1. [Log On to the Cloud Foundry Environment Using the Cloud Foundry Command Line Interface](https://help.sap.com/docs/btp/sap-business-technology-platform/log-on-to-cloud-foundry-environment-using-cloud-foundry-command-line-interface). +2. To bind the application via command line interface, execute the following command: + + ``` + cf bind-service + + ``` + + > ### Note: + > Although the command line interface prompts you to restage the app, the binding takes effect without restaging. + +3. Go to the [Result](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__section_gvg_4k4_xyb) section. + +**Bind the Application Using the SAP BTP Cockpit** + +1. [Log On to the Cloud Foundry Environment Using the SAP BTP Cockpit](https://help.sap.com/docs/btp/sap-business-technology-platform/cloud-foundry-environment). +2. Execute [Bind Service Instances to Applications Using the Cockpit](https://help.sap.com/docs/service-manager/sap-service-manager/binding-service-instances-to-cloud-foundry-applications) without binding parameters. +3. Go to the [Result](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__section_gvg_4k4_xyb) section. + + + +### Share Service Instance Across Different Spaces + +You can share a single service instance across multiple spaces. Skip this step if you don't need to share a single instance across multiple spaces of the same org. + +1. [Log on to the Cloud Foundry Environment Using the Cloud Foundry Command Line Interface](https://help.sap.com/docs/btp/sap-business-technology-platform/log-on-to-cloud-foundry-environment-using-cloud-foundry-command-line-interface). +2. To share the services in an additional space, execute the following command: + + ``` + cf share-service -s + ``` + +3. Go to the [Result](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__section_gvg_4k4_xyb) section. + +> ### Note: +> Be careful when deleting service keys. Credentials are invalidated if the service key is deleted, and there is no automated mechanism to track in which user provided services instances the information of a service key is used. + + + +### Bind the Application to a User Provided Service + +[Bind the Application to the Service Instance](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__bind_the_application) is the recommended approach, because of simplicity in setup and maintenance. Introducing an indirection via service key and [binding to a user-provided service](https://docs.cloudfoundry.org/devguide/services/user-provided.html) to send logs is only advisable: + +- to reduce the dependency on bind operations in automated procedures. +- to ingest across Cloud Foundry orgs to a single Cloud Logging instance. + +**Bind the Application to User Provided Service Using the Command Line Interface** + +1. [Log on to the Cloud Foundry Environment Using the Cloud Foundry Command Line Interface](https://help.sap.com/docs/btp/sap-business-technology-platform/log-on-to-cloud-foundry-environment-using-cloud-foundry-command-line-interface). +2. Execute the following command to list the service instances: + + ``` + cf services + ``` + +3. Execute the following command to create a service key without binding to any application: + + ``` + cf create-service-key + ``` + +4. Extract `ingest-endpoint`, `ingest-username`, and `ingest-password` from the response of executing: + + ``` + cf service-key + ``` + +5. Create a user provided service using the following the template filled with the values of the previous step and a user-provided-service-name of your choice: + + ``` + cf cups -l https://:@/cfsyslog + ``` + +6. Proceed with [Bind the Application to the Service Instance](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__bind_the_application) and bind to the user provided service. + +**Bind the Application to User Provided Service Using SAP BTP Cockpit** + +1. [Log On to the Cloud Foundry Environment Using the SAP BTP Cockpit](https://help.sap.com/docs/btp/sap-business-technology-platform/cloud-foundry-environment). +2. Create a service key according to [Creating Service Keys in Cloud Foundry](https://help.sap.com/viewer/09cc82baadc542a688176dce601398de/Cloud/en-US/6fcac08409db4b0f9ad55a6acd4d31c5.html). +3. Create a User-Provided Service following [Creating User-Provided Service Instances in Cloud Foundry Environment](https://help.sap.com/docs/service-manager/sap-service-manager/creating-user-provided-service-instances-in-cloud-foundry-environment) using `Instance Name` of your choice and the information from the the service key to configure `System Logs Drain URL`: + + ``` + https://:@/cfsyslog + ``` + +4. Proceed with [Bind the Application to the Service Instance](ingest-from-cloud-foundry-runtime-f5a7c99.md#loiof5a7c993743c4ee79722479371b90b37__bind_the_application) and bind to the user provided service. + + + + + +## Result + +The ingested data can be analyzed in OpenSearch Dashboards \(see [Access and Analyze Observability Data](access-and-analyze-observability-data-dad5b01.md)\) based on the `logs-cfsyslog-*` index pattern. + diff --git a/docs/ingest-observability-data-ba16ff7.md b/docs/ingest-observability-data-ba16ff7.md index 6e299ec..32f9691 100644 --- a/docs/ingest-observability-data-ba16ff7.md +++ b/docs/ingest-observability-data-ba16ff7.md @@ -4,10 +4,11 @@ Use one or more of the following options to drain observability data in your Cloud Logging instance. -- [Ingest via Cloud Foundry Runtime](ingest-via-cloud-foundry-runtime-f5a7c99.md) +- [Ingest from Cloud Foundry Runtime](ingest-from-cloud-foundry-runtime-f5a7c99.md) - [Ingest via Kyma Runtime](ingest-via-kyma-runtime-612c7b9.md) - Ship via API - [Ingest via OpenTelemetry API Endpoint](ingest-via-opentelemetry-api-endpoint-fdc78af.md) +- [Ingest via JSON API Endpoint](ingest-via-json-api-endpoint-3416f8f.md) - [Rotate the Ingestion Root CA Certificate](rotate-the-ingestion-root-ca-certificate-bbcb3e7.md) diff --git a/docs/ingest-via-cloud-foundry-runtime-f5a7c99.md b/docs/ingest-via-cloud-foundry-runtime-f5a7c99.md deleted file mode 100644 index bd4c554..0000000 --- a/docs/ingest-via-cloud-foundry-runtime-f5a7c99.md +++ /dev/null @@ -1,140 +0,0 @@ - - -# Ingest via Cloud Foundry Runtime - -Following this guide allows you to benefit from default contents, such as dashboards, index patterns, and retention settings. - - - - - -## Ship Logs from a Cloud Foundry Application - -> ### Note: -> Even without any specific application logs, you can analyze your applications based on the automatically issued request logs from the Cloud Foundry router. - -> ### Note: -> You can [Ingest via OpenTelemetry API Endpoint](ingest-via-opentelemetry-api-endpoint-fdc78af.md). There are no predefined dashboards yet, but you can use the observability plugin by OpenSearch Dashboards. - -Ship logs from applications deployed on SAP BTP Cloud Foundry by binding the application. Bind applications either using the SAP BTP Cockpit or the Cloud Foundry Command Line Interface \(CLI\). - - - - - -## Indirection via Service Key and User-Provided Service - -> ### Note: -> If you delete the service key, the certificates and credentials are invalidated. - -> ### Note: -> Skip this step and bind to your application directly if you are sending with certificates \(as it manages certificate rotation for you\). - -Bind the application directly to an SAP Cloud Logging instance. However, to be resilient against issues during the binding process \(important for automated builds\), we recommend an indirection via service key and binding to a [user-provided service](https://docs.cloudfoundry.org/devguide/services/user-provided.html). Cloud Foundry operations can lead to an implicit rebind, without the need for a rebind. Using service keys provides control over the credential lifecycle. - - - -### Using the Cloud Foundry Command Line Interface - -1. `cf services` lists the service instance. -2. To create a service key without binding to any application via `cf cli`, execute the following command: - - ``` - cf create-service-key - ``` - - > ### Note: - > SAP Cloud Logging needs no configuration parameters during service key creation. - -3. The service key holds all the credentials. To view a service key, execute: - - ``` - cf service-key - - ``` - - and extract ingest-endpoint, ingest-username, and ingest-password. - -4. Create a user-provided service using the following pattern: - - ``` - cf cups -l https://ingest-username:ingest-password@ingest-endpoint/cfsyslog - - ``` - - - - -### Using the SAP BTP Cockpit - -1. Create a service key according to [Creating Service Keys in Cloud Foundry](https://help.sap.com/viewer/09cc82baadc542a688176dce601398de/Cloud/en-US/6fcac08409db4b0f9ad55a6acd4d31c5.html). -2. Create a User-Provided Service following [Creating User-Provided Service Instances in Cloud Foundry Environment](https://help.sap.com/docs/service-manager/sap-service-manager/creating-user-provided-service-instances-in-cloud-foundry-environment), using the information from the service key - -Instance Name:`` - -System Logs Drain URL: `https://ingest-username:ingest-password@ingest-endpoint/cfsyslog` - -> ### Note: -> SAP Cloud Logging needs no configuration parameters during service key creation. - - - - - -## Bind the Application to the Service Instance - - - -### Bind the Application Using the Cloud Foundry Command Line Interface - -To bind the application using CF CLI, execute the following command: - -``` -cf bind-service -``` - -> ### Note: -> CF CLI asks you to restage, but this isn't mandatory to use SAP Cloud Logging. - - - -### Bind the Application Using the SAP BTP Cockpit - -You can bind service instances to applications both at the application view, and at the service-instance view in the cockpit. - -1. [Log On to the Cloud Foundry](https://help.sap.com/docs/BTP/65de2977205c403bbc107264b8eccf4b/350356d1dc314d3199dca15bd2ab9b0e.html) environment using the SAP BTP Cockpit. -2. Navigate to the space in which your application is deployed. For more information, see [Navigate to Global Accounts, Subaccounts, Orgs, and Spaces in the Cockpit](https://help.sap.com/viewer/65de2977205c403bbc107264b8eccf4b/Cloud/en-US/5bf87353bf994819b8803e5910d8450f.html). -3. In the navigation area, choose *Services* \> *Service Marketplace*. -4. Search for SAP Cloud Logging. -5. In the navigation area, choose `Instances`. -6. To create a new instance, choose `New Instance`. In the following steps, you assign an application to this service. This application then writes its logs to the newly created service instance. - 1. Choose the service plan, then choose `Next`. - 2. **Optional**. Browse for the .json file of the app for which you want to write logs. Then choose `Next`. - 3. **Optional**. Choose an application from the dropdown that lists all deployed applications. Then choose `Next`. - 4. Enter the name of the new instance, then choose `Finish`. - -7. **Optional**. If you haven't bound an application to the service instance in the optional steps above, you can bind it from the application’s dashboard. For more information, see [Bind Service Instances to Applications Using the Cockpit](https://help.sap.com/viewer/65de2977205c403bbc107264b8eccf4b/Cloud/en-US/2d2a3e8b2f1348ffbb54eaae10d80b95.html). - - - - - -## Share Service Instance Across Different Spaces - -You can share a single service instance across multiple spaces. Skip this step if you don't need to share a single instance across multiple spaces of the same org. To share the services in an additional space, execute the following command: - -``` -cf share-service -s -``` - - - - - -## Include Logging Libraries - -> ### Caution: -> Ensure that you consider the [SAP BTP Security Recommendation BTP-CLS-0002](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations?seclist-index=BTP-CLS-0002&version=Cloud). - -We recommend using one of the Cloud Foundry open-source logging libraries \([Java](https://github.com/SAP/cf-java-logging-support)/[NodeJS](https://github.com/SAP/cf-nodejs-logging-support)\) within your application. - diff --git a/docs/ingest-via-json-api-endpoint-3416f8f.md b/docs/ingest-via-json-api-endpoint-3416f8f.md new file mode 100644 index 0000000..d7f8aea --- /dev/null +++ b/docs/ingest-via-json-api-endpoint-3416f8f.md @@ -0,0 +1,177 @@ + + +# Ingest via JSON API Endpoint + +You can ship documents to SAP Cloud Logging via `JSON` endpoint. + +> ### Note: +> Alternately, you can also [ingest logs, metrics, and traces in OpenTelemetry format](ingest-via-opentelemetry-api-endpoint-fdc78af.md). + +> ### Caution: +> Ensure that you consider [SAP BTP Security Recommendation BTP-CLS-0003](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations?seclist-index=BTP-CLS-0003). + +You can use arbitrary log shippers to send to the mTLS ingest JSON API endpoint. For example, a Fluent Bit output configuration for sending logs would be: + +``` +[OUTPUT] + Name http + Match * + Host + Port 443 + tls.crt_file + tls.key_file + tls true + Compress gzip + URI / + Format json + +``` + +With `Host`, `tls.crt_file`, and `tls.key_file` configured according to your service instance. + +To find and analyze the data, the payload should contain a `date` field. Use the `msg` field to contain the string information for a document. + + + + + +## Procedure + +The Ingest JSON API allows you to add `JSON` documents to your Cloud Logging instance. + + + +### Single document + +This operation adds a single document in JSON format: + +``` +PUT / +{ + "msg": "Single log ingest", + "date": 1668415176, + "additional_field":"additional value" +} + +``` + +The following fields are expected: + +- `date`: This field must contain the timestamp, under which the record will be indexed into OpenSearch. There are two formats possible: + - `Epoch Unix Timestamp`: This works correctly with all possible precisions, from seconds to nanoseconds. Format it as a number. + - `ISO 8601`: This is the standard json timestamp. Format it as a string. + - `default`: If the date field cannot be extracted from the payload, the current time while parsing is used instead. This timestamp is relevant to find the document in the OpenSearch Dashboard UI. + +- `msg`: This field must contain the string payload of a log. It is added if not present in a log, and defaults to "-". Format it as a string. +- The `additional_data` field is a placeholder for any further fields. There are no restrictions on what data you can add, but keep two things in mind: + - It supports a maximum of 1000 fields. + - Field type collisions \(such as string and object\) result in log rejects, and possibly make indices incoherent when rolling over. + + + + +### Multiple documents + +In addition to sending one document per HTTP call, you can upload multiple documents in one HTTP call using JSON batching: + +``` +PUT / +[ + { + "msg": "Message 1", + "date": 1668415178 + }, + { + "msg": "Message 2", + "date": 1668415185 + } +] + +``` + +Setting the `Content-Encoding` header allows HTTP clients to transfer payload more efficiently by applying compression. Currently, it supports gzip and deflate. You can test gzip using the following command: + +``` +echo 'json={"msg":"foobar"}' | gzip > json.gz +curl --data-binary @json.gz -H "Content-Encoding: gzip" \ + --cert client.crt --key client.key --cacert server-ca.crt \ + https:/// + +``` + +This feature is especially useful for users who handle large amounts of data. + + + +### Host and Authentication + +You can read the `ingest-mtls-endpoint`, as well as the credentials \(`ingest-mtls-username` and `ingest-mtls-password`\) that have to be added to the request header, from the service key or service binding. Find out how to create service keys or bindings from the following: [Creating Service Keys](https://help.sap.com/docs/btp/sap-business-technology-platform/creating-service-keys), [Create Bindings](https://help.sap.com/docs/btp/sap-business-technology-platform/binding-service-instances-to-applications), [Create bindings with SAP BTP Operator](https://github.com/SAP/sap-btp-service-operator/blob/main/README.md#step-2-create-a-service-binding). + +> ### Caution: +> TLS certificates for client authentication are issued with a validity period of 90 days by default. Don't forget to rotate the service key and update the credentials in all sender configurations. Otherwise, ingestion will stop! + +> ### Note: +> The validity period can be configured via service binding parameter within the range of 1 to 180 days by utilizing the certValidityDays. For example, passing `{"ingest":{"certValidityDays":30}}` as a configuration parameter for binding creation sets the validity to 30 days. + +> ### Note: +> Deleting a binding does not revoke the corresponding certificate. [Rotate the Ingestion Root CA Certificate](rotate-the-ingestion-root-ca-certificate-bbcb3e7.md) if the root CA of your service instance is expiring soon, or the private key of a certificate was leaked. + + + +### Request body + +Your request body must contain the information you want to index. + +``` +{ + "date": 123456. + "msg": "This is just a sample document", + ... +} +``` + + + +### Response + +Cloud Logging adheres to HTTP response code standards. If your request was successful, the response looks like: + +``` +HTTP/2 200 +date: Mon, 17 Oct 2022 08:52:17 GMT +content-type: text/plain +content-length: 0 +strict-transport-security: max-age=15724800; includeSubDomains + +``` + +If the payload is not valid JSON, an example response looks like: + +``` +HTTP/2 400 +date: Mon, 17 Oct 2022 08:57:54 GMT +content-type: text/plain +content-length: 51 +strict-transport-security: max-age=15724800; includeSubDomains + +400 Bad Request +Received event is not JSON: {12121 + +``` + +If you cannot find the payload shipped in the respective index, the default dashboards may provide a hint on the root cause. + + + + + +## Result + +You can analyze the ingested OpenTelemetry data in OpenSearch Dashboards \(see [Access and Analyze Observability Data](access-and-analyze-observability-data-dad5b01.md), based on custom dashboards or the Discover view. Indices match the following pattern: `logs-json-.*` + +> ### Note: +> There are OpenSearch specifics restricting ingestion, such as mapping conflicts. Mapping conflicts in OpenSearch occur when indices have conflicting field mappings, leading to ambiguity in data interpretation and query failures. Resolving these conflicts involves aligning field types, analyzers, and other properties across indices to ensure consistent data handling and accurate search results. Cloud Logging performs basic sanitation and unification steps to prevent mapping conflicts. However, sending type-aligned data is mandatory to prevent mapping conflicts. + +> ### Note: +> Cloud Logging provides parsing for w3c headers, sap\_passport, and other correlation mechanisms. + diff --git a/docs/ingest-via-opentelemetry-api-endpoint-fdc78af.md b/docs/ingest-via-opentelemetry-api-endpoint-fdc78af.md index f8b2860..b66d5d1 100644 --- a/docs/ingest-via-opentelemetry-api-endpoint-fdc78af.md +++ b/docs/ingest-via-opentelemetry-api-endpoint-fdc78af.md @@ -61,7 +61,7 @@ OpenTelemetry support in SAP Cloud Logging needs to be enabled with a service in > Ensure that you consider the [SAP BTP Security Recommendation BTP-CLS-0003](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations?seclist-index=BTP-CLS-0003&version=Cloud). > ### Note: - > Use the `certValidityDays` to configure the validity period via a service binding parameter within the range of 1 to 180 days. For example, passing `'{"ingest":{"certValidityDays":30}}` as the configuration parameter for binding creation sets the validity to 30 days. + > Use the *certValidityDays* to configure the validity period via a service binding parameter within the range of 1 to 180 days. For example, passing `'{"ingest":{"certValidityDays":30}}` as the configuration parameter for binding creation sets the validity to 30 days. > ### Note: > Deleting a binding doesn't revoke the corresponding certificate. [Rotate the Ingestion Root CA Certificate](rotate-the-ingestion-root-ca-certificate-bbcb3e7.md) if the root Certification Authority \(Certification Authority\) of your service instance expires soon, or the private key of a certificate was leaked. diff --git a/docs/prerequisites-41d8559.md b/docs/prerequisites-41d8559.md index bb89a50..9482e68 100644 --- a/docs/prerequisites-41d8559.md +++ b/docs/prerequisites-41d8559.md @@ -54,11 +54,11 @@ Create a SAML 2.0 application in your Identity Authentication account based on t ### Configure the SAML 2.0 application -Go to `Applications & Resources`, choose `Applications`, and select your application from the list. Then perform the following steps to configure the SAML 2.0 application within Identity Authentication: +Go to *Applications & Resources*, choose *Applications*, and select your application from the list. Then perform the following steps to configure the SAML 2.0 application within Identity Authentication: -1. [Configure a Self-Defined Attribute](https://help.sap.com/docs/identity-authentication/identity-authentication/user-attributes?version=Cloud) with `Name` "groups," `Source` "Identity Directory," and `Value` "Groups." -2. [Configure Default Name ID Format](https://help.sap.com/docs/identity-authentication/identity-authentication/configure-subject-name-identifier-sent-to-application?version=Cloud) to `E-mail`. -3. Select `SAML 2.0 Configuration` and `Configure Manually`. +1. [Configure a Self-Defined Attribute](https://help.sap.com/docs/identity-authentication/identity-authentication/user-attributes?version=Cloud) with *Name* "groups," *Source* "Identity Directory," and *Value* "Groups." +2. [Configure Default Name ID Format](https://help.sap.com/docs/identity-authentication/identity-authentication/configure-subject-name-identifier-sent-to-application?version=Cloud) to *E-mail*. +3. Select *SAML 2.0 Configuration* and *Configure Manually*. - Set the name with value of the `sp.entity_id` from the Create a SAML 2.0 application step. - Continue with one of the following options. **OPTION 1** is recommended, as it removes the need to specify the IdP SAML application's assertion/logout URL. - **OPTION 1:** Enable request signing. @@ -74,13 +74,13 @@ Go to `Applications & Resources`, choose `Applications`, and select your applica ``` - - Enable request signing in Identity Authentication by setting `Require signed authentication requests` to `ON`, going to the `Signing Certificate` section, clicking `Add`, and uploading the certificate. + - Enable request signing in Identity Authentication by setting *Require signed authentication requests* to *ON*, going to the *Signing Certificate* section, clicking *Add*, and uploading the certificate. - Make sure to provide a signing key to the `sp.signature_private_key` field and set the sp.signature\_private\_key\_password field if the signing key is encrypted. The signing certificate in your Identity Authentication SAML 2.0 application can expire, and Identity Authentication rejects login attempts with the error message, "The digital signature of the received SAML2 message is invalid." - **OPTION 2:** ⚠️ This step can only be done after an SAP Cloud Logging instance has been created and has to be repeated for each new service instance. - Set `Assertion Consumer Service Endpoint` to the OpenSearch Dashboards URL plus`/_opendistro/_security/saml/acs`. - Set `Single Logout Endpoint`: Set binding to HTTP\_REDIRECT and the URL must be the OpenSearch Dashboards URL without any path. - - To store the configuration, click `Save` . + - To store the configuration, click *Save* . diff --git a/docs/service-plans-a9d2d1b.md b/docs/service-plans-a9d2d1b.md index 6332d0a..896efe4 100644 --- a/docs/service-plans-a9d2d1b.md +++ b/docs/service-plans-a9d2d1b.md @@ -7,7 +7,8 @@ The SAP Cloud Logging service plans provide different ingestion and storage capa > ### Note: > Updating service plans isn't supported. The recommended migration procedure involves running instances side-by-side during a transition. -For production service plans, service instances scale automatically within the configured limits. To avoid disk overflow, there is time-based and disk-utilization-based data curation. If the disk usage watermark has been exceeded and the instance is scaled to its maximum, the system automatically deletes the oldest indices. The term `net storage capacity` used in service plan descriptions refers to the usable disk-size up to the watermark and has subtracted the disk space required for replicas. Service plans can handle peak load in relation to their storage volumes. However, service quality degradation can happen if the load exceeds the non-scaled disk capacity within one day. +> ### Caution: +> For production service plans, service instances scale automatically within the configured limits. To avoid disk overflow, there is time-based and disk-utilization-based data curation. If the disk usage watermark has been exceeded and the instance is scaled to its maximum, the system automatically deletes the oldest indices. The term `net storage capacity` used in service plan descriptions refers to the usable disk-size up to the watermark and has subtracted the disk space required for replicas. Service plans can handle peak load in relation to their storage volumes. However, service quality degradation can happen if the load exceeds the non-scaled disk capacity within one day. To get an overview on the availability of SAP Cloud Logging according to region, infrastructure provider, and release status, visit the [SAP Discovery Center](https://discovery-center.cloud.sap/protected/index.html#/serviceCatalog/cloud-logging).