diff --git a/docs/shipping/Code/http.md b/docs/shipping/Code/http.md index ea0f38c1..7b6901a6 100644 --- a/docs/shipping/Code/http.md +++ b/docs/shipping/Code/http.md @@ -1,10 +1,10 @@ --- id: HTTP title: HTTP -overview: Ship logs from your code directly to the Logz.io listener as a minified JavaScript Object Notation (JSON) file, a standard text-based format for representing structured data based on JavaScript object syntax. -product: ['logs'] -os: ['windows', 'linux'] -filters: ['Code', 'Most Popular'] +overview: Ship data from your code directly to the Logz.io listener as a minified JavaScript Object Notation (JSON) file, a standard text-based format for representing structured data based on JavaScript object syntax. +product: ["logs", "traces"] +os: ["windows", "linux"] +filters: ["Code", "Most Popular"] logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/json.svg logs_dashboards: [] logs_alerts: [] @@ -138,5 +138,173 @@ Allow some time for data ingestion, then open [Open Search Dashboards](https://a Encounter an issue? See our [log shipping troubleshooting](https://docs.logz.io/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/) guide. + + + +This guide provides step-by-step instructions to Logz.io users on how to send logs in Protobuf format using the OTLP listener. Follow these steps to set up your environment and send logs via the OTLP protocol using the protocurl tool. + +## Download `protocurl` + +`protocurl` is a tool based on curl and Protobuf, designed for working with Protobuf-encoded requests over HTTP. Follow the instructions on the [`protocurl` GitHub page](https://github.com/qaware/protocurl) to download and install the tool on your machine. + +Once installed, verify the installation with: + +```bash +protocurl --version +``` + +## Download OpenTelemetry Protobuf Definitions + +Download the OpenTelemetry Protobuf definitions from the [OpenTelemetry-proto GitHub repository](https://github.com/open-telemetry/opentelemetry-proto/tree/main). + +You need the `.proto` files to compile Protobuf messages and send logs. Download the repository to a local folder, for example: + +```bash +git clone ~/Downloads/proto/opentelemetry-proto +``` + +## Prepare the Command + +Once `protocurl` is installed and the OpenTelemetry Protobuf files are downloaded, you can start sending logs. + +Here is the basic structure of the command: + +```bash +protocurl -v \\ + -I ~/Downloads/proto/opentelemetry-proto \\ + -i opentelemetry.proto.collector.logs.v1.ExportLogsServiceRequest \\ + -o opentelemetry.proto.collector.logs.v1.ExportLogsServiceResponse \\ + -u '' \\ + -H 'Authorization: Bearer ' \\ + -H 'user-agent: logzio-protobuf-logs' \\ + -d @export_logs_request.json +``` + +Breakdown: + +* `I`: Points to the location of the OpenTelemetry Protobuf definitions. +* `i`: Specifies the Protobuf request type (`ExportLogsServiceRequest`). +* `o`: Specifies the Protobuf response type (`ExportLogsServiceResponse`). +* `u`: URL of the Logz.io OTLP listener endpoint. Adjust the URL for your region using Logz.io [region settings](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/). +* `H`: Include headers like the Authorization token and user-agent. +* `d`: Specifies the JSON file containing the log data. + +## Prepare the JSON Data + +You need to create the `export_logs_request.json` file, which contains the structure of the log data to be sent to the OTLP listener. The required fields in the log request are as follows: + + +```bash +{ + "resourceLogs": [ + { + "resource": { + "attributes": [ + { + "key": "service.name", + "value": { "stringValue": "example-service" } + } + ] + }, + "scopeLogs": [ + { + "scope": { + "name": "example-scope" + }, + "logRecords": [ + { + "timeUnixNano": "", // e.g., 1727270794000000000 + "severityNumber": "SEVERITY_NUMBER_INFO", + "severityText": "INFO", + "body": { "stringValue": "Log message here" } + } + ] + } + ] + } + ] +} +``` + +Key fields: + +* `timeUnixNano`: A required field that represents the timestamp in nanoseconds since epoch (e.g., `1727270794000000000` for a future time). This needs to be manually set, but you can automate it in future feature requests. +* `severityNumber`: Log severity level (e.g., `SEVERITY_NUMBER_INFO`). +* `body`: The log message content. + +## Adjust the Region URL + +You must adjust the OTLP listener URL based on your Logz.io account region. Find the correct endpoint for your region [here](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/). + +For example, if you're in the US region, your endpoint might be: + +```bash +-u '' +``` + +For EU region: + +```bash +-u '' +``` + +## Full Command Example + +```bash +protocurl -v \\ + -I ~/Downloads/proto/opentelemetry-proto \\ + -i opentelemetry.proto.collector.logs.v1.ExportLogsServiceRequest \\ + -o opentelemetry.proto.collector.logs.v1.ExportLogsServiceResponse \\ + -u '' \\ + -H 'Authorization: Bearer ' \\ + -H 'user-agent: logzio-protobuf-logs' \\ + -d @export_logs_request.json +``` + +## Sample Output in Console + +When you run the command, you should see an output like this in your console: + +```bash +=========================== POST Request JSON =========================== >>> +{ + "resource_logs": [ + { + "resource": { + "attributes": [ + { + "key": "service.name", + "value": {"string_value": "example-service"} + } + ] + }, + "scope_logs": [ + { + "scope": { + "name": "example-scope" + }, + "log_records": [ + { + "time_unix_nano": "1727065212000000000", + "severity_number": "SEVERITY_NUMBER_INFO", + "severity_text": "INFO", + "body": {"string_value": "Log message here"} + } + ] + } + ] + } + ] +} +=========================== POST Request Binary =========================== >>> +00000000 0a 5c 0a 23 0a 21 0a 0c 73 65 72 76 69 63 65 2e |.\\.#.!..service.| +00000010 6e 61 6d 65 12 11 0a 0f 65 78 61 6d 70 6c 65 2d |name....example-| +... +=========================== POST Response Headers =========================== <<< +HTTP/1.1 200 OK +``` + +If everything is set correctly, you should see an HTTP status code `200 OK`, indicating the logs were successfully sent. +