Skip to content

Commit

Permalink
Merge branch 'grafana:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
duncan485 authored Oct 22, 2024
2 parents 0fd9f3d + 1db3b0e commit 1140985
Show file tree
Hide file tree
Showing 5 changed files with 51 additions and 5 deletions.
2 changes: 1 addition & 1 deletion docs/developer/writing-exporter-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ prometheus.exporter.blackbox "example" {

## Registering the component

In order to make the component visible to Alloy configurations, it needs to be added to [all.go](../../component/all/all.go) file.
In order to make the component visible to Alloy configurations, it needs to be added to [all.go](../../internal/component/all/all.go) file.

## Documentation

Expand Down
16 changes: 16 additions & 0 deletions docs/sources/reference/components/loki/loki.secretfilter.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ The detection is based on regular expression patterns, defined in the [Gitleaks

{{< admonition type="caution" >}}
Personally Identifiable Information (PII) or undefined secret types could remain undetected.
This component may generate false positives.
Don't rely solely on this component to redact sensitive information.
{{< /admonition >}}

[gitleaks]: https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml
Expand Down Expand Up @@ -49,6 +51,20 @@ The Gitleaks configuration file embedded in the component is used if you don't p

The `types` argument is a map of secret types to look for. The values are used as prefixes for the secret types in the Gitleaks configuration. If you don't provide this argument, all types are used.

{{< admonition type="note" >}}
Configuring this argument with the secret types you want to look for is strongly recommended.
If you don't, the component will look for all known types, which is resource-intensive.
{{< /admonition >}}

{{< admonition type="caution" >}}
Some secret types in the Gitleaks configuration file rely on regular expression patterns that don't detect the secret itself but rather the context around it.
For example, the `aws-access-token` type detects AWS key IDs, not the keys themselves.
This is because the keys don't have a unique pattern that can easily be detected with a regular expression.
As a result, with this secret type enabled, the component will redact key IDs but not actual secret keys.
This behavior is consistent with the Gitleaks redaction feature but may not be what you expect.
Currently, the secret types known to have this behavior are: `aws-access-token`.
{{< /admonition >}}

The `redact_with` argument is a string that can use variables such as `$SECRET_NAME` (replaced with the matching secret type) and `$SECRET_HASH`(replaced with the sha1 hash of the secret).

The `include_generic` argument is a boolean that includes the generic API key rule in the Gitleaks configuration file if set to `true`. It's disabled by default because it can generate false positives.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,37 @@ prometheus.remote_write "default" {
}
```

## Troubleshooting

### Out of order errors

You may sometimes see an "out of order" error in the {{< param "PRODUCT_NAME" >}} log files.
This means that {{< param "PRODUCT_NAME" >}} sent a metric sample that has an older timestamp than a sample that the database already ingested.
If your database is Mimir, the exact name of the [Mimir error][mimir-ooo-err] is `err-mimir-sample-out-of-order`.

The most common cause for this error is that there is more than one {{< param "PRODUCT_NAME" >}} instance scraping the same target.
To troubleshoot, take the following steps in order:
1. If you use clustering, check if the number of {{< param "PRODUCT_NAME" >}} instances changed at the time the error was logged.
This is the only situation in which it is normal to experience an out of order error.
The error would only happen for a short period, until the cluster stabilizes and all {{< param "PRODUCT_NAME" >}} instances have a new list of targets.
Since the time duration for the cluster to stabilize is expected to be much shorter than the scrape interval, this isn't a real problem.
If the out of order error you see is not related to scaling of clustered collectors, it must be investigated.
1. Check if there are active {{< param "PRODUCT_NAME" >}} instances which should not be running.
There may be an older {{< param "PRODUCT_NAME" >}} instance that wasn't shut down before a new one was started.
1. Inspect the configuration to see if there could be multiple {{< param "PRODUCT_NAME" >}} instances which scrape the same target.
1. Inspect the WAL to see which {{< param "PRODUCT_NAME" >}} instance sent those metric samples.
The WAL is located in a directory set by the [run command][run-cmd] `--storage.path` argument.
You can use [Promtool][promtool] to inspect it and find out which metric series were sent by this {{< param "PRODUCT_NAME" >}} instance since the last WAL truncation event.
For example:
```
./promtool tsdb dump --match='{__name__=\"otelcol_connector_spanmetrics_duration_seconds_bucket\", http_method=\"GET\", job=\"ExampleJobName\"' /path/to/wal/
```

[clustering]: ../../configure/clustering
[mimir-ooo-err]: https://grafana.com/docs/mimir/latest/manage/mimir-runbooks/#err-mimir-sample-out-of-order
[run-cmd]: ../../cli/run
[promtool]: https://prometheus.io/docs/prometheus/latest/command-line/promtool/#promtool-tsdb

## Technical details

`prometheus.remote_write` uses [snappy][] for compression.
Expand Down
4 changes: 0 additions & 4 deletions docs/sources/reference/config-blocks/import.file.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,8 @@ description: Learn about the import.file configuration block
title: import.file
---

<span class="badge docs-labels__stage docs-labels__item">Public preview</span>

# import.file

{{< docs/shared lookup="stability/public_preview.md" source="alloy" version="<ALLOY_VERSION>" >}}

The `import.file` block imports custom components from a file or a directory and exposes them to the importer.
`import.file` blocks must be given a label that determines the namespace where custom components are exposed.

Expand Down
3 changes: 3 additions & 0 deletions docs/sources/set-up/migrate/from-static.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ This conversion allows you to take full advantage of the many additional feature

* _`<INPUT_CONFIG_PATH>`_: The full path to the configuration file for Grafana Agent Static.
* _`<OUTPUT_CONFIG_PATH>`_: The full path to output the {{< param "PRODUCT_NAME" >}} configuration.

1. [Stop][] Grafana Agent static.

1. [Run][run alloy] {{< param "PRODUCT_NAME" >}} using the new {{< param "PRODUCT_NAME" >}} configuration from _`<OUTPUT_CONFIG_PATH>`_:

Expand Down Expand Up @@ -387,3 +389,4 @@ The following list is specific to the convert command and not {{< param "PRODUCT
[otelcol.receiver.otlp]: ../../../reference/components/otelcol/otelcol.receiver.otlp/
[otelcol.processor.batch]: ../../../reference/components/otelcol/otelcol.processor.batch/
[otelcol.exporter.otlp]:../../../reference/components/otelcol/otelcol.exporter.otlp/
[Stop]: https://grafana.com/docs/agent/latest/static/set-up/start-agent

0 comments on commit 1140985

Please sign in to comment.