Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource attributes from container operator are not passed to downstream connector #36254

Open
zeitlinger opened this issue Nov 7, 2024 · 3 comments
Labels
connector/otlpjson discussion needed Community discussion needed enhancement New feature or request receiver/filelog

Comments

@zeitlinger
Copy link
Member

Component(s)

connector/otlpjson, receiver/filelog

Is your feature request related to a problem? Please describe.

With this setup

receivers:
  filelog/otlp-json-logs:
    include:
      - /var/log/pods/*/*/*.log
    include_file_path: true
    operators:
      - id: container-parser
        type: container

connectors:
  otlpjson:

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    logs/raw_otlpjson:
      receivers: [ filelog/otlp-json-logs ]
      processors: [ ]
      exporters: [ otlpjson ]
    logs/otlp:
      receivers: [ otlpjson ]
      processors: [ ]
      exporters: [ debug ]

Describe the solution you'd like

The detected resource attributes from the container parser (e.g. k8s.container.name) should be added to the resource attributes of the log entries.

However, there is no way to propagate resource attributes from the file receiver to a connector IIUC.
See https://github.com/open-telemetry/opentelemetry-collector/blob/720f3a86a3de10bc679e3cb8f8f5de4b868455e6/service/internal/graph/graph.go#L296

Describe alternatives you've considered

No response

Additional context

No response

@zeitlinger zeitlinger added enhancement New feature or request needs triage New item requiring triage labels Nov 7, 2024
Copy link
Contributor

github-actions bot commented Nov 7, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@ChrsMark ChrsMark added discussion needed Community discussion needed and removed needs triage New item requiring triage labels Nov 7, 2024
@ChrsMark
Copy link
Member

ChrsMark commented Nov 7, 2024

I think that's a known generic "limitation" of this connector. Right now the connector just extracts the otlpjson from the log records' body and forwards it to the next metrics'/traces'/logs' consumer accordingly. So any attributes or resource attributes of the original log record are discarded.

We could probably provide the option to keep the original resource attributes as well but that should be optional because that assumption might not apply to every use-case.

@zeitlinger
Copy link
Member Author

We could probably provide the option to keep the original resource attributes as well but that should be optional because that assumption might not apply to every use-case.

yes, that would be great

can you provide a hint how this option could be implemented?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
connector/otlpjson discussion needed Community discussion needed enhancement New feature or request receiver/filelog
Projects
None yet
Development

No branches or pull requests

2 participants