Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chunk_trace: use a threaded pipeline. #8097

Merged
merged 6 commits into from
Oct 29, 2023
Merged

Conversation

pwhelan
Copy link
Contributor

@pwhelan pwhelan commented Oct 26, 2023

Summary

Create a separate thread to run the pipeline for chunk traces.

With this pull request traces now use a threaded input so traces can be logged using a ring buffer. This bypasses the need to call in_emitter_add_record from the in_emitter plugin as well.

This should fix the occasional crash ocurring during the flb-rt-core_chunk_trace test as well as making traces more stable.

As an extra bonus, the use of a threaded input with ring buffers now allows for logging the output return value from outputs.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

… thread.

Use a pthread cond to coordinate the startup and then shutdown of the
pipeline thread, avoiding deadlocks and unfreed resources.

Signed-off-by: Phillip Whelan <[email protected]>
@pwhelan pwhelan temporarily deployed to pr October 26, 2023 20:07 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr October 26, 2023 20:07 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr October 26, 2023 20:07 — with GitHub Actions Inactive
@pwhelan pwhelan changed the title Pwhelan chunk trace thread chunk_trace: use a threaded pipeline. Oct 26, 2023
@pwhelan pwhelan temporarily deployed to pr October 26, 2023 20:37 — with GitHub Actions Inactive
@edsiper edsiper merged commit 06da717 into master Oct 29, 2023
44 of 45 checks passed
@edsiper edsiper deleted the pwhelan-chunk-trace-thread branch October 29, 2023 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants