in_kubernetes_events: Efficiently stream kubernetes events via watch #8351
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 tasks
edsiper
requested changes
Jan 9, 2024
ryanohnemus
force-pushed
the
feature/in_k8s_watch
branch
from
14b804c to
f4093d7
Compare
|
@edsiper - just force pushed a new version of this. I still have this PR draft mode because it was branched off of 2 other PRs: #8316 & #8323. I was assuming it would be easier to review those each individually and then I'd rebase this one with a smaller set of changes to review, but if you'd prefer to just do them all in one change PR review/merge, I can move this pr to ready to review. |
ryanohnemus
force-pushed
the
feature/in_k8s_watch
branch
from
f4093d7 to
8809b42
Compare
ryanohnemus
force-pushed
the
feature/in_k8s_watch
branch
2 times, most recently
from
8639849 to
589a7cb
Compare
ryanohnemus
force-pushed
the
feature/in_k8s_watch
branch
from
589a7cb to
9e61635
Compare
7 tasks
Potentially breaking change as it now requires the rbac used by fluent-bit to have 'watch'. Uses a k8s watch instead of http api polling to stream k8s events from the kube api server Signed-off-by: ryanohnemus <[email protected]>
Signed-off-by: ryanohnemus <[email protected]>
Signed-off-by: ryanohnemus <[email protected]>
ryanohnemus
force-pushed
the
feature/in_k8s_watch
branch
from
15cf21a to
f526aa4
Compare
|
We probably should update docs as well, particularly with the RBAC change. Could you link a docs PR @ryanohnemus ? Do we have any int tests for this btw? |
ryanohnemus
added a commit
to ryanohnemus/fluent-bit-docs
that referenced
this pull request
Jun 19, 2024
fluent/fluent-bit#8351 Signed-off-by: ryanohnemus <[email protected]>
|
@patrick-stephens added doc via fluent/fluent-bit-docs#1396 No int tests, but I added unit tests for the plugin in this PR. |
lecaros
pushed a commit
to fluent/fluent-bit-docs
that referenced
this pull request
Aug 16, 2024
* in_kubernetes_events: add k8s permission details fluent/fluent-bit#8351 Signed-off-by: ryanohnemus <[email protected]> * Update pipeline/inputs/kubernetes-events.md Co-authored-by: Adam Locke <[email protected]> Signed-off-by: Ryan Ohnemus <[email protected]> * Update pipeline/inputs/kubernetes-events.md Co-authored-by: Adam Locke <[email protected]> Signed-off-by: Ryan Ohnemus <[email protected]> * Add hard return to break section Signed-off-by: Adam Locke <[email protected]> --------- Signed-off-by: ryanohnemus <[email protected]> Signed-off-by: Ryan Ohnemus <[email protected]> Signed-off-by: Adam Locke <[email protected]> Co-authored-by: Adam Locke <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change
in_kubernetes_eventsplugin to watch kubernetes events after requesting the event list. Instead of polling for the full event list every 500ms (default), an initial full events list is requested and then is a watch is requested. The watch will create an efficient http chunked stream that will push events as they are added, modified, or deleted from the cluster. The interval_sec and interval_nsec plugin config options now act as a reconnect timer if the watch stream is ended, instead of timer to re-poll the k8s cluster.Potentially Breaking: this will require the kubernetes role used by fluent-bit to have
watchpermission in addition to the currentlistandgetpermissions.Fixes #8315
Leaving in draft as this is dependent on both #8316 & #8323, will rebase and move out of draft after those are reviewed/merged.
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.