Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/fluentd #59

Merged
merged 38 commits into from
Aug 4, 2022
Merged

Feature/fluentd #59

merged 38 commits into from
Aug 4, 2022

Conversation

ricsanfre
Copy link
Owner

@ricsanfre ricsanfre commented Aug 3, 2022 •
edited
Loading

Changes Introduced

  • Logging solution (EFK) enhancements
    • Logging collection and distribution architecture based on fluentd/fluentbit re-architected
    • Prometheus integration: Adding Elasticsearch and Fluentbit/Fluentd metrics
    • Logging parsing enhancements and log fields standardization
  • Configure Kubernetes' graceful node shutdown feature
  • Monitor external nodes (gateway) in Prometheus using fluent-bit agent.
  • Upgrade software components to latest stable version
  • Update documentation

Issues fixed by this PR:

ricsanfre added 30 commits June 24, 2022 16:44
@ricsanfre
Initial commit forwarder/aggregator architecture
4175273
@ricsanfre
Changing Fluentd-aggregator imagePullPolicy and keeping fluentd-daemo…
dc42dfa 
…nset image variable
@ricsanfre
Adding custom fluentd-aggregator image and refactoring ansible tasks …
3bfd3b0 
…to install all components
@ricsanfre
Configuring fluent-elasticsearh-plugin with log_es_400_reason set to …
0a7beb0 
…true
@ricsanfre
Remove ansible variables and jinja2 code to create fluentbit configur…
46e5030 
…ation automatically from variables. Not supported FILTER configuration with duplicate keys (i.e: Rename, Remove)
@ricsanfre
Indentation issue solved in prometheus-stack chart config
b7eff6f
@ricsanfre
Updating logging documentation to include forward/aggregator architec…
f8d308a 
…ture based in fluentbit and fluentd. Separate document to include alternative fluentbit or fluentd configuration without aggregation layer.
@ricsanfre
Securing communications between fluentbit forwarders and fluentd aggr…
13736a5 
…egator
@ricsanfre
Exposing fluentd forwarding port as cluster service and configuring f…
71b363e 
…luentbit in gateway to use fluentd aggregator
@ricsanfre
Updating fluentd fluentbit configuration
099b60a
@ricsanfre
Restructuring log management documentation. Split in several pages
44f35d3
@ricsanfre
Changing fluent agent/sidecar pattern name to forwarder-only
db0bd68
@ricsanfre
Update logging documentation: adding different log forwarding archite…
f365afd 
…ctures images
@ricsanfre
Fix #54. Updating Metallb installation procedure to make it work with…
965f27b 
… release v0.13
@ricsanfre
Removing fluentd forwarding rule. It is not needed. Updating firewall…
255c509 
… configuration doc
@ricsanfre
Updating fluentbit role to release 1.0.4
c396539
@ricsanfre
Fixing typo in SAN installation document
e2eb881
@ricsanfre
Logging solution documentation update
df77851
@ricsanfre
Fixing typo in hardware documentation
3279e25
@ricsanfre
Fix #55. Removing K3s controller-manager and scheduler deprecated args
99240fc
@ricsanfre
Fluentbit: enabling filesystem buffering
9e3f090
@ricsanfre
Prometheus: adding fluent-bit monitoring
a39d4d5
@ricsanfre
Adding monitoring of fluentbit storage metrics
fcfb92a
@ricsanfre
Prometheus adding fluentd metrics
1688315
@ricsanfre
Prometheus elasticsearch exporter installation and prometheus monitor…
df9eb1d 
…ing configuration
@ricsanfre
Updating logging documentation
320da09
@ricsanfre
Closes #53: Configuring Kubernetes' graceful shutdown feature
4c230da
@ricsanfre
Fix #58: Solving issue with confliting fields in ES
8f729af
@ricsanfre
Fluentd: Update installation using official helm chart
52c5d53
@ricsanfre
Logging doc update: Adding forwarder only architecture installation i…
1f49b4a 
…nstructions
@ricsanfre ricsanfre merged commit 7a9833d into master Aug 4, 2022
@ricsanfre ricsanfre deleted the feature/fluentd branch October 18, 2022 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@ricsanfre