Skip to content

Commit

Permalink
Deduplicate status messages (rancher#2859)
Browse files Browse the repository at this point in the history
Prevents fleet from crashing due to resources exceeding etcd's
configured size limit.

Deduplicate messages should only be necessary for edge cases which are
not officially supported by fleet but result in ever increasing message
sizes.

This is due to the messages being copied from one resource to another
and back again. Every resource adds its status to the message. This only
happens if a cluster group is deployed by a GitRepo, which results in a
bundle containing a cluster group. This bundle can only become ready if
the cluster group is ready, but if the cluster group points to the
cluster of the bundle that deployed the cluster group, this cannot ever
happen. The user is expected to fix this situation but deduplicating the
messages prevents the message from growing up to the point where etcd's
limit is reached and fleet crashes.

Deduplicating the messages also has the effect of not changing the
status of resources frequently, which results in less controllers being
triggered.
  • Loading branch information
p-se committed Oct 31, 2024
1 parent 615f45c commit 94a3fe9
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion internal/helmdeployer/kustomize/kstatus.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package kustomize

import (
"strings"

"github.com/rancher/fleet/internal/cmd/agent/deployer/data"
"github.com/rancher/fleet/internal/cmd/agent/deployer/summary"
fleetv1 "github.com/rancher/fleet/pkg/apis/fleet.cattle.io/v1alpha1/summary"
Expand Down Expand Up @@ -31,7 +33,18 @@ func KStatusSummarizer(obj data.Object, conditions []summary.Condition, summary
}

if result.Message != "" {
summary.Message = append(summary.Message, result.Message)
// Deduplicate status messages (https://github.com/rancher/fleet/issues/2859)
messages := make(map[string]bool)
var resultMessages []string
for _, message := range strings.Split(result.Message, ";") {
if _, ok := messages[message]; ok {
continue
}
messages[message] = true
resultMessages = append(resultMessages, message)
}

summary.Message = append(summary.Message, strings.Join(resultMessages, ";"))
}

return summary
Expand Down

0 comments on commit 94a3fe9

Please sign in to comment.