Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[management] Skip account peers update if no changes affect peers #2310

Open
wants to merge 61 commits into
base: feature/optimize-network-map-updates
Choose a base branch
from

Conversation

bcmmbaga
Copy link
Contributor

@bcmmbaga bcmmbaga commented Jul 23, 2024

Describe your changes

This PR optimizes account peer updates to trigger only when necessary, reducing unnecessary network map processing and updates. These changes improve efficiency by avoiding redundant updates while ensuring all necessary updates are still performed.

Note: Network map updates are sent only if the new network map differs from the last one sent, except for TURN credential refresh updates. This occurs when the network map has been updated and has a higher serial number than the previous one.

Changes:

  1. Posture Checks:

    • Creating new posture checks no longer triggers account peer updates or sends network map updates.
    • Updating posture checks only triggers account peer updates when they are attached to a policy.
  2. Policies:

    • Saving a policy with empty rules (no source or destination groups) does not trigger account peer updates.
    • Updating a policy triggers account peer updates and sends network maps to peers, but only if the new network map differs from the last sent one.
    • Deleting a policy always triggers peer updates and sends the updated network map to peers.
  3. Setup Keys:

    • Saving a setup key does not trigger account peer updates.
  4. Groups:

    • Saving a group not used in nameserver groups, policies, or routes does not trigger account peer updates.
    • Adding or removing a peer to a group triggers account peer updates if the group is in use with DNS, policies, or routes.
    • Deleting a group does not trigger account peer updates.
  5. DNS Settings:

    • Saving DNS settings triggers account peer updates when before/after groups are in use by nameserver groups, policies or routes and if it has peers.
  6. NameServer Groups:

    • Creating a new nameserver group triggers account peer updates when distribution groups have peers.
    • Updating a nameserver group triggers account peer updates when the before or after state of distribution groups have peers.
    • Deleting a nameserver group triggers account peer updates when distribution groups have peers.
  7. Routes:

    • Creating a route triggers account peer updates when distribution groups or routing groups are not empty or there is a routing peer.
    • Saving (updating) a route triggers account peer updates when before and after distribution groups or routing groups are not empty or there is a routing peer.
    • Deleting a route triggers account peer updates when distribution groups or routing groups are not empty or there is a routing peer.
  8. Users:

    • Creating new users no longer triggers account peer updates.
    • Updating existing users triggers account peer updates if the user has a linked peer and groups propagation is enabled.
    • Deleting a regular user triggers account peer updates if there is a peer linked to the user and groups propagation is enabled.
  9. Peers:

    • Adding a peer triggers account peer updates if the peer is part of a group used in active DNS, routes, or ACL.
    • Updating a peer does not trigger account peer updates if the peer is not expired and peer login expiration is not enabled.
    • Deleting a peer triggers account peer updates if the peer was part of a group used in active DNS, routes, or policies.
  10. Removed unused UpdatePeerSSHKey method.

Issue ticket number and link

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)
  • Extended the README / documentation, if necessary

@bcmmbaga bcmmbaga changed the title Skip account peer update if no changes affect peers Skip account peers update if no changes affect peers Jul 23, 2024
Refactor the condition for updating account peers to remove redundant checks

Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from beb8bbd to 4e2cf9c Compare September 6, 2024 09:15
management/server/group_test.go Show resolved Hide resolved
Comment on lines 480 to 495
// adding a group to policy
err = manager.SavePolicy(context.Background(), account.Id, userID, &Policy{
ID: "policy",
Enabled: true,
Rules: []*PolicyRule{
{
Enabled: true,
Sources: []string{"groupA"},
Destinations: []string{"groupA"},
Bidirectional: true,
Action: PolicyTrafficActionAccept,
},
},
})
assert.NoError(t, err)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is ugly but I think we should copy the group chnaged tests for each resource type?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also for policy we should copy the tests for all 3 cases, source and dest group have peers and sorce has peers but not dest and dest has peers but not source

Comment on lines 960 to 997
t.Run("creating nameserver group with distribution group no peers", func(t *testing.T) {
done := make(chan struct{})
go func() {
peerShouldNotReceiveUpdate(t, updMsg)
close(done)
}()

newNameServerGroup, err = manager.CreateNameServerGroup(
context.Background(), account.Id, "ns-group-1", "ns-group-1", []nbdns.NameServer{{
IP: netip.MustParseAddr(peer1.IP.String()),
NSType: nbdns.UDPNameServerType,
Port: nbdns.DefaultDNSPort,
}},
[]string{"group-id"},
true, []string{}, true, userID, false,
)
assert.NoError(t, err)

select {
case <-done:
case <-time.After(time.Second):
t.Error("timeout waiting for peerShouldNotReceiveUpdate")
}
})

err = manager.GroupAddPeer(context.Background(), account.Id, "group-id", peer1.ID)
assert.NoError(t, err)

// saving a nameserver group with a distribution group with peers should update account peers and send peer update
t.Run("saving nameserver group with distribution group has peers", func(t *testing.T) {
done := make(chan struct{})
go func() {
peerShouldReceiveUpdate(t, updMsg)
close(done)
}()

err = manager.SaveNameServerGroup(context.Background(), account.Id, userID, newNameServerGroup)
assert.NoError(t, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should redo both tests for creating nameserver with group that has peers and saving nameserver with group that has no peers

Comment on lines 1091 to 1123
// use the group-id in policy
err = manager.SavePolicy(context.Background(), account.Id, userID, &Policy{
ID: "policy",
Enabled: true,
Rules: []*PolicyRule{
{
Enabled: true,
Sources: []string{"group-id"},
Destinations: []string{"group-id"},
Bidirectional: true,
Action: PolicyTrafficActionAccept,
},
},
})
require.NoError(t, err)

// Adding peer with a used group in active dns, route or policy should update account peers and send peer update
t.Run("adding peer with used group", func(t *testing.T) {
done := make(chan struct{})
go func() {
peerShouldReceiveUpdate(t, updMsg)
close(done)
}()

key, err := wgtypes.GeneratePrivateKey()
require.NoError(t, err)

expectedPeerKey := key.PublicKey().String()
peer4, _, _, err = manager.AddPeer(context.Background(), "", "regularUser1", &nbpeer.Peer{
Key: expectedPeerKey,
LoginExpirationEnabled: true,
Meta: nbpeer.PeerSystemMeta{Hostname: expectedPeerKey},
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we should also copy the tests for all resource types

Comment on lines +1031 to +1047
t.Run("updating not expired peer and peer expiration is enabled", func(t *testing.T) {
done := make(chan struct{})
go func() {
peerShouldNotReceiveUpdate(t, updMsg)
close(done)
}()

_, err := manager.UpdatePeer(context.Background(), account.Id, userID, peer2)
require.NoError(t, err)

select {
case <-done:
case <-time.After(time.Second):
t.Error("timeout waiting for peerShouldNotReceiveUpdate")
}
})

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we need more tests. E.g. peer label chnaged and login expired and the related ones with expiration disabled maybe

management/server/route_test.go Outdated Show resolved Hide resolved
require.NoError(t, err)

// updating user with linked peers should update account peers and send peer update
t.Run("updating user with no linked peers", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
t.Run("updating user with no linked peers", func(t *testing.T) {
t.Run("updating user with linked peers", func(t *testing.T) {

}
})

_ = peer4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do?

manager.peersUpdateManager.CloseChannel(context.Background(), peer4.ID)
})

// deleting user with linked peers should update account peers and no send peer update
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// deleting user with linked peers should update account peers and no send peer update
// deleting user with linked peers should update account peers and send peer update

Comment on lines +302 to +313
t.Run("Updating firewall rule", func(t *testing.T) {
newUpdateMessage1 := createMockUpdateMessage(t)
newUpdateMessage2 := createMockUpdateMessage(t)

newUpdateMessage2.NetworkMap.FirewallRules[0].Port = "443"
newUpdateMessage2.Update.NetworkMap.Serial++

message, err := isNewPeerUpdateMessage(newUpdateMessage1, newUpdateMessage2)
assert.NoError(t, err)
assert.True(t, message)
})

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either we need to check each single field for changes in the test or preferably add tests for the diff classes. We need to test for diff in each possible field while making sure that if we add a field that the test is not covering we need to notice. Or we are able to do the diffs generic whitout the need of checking each field separately

@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from d37e482 to 4e2cf9c Compare October 3, 2024 11:21
…date-group-association

# Conflicts:
#	management/server/account.go
#	management/server/peer.go
#	management/server/peer_test.go
#	management/server/policy.go
#	management/server/route.go
#	management/server/route_test.go
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
@bcmmbaga bcmmbaga changed the title Skip account peers update if no changes affect peers [management] Skip account peers update if no changes affect peers Oct 9, 2024
Signed-off-by: bcmmbaga <[email protected]>
Signed-off-by: bcmmbaga <[email protected]>
Copy link

sonarcloud bot commented Oct 9, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants