Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: choose peers from the connected pool, instead of dialing unconnected peers #2096

Closed
wants to merge 3 commits into from

Conversation

danisharora099
Copy link
Collaborator

@danisharora099 danisharora099 commented Aug 6, 2024

Problem

With #2065, we changed the logic to fetch peers for a protocol from choosing from connected peers to finding all available peers (unconnected), and attempting to dial them instead.

This does not seem correct considering if the peer was dialable, it should have been connected already. If the peer is actually healthy, we can assume it has a connection open.

Solution

We should find connected peers that support the given protocol, and do not have a stream opened for that protocol yet, and then connect over that protocol.

Notes

I understand this change in #2065 was introduced because it was seen that in practicality peers weren't found when using this strategy, which seems like there is actually something else preventing it.
Will do further investigations.

This is also similar to #1758.

Copy link

github-actions bot commented Aug 6, 2024

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 87.08 KB (+0.08% 🔺) 1.8 s (+0.08% 🔺) 9.7 s (-42.16% 🔽) 11.4 s
Waku Simple Light Node 154.66 KB (-0.1% 🔽) 3.1 s (-0.1% 🔽) 28 s (+72.24% 🔺) 31 s
ECIES encryption 22.94 KB (0%) 459 ms (0%) 6.9 s (+65.18% 🔺) 7.4 s
Symmetric encryption 22.39 KB (0%) 448 ms (0%) 2.5 s (-28.65% 🔽) 2.9 s
DNS discovery 72.38 KB (0%) 1.5 s (0%) 17.7 s (+52% 🔺) 19.1 s
Peer Exchange discovery 73.73 KB (0%) 1.5 s (0%) 10.2 s (-9.8% 🔽) 11.7 s
Local Peer Cache Discovery 67.63 KB (0%) 1.4 s (0%) 16.7 s (+55.56% 🔺) 18.1 s
Privacy preserving protocols 38.97 KB (0%) 780 ms (0%) 5.7 s (+0.31% 🔺) 6.5 s
Waku Filter 81.36 KB (+0.01% 🔺) 1.7 s (+0.01% 🔺) 19 s (+6.01% 🔺) 20.7 s
Waku LightPush 78.62 KB (+0.03% 🔺) 1.6 s (+0.03% 🔺) 14.8 s (-16.56% 🔽) 16.4 s
History retrieval protocols 79.12 KB (-0.06% 🔽) 1.6 s (-0.06% 🔽) 14.1 s (+29.76% 🔺) 15.7 s
Deterministic Message Hashing 7.38 KB (0%) 148 ms (0%) 5.5 s (+134.63% 🔺) 5.7 s

@weboko
Copy link
Collaborator

weboko commented Aug 13, 2024

Good initiative, we discussed this 2 weeks ago and reached some solution.

I understand this change in #2065 was introduced because it was seen that in practicality peers weren't found when using this strategy, which seems like there is actually something else preventing it.

The problem was that when a protocol was trying to renew a peer - there were no connected peers (because connection manager didn't kick in yet) and so that new peer was not added.

Also, there were some other problems that #2065 has solved, e.g it happened that renewal degraded over time and was switching between couple of peers.

As for solution we agreed that following logic makes sense:

  • try to find a suitable peer in connected pool;
  • if previous step doesn't happen - try to establish new connection;

@danisharora099
Copy link
Collaborator Author

Also, there were some other problems that #2065 has solved, e.g it happened that renewal degraded over time and was switching between couple of peers

The errors received from nodes are RLN limits being hit. I suggest to iteratively dogfood and improve once we have infra sorted + bug tickets filed.

@danisharora099 danisharora099 marked this pull request as ready for review August 23, 2024 09:22
@danisharora099 danisharora099 requested a review from a team as a code owner August 23, 2024 09:22
this.log.warn("No new peers found.");
this.log.warn("No new peers found, trying with bootstrap peers");
newPeers = await this.core.getPeers({
maxBootstrapPeers: numPeers,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't previous line return all bootstrap peers already?
I mean 0 is for returning all peers, right?

@@ -198,11 +198,6 @@ export class BaseProtocolSDK implements IBaseProtocolSDK {
this.log.info(`Finding and adding ${numPeers} new peers`);
try {
const additionalPeers = await this.findAdditionalPeers(numPeers);
const dials = additionalPeers.map((peer) =>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is it removed?

@@ -227,10 +222,17 @@ export class BaseProtocolSDK implements IBaseProtocolSDK {
private async findAdditionalPeers(numPeers: number): Promise<Peer[]> {
this.log.info(`Finding ${numPeers} additional peers`);
try {
let newPeers = await this.core.allPeers();
let newPeers = await this.core.getPeers({
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you try this in practice, e.g dogfooding app?
I used allPeers because getPeers were not returning everything, and was limited for some reason

Copy link
Collaborator

@weboko weboko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left questions but looks good
let's dogfood it before merging, please

@weboko
Copy link
Collaborator

weboko commented Sep 11, 2024

@danisharora099 should this PR be marked as draft as per waku-org/pm#186 (comment) ?

@danisharora099 danisharora099 marked this pull request as draft October 8, 2024 08:34
@danisharora099
Copy link
Collaborator Author

pretty much continued by newer efforts related to protocol peer management #2137

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants