Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example: Count all IP addresses close to a target key #33

Merged
merged 13 commits into from
Nov 27, 2024

Conversation

SeverinAlexB
Copy link
Contributor

@SeverinAlexB SeverinAlexB commented Nov 15, 2024

Adds an example to count all unique IP address around a target key. Used to test how sloppy the algorithm is in finding a key.

The algorithm

  • Tries to find all IP addresses that are close to a random target key.
  • Does multiple lookups, starting with a fresh routing table.
  • Randomized boostrap nodes optionally.
Random Boostrap Nodes 1
Count all IP addresses around a random target_key=e4596f91c46be7ce5026c681d35952668d5a52ad k=20 max_distance=150 random_boostrap=true.
lookup=2 Ips found 20. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 0%
lookup=3 Ips found 25. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=4 Ips found 26. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=5 Ips found 27. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=6 Ips found 29. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=7 Ips found 31. Closest node distance: 141, furthest node distance: 145, overlap with previous lookup 80%
lookup=8 Ips found 32. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 70%
lookup=9 Ips found 33. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=10 Ips found 35. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=11 Ips found 35. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 80%
lookup=12 Ips found 35. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=13 Ips found 35. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=14 Ips found 35. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=15 Ips found 35. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=16 Ips found 36. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=17 Ips found 36. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=18 Ips found 36. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=19 Ips found 38. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=20 Ips found 39. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=21 Ips found 41. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 55%
lookup=22 Ips found 41. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 80%
lookup=23 Ips found 41. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=24 Ips found 41. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=25 Ips found 41. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=26 Ips found 42. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 70%
lookup=27 Ips found 42. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 70%
lookup=28 Ips found 42. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 80%
lookup=29 Ips found 44. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 60%
lookup=30 Ips found 44. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=31 Ips found 44. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=32 Ips found 44. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=33 Ips found 44. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 70%
lookup=34 Ips found 46. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 70%
lookup=35 Ips found 46. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=36 Ips found 46. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=37 Ips found 46. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 70%
lookup=38 Ips found 46. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=39 Ips found 46. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 75%
lookup=40 Ips found 46. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 80%
lookup=41 Ips found 46. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 75%
lookup=42 Ips found 47. Closest node distance: 141, furthest node distance: 145, overlap with previous lookup 75%
lookup=43 Ips found 47. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 80%
lookup=44 Ips found 47. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 75%
lookup=45 Ips found 47. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=46 Ips found 47. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 75%
lookup=47 Ips found 47. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=48 Ips found 47. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 75%
lookup=49 Ips found 47. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 75%
lookup=50 Ips found 47. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 85%
lookup=51 Ips found 47. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 70%
lookup=52 Ips found 47. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 75%
lookup=53 Ips found 47. Closest node distance: 140, furthest node distance: 146, overlap with previous lookup 65%
lookup=54 Ips found 47. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 75%
lookup=55 Ips found 47. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 70%
Random Boostrap Nodes 2

Count all IP addresses around a random target_key=ba852044276bffd0a93e7f1afffbfa9c449dda46 k=20 max_distance=150 random_bootstrap=true.
lookup=1 Ips found 20. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 0%
lookup=2 Ips found 20. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=3 Ips found 20. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=4 Ips found 20. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=5 Ips found 21. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=6 Ips found 21. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=7 Ips found 22. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=8 Ips found 22. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=9 Ips found 23. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=10 Ips found 24. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 85%
lookup=11 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=12 Ips found 24. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=13 Ips found 26. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=14 Ips found 27. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=15 Ips found 28. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 95%
lookup=16 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=17 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=18 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=19 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=20 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=21 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=22 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=23 Ips found 28. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=24 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=25 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=26 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=27 Ips found 29. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=28 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 85%
lookup=29 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=30 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=31 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=32 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=33 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=34 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=35 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=36 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=37 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 85%
lookup=38 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=39 Ips found 29. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=40 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%
lookup=41 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 70%
lookup=42 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=43 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=44 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=45 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=46 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=47 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=48 Ips found 30. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=49 Ips found 30. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=50 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=51 Ips found 30. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=52 Ips found 31. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=53 Ips found 31. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=54 Ips found 31. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=55 Ips found 31. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=56 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 85%
lookup=57 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 85%
lookup=58 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=59 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 100%
lookup=60 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=61 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=62 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%
lookup=63 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%
lookup=64 Ips found 32. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=65 Ips found 33. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=66 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=67 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=68 Ips found 33. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=69 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=70 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%
lookup=71 Ips found 33. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=72 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=73 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 90%
lookup=74 Ips found 33. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=75 Ips found 37. Closest node distance: 141, furthest node distance: 146, overlap with previous lookup 70%
lookup=76 Ips found 37. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 65%
lookup=77 Ips found 37. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=78 Ips found 37. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 95%
lookup=79 Ips found 37. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%
lookup=80 Ips found 37. Closest node distance: 140, furthest node distance: 144, overlap with previous lookup 80%

Default Boostrap Nodes 1

Count all IP addresses around a random target_key=d4573f7131c0487eb91f4cf83ad98d0d0a162d32 k=20 max_distance=150 random_bootstrap=false.
lookup=1 Ips found 20. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 0%
lookup=2 Ips found 21. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=3 Ips found 23. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=4 Ips found 23. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=5 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=6 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=7 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=8 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=9 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=10 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=11 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=12 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=13 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=14 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=15 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=16 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=17 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=18 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 80%
lookup=19 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=20 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=21 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=22 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=23 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=24 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=25 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=26 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=27 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=28 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=29 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=30 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=31 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=32 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=33 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=34 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=35 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=36 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=37 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=38 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=39 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=40 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=41 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=42 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 95%
lookup=43 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%
lookup=44 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=45 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 85%
lookup=46 Ips found 24. Closest node distance: 140, furthest node distance: 145, overlap with previous lookup 90%

Default Boostrap Nodes 2
Count all IP addresses around a random target_key=11b0954d05c206d38d4fac9116e51af51ec30c19 k=20 max_distance=150 random_boostrap=false.
lookup=1 Ips found 19. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 0%
lookup=2 Ips found 21. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 85%
lookup=3 Ips found 24. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 80%
lookup=4 Ips found 24. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 80%
lookup=5 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=6 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=7 Ips found 25. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 85%
lookup=8 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=9 Ips found 25. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 85%
lookup=10 Ips found 25. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 85%
lookup=11 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 80%
lookup=12 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 80%
lookup=13 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 75%
lookup=14 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=15 Ips found 25. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 80%
lookup=16 Ips found 25. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 95%
lookup=17 Ips found 25. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 85%
lookup=18 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=19 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 95%
lookup=20 Ips found 25. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 90%
lookup=21 Ips found 25. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 85%
lookup=22 Ips found 25. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 90%
lookup=23 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=24 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 95%
lookup=25 Ips found 25. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 85%
lookup=26 Ips found 25. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 90%
lookup=27 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 80%
lookup=28 Ips found 25. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 85%
lookup=29 Ips found 25. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=30 Ips found 26. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 75%
lookup=31 Ips found 26. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 70%
lookup=32 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=33 Ips found 26. Closest node distance: 139, furthest node distance: 144, overlap with previous lookup 85%
lookup=34 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 90%
lookup=35 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 90%
lookup=36 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=37 Ips found 26. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 90%
lookup=38 Ips found 26. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 85%
lookup=39 Ips found 26. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 95%
lookup=40 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=41 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 90%
lookup=42 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 90%
lookup=43 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 85%
lookup=44 Ips found 26. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 85%
lookup=45 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 90%
lookup=46 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 95%
lookup=47 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 95%
lookup=48 Ips found 26. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 90%
lookup=49 Ips found 26. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 85%
lookup=50 Ips found 26. Closest node distance: 138, furthest node distance: 144, overlap with previous lookup 90%
lookup=51 Ips found 26. Closest node distance: 139, furthest node distance: 145, overlap with previous lookup 80%
lookup=52 Ips found 26. Closest node distance: 138, furthest node distance: 145, overlap with previous lookup 80%

Findings:

  • Using the default bootstrap nodes limits the jitter. Randomising bootstraps helps.
  • After lastest 50 lookups, I found close to all IPs that the client will ever see.

@SeverinAlexB SeverinAlexB marked this pull request as draft November 15, 2024 11:39
@SeverinAlexB SeverinAlexB marked this pull request as ready for review November 15, 2024 13:27
@SeverinAlexB SeverinAlexB marked this pull request as draft November 15, 2024 15:23
@SeverinAlexB
Copy link
Contributor Author

Added a histogram at the end of the lookups. This gives us insights into the probability a nodes is found within a given search. The graph is not the easiest to read. I am constrained by the library. Nonetheless, it shows the distribution quite well.

Here is short explanation on how you can read this histogram:

Prints a histogram with the collected nodes
First column are the buckets indicating the hit rate. 3 .. 12 summerizes the nodes that get hit with a probability of 3 to 12% in each lookup.
Second column indicates the number of nodes that this bucket contains. [19] means 19 nodes got hit with a probability of 3 to 12%.
Third column is a visualization of the number of nodes [19].

Example1:
 3 .. 12 [ 19 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
Within one lookup, there is a chance of 3 to 12% of 19 nodes being hit. These are rarely found therefore.

Example2: 
84 .. 93 [ 15 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
Within one lookup, 15 nodes got hit in 84 to 93% of the cases. These nodes are therefore found in almost all lookups.

Full example:
 3 .. 12 [ 19 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
12 .. 21 [  2 ]: ∎∎
21 .. 30 [  3 ]: ∎∎∎
30 .. 39 [  2 ]: ∎∎
39 .. 48 [  3 ]: ∎∎∎
48 .. 57 [  0 ]: 
57 .. 66 [  0 ]: 
66 .. 75 [  0 ]: 
75 .. 84 [  1 ]: ∎
84 .. 93 [ 15 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎

Default Bootstrap Nodes - 23 IPs found - 55 lookups - Example 1

It's almost always the 17+1=18 same nodes. One node has a 50% chance and then there is a long tail.

  1 ..  11 [  2 ]: ∎∎
 11 ..  21 [  0 ]: 
 21 ..  31 [  1 ]: ∎
 31 ..  41 [  1 ]: ∎
 41 ..  51 [  0 ]: 
 51 ..  61 [  1 ]: ∎
 61 ..  71 [  0 ]: 
 71 ..  81 [  0 ]: 
 81 ..  91 [  1 ]: ∎
 91 .. 101 [ 17 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎

Default Bootstrap Nodes - 21 IPs found - 51 lookups - Example 2

Similar to the first example. 17+1+1=19 nodes are almost always found. Additional 2 nodes are found
in every ~10th lookup.

  1 ..  11 [  2 ]: ∎∎
 11 ..  21 [  0 ]: 
 21 ..  31 [  0 ]: 
 31 ..  41 [  0 ]: 
 41 ..  51 [  0 ]: 
 51 ..  61 [  0 ]: 
 61 ..  71 [  0 ]: 
 71 ..  81 [  1 ]: ∎
 81 ..  91 [  1 ]: ∎
 91 .. 101 [ 17 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎

Randomized Bootstrap Nodes - 50 IPs found - 50 lookups - Example 3

Randomizing bootstrap nodes increases the sloppyness heavily. 10+2+3=15 nodes are hit with a chance of >75% for each lookup. 1+2+3+1+2=9 nodes have an average chance of ~40% of being hit and then there is a long tail of 23+3=26 nodes that might be hit occasionally.

  2 ..  12 [ 23 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
 12 ..  22 [  3 ]: ∎∎∎
 22 ..  32 [  2 ]: ∎∎
 32 ..  42 [  1 ]: ∎
 42 ..  52 [  3 ]: ∎∎∎
 52 ..  62 [  2 ]: ∎∎
 62 ..  72 [  1 ]: ∎
 72 ..  82 [  3 ]: ∎∎∎
 82 ..  92 [  2 ]: ∎∎
 92 .. 102 [ 10 ]: ∎∎∎∎∎∎∎∎∎∎

Randomized Bootstrap Nodes - 44 IPs found - 54 lookups - Example 4

The same 11+6=17 nodes are basically hit all the time. 3+1+1=5 nodes are hit every ~50% and the long tail of 4+18=22 nodes are hit rarely.

  1 ..  11 [ 18 ]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
 11 ..  21 [  4 ]: ∎∎∎∎
 21 ..  31 [  1 ]: ∎
 31 ..  41 [  0 ]: 
 41 ..  51 [  1 ]: ∎
 51 ..  61 [  0 ]: 
 61 ..  71 [  3 ]: ∎∎∎
 71 ..  81 [  0 ]: 
 81 ..  91 [  6 ]: ∎∎∎∎∎∎
 91 .. 101 [ 11 ]: ∎∎∎∎∎∎∎∎∎∎∎

I hope this gives enough insights into how sloppy the algorithm is. Randomizing bootstrap nodes definitely adds randomness and should be added as an option in the library.

@SeverinAlexB SeverinAlexB marked this pull request as ready for review November 18, 2024 12:56
@SeverinAlexB
Copy link
Contributor Author

All checks pass. Ready to merge IMO.

@Nuhvi Nuhvi merged commit d6fc36e into pubky:main Nov 27, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants