Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

P2P: connection monitor running more often than configured #830

Closed
heifner opened this issue Sep 27, 2024 · 12 comments · Fixed by #838 or #844
Closed

P2P: connection monitor running more often than configured #830

heifner opened this issue Sep 27, 2024 · 12 comments · Fixed by #838 or #844
Assignees
Labels
bug The product is not working as was intended. 👍 lgtm OCI Work exclusive to OCI team
Milestone

Comments

@heifner
Copy link
Member

heifner commented Sep 27, 2024

With a connection-cleanup-period = 30 which is the default:

info  2024-09-27T13:53:11.169 net-3     net_plugin.cpp:4899           connection_monitor   ] p2p client connections: 0/4, peer connections: 10/10, block producer peers: 0
warn  2024-09-27T13:53:11.224 net-1     net_plugin.cpp:4711           operator()           ] Unable to resolve p2p.bitmars.one:8080 Host not found (authoritative)
info  2024-09-27T13:53:11.318 net-2     net_plugin.cpp:2862           operator()           ] connection failed to peer.eosn.io:9876, Connection refused
info  2024-09-27T13:53:11.445 net-3     net_plugin.cpp:1596           operator()           ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Sending handshake generation 1, lib 396158153, fhead 396158354, id f7b41de76d6e8ef5
warn  2024-09-27T13:53:11.587 net-3     net_plugin.cpp:3608           handle_message       ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Clock offset is 3495269us, calculation: (rec 1727445195011927000 - org 1727445191445814595 + xmt 1727445195011928000 - dst 1727445191587501533)/2
info  2024-09-27T13:53:11.587 net-3     net_plugin.cpp:1596           operator()           ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Sending handshake generation 2, lib 396158153, fhead 396158354, id f7b41de76d6e8ef5
info  2024-09-27T13:53:11.588 net-1     net_plugin.cpp:3465           handle_message       ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Local network version: 9
info  2024-09-27T13:53:11.588 net-1     net_plugin.cpp:2331           recv_handshake       ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] handshake lib 396440434, fhead 396440436, id fb0598a488a18012.. sync 1, fhead 396158354, lib 396158153
info  2024-09-27T13:53:12.582 net-2     net_plugin.cpp:3038           operator()           ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Peer closed connection
info  2024-09-27T13:53:12.582 net-2     net_plugin.cpp:3066           operator()           ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] Closing connection
info  2024-09-27T13:53:12.582 net-2     net_plugin.cpp:1455           _close               ] ["p2p.mainnet.eosrio.io:9876 - 259659b" - 6 187.16.79.235:9876] closing
info  2024-09-27T13:53:12.682 net-1     net_plugin.cpp:4899           connection_monitor   ] p2p client connections: 0/4, peer connections: 10/10, block producer peers: 0
@luoxiandong
Copy link

luoxiandong commented Sep 27, 2024

I have same problem

generation 540, lib 396414235, fhead 396415237, id a3f620c77e66ea08
info  2024-09-27T15:17:34.216 net-1     net_plugin.cpp:4863           connection_monitor   ] p2p client connections: 0/25, peer connections: 34/34, block producer peers: 0
info  2024-09-27T15:17:34.216 net-2     net_plugin.cpp:2833           operator()           ] connection failed to fn001.eossv.org:443, Element not found
info  2024-09-27T15:17:34.216 net-0     net_plugin.cpp:2833           operator()           ] connection failed to node.eosflare.io:1883, Element not found
info  2024-09-27T15:17:34.216 net-0     net_plugin.cpp:2833           operator()           ] connection failed to seed2.greymass.com:9876, Element not found
info  2024-09-27T15:17:34.216 net-1     net_plugin.cpp:2833           operator()           ] connection failed to node1.eoscannon.io:59876, Element not found
info  2024-09-27T15:17:34.216 net-2     net_plugin.cpp:2833           operator()           ] connection failed to publicnode.cypherglass.com:9876, Element not found
info  2024-09-27T15:17:34.216 net-3     net_plugin.cpp:2833           operator()           ] connection failed to bp.eosbeijing.one:8080, Element not found
info  2024-09-27T15:17:34.216 net-0     net_plugin.cpp:2833           operator()           ] connection failed to seed1.greymass.com:9876, Element not found
info  2024-09-27T15:17:34.217 net-1     net_plugin.cpp:1604           operator()           ] ["fullnode.eoslaomao.com:443" - 12 172.67.220.33:443] Sending handshake generation 1, lib 396414235, fhead 396415237, id a3f620c77e66ea08
error 2024-09-27T15:17:34.219 net-3     net_plugin.cpp:2981           operator()           ] ["fullnode.eoslaomao.com:443" - 12 172.67.220.33:443] incoming message length unexpected (1347703880)
info  2024-09-27T15:17:34.219 net-3     net_plugin.cpp:3039           operator()           ] ["fullnode.eoslaomao.com:443" - 12 172.67.220.33:443] Closing connection
info  2024-09-27T15:17:34.219 net-3     net_plugin.cpp:1463           _close               ] ["fullnode.eoslaomao.com:443" - 12 172.67.220.33:443] closing
info  2024-09-27T15:17:34.319 net-0     net_plugin.cpp:4863           connection_monitor   ] p2p client connections: 0/25, peer connections: 34/34, block producer peers: 0
info  2024-09-27T15:17:34.338 net-2     net_plugin.cpp:2308           recv_handshake       ] ["0.0.0.0:9876:blk - 25c26e4" - 5 136.243.72.242:9876] handshake lib 396450196, fhead 396450198, id 55b1631b65ef05a9.. sync 1, fhead 396415237, lib 396414235
info  2024-09-27T15:17:34.338 net-2     net_plugin.cpp:1604           operator()           ] ["0.0.0.0:9876:blk - 25c26e4" - 5 136.243.72.242:9876] Sending handshake generation 601, lib 396414235, fhead 396415237, id a3f620c77e66ea08
info  2024-09-27T15:17:34.468 net-3     net_plugin.cpp:2833           operator()           ] connection failed to peer.eosn.io:9876, Connection refused

Starting from tonight, how fix this problem?

@heifner
Copy link
Member Author

heifner commented Sep 27, 2024

Hopefully we can get a fix in a patch release soon. Until then, remove fullnode.eoslaomao.com from your peer list. Keep removing ones that cause the issue.

@luoxiandong
Copy link

Hopefully we can get a fix in a patch release soon. Until then, remove fullnode.eoslaomao.com from your peer list. Keep removing ones that cause the issue.
Can provide some p2p-peer-address list? tks

@heifner
Copy link
Member Author

heifner commented Sep 27, 2024

Can provide some p2p-peer-address list? tks

https://validate.eosnation.io/eos/reports/endpoints.html

@bhazzard bhazzard added bug The product is not working as was intended. 👍 lgtm and removed triage labels Sep 27, 2024
@bhazzard bhazzard added this to the Spring v1.0.2 milestone Sep 27, 2024
@heifner heifner self-assigned this Sep 27, 2024
@heifner heifner added the OCI Work exclusive to OCI team label Sep 27, 2024
@luoxiandong
Copy link

After the node starts, the RPC is requested to have no response
curl -x post http://127.0.0.1:8888/v1/chain/get_info

@luoxiandong
Copy link

image image

@heifner
Copy link
Member Author

heifner commented Sep 27, 2024

After the node starts, the RPC is requested to have no response curl -x post http://127.0.0.1:8888/v1/chain/get_info

Be very patient with that call. See #284 and #527

When syncing get_info can take seconds/minutes to respond.

@heifner
Copy link
Member Author

heifner commented Sep 27, 2024

Just tested a call to get_info while syncing to EOS Mainnet. It took 9 minutes to respond.

@luoxiandong
Copy link

Yes, the rpc is too long, which will cause our Dapp service to be unavailable. How to improve the RPC call response speed?

@heifner
Copy link
Member Author

heifner commented Sep 27, 2024

We plan on doing both #284 and #527 both of which will fix this issue.
Also when a node is in-sync the repsonse time should be as expected.

@luoxiandong
Copy link

How long does it take to have a new version?

@bhazzard
Copy link

How long does it take to have a new version?

Any date I can provide is a guess. But we're currently targeting this for 1.1.0, which is likely sometime in Q1 2025.

@heifner heifner moved this from Todo to In Progress in Team Backlog Sep 30, 2024
@heifner heifner moved this from In Progress to Awaiting Review in Team Backlog Sep 30, 2024
heifner added a commit that referenced this issue Sep 30, 2024
[1.0.2] P2P: Normally wait for connection monitor to reconnect
heifner added a commit that referenced this issue Sep 30, 2024
[1.0.2 -> main] P2P: Normally wait for connection monitor to reconnect
@github-project-automation github-project-automation bot moved this from Awaiting Review to Done in Team Backlog Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug The product is not working as was intended. 👍 lgtm OCI Work exclusive to OCI team
Projects
Archived in project
4 participants