Verify minimum peer staking

[helins]

The minimum stake is currently set to 1e9 CVX.

Before this requirement, I was already noticing situations where finality was 10 - 20x slower when at least some peers where fairly low staked.

Using Convex 0.7.11 with a local network of 5 peers:

• Staked equally with 1e9 CVX, slow finality
• Staked equally with 1e9 + 1 CVX, fast finality
• At least 2 staked at 1e9 + 1, rest at 1e9, fast finality with occasional degradation
• Different combinations, some being staked even below 1e9, weirdly behaves a bit better than the last one

Where fast finality is below 100 ms.

Recommendations:

• Forbid setting stake below the minimum in set-peer-stake
• Ensure the minimum is enforced whenever relevant
• Ensure stakes are computed right during consensus, behaviour looks a bit odd overall

[mikera]

Not sure about forbidding setting below minimum at the CVM level: it's not exactly a hard constraint. Peers with stakes below that are likely to get ejected so it isn't sensible of course but not sure if it should be illegal. Similar to rm -rf: you can shoot yourself in the foot if you really want....

[mikera]

Please create a separate issue for places where you identify that either:
a) A minimum should be enforced but isn't
b) Stakes are computed wrong

I don't think we should make any guarantees on behaviour if a significant proportion of Peers set stake below the minimum. There should be at least a few Peers with at least e.g. 1e12 stake (which might in fact end up being the new minimum) and I'm expecting total Peer stake in the 1e14-1e16 range.

[helins]

Putting the minimum stake aside, isn't it weird how performance degrade under different stake repartition? Unclear as to what is causing it and when.

[mikera]

Not really strange, Peers ignore and stop sending messages to Peers with insufficient stake. So the network is effectively shut down if nobody has significant stake in running it (which is of course as it should be!)