[1.0.3] Finality violation tests 1.0.0

[systemzax]

This PR adds tests to cover violation of finality rules #1, #2 and #3 and addresses issue : #91

[arhag]

The expected bitset_strings seem very strange to me. I get what it is doing, but it does not seem intuitive at all if we were trying to mentally map it to which finalizer is present or not when just looking at the hex string.

My hope with this string representation was following:
If I want to figure out whether finalizer i is present or not in the QC, I first find i/4th nibble (using zero-indexing) in the hex string and convert that nibble into string of 4 bits. I then look at the i%4th bit (using zero-indexing) in that string of 4 bits to determine whether the finalizer is present or not.

[arhag]

For all three rules, you will need to check that the two QCs are for distinct blocks. So just check that the two computed finality digests are different here rather than having to recompute the finality digest in the different rule checking implementations.

[systemzax]

Addressed in 68c2c1d

[arhag]

This is not sufficient.

Look at the qc_sig_t type in Spring. Two separate (optional) bitsets are required. One to track whether a finalizer voted strong, and another one to track whether a finalizer voted weak.

If the second bitset is not present, then you know that if this is a valid QC then it must be a strong QC and all of the present finalizers (which can be checked in the first bitset which must be present) voted strong.

If the second bitset is present, then you know that if this is a valid QC then it must be a weak QC. A finalizer present in the QC must have either voted strong (if so they are present in the first bitset which must be present) or voted weak (if so they are present in the second bitset). They cannot be present in both bitsets. If a finalizer is not present in either bitset, then it its vote (assuming it even voted) was not present as part of this QC.

This level of detail is necessary to properly validate the signature of a weak QC because some finalizers may have voted weak while others voted strong.

[systemzax]

Addressed in eca9352

[arhag]

You must create up to two messages. A strong_message capturing the finality_digest if a strong bitset is present. And a weak_digest capturing the result of create_weak_digest(finality_digest) if a weak bitset is present.

[arhag]

You would have also needed to construct up to two different aggregated public keys. A strong_agg_pub_key would aggregate the public keys of finalizers that voted strong (present in the strong bitset) and a weak_agg_pub_key would aggregate the public keys of finalizers that voted weak (present in the weak bitset).

[systemzax]

Addressed in eca9352

[arhag]

The _verify call needs to be updated to support verifying the signature with a vector of public keys and a vector of messages where the length of both vectors is the same (in this case either length of 1 or 2).

See:

spring/libraries/chain/finality/qc.cpp

Lines 118 to 152 in 53b2392

	// no reason to use bls_public_key wrapper
	std::vector<bls12_381::g1> pubkeys;
	pubkeys.reserve(2);
	std::vector<std::vector<uint8_t>> digests;
	digests.reserve(2);
	// utility to aggregate public keys for verification
	auto aggregate_pubkeys = [&](const auto& votes_bitset) -> bls12_381::g1 {
	const auto n = std::min(num_finalizers, votes_bitset.size());
	std::vector<bls12_381::g1> pubkeys_to_aggregate;
	pubkeys_to_aggregate.reserve(n);
	for(auto i = 0u; i < n; ++i) {
	if (votes_bitset[i]) { // ith finalizer voted
	pubkeys_to_aggregate.emplace_back(finalizers[i].public_key.jacobian_montgomery_le());
	}
	}
	return bls12_381::aggregate_public_keys(pubkeys_to_aggregate);
	};
	// aggregate public keys and digests for strong and weak votes
	if( strong_votes ) {
	pubkeys.emplace_back(aggregate_pubkeys(*strong_votes));
	digests.emplace_back(std::vector<uint8_t>{strong_digest.data(), strong_digest.data() + strong_digest.data_size()});
	}
	if( weak_votes ) {
	pubkeys.emplace_back(aggregate_pubkeys(*weak_votes));
	digests.emplace_back(std::vector<uint8_t>{weak_digest.begin(), weak_digest.end()});
	}
	// validate aggregated signature
	EOS_ASSERT( bls12_381::aggregate_verify(pubkeys, digests, sig.jacobian_montgomery_le()),
	invalid_qc_claim, "qc signature validation failed" );

[systemzax]

Addressed in eca9352

[arhag]

For rule 2 violation, we only want to count a finalizer as a bad actor if it has voted in the low_proof (weak or strong doesn't matter) and it has voted strong in the high_proof.

[arhag]

For rule 3 violation, we only want to count a finalizer as a bad actor if it has voted in the high_proof (weak or strong doesn't matter) and it has voted strong in the low_proof.

[arhag]

This check is not correct.

The digests used to construct the Merkle tree of reversible blocks are not just the finality digests.

See:

spring/libraries/chain/finality/finality_core.cpp

Lines 166 to 173 in 53b2392

	block_ref_digest_data data = {
	.block_num = refs[i].block_num(),
	.timestamp = refs[i].timestamp,
	.finality_digest = refs[i].finality_digest,
	.parent_timestamp = refs[i-1].timestamp
	};
	block_ref_digests.emplace_back(fc::sha256::hash(data));

So the check within this action at this line should never fail. And related to that, you need test cases where the rule proof fails because the conditions are not met. We need to verify that false proofs are properly rejected.

The proper way to check this is as follows:

First, the user must submit the contents of the leaf node in the reversible block Merkle tree (along with the proof of inclusion up to the reversible blocks root) for the block that has a timestamp greater than or equal to low_proof_timestamp and a parent_timestamp strictly less than low_proof_timestamp. Both of those inequalities must be validated otherwise the submitted proof is not correct.

Then, if the timestamp for the submitted reversible blocks leaf node is strictly greater than low_proof_timestamp, then you know that the low proof block is not an ancestor of the high proof block and so a rule 2 violation has occurred.

Otherwise, in the case where the timestamp for the submitted reversible blocks leaf node is exactly equal to low_proof_timestamp, you then need to calculate computed_digest (the finality digest of the low proof block) and compare it to the finality_digest of the submitted reversible blocks leaf node to check that they are not the same. If they are the same, then again the submitted proof is not correct. But if they are not the same, then you know that the low proof block is not an ancestor of the high proof block and so a rule 2 violation has occurred.

Better yet, as an optimization, don't calculate computed_digest again since you would have already calculated once before to validate the QC of low_proof. So just cache that and reuse it if necessary.

[arhag]

This should be:

bool lock_violation = 
       (high_proof_last_claim_timestamp <= low_proof_last_claim_timestamp) &&
       (low_proof_timestamp < high_proof_timestamp);

[systemzax]

Addressed in : bae1c27

[arhag]

Again, this check is not correct for the same reasons described earlier for the rule 2 proof action. But in addition to that it is attempting to show that the low proof block is not an ancestor of the high proof block, but rule 3 requires that the block claimed by the low proof block is not an ancestor of the high proof block.

A similar process as before needs to be carried out here:

The proper way to check this is as follows:

First, the user must submit the contents of the leaf node in the reversible block Merkle tree (along with the proof of inclusion up to the reversible blocks root) for the block that has a timestamp greater than or equal to low_proof_last_claim_timestamp and a parent_timestamp strictly less than low_proof_last_claim_timestamp. Both of those inequalities must be validated otherwise the submitted proof is not correct.

Then, if the timestamp for the submitted reversible blocks leaf node is strictly greater than low_proof_last_claim_timestamp, then you know that the block claimed by the low proof block is not an ancestor of the high proof block and so a rule 3 violation has occurred.

Otherwise, in the case where the timestamp for the submitted reversible blocks leaf node is exactly equal to low_proof_last_claim_timestamp, you then need to get finality digest of the block claimed by the low proof block (low_proof.qc_block.level_3_commitments->latest_qc_claim_finality_digest) and compare it to the finality_digest of the submitted reversible blocks leaf node to check that they are not the same. If they are the same, then again the submitted proof is not correct. But if they are not the same, then you know that the block claimed by the low proof block is not an ancestor of the high proof block and so a rule 3 violation has occurred.

[ericpassmore]

Note:start
group: STABILITY
category: TEST
summary: Tests to cover violation of finality rule.
Note:end