Policy transitions should use joint policies

[arhag]

When transitioning from one finalizer policy to another there is a stage in which the next policy is in the pending stage before becoming active. During this time, the code as it currently exists requires only QCs from the active finalizer policy to make progress with finality. However, for safety reasons, we require QCs from both the active and pending finalizer policies.

The rules for when the pending policy gets promoted to active still remain the same.

The QC claims also remain the same. There is still only one claim on a prior block (by block number) with an additional flag to indicate whether that QC was weak or strong. There are no changes to the block_header. This remains true even if that claimed block required a joint policy (i.e. had both an active policy and a pending policy) and therefore two different QCs are required to be attached to the block to validate its claim.

When a new QC claim is made against a block with a joint policy, the following needs to be true of quorum_certificate included in the block extension:

• two QCs are needed, one for the active policy and one for the pending policy;
• both QCs need to be valid, however they can independently be weak or strong QCs.

If the QC claim is that it is strong, then both the QCs attached in the block extension need to be strong QCs. If the QC claim is that it is weak, then there are no additional requirements on the two QCs other than they are valid QCs (weak or strong) that meet their respective thresholds.

If a new QC claim is made against a block with a single policy, then the quorum_certificate included in the block extension should only have a QC for the active policy.

We still only expect (and in fact require) at most one vote message per block from each finalizer even if the block has a joint policy in which the finalizer exists in both the active and pending policy. But when integrating the signatures from the validated vote message into pending QCs, they must be integrated appropriately into the pending QC for both the active policy and the pending policy. For vote messages on a block with a joint policy that are from finalizers that are only in one of the two policies, the signature should only be integrated into the appropriate pending QC (either active or pending).

When producing a block and searching for the best QC in its branch to include in the new block, the search must respect the rules for joint policies. If a block has a joint policy and only one of its pending QCs are valid, then it cannot be considered for inclusion as the claim for the new block.

Add a new field last_pending_finalizer_policy_start_num at the same level as last_pending_finalizer_policy_digest which tracks the block number at which the last pending finalizer policy first became pending. This is the second level which should be a struct with the name level_2_commitments. For now there are only two levels and so the base_digest will also be in this second level. Later we will add a third-level.