Merge pull request #4253 from IntersectMBO/lehins/hardfork-initiation

Hardfork Initiation into a new era

docs/adr/2024-04-30_008-pparams-update.md

@@ -0,0 +1,83 @@
+---
+slug: 8
+title: |
+  8. Update of protocol parameters
+authors: [Alexey Kuleshevich]
+tags: [Accepted]
+---
+
+## Status
+
+Accepted
+
+## Context
+
+We need an unified approach for predicting and updating protocol parameters.
+
+Outline of the new approach.
+
+### Shelley through Babbage:
+
+1. Genesis key holders submit votes with proposals for new `PParams`. Depending on the
+   timeing within the epoch those votes will either go into the current or the future
+   proposals bucket.
+2. During the first `4k/f` slots in the the PPUP rule PParamUpdate proposals added to
+   current proposals. Besides keeping the proposals we also now keep potential protocol
+   parameters in the future pparms, but only quorum of genesis votes is reached. Potential
+   values for new `PParams` can change if genesis key holders change their votes during
+   this period, therefore they cannot yet be considered stable.
+3. The very first `TICK` that happens during the last two stability windows before the end
+   of the epoch we solidify the proposed `PParams`, thus ensuring they will be applied at
+   the next epoch boundary. (See `solidifyNextEpochPParams`)
+4. At the epoch boundary during the `NEWPP` rule, instead of counting the votes on the
+   proposals, we just looked up in the next `PParams` that were decided earlier and apply
+   them. All of the future votes that where potentially submitted before during the last
+   two stability windows of past epoch are converted to current votes, which will be
+   treated in the same way as if they were submitted in the 2nd step. This also resets the
+   future PParams for the next epoch.
+
+### Conway era forward
+
+1. Either `ParameterChange` or `HardForkInitiation` proposal is submitted and votes are
+   collected.
+2. At the epoch boundary all of the votes and proposals are snapshotted.
+3. Pulser starts the work on every `TICK` figuring out the stake distribution for `DReps`,
+   calculating the votes and ratifying the proposals. Whenever `ParameterChange` or
+   `HardForkInitiation` gets ratified in the `RATIFY` rule, the new values are immediately
+   applied to the future enact state by the `ENACT` rule. Therefore we have future
+   `PParams` at the latest two stability windows before the end of the epoch.
+4. During the first `4k/f` slots on every tick we also lazily update future `PParams`.
+5. Just as in Shelley era the very first `TICK` that happens during the last two stability
+   windows (`6k/f`) before the end of the epoch we solidify the proposed `PParams`. (See
+   `solidifyNextEpochPParams`). Unlike previous eras, in Conway this step is safe to do at
+   any point during the initial part of the epoch, because they are considered stable as
+   soon as we enter new epoch, however they are expensive to compute during that period,
+   that is why we solidify them only when we are pretty confident that the DRep pulser is
+   done and `RATIFY` with `ENACT` rules got a chance to be executed.
+6. At the epoch boundary we apply the new PParams that where solidified in the previous
+   step and reset the future pparams, thus making it ready for the next epoch. The
+   important part here is that we do not use the values from the Enact state directly, but
+   we take the futurePParams as the source of truth. This allows us to correctly update
+   the `PParams` not only using the voting process of Conway era, but allows us to apply
+   the `PParams` update from Babbage era.
+
+### Forecast
+
+It is very important that the TICKF rule does the same steps as the TICK and EPOCH (for
+Conway) or NEWPP (for pre Conway) rules. In particular same solidification and rotation of
+pparams process as in the stepsabove should happen for forcasting to work correctly.
+
+## Decision
+
+New approach to update has been implemented.
+
+## Consequences
+
+* We have a unified approach to update PParams throughout all eras starting with Shelley,
+  thus making HFC combinator much more robust and correct.
+* We remove duplicate logic from consensus that used to count up the genesis key holder votes.
+* We solve a problem where HFC is triggered in previous era, while TICK happens in the new
+  era after translation. Which previously caused protocol version not being updated
+  correctly in Conway, since protocol parameter update mechanism was vastly different from
+  the one in Babbage.
+* We finish implementation of HardForkInitiation into a new era.

eras/allegra/impl/src/Cardano/Ledger/Allegra/PParams.hs

@@ -43,17 +43,14 @@ instance Crypto c => EraPParams (AllegraEra c) where
 
 instance Crypto c => EraGov (AllegraEra c) where
   type GovState (AllegraEra c) = ShelleyGovState (AllegraEra c)
-  emptyGovState =
-    ShelleyGovState
-      emptyPPPUpdates
-      emptyPPPUpdates
-      emptyPParams
-      emptyPParams
+  emptyGovState = emptyShelleyGovState
 
   getProposedPPUpdates = Just . sgsCurProposals
 
   curPParamsGovStateL = curPParamsShelleyGovStateL
 
   prevPParamsGovStateL = prevPParamsShelleyGovStateL
 
+  futurePParamsGovStateL = futurePParamsShelleyGovStateL
+
   obligationGovState = const mempty

eras/allegra/impl/src/Cardano/Ledger/Allegra/Translation.hs

@@ -1,6 +1,7 @@
 {-# LANGUAGE DataKinds #-}
 {-# LANGUAGE FlexibleContexts #-}
 {-# LANGUAGE FlexibleInstances #-}
+{-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE MultiParamTypeClasses #-}
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE RecordWildCards #-}
@@ -88,6 +89,12 @@ instance Crypto c => TranslateEra (AllegraEra c) PParams
 
 instance Crypto c => TranslateEra (AllegraEra c) PParamsUpdate
 
+instance Crypto c => TranslateEra (AllegraEra c) FuturePParams where
+  translateEra ctxt = \case
+    NoPParamsUpdate -> pure NoPParamsUpdate
+    DefinitePParamsUpdate pp -> DefinitePParamsUpdate <$> translateEra ctxt pp
+    PotentialPParamsUpdate mpp -> PotentialPParamsUpdate <$> mapM (translateEra ctxt) mpp
+
 instance Crypto c => TranslateEra (AllegraEra c) ProposedPPUpdates where
   translateEra ctxt (ProposedPPUpdates ppup) =
     return $ ProposedPPUpdates $ Map.map (translateEra' ctxt) ppup
@@ -100,6 +107,7 @@ instance Crypto c => TranslateEra (AllegraEra c) ShelleyGovState where
         , sgsFutureProposals = translateEra' ctxt $ sgsFutureProposals ps
         , sgsCurPParams = translateEra' ctxt $ sgsCurPParams ps
         , sgsPrevPParams = translateEra' ctxt $ sgsPrevPParams ps
+        , sgsFuturePParams = translateEra' ctxt $ sgsFuturePParams ps
         }
 
 instance Crypto c => TranslateEra (AllegraEra c) ShelleyTxOut where

eras/alonzo/impl/src/Cardano/Ledger/Alonzo/PParams.hs

@@ -120,7 +120,6 @@ import Cardano.Ledger.Plutus.ExUnits (
 import Cardano.Ledger.Plutus.Language (Language (..))
 import Cardano.Ledger.Shelley.PParams (
   ShelleyPParams (..),
-  emptyPPPUpdates,
   shelleyCommonPParamsHKDPairs,
   shelleyCommonPParamsHKDPairsV6,
   shelleyCommonPParamsHKDPairsV8,
@@ -378,19 +377,16 @@ instance Crypto c => AlonzoEraPParams (AlonzoEra c) where
 
 instance Crypto c => EraGov (AlonzoEra c) where
   type GovState (AlonzoEra c) = ShelleyGovState (AlonzoEra c)
-  emptyGovState =
-    ShelleyGovState
-      emptyPPPUpdates
-      emptyPPPUpdates
-      emptyPParams
-      emptyPParams
+  emptyGovState = emptyShelleyGovState
 
   getProposedPPUpdates = Just . sgsCurProposals
 
   curPParamsGovStateL = curPParamsShelleyGovStateL
 
   prevPParamsGovStateL = prevPParamsShelleyGovStateL
 
+  futurePParamsGovStateL = futurePParamsShelleyGovStateL
+
   obligationGovState = const mempty
 
 instance Era era => EncCBOR (AlonzoPParams Identity era) where

eras/alonzo/impl/src/Cardano/Ledger/Alonzo/Translation.hs

@@ -1,5 +1,6 @@
 {-# LANGUAGE DerivingStrategies #-}
 {-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE MultiParamTypeClasses #-}
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE RecordWildCards #-}
@@ -68,6 +69,12 @@ instance Crypto c => TranslateEra (AlonzoEra c) NewEpochState where
 instance Crypto c => TranslateEra (AlonzoEra c) PParams where
   translateEra (AlonzoGenesisWrapper upgradeArgs) = pure . upgradePParams upgradeArgs
 
+instance Crypto c => TranslateEra (AlonzoEra c) FuturePParams where
+  translateEra ctxt = \case
+    NoPParamsUpdate -> pure NoPParamsUpdate
+    DefinitePParamsUpdate pp -> DefinitePParamsUpdate <$> translateEra ctxt pp
+    PotentialPParamsUpdate mpp -> PotentialPParamsUpdate <$> mapM (translateEra ctxt) mpp
+
 newtype Tx era = Tx {unTx :: Core.Tx era}
 
 instance Crypto c => TranslateEra (AlonzoEra c) Tx where
@@ -152,6 +159,7 @@ instance Crypto c => TranslateEra (AlonzoEra c) ShelleyGovState where
         , sgsFutureProposals = translateEra' ctxt $ sgsFutureProposals ps
         , sgsCurPParams = translateEra' ctxt $ sgsCurPParams ps
         , sgsPrevPParams = translateEra' ctxt $ sgsPrevPParams ps
+        , sgsFuturePParams = translateEra' ctxt $ sgsFuturePParams ps
         }
 
 instance Crypto c => TranslateEra (AlonzoEra c) ProposedPPUpdates where

eras/alonzo/impl/testlib/Test/Cardano/Ledger/Alonzo/Arbitrary.hs

@@ -337,7 +337,6 @@ instance Era era => Arbitrary (AlonzoContextError era) where
 
 instance
   ( EraTxOut era
-  , Era era
   , Arbitrary (Value era)
   , Arbitrary (TxOut era)
   , Arbitrary (PredicateFailure (EraRule "UTXOS" era))

eras/babbage/impl/src/Cardano/Ledger/Babbage/PParams.hs

@@ -95,7 +95,7 @@ import Cardano.Ledger.Core (EraPParams (..))
 import Cardano.Ledger.Crypto (Crypto)
 import Cardano.Ledger.HKD (HKD, HKDFunctor (..))
 import Cardano.Ledger.Orphans ()
-import Cardano.Ledger.Shelley.PParams (emptyPPPUpdates, shelleyCommonPParamsHKDPairsV8)
+import Cardano.Ledger.Shelley.PParams (shelleyCommonPParamsHKDPairsV8)
 import Control.DeepSeq (NFData)
 import Data.Aeson as Aeson (
   FromJSON (..),
@@ -263,19 +263,16 @@ instance Crypto c => BabbageEraPParams (BabbageEra c) where
 
 instance Crypto c => EraGov (BabbageEra c) where
   type GovState (BabbageEra c) = ShelleyGovState (BabbageEra c)
-  emptyGovState =
-    ShelleyGovState
-      emptyPPPUpdates
-      emptyPPPUpdates
-      emptyPParams
-      emptyPParams
+  emptyGovState = emptyShelleyGovState
 
   getProposedPPUpdates = Just . sgsCurProposals
 
   curPParamsGovStateL = curPParamsShelleyGovStateL
 
   prevPParamsGovStateL = prevPParamsShelleyGovStateL
 
+  futurePParamsGovStateL = futurePParamsShelleyGovStateL
+
   obligationGovState = const mempty
 
 instance Era era => EncCBOR (BabbagePParams Identity era) where

eras/babbage/impl/src/Cardano/Ledger/Babbage/Translation.hs

@@ -1,5 +1,6 @@
 {-# LANGUAGE DerivingStrategies #-}
 {-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE MultiParamTypeClasses #-}
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE RecordWildCards #-}
@@ -93,6 +94,12 @@ instance Crypto c => TranslateEra (BabbageEra c) Tx where
 instance Crypto c => TranslateEra (BabbageEra c) PParams where
   translateEra _ = pure . upgradePParams ()
 
+instance Crypto c => TranslateEra (BabbageEra c) FuturePParams where
+  translateEra ctxt = \case
+    NoPParamsUpdate -> pure NoPParamsUpdate
+    DefinitePParamsUpdate pp -> DefinitePParamsUpdate <$> translateEra ctxt pp
+    PotentialPParamsUpdate mpp -> PotentialPParamsUpdate <$> mapM (translateEra ctxt) mpp
+
 instance Crypto c => TranslateEra (BabbageEra c) EpochState where
   translateEra ctxt es =
     pure
@@ -158,6 +165,7 @@ instance Crypto c => TranslateEra (BabbageEra c) ShelleyGovState where
         , sgsFutureProposals = translateEra' ctxt $ sgsFutureProposals ps
         , sgsCurPParams = translateEra' ctxt $ sgsCurPParams ps
         , sgsPrevPParams = translateEra' ctxt $ sgsPrevPParams ps
+        , sgsFuturePParams = translateEra' ctxt $ sgsFuturePParams ps
         }
 
 instance Crypto c => TranslateEra (BabbageEra c) ProposedPPUpdates where