Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update dependencies in sync with node v9.1.0 & Conway support #327

Merged
merged 52 commits into from
Aug 29, 2024

Conversation

sourabhxyz
Copy link
Member

@sourabhxyz sourabhxyz commented Jul 30, 2024

Closes #326, closes #334, closes #332, closes #330, closes #314.

@eddiemundo
Copy link

Is the remaining work here adding another GYEra, GYConway, then adding a GYConway branch to every place GYBabbage exists and doing the analogous thing?

@sourabhxyz
Copy link
Member Author

Hey @eddiemundo, there are huge number of changes made that I haven't yet pushed to this branch as it's a bit of a mess atm. It's priority on our end to bring Conway support in Atlas asap.

.gitignore Outdated Show resolved Hide resolved
@sourabhxyz
Copy link
Member Author

Migration guide

General changes

  • Atlas' GYTxMonad has been revamped, with details in this PR. In particular, to handle these changes, you might need to do following changes in your codebase:
    • Use runGYTxQueryMonadIO in place of runGYTxQueryMonadNode. I'll use runGYTxQueryMonadNode -> runGYTxQueryMonadIO to denote such a change.

    • GYTxQueryMonadNode -> GYTxQueryMonadIO.

    • GYTxMonadNode -> GYTxBuilderMonadIO.

    • GYTxBuildResult Identity -> GYTxBuildResult.

    • GYTxMonad can likely be replaced with GYTxUserQueryMonad in your code.

    • Some of the old functions such as runGYTxMonadNode are removed but these can be defined like so:

      runGYTxMonadNode :: GYNetworkId -> GYProviders -> [GYAddress] -> GYAddress -> Maybe (GYTxOutRef, Bool) -> GYTxBuilderMonadIO (GYTxSkeleton v) -> IO GYTxBody
      runGYTxMonadNode nid providers addrs change collateral act = runGYTxBuilderMonadIO nid providers addrs change collateral $ act >>= buildTxBody
      
      runGYTxMonadNodeF :: forall t v. Traversable t => GYCoinSelectionStrategy -> GYNetworkId -> GYProviders -> [GYAddress] -> GYAddress -> Maybe (GYTxOutRef, Bool) -> GYTxBuilderMonadIO (t (GYTxSkeleton v)) -> IO (t GYTxBody)
      runGYTxMonadNodeF strat nid providers addrs change collateral act = runGYTxBuilderMonadIO nid providers addrs change collateral $ act >>= traverse (buildTxBodyWithStrategy strat)
      
      runGYTxMonadNodeParallel :: GYNetworkId -> GYProviders -> [GYAddress] -> GYAddress -> Maybe (GYTxOutRef, Bool) -> GYTxBuilderMonadIO [GYTxSkeleton v] -> IO GYTxBuildResult
      runGYTxMonadNodeParallel nid providers addrs change collateral act = runGYTxBuilderMonadIO nid providers addrs change collateral $ act >>= buildTxBodyParallel
      
      runGYTxMonadNodeParallelWithStrategy :: GYCoinSelectionStrategy -> GYNetworkId -> GYProviders -> [GYAddress] -> GYAddress -> Maybe (GYTxOutRef, Bool) -> GYTxBuilderMonadIO [GYTxSkeleton v] -> IO GYTxBuildResult
      runGYTxMonadNodeParallelWithStrategy strat nid providers addrs change collateral act = runGYTxBuilderMonadIO nid providers addrs change collateral $ act >>= buildTxBodyParallelWithStrategy strat
    • GYTxQueryMonadIO and GYTxBuilderMonadIO do not have MonadIO instance, but these can be defined as shown:

      import           GeniusYield.Unsafe                    (unsafeIOToQueryMonad, unsafeIOToTxBuilderMonad)
      
      instance MonadIO GYTxQueryMonadIO where
          liftIO = unsafeIOToQueryMonad
      
      instance MonadIO GYTxBuilderMonadIO where
          liftIO = unsafeIOToTxBuilderMonad

Privnet

How to run?

To run the privnet tests, first one needs to install cardano-node & cardano-cli like so:

cabal install --package-env=$(pwd) --overwrite-policy=always cardano-cli cardano-node

Then the tests can be ran, like for Atlas: cabal run atlas-privnet-tests -- -j1 --hide-successes

How to structure?

  • Due to redesign of monads related to transaction building & query, functions such as ctxRunI are no longer present, but they (and some of the related utilities) can be implemented like so:
ctxRunI :: Ctx -> User -> GYTxBuilderMonadIO (GYTxSkeleton v) -> IO GYTxBody
ctxRunI ctx user act = ctxRunBuilder ctx user $ act >>= buildTxBody

ctxRunC :: Ctx -> User -> GYTxBuilderMonadIO a -> IO a
ctxRunC = ctxRunBuilder

ctxRunF :: Traversable t => Ctx -> User -> GYTxBuilderMonadIO (t (GYTxSkeleton v)) -> IO (t GYTxBody)
ctxRunF ctx user act = ctxRunBuilder ctx user $ act >>= traverse buildTxBody

submitTxCtx :: Ctx -> User -> GYTxBody -> IO GYTxId
submitTxCtx ctx user txBody = ctxRun ctx user $ signAndSubmitConfirmed txBody

submitTxCtx' :: Ctx -> User -> GYTx -> IO GYTxId
submitTxCtx' ctx user tx = ctxRun ctx user $ submitTxConfirmed tx

addRefScriptCtx :: Ctx -> User -> GYScript PlutusV2 -> IO GYTxOutRef
addRefScriptCtx ctx user script = ctxRun ctx user $ addRefScriptToLimbo script
  • Earlier the tests might expect the argument IO Setup but now only Setup is required which can be generated with following continuation: withPrivnet cardanoDefaultTestnetOptionsConway $ \setup. Please see privnet tests structured inside Atlas here for examples.

  • Arguments of withSetup are flipped, so instead of withSetup setup info, now withSetup info setup is required.

  • Use userChangeAddress instead of userAddr.

  • submitTx is removed, can use above submitTxCtx function.

  • In case you were catching of BuildTxException (now renamed to GYBuildTxError and it no longer has instance for Exception), since now these are made part of GYBuildTxException constructor inside GYTxMonadException type, catch for those instead. For example:

    - isTxBodyScriptExecutionError :: BuildTxException -> Bool
    - isTxBodyScriptExecutionError (BuildTxBodyErrorAutoBalance (Api.TxBodyScriptExecutionError _)) = True
    - isTxBodyScriptExecutionError _                                                                = False
    + isTxBodyScriptExecutionError :: GYTxMonadException -> Bool
    + isTxBodyScriptExecutionError (GYBuildTxException (GYBuildTxBodyErrorAutoBalance (Api.TxBodyScriptExecutionError _))) = True
    + isTxBodyScriptExecutionError _                                                                  = False
  • GYPrivnet now has an additional field to store of the network information used internally, you can largely ignore those.

  • Note that these privnet run against Conway era and protocol parameters can be check like so:

      pp <- ctxRunQuery ctx protocolParams
      info $ printf "Protocol parameters: %s" (show pp)

Let us know if you think some of these should be changed!

CLB (cardano-ledger-backend, replacement of plutus-simple-model employed earlier) & new testing mechanism

Note that we now have unified testing in place, i.e., same test file can be ran against multiple backend interpreters, i.e., either utilising CLB, private testnet machinery or any of cardano network (testnet/mainnet).

It is best to see tests-unified directory to see how to structure these. In particular you don't need to write privnet tests separately!

  • There is no longer Wallet type, use User instead.

  • If you earlier had

    runWallet w1 $ withWalletBalancesCheckSimple walletDiffList $ do ...

    You should instead do

    withWalletBalancesCheckSimple walletDiffList $ asUser w1 $ do ...
  • Note that since the testing mechanism is unified, you should no longer require to import types such as GYTxMonadClb in your backend code. I.e., if you had

    myTrace :: SomeParameters -> Wallets -> GYTxMonadClb ()
    

    You should instead do

    myTrace :: GYTxGameMonad m => SomeParameters -> Wallets -> m ()
    

    Note that GYTxGameMonad constraint should be added if you are employing asUser in your trace, otherwise make use of GYTxMonad etc.

  • Use ownChangeAddress instead of ownAddress.

  • Instead of fail ... use throwAppError $ someBackendError $ ....

  • sendSkeleton and sendSkeleton' can be defined like so:

    -- | This is simply defined as @buildTxBody skeleton >>= signAndSubmitConfirmed@.
    sendSkeleton :: GYTxMonad m => GYTxSkeleton v -> m GYTxId
    sendSkeleton skeleton = snd <$> sendSkeleton' skeleton
    
    sendSkeleton' :: GYTxMonad m => GYTxSkeleton v -> m (GYTxBody, GYTxId)
    sendSkeleton' skeleton = buildTxBody skeleton >>= \tx -> signAndSubmitConfirmed tx >>= \txId -> pure (tx, txId)

@sourabhxyz
Copy link
Member Author

Single test inside atlas-tests test-suite fails as there is a bug in Maestro where it returns extra registered stake pools (on preprod) for it's /pools endpoint.

@@ -105,3 +112,17 @@ hashSimpleScript = scriptHashFromApi . hashSimpleScript'

hashSimpleScript' :: GYSimpleScript -> Api.ScriptHash
hashSimpleScript' = Api.hashScript . Api.SimpleScript . simpleScriptToApi

-- FIXME: Need to test this.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in this issue.

@sourabhxyz sourabhxyz marked this pull request as ready for review August 27, 2024 15:44
@sourabhxyz sourabhxyz requested a review from a team as a code owner August 27, 2024 15:44
@TotallyNotChase
Copy link
Contributor

TotallyNotChase commented Aug 27, 2024

Good migration guide. I'd add that there is a withSetupOld that is there for compatibility. Furthermore, there is also a pattern User' with the old fields such as userAddr and such for compatibility.

I think the migration guide should also request people to move to the new tx monad hierarchy design (i.e don't use more powerful monad than needed), before opting for the drop in functions like runGYTxMonadNodeF, ctxRunF etc.

Could you also put the short brief on the tx monad hierarchy in the migration guide (when creating a new release?)

Copy link
Contributor

@TotallyNotChase TotallyNotChase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done. Most design changes look fine to me.

The conway update specific changes seem fine. However, they require testing to make sure they do indeed follow the protocol. Private testnet running conway is a start. But there ought to be some manual sanity checking transactions submitted to Preprod, and finally to mainnet later on.

The hardcoded PParams related things need to have a tracking issue (so they can be updated later).

cabal.project Outdated Show resolved Hide resolved
cabal.project Outdated Show resolved Hide resolved
src/GeniusYield/Test/Privnet/Setup.hs Show resolved Hide resolved
src/GeniusYield/Test/Privnet/Setup.hs Outdated Show resolved Hide resolved
src/GeniusYield/Types/PlutusVersion.hs Outdated Show resolved Hide resolved
src/GeniusYield/Types/Providers.hs Show resolved Hide resolved
src/GeniusYield/Types/Script.hs Show resolved Hide resolved
src/GeniusYield/Unsafe.hs Outdated Show resolved Hide resolved
@sourabhxyz
Copy link
Member Author

@TotallyNotChase Thanks for your review! It's highly appreciated.

@sourabhxyz
Copy link
Member Author

@TotallyNotChase

But there ought to be some manual sanity checking transactions submitted to Preprod, and finally to mainnet later on.

Yes, we have been testing these changes with our dapp.

@sourabhxyz
Copy link
Member Author

@TotallyNotChase

The hardcoded PParams related things need to have a tracking issue (so they can be updated later).

Yes, this was on radar! I have created issue for it here.

@eddiemundo
Copy link

eddiemundo commented Aug 28, 2024

I tried pulling from the branch update-node-9.1.0 and it doesn't compile for me because

• Not in scope: data constructor ‘Blockfrost.PlutusV3’
  NB: the module ‘Blockfrost.Client’ does not export ‘PlutusV3’.
• Perhaps use one of these:
    ‘Blockfrost.PlutusV1’ (imported from Blockfrost.Client),
    ‘Blockfrost.PlutusV2’ (imported from Blockfrost.Client)

which seems to be because blockfrost-client ^>=0.8.0 depends on blockfrost-api < 0.11 and Blockfrost.PlutusV3 comes from blockfrost-api 0.11.0

How do I get it to use blockfrost-api 0.11.0...

@eddiemundo
Copy link

How do I get it to use blockfrost-api 0.11.0...

Sorry, I just needed to cabal update.

@sourabhxyz
Copy link
Member Author

@eddiemundo Yes, please make use of cabal.project file as given in this PR and get latest index state using cabal update.

Copy link
Contributor

@4TT1L4 4TT1L4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Excellent work!

LGTM!

@4TT1L4 4TT1L4 merged commit b05a986 into main Aug 29, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment