[Discussion] Aptos Workspace - Better Infrastructure for E2E Testing

[vgao1996]

Recently we’ve been looking into improving our developer workflow and have identified a few week links. As of today, Aptos developers

1. Need to manually download and install the required tools (e.g. Aptos CLI), and sometimes compile from source.
2. Need to manually run a local node
3. Need to manually compile and publish Move modules before testing them
4. Do not have an easy way to write integration tests involving multiple transactions

To address these challenges, we propose to build Aptos Workspace (tentative name subject to change), a new set of tools to enable more automation, similar to Ethereum’s hardhat. More specifically, this includes

1. A workspace environment that streamlines source management & compilation
   1. No need to manually compile the Move code
   2. Move packages will be built automatically before use
2. A testing framework/harness that automatically runs a local node and enables developers to simulate a series of transactions using scripts (e.g. js/ts)

Here is a more detailed design:
e39f3c2a-36f4-4ebe-aad8-5cd018476288_(Draft)_Aptos_Workspace.pdf

(Please be mindful that this is still a draft, and as such, there may be potential changes in the future.)

Additionally, we would also like to enhance our existing create-aptos-dapp script so that it can install the correct tools and set up the workspace template for developers.

What’s your experience with our developer workflow so far? What are your pain points? Please join the discussion and share your thoughts!

[alnoki]

Thank you and @vgao1996 and team for starting this discussion

Right now some of the biggest pain points for Econia developers:

• Having to wait a few hours to sync to chain tip every time we redeploy the Econia indexer stack
• Compile times on Aptos crates
• Bizarre dependencies like tokio-unstable in some crates
• Having to include a fork of aptos-indexer-processors as a submodule in the Econia core repo
• Private view functions not rendering on the Aptos explorer
• View functions erroring out due to execution gas limits
• No opinionated Move linter (unless I'm missing something)
• No enums in Move
• Uncertainty around storage gas costs
• Uncertainty around Coin vs FungibleAsset as a de facto standard

@CRBl69 anything else?

[CRBl69]

• Aptos crates documentation does not compile correctly, which makes it impossible to see a crate's API and doc on docs.rs. This is a problem when working with the API, as we have to manually search source code.
• Compile times for the aptos-sdk crate is very high and has hundreds of dependencies. I'm sure most people don't use all of those. How about separating the aptos-sdk crate with features (see official rust doc).

[wrwg]

Thanks for the feedback, @alnoki . It might be useful to collect most of this in different issues than this one, which is about e2e testing. Some more comments below.

Thank you and @vgao1996 and team for starting this discussion

Right now some of the biggest pain points for Econia developers:

• Having to wait a few hours to sync to chain tip every time we redeploy the Econia indexer stack

My understanding is that this is a matter of how sync is configured (execution based or writeset based), but @JoshLind will know better.

• Compile times on Aptos crates

This is hard to avoid with Rust, which is known for slow compilation. I any case on my Max I'm looking at a couple of minutes, so doesn't seem that bad.

• Bizarre dependencies like tokio-unstable in some crates

Maybe @zekun000 can say something about this, but doesn't seem to be a DevX problem but more a problem around security?

• Having to include a fork of aptos-indexer-processors as a submodule in the Econia core repo
• Private view functions not rendering on the Aptos explorer
• View functions erroring out due to execution gas limits

Would you mind to open bugs about this?

• No opinionated Move linter (unless I'm missing something)

A foundation grant has been approved for a team which now works on the linter, so stay tuned.

• No enums in Move

We are 80% done with a new Move compiler which will make it much easier to do language extensions like this and multiple others. @vgao1996 gave a preso in Singapore about the planned extensions, enums are a top priority. However, it may take another quarter to have this ready for prod

• Uncertainty around storage gas costs
• Uncertainty around Coin vs FungibleAsset as a de facto standard

Again, perhaps open individual issues for this?

@CRBl69 anything else?

[wrwg]

• Aptos crates documentation does not compile correctly, which makes it impossible to see a crate's API and doc on docs.rs. This is a problem when working with the API, as we have to manually search source code.

The documentation of most of our code is not intended for doc generation. I'd recommend to use an IDE like VS Code or Jetbrains family which makes it very easy to navigate code, much easier than generated docs would do. However, if there are particular public APIs you'd like to see accessible via generated doc, please open an issue for this,

• Compile times for the aptos-sdk crate is very high and has hundreds of dependencies. I'm sure most people don't use all of those. How about separating the aptos-sdk crate with features (see official rust doc).

Maybe @gregnazario can say more about this, but as said above, if you have a decent machine, its not that bad. Its rather difficult to constraint deps and code via features.

[gregnazario]

We recently split the repo for the typescript SDK, we may in the future look at similar for the Rust SDK, but we internally use the Rust SDK for all automated testing in the current repo.

[alnoki]

@wrwg thanks for all the responses above. I've opened a few issues where applicable (e.g. aptos-labs/explorer#571) and have been in touch with core devs on others (e.g. @banool re: indexer sync times and view function execution gas errors etc.)

I'm looking forward in particular to the linter and especially to enums and some of the other Move features coming down the pipeline. Re: uncertainty for things like storage gas and asset standards, I understand that much of this is in early stages and I try to follow on GitHub (e.g. #10092), though the AIP updates in the group channels have been especially helpful too as of late, and I'd certainly appreciate more of this