chore(aptos): Release 1.0.1 (#135)

* chore(aptos): Release 1.0.1 * feat: bootstrap client (#75) * feat: fetch checkpoint and bootstrap Signed-off-by: Thomas Chataigner <[email protected]> * docs: started doc for client * docs: rust doc * chore: clean deps + fmt + xclippy * refactor: simplify readability for serde methods * refactor: apply suggestion from review Co-authored-by: wwared <[email protected]> * refactor: rename const var & check no leftover bytes * fix: fix base length for LightClientHeader --------- Signed-off-by: Thomas Chataigner <[email protected]> Co-authored-by: wwared <[email protected]> * ci: Add `cargo-deny` workflow (#79) * ci: Add `cargo-deny` workflow (WIP) * chore: use workspace dependencies * chore: Refactor uneeded dependencies in aptos core Cargo.toml - Removed `move-core-types` from workspace dependencies and the aptos feature in Cargo.toml file - Stripped `reqwest` from the dev-dependencies within the aptos/core module * fix: update deny.toml exceptions - Modified various Cargo.toml files within the aptos project to standardize the `edition` field and reference the workspace. - Added the "BUSL-1.1" license to multiple packages throughout the aptos codebase, including `proof-server`, `aptos-lc-core`, `aptos-programs`, `fixture-generator`, and `light-client`. - Updated the deny.toml file, removing redundant sections and updating the list of exceptions to allow `aptos-lc-core` to use the "BUSL-1.1" license. - Set edition 2021 in the workspace package configuration within the main aptos project's Cargo.toml. * chore: add exception for number_range * ci: fix deny action * ci: Use `ssh-key` input for `cargo-deny-action` * Test regular `cargo-deny` CLI * Ignore `aptos` advisories --------- Co-authored-by: François Garillot <[email protected]> * ci: Update licenses (#86) * ci: Update fixtures in nightly job (#92) * ci: Update fixtures on a nightly job * Parallelize and test flow * Test E2E * Prep for review * Set to 1 retention day instead of the default (90) * chore: Update fixtures (#94) Co-authored-by: samuelburnham <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * chore: Update fixtures (#95) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#98) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#107) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#114) Co-authored-by: tchataigner <[email protected]> * feat: Wrapper contract with additional LC protocol-specific checks (#111) * feat: Add stub for Wrapper contract * feat: Refactoring * feat: Add logic for updating the signer hash * chore: Fix compiler warnings * chore: Add access control to signer hash setter * feat: Include block_id, key and value into inclusion program output * chore: Update wrapper contract testing * feat: Use block hash as unique block identifier * chore: Requested changes * chore: Update fixtures (#116) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#120) Co-authored-by: tchataigner <[email protected]> * feat: Lookahead Move verifier (#119) * feat: Restructured fixture generator * feat: Add epoch change fixture generating using ethereum program * chore: Bind foundry/forge dependencies to particular versions * chore: Update CI to generate Move fixtures as well * feat: Add initial Move wrapper that implements lookahead verification * feat: Generating inclusion proof fixture using Ethereum program * chore: Update Move fixtures * feat: Update core verifier * feat: Update Move wrapper contract * chore: Update Move Readme * chore: Properly update submodules * ci: Update fixture CI workflow * chore: Move fixture-generator to root of the repository * docs: Update documentation about on-chain verification (Aptos / Ethereum) * chore: Move test utilities to separate module * chore: Addressing review comments related to Move wrapper contract * chore: Address review comments * feat: update deps to fit release branch * chore: Update fixtures (#124) Co-authored-by: tchataigner <[email protected]> * feat: compact data inclusion (#125) * feat: compact data * feat: more clippy rules * refactor: move test utils to core * fix: needed feature for eth-lc-core * refactor: refactor visibility to move * Revert "chore: Update fixtures (#124)" (#129) This reverts commit 73bd552. * fix: fixtures (#130) * fix: fixture generation * ci: Fix `fixtures.yml` * Test `fixtures.yml` * Fixup and test `fixtures.yml` * fix: update fixtures * Fixup * Prep for review --------- Co-authored-by: Samuel Burnham <[email protected]> * docs: update doc (#132) * docs: update doc * refactor: apply suggestions Co-authored-by: wwared <[email protected]> * refactor: second batch of suggestion Co-authored-by: wwared <[email protected]> --------- Co-authored-by: wwared <[email protected]> --------- Signed-off-by: Thomas Chataigner <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: tchataigner <[email protected]> Co-authored-by: wwared <[email protected]> Co-authored-by: Samuel Burnham <[email protected]> Co-authored-by: François Garillot <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: tchataigner <[email protected]> Co-authored-by: Artem Storozhuk <[email protected]>
argumentcomputer · Jul 30, 2024 · bcaf38f · bcaf38f
1 parent 3d2d273
commit bcaf38f
Show file tree

Hide file tree

Showing 47 changed files with 13,127 additions and 763 deletions.
diff --git a/.github/workflows/fixtures.yml b/.github/workflows/fixtures.yml
@@ -0,0 +1,82 @@
+# Rebuilds proof fixtures for Solidity and Move for smart contract verification tests
+# Then opens a pull request with the changes
+# Note: This workflow takes over 30 minutes due to parallel E2E proof generation for `inclusion` and `epoch_change`
+name: Update fixtures
+
+on:
+  workflow_dispatch: {}
+  # Once per day at 00:00 UTC
+  schedule:
+    - cron: "0 0 * * *"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  generate-fixtures:
+    runs-on: warp-ubuntu-latest-x64-32x
+    timeout-minutes: 90
+    strategy:
+      matrix:
+        fixture: [inclusion, epoch_change]
+        language: [solidity, move]
+        include:
+          - fixture-path: aptos/solidity/contracts/src/plonk_fixtures
+            language: solidity
+          - fixture-path: ethereum/move/sources/fixtures
+            language: move
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          repository: lurk-lab/ci-workflows
+      - uses: ./.github/actions/ci-env
+      - uses: actions/checkout@v4
+      - name: Setup CI
+        uses: ./.github/actions/setup
+        with:
+          pull_token: ${{ secrets.REPO_TOKEN }}
+      - name: Generate ${{ matrix.language }} fixtures
+        run: |
+          cargo run --release -- --program ${{ matrix.fixture }} --language ${{ matrix.language }}
+        working-directory: ${{ github.workspace }}/fixture-generator
+      - name: Upload fixture artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.language }}_${{ matrix.fixture }}_fixture.json
+          path: ${{ github.workspace }}/${{ matrix.fixture-path }}/${{ matrix.fixture }}_fixture.json
+          if-no-files-found: error
+          overwrite: true
+          retention-days: 1
+
+  create-pull-request:
+    needs: generate-fixtures
+    runs-on: ubuntu-latest
+    env:
+      SOLIDITY_FIXTURE_PATH: aptos/soldity/contracts/src/plonk_fixtures
+      MOVE_FIXTURE_PATH: ethereum/move/sources/fixtures
+    steps:
+      - uses: actions/checkout@v4
+      - name: Download Solidity fixtures
+        uses: actions/download-artifact@v4
+        with:
+          path: ${{ github.workspace }}/${{ env.SOLIDITY_FIXTURE_PATH }}
+          pattern: "solidity_*"
+          merge-multiple: true
+      - name: Download Move fixtures
+        uses: actions/download-artifact@v4
+        with:
+          path: ${{ github.workspace }}/${{ env.MOVE_FIXTURE_PATH }}
+          pattern: "move_*"
+          merge-multiple: true
+      - name: Create Pull Request
+        uses: peter-evans/create-pull-request@v6
+        with:
+          branch: "ci-update-fixtures"
+          title: "chore: Update fixtures"
+          commit-message: "chore: Update fixtures"
+          labels: "automated-issue"
+          body: |
+            This is an automated PR updating the proof fixtures for Solidity and Move, which are used for smart contract verification tests.
+
+            Workflow run: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml
@@ -8,44 +8,40 @@ on:
   pull_request:
     types: [ opened, synchronize, reopened, ready_for_review ]
 
-env:
-  CARGO_TERM_COLOR: always
-  # Disable incremental compilation.
-  #
-  # Incremental compilation is useful as part of an edit-build-test-edit cycle,
-  # as it lets the compiler avoid recompiling code that hasn't changed. However,
-  # on CI, we're not making small edits; we're almost always building the entire
-  # project from scratch. Thus, incremental compilation on CI actually
-  # introduces *additional* overhead to support making future builds
-  # faster...but no future builds will ever occur in any given CI environment.
-  #
-  # See https://matklad.github.io/2021/09/04/fast-rust-builds.html#ci-workflow
-  # for details.
-  CARGO_INCREMENTAL: 0
-  # Allow more retries for network requests in cargo (downloading crates) and
-  # rustup (installing toolchains). This should help to reduce flaky CI failures
-  # from transient network timeouts or other issues.
-  CARGO_NET_RETRY: 10
-  RUSTUP_MAX_RETRIES: 10
-  # Don't emit giant backtraces in the CI logs.
-  RUST_BACKTRACE: short
-
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 
 jobs:
+  # Detect changes in each light client crate
+  changes:
+    runs-on: ubuntu-latest
+    outputs:
+      # Expose matched filters as job 'packages' output variable
+      packages: ${{ steps.filter.outputs.changes }}
+      aptos: ${{ steps.filter.outputs.aptos }}
+      ethereum: ${{ steps.filter.outputs.ethereum }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dorny/paths-filter@v3
+        id: filter
+        with:
+          filters: |
+            aptos:
+              - 'aptos/**'
+            ethereum:
+              - 'ethereum/**'
+
   test:
-    runs-on: ${{ matrix.os }}
+    needs: changes
+    runs-on: buildjet-16vcpu-ubuntu-2204
+    if: ${{ needs.changes.outputs.packages != '[]' && needs.changes.outputs.packages != '' }}
     strategy:
       matrix:
-        os:
-          - buildjet-16vcpu-ubuntu-2204
-        package:
-          - "aptos"
+        # Parse JSON array containing names of all changed light client packages
+        # e.g. ['aptos', 'ethereum'] if both directories contain changes
+        package: ${{ fromJSON(needs.changes.outputs.packages) }}
       fail-fast: false
-    env:
-      CARGO_NET_GIT_FETCH_WITH_CLI: "true"
     steps:
       - uses: actions/checkout@v4
         with:
@@ -59,33 +55,31 @@ jobs:
       # make sure benches don't bit-rot
       - name: build benches
         run: |
-          cargo check --benches --features aptos
+          cargo check --benches --all-features
         working-directory: ${{ github.workspace }}/${{ matrix.package }}/light-client
       - name: Run cargo test in workspace
         run: |
           cargo nextest run --workspace --release --profile ci --all-features
         working-directory: ${{ github.workspace }}/${{ matrix.package }}
-      - name: Doctests
-        run: |
-          cargo test --doc
-        env:
-          RUSTFLAGS: "--cfg tokio_unstable"
-        working-directory: ${{ github.workspace }}/${{ matrix.package }}
 
   clippy:
+    needs: changes
     runs-on: buildjet-16vcpu-ubuntu-2204
+    if: ${{ needs.changes.outputs.packages != '[]' && needs.changes.outputs.packages != '' }}
     strategy:
       matrix:
-        package:
-          - "aptos"
-    env:
-      CARGO_NET_GIT_FETCH_WITH_CLI: "true"
+        # Parse JSON array containing names of all changed light client packages
+        # e.g. ['aptos', 'ethereum'] if both directories contain changes
+        package: ${{ fromJSON(needs.changes.outputs.packages) }}
     steps:
       - uses: actions/checkout@v4
         with:
           repository: lurk-lab/ci-workflows
       - uses: ./.github/actions/ci-env
       - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+          token: ${{ secrets.REPO_TOKEN }}
       - name: Setup CI
         uses: ./.github/actions/setup
         with:
@@ -97,9 +91,22 @@ jobs:
       - name: cargo clippy
         run: cargo xclippy -D warnings
         working-directory: ${{ github.workspace }}/${{ matrix.package }}
+      - name: Doctests
+        run: |
+          cargo test --doc
+        working-directory: ${{ github.workspace }}/${{ matrix.package }}
+      - run: cargo install --locked cargo-deny
+      - name: Cargo-deny check
+        run: |
+          cargo deny --manifest-path ${{ matrix.package }}/Cargo.toml check
+      - name: Cargo-deny check programs
+        run: |
+          find ${{ matrix.package }}/programs -type d -name "target" -prune -o -type f -name "Cargo.toml" -exec cargo deny --manifest-path {} check \;
 
   solidity-unit-tests:
+    needs: changes
     runs-on: buildjet-16vcpu-ubuntu-2204
+    if: needs.changes.outputs.aptos == 'true'
     steps:
       - uses: actions/checkout@v4
         with:
@@ -132,7 +139,14 @@ jobs:
         working-directory: ${{ github.workspace }}/aptos/solidity/contracts/
 
   cycle-count-regression:
+    needs: changes
     runs-on: warp-ubuntu-latest-x64-32x
+    if: ${{ needs.changes.outputs.packages != '[]' && needs.changes.outputs.packages != '' }}
+    strategy:
+      matrix:
+        # Parse JSON array containing names of all changed light client packages
+        # e.g. ['aptos', 'ethereum'] if both directories contain changes
+        package: ${{ fromJSON(needs.changes.outputs.packages) }}
     steps:
       - uses: actions/checkout@v4
         with:
@@ -143,22 +157,34 @@ jobs:
         uses: ./.github/actions/setup
         with:
           pull_token: ${{ secrets.REPO_TOKEN }}
+      - name: Set env
+        run: |
+          if [[ "${{ matrix.package }}" == "aptos" ]]; then
+            # TODO: Remove hardcoded test names
+            TESTS="test_execute_inclusion test_execute_epoch_change test_execute_sig"
+            FEATURES="--features aptos"
+          elif [[ "${{ matrix.package }}" == "ethereum" ]]; then
+            # TODO: Add execution tests
+            TESTS=""
+            FEATURES=""
+          fi
+          echo "TESTS=$TESTS" | tee -a $GITHUB_ENV
+          echo "FEATURES=$FEATURES" | tee -a $GITHUB_ENV
       - name: Get cycle counts for PR
         id: get_cycles_pr
         run: |
           CYCLE_COUNTS='[]'
           set -o pipefail
 
-          # TODO: Remove hardcoded test names
-          for test_name in "test_execute_inclusion" "test_execute_epoch_change" "test_execute_sig"; do
-            cargo nextest run --verbose --release --profile ci --features aptos --package aptos-lc --no-capture -E "test($test_name)" 2>&1 | tee out.txt
+          for test_name in ${{ env.TESTS }}; do
+            cargo nextest run --verbose --release --profile ci ${{ env.FEATURES }} --package ${{ matrix.package }}-lc --no-capture -E "test($test_name)" 2>&1 | tee out.txt
             num_cycles=$(cat out.txt | grep -o 'finished execution clk = [0-9]\+' | awk -F'= ' '{ print $2 }')
             CYCLE_COUNTS=$(echo $CYCLE_COUNTS | jq -c ". += [{\"${test_name}\": \"$num_cycles\"}]")
           done
 
           set +o pipefail
           echo "CYCLE_COUNTS=$CYCLE_COUNTS" | tee -a "$GITHUB_OUTPUT"
-        working-directory: ${{ github.workspace }}/aptos/light-client
+        working-directory: ${{ github.workspace }}/${{ matrix.package }}/light-client
         env:
           RUST_LOG: debug
       - uses: actions/checkout@v4
@@ -175,8 +201,8 @@ jobs:
           set -o pipefail
 
           # TODO: Remove hardcoded test names
-          for test_name in "test_execute_inclusion" "test_execute_epoch_change" "test_execute_sig"; do
-            cargo nextest run --verbose --release --profile ci --features aptos --package aptos-lc --no-capture -E "test($test_name)" 2>&1 | tee out.txt
+          for test_name in ${{ env.TESTS }}; do
+            cargo nextest run --verbose --release --profile ci ${{ env.FEATURES }} --package ${{ matrix.package }}-lc --no-capture -E "test($test_name)" 2>&1 | tee out.txt
             num_cycles_base=$(cat out.txt | grep -o 'finished execution clk = [0-9]\+' | awk -F'= ' '{ print $2 }')
             num_cycles_pr=$(echo "$CYCLE_COUNTS" | jq ".[$counter] | to_entries | .[0].value")
             echo "$test_name summary"
@@ -197,7 +223,7 @@ jobs:
           echo -e "$FAILING_TESTS" | tee -a $GITHUB_OUTPUT
           echo "EOF" >> $GITHUB_OUTPUT
           echo "WORKFLOW_URL=https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}" | tee -a $GITHUB_ENV
-        working-directory: ${{ github.workspace }}/aptos/light-client
+        working-directory: ${{ github.workspace }}/${{ matrix.package }}/light-client
         env:
           RUST_LOG: debug
       - uses: actions/checkout@v4

diff --git a/.github/workflows/switch-license.yml b/.github/workflows/switch-license.yml
@@ -22,6 +22,11 @@ jobs:
         run: |
           cp .github/workflows/assets/APACHE.md aptos/LICENSE.md
 
+      - name: Replace Cargo.toml licenses
+        run: |
+          sed -i 's/license = "BUSL-1.1"/license = "Apache-2.0"/g' "aptos/Cargo.toml"
+          find aptos/programs -type d -name "target" -prune -o -type f -name "Cargo.toml" -exec sed -i 's/license = "BUSL-1.1"/license = "Apache-2.0"/g' {} \;
+
       - name: Replace SPDX license headers
         run: |
           find aptos -type f -exec sed -i 's|// SPDX-License-Identifier: BUSL-1.1|// SPDX-License-Identifier: Apache-2.0, MIT|g' {} \;
@@ -36,7 +41,7 @@ jobs:
         run: |
           {
             echo 'PR_BODY<<EOF'
-            This is an automated PR to update our licenses to Apache2.0.
+            This is an automated PR to update the Aptos licenses to Apache 2.0.
           
             [Workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
             EOF
@@ -48,4 +53,4 @@ jobs:
         with:
           body: ${{ env.PR_BODY }}
           branch: "feature/license-switch"
-          title: "Switch to Apache2.0 License"
+          title: "Switch to Apache2.0 License"
diff --git a/.gitmodules b/.gitmodules
@@ -1,8 +1,9 @@
 [submodule "aptos/solidity/contracts/lib/forge-std"]
 	path = aptos/solidity/contracts/lib/forge-std
 	url = https://github.com/foundry-rs/forge-std
-	# https://stackoverflow.com/questions/5126765/how-to-get-rid-of-git-submodules-untracked-status
-	ignore = dirty
+[submodule "aptos/solidity/contracts/lib/openzeppelin-contracts"]
+	path = aptos/solidity/contracts/lib/openzeppelin-contracts
+	url = https://github.com/openzeppelin/openzeppelin-contracts
 [submodule "aptos/solidity/contracts/lib/sphinx-contracts"]
 	path = aptos/solidity/contracts/lib/sphinx-contracts
 	url = https://github.com/lurk-lab/sphinx-contracts