chore(ethereum): Release ethereum-v1.1.0 (#260)

* chore(ethereum): Release `ethereum-v1.1.0` * fix: Use `Box::pin` for large futures in ethereum's client binary (#177) * chore: Add `large_futures` clippy lint to xclippy * fix: Use `Box::pin` to heap allocate large futures --------- Co-authored-by: wwared <[email protected]> * refactor: Adapt to Sphinx (Refactor proof handling) (#161) * refactor: Refactor proof handling across all components - Update Sphinx dependencies from tag v1.0.0 to branch forward_ports_43 across all components. - Modification of all instances of `SphinxProof` in favor of `SphinxProofWithPublicValues`. - Inclusion of the run() method call chained after the `prove` and `execute` functions across system (enacting builder pattern). - Removal and integration of separate `prove_plonk` and `verify_plonk` functions into main proving and verification functions using the method `.plonk()` where required. - Direct passing of `stdin` as an argument in the `execute` function across applications. - Update of the Rust Toolchain from "nightly-2024-07-30" to "nightly-2024-08-13" in both Aptos and Ethereum components. - Addition of `bytes` function for supporting different proof types in the fixture-generator. - Updating proof creation and validation in the fixture generation process with revised function calls. - Revisions in client and server operations to support `SphinxProofWithPublicValues` and updated function calls. - Update of `tokio` version from `"1.39.0"` to `"1.39.2"` and Hex dependency addition at version `0.4.3`. * fix: Adapt fixture-generator to prepend vkey hash * fix: Fix small typo * chore: Update forge dependencies * fix: Update Move verifier constants and vkey logic * chore: clippy * fix: Fix the script.move version header logic * chore: Update inclusion fixtures * chore: Update fixtures * chore: Update all solidity fixtures * feat: Update Move codebase --------- Co-authored-by: wwared <[email protected]> Co-authored-by: Artem Storozhuk <[email protected]> * chore: Update dependencies (#178) * fix: Pass stdin as a ref (#183) Co-authored-by: wwared <[email protected]> * chore: Remove superfluous nightly versions (#185) * chore: Remove superfluous nightly versions * Use `rust-toolchain.toml` version in CI * ci: Use custom runner for `bench.yml` (#187) * ci: Use custom runner for `bench.yml` * Address review * Address review * ci: Fix cycle regression checker (#188) * ci: Fix cycle regression checker * Test workflow * Prep for review * chore: Update fixtures (#160) Co-authored-by: tchataigner <[email protected]> * feat: docker & k8s (#12) * feat: docker & k8s Signed-off-by: Thomas Chataigner <[email protected]> * feat: wip docker * feat: wip docker flexible lc * feat: simplify k8s conf * feat: refactor aptos proof server to one bin * feat: one server bin eth + k8s aptos * feat: ethereum client configuration * chore: lint * ci: revise docker publish * refactor: base review integrated * chore: lint * refactor: router for proof server + health check * refactor: ethereum health check * refactor: probes * refactor: multiple routes eth proof server * refactor: fix compilation * refactor: accept octet stream * refactor: change handling request proof server * refactor: all routes working * refactor: not using serde json * refactor: only one request * refactor: health does not count as increment * fix: fix middleware * refactor: working aptos proof_server * chore: use let-else more effectively (#197) * refactor: replicas --------- Signed-off-by: Thomas Chataigner <[email protected]> Co-authored-by: François Garillot <[email protected]> * chore: Update fixtures (#198) Co-authored-by: tchataigner <[email protected]> * chore: Update `ethereum` Rust version to `nightly-2024-08-29` (#199) * refactor: lint * chore: Update `ethereum` Rust version to `nightly-2024-08-29` * chore: authorize too long first paragraph --------- Co-authored-by: Thomas Chataigner <[email protected]> * chore: Update fixtures (#203) Co-authored-by: tchataigner <[email protected]> * feat: Initial Pact contract (#205) * wip: Initial pact skeleton * chore: Add function for inclusion event processing * chore: Add correct proof version handling * chore: Refactoring * chore: Add test.repl --------- Co-authored-by: wwared <[email protected]> * chore: Update fixtures (#206) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#208) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#212) Co-authored-by: tchataigner <[email protected]> * fix: ethereum lc (#214) * fix: fix forward endpoint * fix: fix not used update * fix: boxed store * chore: Update fixtures (#215) Co-authored-by: tchataigner <[email protected]> * feat: fixture raw proof (#216) * feat: revamp fixture generate * feat: generate fixtures * refactor: add fixture test * fix: remove proof version check * chore: Update fixtures (#218) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#222) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#223) Co-authored-by: tchataigner <[email protected]> * chore: Update `ethereum` Rust version to `nightly-2024-09-13` (#224) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#229) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#230) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#232) Co-authored-by: tchataigner <[email protected]> * feat: helm chart (#228) * feat: wip helm * refactor: remove client from dockerfile * feat: helm icon * refactor: refactor helm & introduce eksctl conf * refactor: helm with node selector * refactor: external IP for cluster * refactor: use published docker image * refactor: fix command * refactor: remove secondary dep * refactor: proper comp config dockerfile * refactor: remove not needed file * refactor: always pull image * refactor: pull image for docker compose * refactor: docker compose file * refactor: proper command * refactor: liveness & readiness * refactor: fix middleware * refactor: pas s rust flags * fix: fix stack overflow * fix: stack limit as str & change build target * refactor: lint * fix: docker file * refactor: change target for build * refactor: idle time limit load balancer + docs * refactor: lint * docs: review typo fix Co-authored-by: Samuel Burnham <[email protected]> * refactor: integrate review --------- Co-authored-by: Samuel Burnham <[email protected]> * chore: Update fixtures (#235) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#238) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#242) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#245) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#248) Co-authored-by: tchataigner <[email protected]> * chore: Update fixtures (#252) Co-authored-by: tchataigner <[email protected]> * feat: eth cleanup (#254) * refactor: makefile 32xl * docs: update benchamrks + fixed a few commands * docs: refactor linebreaks * refactor: latest sphinx + compile programs * refactor: argumentcomputer * refactor: xclippy * fix: deny (#250) * refactor: fix aptos deps * ci: Refactor release workflow (#236) * ci: Refactor release workflow * Support manual release on `workflow_dispatch` * Fix token * refactor: fix fixture generator * ci: Fix `release-pr.yml` version (#255) * refactor: deny file * update licenses (#142) * update licenses * Update Rust licenses * Remove MIT * Update license * Update Ethereum license * rename Yatima -> Argument * chore: modify forgotten file --------- Co-authored-by: Samuel Burnham <[email protected]> Co-authored-by: François Garillot <[email protected]> --------- Signed-off-by: Thomas Chataigner <[email protected]> Co-authored-by: tchataigner <[email protected]> Co-authored-by: wwared <[email protected]> Co-authored-by: wwared <[email protected]> Co-authored-by: François Garillot <[email protected]> Co-authored-by: Artem Storozhuk <[email protected]> Co-authored-by: Samuel Burnham <[email protected]> Co-authored-by: tchataigner <[email protected]> Co-authored-by: argument-ci-bot[bot] <178725181+argument-ci-bot[bot]@users.noreply.github.com> Co-authored-by: John Chandler Burnham <[email protected]> Co-authored-by: François Garillot <[email protected]>
argumentcomputer · Sep 27, 2024 · bdc2073 · bdc2073
1 parent cf15026
commit bdc2073
Show file tree

Hide file tree

Showing 159 changed files with 4,815 additions and 2,934 deletions.
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
@@ -21,9 +21,15 @@ runs:
       with:
         go-version: '1.22'
         cache-dependency-path: "**/go.sum"
-    - uses: dtolnay/rust-toolchain@master
+    # Assumes the `fixture-generator` Rust version is reasonably up to date with all of the light clients
+    - name: Get Rust toolchain version
+      shell: bash
+      run: |
+        echo "TOOLCHAIN_VERSION=$(grep -E '^channel\s*=' ./fixture-generator/rust-toolchain.toml | awk -F'"' '{ print $2 }')" | tee -a $GITHUB_ENV
+    - name: Install Rust
+      uses: dtolnay/rust-toolchain@master
       with:
-        toolchain: nightly-2024-05-31
+        toolchain: ${{ env.TOOLCHAIN_VERSION }}
     - uses: Swatinem/rust-cache@v2
       with:
         workspaces: |

diff --git a/.github/workflows/assets/APACHE.md b/.github/workflows/assets/APACHE.md
diff --git a/.github/workflows/bench.yml b/.github/workflows/bench.yml
@@ -1,74 +1,97 @@
 # Runs benchmarks on self-hosted infra via `workflow_dispatch`
-# This trigger can be found at https://github.com/wormhole-foundation/example-zk-light-clients-internal/actions/workflows/bench.yml
+# This trigger can be found at https://github.com/argumentcomputer/zk-light-clients/actions/workflows/bench.yml
 #
-# The output can be found in the logs or in a comment on the latest commit. This can be viewed on GitHub at the bottom of the commit page.
-# See https://github.com/wormhole-foundation/example-zk-light-clients-internal/commit/3d06c3585e94fe027bf7dacf865106c259994c85#comments
-name: Manual benchmark
+# The benchmark report can be found in the logs and as a comment on the latest commit on `dev`.
+# The report can also be sent as a Zulip message to https://zulip.argument.xyz
+name: Light client benchmark
 on:
   workflow_dispatch:
     inputs:
-      # Which light client to bench, e.g. `aptos` or `ethereum`
-      package:
+      # Which light client to bench, e.g. `aptos`, `ethereum` or `kadena`
+      light-client:
+        description: 'Name of the light client to benchmark'
         type: string
         required: true
-      # Name of the `light-client` benchmark to run
+      # Name of the `light-client` benchmark to run, e.g. `inclusion`
+      # Runs in the `light-client` directory, so it cannot benchmark `proof_server` or `programs`
       bench-name:
+        description: 'Name of the benchmark to run'
         type: string
         required: true
+      # List of comma-separated env vars, e.g. `RUST_LOG=debug,SNARK=1`
+      # `RUSTFLAGS="-C target-cpu=native --cfg tokio_unstable -C opt-level=3"` is set by default
+      env:
+        description: 'List of comma-separated environment variables'
+        type: string
+        required: false
       # Optionally send a message to the below Zulip streams
       # Defaults to false
       zulip:
+        description: 'Send the report to Zulip'
         type: boolean
         required: false
       # User(s) to whom to send a private DM (optional)
       # Comma-separated list of user ID integers, e.g. `11,12` (IDs can be found in user profiles)
       # If not specified, sends to a stream/topic pair instead
       private:
-        description: 'DM given user ID(s)'
+        description: 'Send DM to given user ID(s)'
         type: string
         required: false
       # Zulip stream in which to send the message (optional)
       # Ignored if `private` input is specified
       # Defaults to `light_client` stream
-      stream:
+      channel:
+        description: 'Send message to channel (default is `light-client`). Ignored if `private`'
         type: string
         required: false
       # Zulip topic in which to send the message (optional)
       # Ignored if `private` input is specified
       # Defaults to `chat`
       topic:
+        description: 'Send message to topic (default is `chat`). Ignored if `private`'
         type: string
         required: false
   schedule:
     # Bench report on Tuesdays at 11pm EST/12pm EDT
-    - cron:  '0 16 * * 4'
+    - cron: '0 16 * * 4'
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 
 jobs:
   benchmark:
-    name: Manual benchmark
-    runs-on: [self-hosted, bench, avx512]
+    name: Light client benchmark
+    runs-on: warp-custom-r7iz-metal-16xl
     steps:
       - uses: actions/checkout@v4
         with:
-          repository: lurk-lab/ci-workflows
+          repository: argumentcomputer/ci-workflows
       - uses: ./.github/actions/ci-env
       - uses: actions/checkout@v4
       - name: Setup CI
         uses: ./.github/actions/setup
-        with:
-          pull_token: ${{ secrets.REPO_TOKEN }}
-      - uses: dtolnay/rust-toolchain@master
-        with:
-          toolchain: nightly-2024-05-31
       - name: Install extra deps
         run: |
           sudo apt-get update && sudo apt-get install -y python3-pip
           pip3 install jtbl
-      - name: Parse inputs
+      - name: Set env
+        run: |
+          # Default benchmark settings optimized for light clients, can be overwritten with `env` input
+          echo "RUSTFLAGS='-C target-cpu=native --cfg tokio_unstable -C opt-level=3'" | tee -a $GITHUB_ENV
+          echo "SHARD_SIZE=4194304" | tee -a $GITHUB_ENV
+          echo "SHARD_BATCH_SIZE=0" | tee -a $GITHUB_ENV
+          echo "RECONSTRUCT_COMMITMENTS=false" | tee -a $GITHUB_ENV
+          echo "SHARD_CHUNKING_MULTIPLIER=256" | tee -a $GITHUB_ENV
+          echo "SNARK=1" | tee -a $GITHUB_ENV
+
+          IFS=',' read -ra ENV_VARS <<< "${{ inputs.env }}"
+          for VAR in "${ENV_VARS[@]}"; do
+            VAR_NAME="${VAR%%=*}"
+            VAR_VALUE="${VAR#*=}"
+            echo "${VAR_NAME}=${VAR_VALUE}" | tee -a $GITHUB_ENV
+          done
+      - name: Parse Zulip inputs
         run: |
           if [[ "${{ inputs.zulip }}" == "true" ]]; then
             if [[ ! -z "${{ inputs.private }}" ]]; then
@@ -97,8 +120,8 @@ jobs:
         run: |
           make bench-ci BENCH=${{ inputs.bench-name }} 2>&1 | tee out.txt
 
-          grep 'cycles=' out.txt >> cycles.txt
-          grep 'proving_time' out.txt >> timings.txt
+          grep 'cycles=' out.txt > cycles.txt
+          grep 'proving_time' out.txt > timings.txt
 
           while IFS=$'\t' read -r f1 f2
           do