Skip to content

Commit

Permalink
Merge branch 'main' into rel/0.1.4-test25
Browse files Browse the repository at this point in the history
  • Loading branch information
fab-10 committed May 3, 2024
2 parents 1bdc317 + 6e52d3e commit 70b35b4
Show file tree
Hide file tree
Showing 68 changed files with 3,370 additions and 990 deletions.
70 changes: 43 additions & 27 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v4

- name: Set up GCC
uses: egor-tensin/setup-gcc@v1

- name: Checkout repository
uses: actions/checkout@v3

Expand All @@ -36,6 +42,12 @@ jobs:
acceptanceTest:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v4

- name: Set up GCC
uses: egor-tensin/setup-gcc@v1

- name: Checkout repository
uses: actions/checkout@v3

Expand All @@ -54,46 +66,50 @@ jobs:
if: always()
uses: actions/upload-artifact@v3
with:
name: test-report
name: acceptance-test-report
path: acceptance-tests/build/reports/tests/

tests:
libCompressTest:
runs-on: ubuntu-latest
steps:
- uses: webfactory/[email protected]
with:
ssh-private-key: |
${{ secrets.CONSTRAINTS_SSH_KEY }}
- name: Install Go
uses: actions/setup-go@v4

- name: Set up GCC
uses: egor-tensin/setup-gcc@v1

- name: Checkout repository
uses: actions/checkout@v3
with:
submodules: recursive

- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'

# The asset URL for the latest release can be found with:
# curl -L -H "Accept: application/vnd.github+json" \
# -H "Authorization: Bearer YOUR_GH_API_TOKEN" \
# -H "X-GitHub-Api-Version: 2022-11-28" \
# https://api.github.com/repos/ConsenSys/corset/releases/latest
# | jq '.assets[] | select(.name|endswith("x86_64-unknown-linux-musl.tar.gz")) | .url'
- name: Install Corset
run: |
curl -L \
-H "Accept: application/octet-stream" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
-o corset.tar.gz \
https://api.github.com/repos/Consensys/corset/releases/assets/147491099
tar xzf corset.tar.gz
mv corset $HOME
echo $HOME >> $GITHUB_PATH
- name: Run libcompress JNI tests
run: ./gradlew :native:compress:test
env:
JAVA_OPTS: -Dorg.gradle.daemon=false

- name: Upload test report
if: always()
uses: actions/upload-artifact@v3
with:
name: compress-test-report
path: compress/build/reports/tests/

tests:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'

- name: Run unit tests
run: ./gradlew :arithmetization:test
Expand All @@ -103,8 +119,8 @@ jobs:
- name: Upload test report
uses: actions/upload-artifact@v3
with:
name: test-report
path: build/reports/tests/
name: unit-test-report
path: arithmetization/build/reports/tests/

spotless:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
run: echo ::set-output name=VERSION::${GITHUB_REF/refs\/tags\//}

- name: Build without tests
run: ./gradlew build -PreleaseVersion=${{ steps.get_version.outputs.VERSION }} -x test -x spotlessCheck
run: ./gradlew build -PreleaseNativeLibs -PreleaseVersion=${{ steps.get_version.outputs.VERSION }} -x test -x spotlessCheck
env:
JAVA_OPTS: -Xmx2g -Dorg.gradle.daemon=false

Expand Down
32 changes: 32 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,37 @@
# Changelog

## 0.1.4-test24
Test pre-release 24 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* Improve linea_estimateGas error response [#650](https://github.com/Consensys/besu-sequencer-plugins/pull/650)
* On Windows also build Linux native lib so it can run on WSL [#651](https://github.com/Consensys/besu-sequencer-plugins/pull/651)

## 0.1.4-test23
Test pre-release 23 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* linea_estimateGas compatibility mode multiplier https://github.com/Consensys/besu-sequencer-plugins/pull/646

## 0.1.4-test22
Test pre-release 22 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* linea_estimateGas compatibility switch https://github.com/Consensys/besu-sequencer-plugins/pull/634
* Update profitability formula with gas price adjustment option https://github.com/Consensys/besu-sequencer-plugins/pull/638
* Update code to latest plugin API https://github.com/Consensys/besu-sequencer-plugins/pull/640
* Txpool profitability check https://github.com/Consensys/besu-sequencer-plugins/pull/603
* Fix price adjustment in profitability formula https://github.com/Consensys/besu-sequencer-plugins/pull/642

## 0.1.4-test21
Test pre-release 21 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* fix: capture SSTORE-touched storage slots for correct gas computations [#606](https://github.com/Consensys/besu-sequencer-plugins/pull/606)
* build: make the build script portable, explicit dependency on Go & GCC, test libcompress build [#621](https://github.com/Consensys/besu-sequencer-plugins/pull/621)
* Update after the refactor of transaction selection service [#626](https://github.com/Consensys/besu-sequencer-plugins/pull/626)
* Use the right classloader to load the native library [#628](https://github.com/Consensys/besu-sequencer-plugins/pull/628)

## 0.1.4-test20
Test pre-release 20 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* Get L2L1 settings from CLI options [#591](https://github.com/Consensys/besu-sequencer-plugins/pull/591)
* feat: add a replay capture script [#600](https://github.com/Consensys/besu-sequencer-plugins/pull/600)
* move compress native into plugin repo [#604](https://github.com/Consensys/besu-sequencer-plugins/pull/604)
* Add compression [#605](https://github.com/Consensys/besu-sequencer-plugins/pull/605)
* Update for the new bad block manager [#607](https://github.com/Consensys/besu-sequencer-plugins/pull/607)

## 0.1.4-test19
Test pre-release 19 from [temp/issue-248/count-stack-only](https://github.com/Consensys/besu-sequencer-plugins/tree/temp/issue-248/count-stack-only)
* Avoid returning an estimated priority fee that is less than the min gas price [#598](https://github.com/Consensys/besu-sequencer-plugins/pull/598)
Expand Down
75 changes: 56 additions & 19 deletions PLUGINS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,37 @@
# Linea plugins
# Linea plugins

## Shared components
### Profitability calculator
The profitability calculator is a shared component, that is used to check if a tx is profitable.
It is applied, with different configuration to:
1. `linea_estimateGas` endpoint
2. Tx validation for the txpool
3. Tx selection during block creation

#### CLI Options

| Option Name | Default Value | Command Line Argument |
|--------------------------|---------------|-------------------------------------------|
| L1_VERIFICATION_GAS_COST | 1_200_000 | `--plugin-linea-verification-gas-cost` |
| L1_VERIFICATION_CAPACITY | 90_000 | `--plugin-linea-verification-capacity` |
| L1_L2_GAS_PRICE_RATIO | 15 | `--plugin-linea-gas-price-ratio` |
| L2_GAS_PRICE_ADJUSTMENT | 0 wei | `--plugin-linea-gas-price-adjustment` |
| MIN_MARGIN | 1.0 | `--plugin-linea-min-margin` |
| ESTIMATE_GAS_MIN_MARGIN | 1.0 | `--plugin-linea-estimate-gas-min-margin` |
| TX_POOL_MIN_MARGIN | 0.5 | `--plugin-linea-tx-pool-min-margin` |
| UNPROFITABLE_CACHE_SIZE | 100_000 | `--plugin-linea-unprofitable-cache-size` |
| UNPROFITABLE_RETRY_LIMIT | 10 | `--plugin-linea-unprofitable-retry-limit` |
| TX_POOL_ENABLE_CHECK_API | true | `--plugin-linea-tx-pool-profitability-check-api-enabled` |
| TX_POOL_ENABLE_CHECK_P2P | false | `--plugin-linea-tx-pool-profitability-check-p2p-enabled` |

### L1 L2 Bridge

#### CLI Options

| Option Name | Default Value | Command Line Argument |
|------------------------------|---------------|---------------------------------------------|
| L1L2_BRIDGE_CONTRACT_ADDRESS | | `--plugin-linea-l1l2-bridge-contract` |
| L1L2_BRIDGE_LOG_TOPIC | | `--plugin-linea-l1l2-bridge-topic` |

## Sequencer
### Transaction Selection - LineaTransactionSelectorPlugin
Expand All @@ -11,19 +44,14 @@ of a transaction.

#### CLI Options

| Option Name | Default Value | Command Line Argument |
|--------------------------|---------|----------------------------------------|
| MAX_BLOCK_CALLDATA_SIZE | 70000 | `--plugin-linea-max-block-calldata-size` |
| MODULE_LIMIT_FILE_PATH | moduleLimitFile.toml | `--plugin-linea-module-limit-file-path` |
| OVER_LINE_COUNT_LIMIT_CACHE_SIZE | 10_000 | `--plugin-linea-over-line-count-limit-cache-size` |
| MAX_GAS_PER_BLOCK | 30_000_000L | `--plugin-linea-max-block-gas` |
| L1_VERIFICATION_GAS_COST | 1_200_000 | `--plugin-linea-verification-gas-cost` |
| L1_VERIFICATION_CAPACITY | 90_000 | `--plugin-linea-verification-capacity` |
| L1_L2_GAS_PRICE_RATIO | 15 | `--plugin-linea-gas-price-ratio` |
| MIN_MARGIN | 1.0 | `--plugin-linea-min-margin` |
| ADJUST_TX_SIZE | -45 | `--plugin-linea-adjust-tx-size` |
| UNPROFITABLE_CACHE_SIZE | 100_000 | `--plugin-linea-unprofitable-cache-size` |
| UNPROFITABLE_RETRY_LIMIT | 10 | `--plugin-linea-unprofitable-retry-limit` |
| Option Name | Default Value | Command Line Argument |
|-------------------------------------|----------------------|-------------------------------------------------------|
| MAX_BLOCK_CALLDATA_SIZE | 70000 | `--plugin-linea-max-block-calldata-size` |
| MODULE_LIMIT_FILE_PATH | moduleLimitFile.toml | `--plugin-linea-module-limit-file-path` |
| OVER_LINE_COUNT_LIMIT_CACHE_SIZE | 10_000 | `--plugin-linea-over-line-count-limit-cache-size` |
| MAX_GAS_PER_BLOCK | 30_000_000L | `--plugin-linea-max-block-gas` |
| TX_POOL_ENABLE_SIMULATION_CHECK_API | true | `--plugin-linea-tx-pool-simulation-check-api-enabled` |
| TX_POOL_ENABLE_SIMULATION_CHECK_P2P | false | `--plugin-linea-tx-pool-simulation-check-p2p-enabled` |


### Transaction validation - LineaTransactionValidatorPlugin
Expand All @@ -35,14 +63,23 @@ that are not allowed to add transactions to the pool.

#### CLI Options

| Option Name | Default Value | Command Line Argument |
| --- | --- | --- |
| DENY_LIST_PATH | lineaDenyList.txt | `--plugin-linea-deny-list-path` |
| MAX_TX_GAS_LIMIT_OPTION | 30_000_000 | `--plugin-linea-max-tx-gas-limit` |
| MAX_TX_CALLDATA_SIZE | 60_000 | `--plugin-linea-max-tx-calldata-size` |
| Option Name | Default Value | Command Line Argument |
|-------------------------|-------------------|---------------------------------------|
| DENY_LIST_PATH | lineaDenyList.txt | `--plugin-linea-deny-list-path` |
| MAX_TX_GAS_LIMIT_OPTION | 30_000_000 | `--plugin-linea-max-tx-gas-limit` |
| MAX_TX_CALLDATA_SIZE | 60_000 | `--plugin-linea-max-tx-calldata-size` |

## RPC

### Linea Estimate Gas
#### `linea_estimateGas`

This endpoint simulates a transaction and returns the estimated gas used ( as the standard `eth_estimateGas`) plus the estimated gas price to be used when submitting the tx.

#### Parameters

same as `eth_estimateGas`

### Counters - CountersEndpointServicePlugin
#### `rollup_getTracesCountersByBlockNumberV0`

Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,19 @@ an [existing implementation in Go](https://github.com/Consensys/zk-evm/).
brew install openjdk@17
```

### Native Lib Prerequisites

Linux/MacOs
* Install the relevant CGo compiler for your platform
* Install the Go toolchain

Windows
* Requirement [Docker Desktop WSL 2 backend on Windows](https://docs.docker.com/desktop/wsl/)

On release native libs are build for all the supported platforms,
if you want to test this process locally run `./gradlew -PreleaseNativeLibs jar`,
jar is generated in `arithmetization/build/libs`.

### Install Rust

```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
Expand Down Expand Up @@ -149,7 +150,8 @@ private void assertTransactionsInCorrectBlocks(Web3j web3j, List<String> hashes,
protected SimpleStorage deploySimpleStorage() throws Exception {
final Web3j web3j = minerNode.nodeRequests().eth();
final Credentials credentials = Credentials.create(Accounts.GENESIS_ACCOUNT_ONE_PRIVATE_KEY);
TransactionManager txManager = new RawTransactionManager(web3j, credentials, CHAIN_ID);
TransactionManager txManager =
new RawTransactionManager(web3j, credentials, CHAIN_ID, createReceiptProcessor(web3j));

final RemoteCall<SimpleStorage> deploy =
SimpleStorage.deploy(web3j, txManager, new DefaultGasProvider());
Expand Down Expand Up @@ -198,11 +200,15 @@ protected void assertTransactionNotInThePool(String hash) {
.notInTransactionPool(Hash.fromHexString(hash)));
}

protected List<Map<String, String>> getTxPoolContent() {
return minerNode.execute(new TxPoolTransactions().getTxPoolContents());
}

private TransactionReceiptProcessor createReceiptProcessor(Web3j web3j) {
return new PollingTransactionReceiptProcessor(
web3j,
TransactionManager.DEFAULT_POLLING_FREQUENCY,
TransactionManager.DEFAULT_POLLING_ATTEMPTS_PER_TX_HASH);
Math.max(1000, LINEA_CLIQUE_OPTIONS.blockPeriodSeconds() * 1000 / 5),
LINEA_CLIQUE_OPTIONS.blockPeriodSeconds() * 3);
}

protected String sendTransactionWithGivenLengthPayload(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ public List<String> getTestCliOptions() {
.set(
"--plugin-linea-module-limit-file-path=",
getResourcePath("/txOverflowModuleLimits.toml"))
.set("--plugin-linea-tx-pool-simulation-check-api-enabled=", "false")
.build();
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
* Copyright Consensys Software Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package linea.plugin.acc.test.rpc.linea;

import static org.assertj.core.api.Assertions.assertThat;

import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.List;

import org.hyperledger.besu.datatypes.Wei;
import org.hyperledger.besu.ethereum.core.Transaction;

public class EstimateGasCompatibilityModeTest extends EstimateGasTest {
private static final BigDecimal PRICE_MULTIPLIER = BigDecimal.valueOf(1.2);

@Override
public List<String> getTestCliOptions() {
return getTestCommandLineOptionsBuilder()
.set("--plugin-linea-estimate-gas-compatibility-mode-enabled=", "true")
.set(
"--plugin-linea-estimate-gas-compatibility-mode-multiplier=",
PRICE_MULTIPLIER.toPlainString())
.build();
}

@Override
protected void assertIsProfitable(
final Transaction tx,
final Wei baseFee,
final Wei estimatedPriorityFee,
final Wei estimatedMaxGasPrice,
final long estimatedGasLimit) {
final var minGasPrice = minerNode.getMiningParameters().getMinTransactionGasPrice();
final var minPriorityFee = minGasPrice.subtract(baseFee);
final var compatibilityMinPriorityFee =
Wei.of(
PRICE_MULTIPLIER
.multiply(new BigDecimal(minPriorityFee.getAsBigInteger()))
.setScale(0, RoundingMode.CEILING)
.toBigInteger());

// since we are in compatibility mode, we want to check that returned profitable priority fee is
// the min priority fee per gas * multiplier + base fee
final var expectedMaxGasPrice = baseFee.add(compatibilityMinPriorityFee);
assertThat(estimatedMaxGasPrice).isEqualTo(expectedMaxGasPrice);
}

@Override
protected void assertMinGasPriceLowerBound(final Wei baseFee, final Wei estimatedMaxGasPrice) {
// since we are in compatibility mode, we want to check that returned profitable priority fee is
// the min priority fee per gas * multiplier + base fee
assertIsProfitable(null, baseFee, null, estimatedMaxGasPrice, 0);
}
}
Loading

0 comments on commit 70b35b4

Please sign in to comment.