+ + + +
+ +Private transactions pass from the Besu node to the associated Tessera node. The Tessera node encrypts and directly distributes (that is, point-to-point) the private transaction to the Tessera nodes participating in the transaction. + +By default, each participant in a privacy-enabled network uses its own Besu and Tessera node. [Multi-tenancy](multi-tenancy.md) allows more than one participant to use the same Besu and Tessera node. + +:::tip + +Private Transaction Managers are also known as Enclaves. + +::: + +## Privacy-enabled networks + +When enabling privacy in a [private network](../../get-started/system-requirements.md), there's an assumed level of trust among the node operators, since all are members of the private network. + +:::caution + +Inefficient contracts deployed accidentally or deliberately can cause performance issues in privacy-enabled networks because gas isn't required in private transactions. + +::: + +In contrast, gas is required in Ethereum Mainnet and public testnets because they are trustless environments. + +Privacy-enabled networks should have a mechanism to establish trust offchain. Node operators should be informed on: + +- Guidelines for use, responsibilities, and good behavior. +- Smart contract security, so contracts deployed on the network use resources efficiently. +- Consequences for malicious activity. + +Privacy-enabled networks should run development and test environments that closely resemble production, so contracts can be tested, and potential issues can be found before they're deployed in production. + +## Reorg-compatible privacy + +In v1.4, using private transactions in a network using a consensus mechanism where forks occur (that is, PoW algorithms or Clique) is an early access feature. + +Do not use private transactions in production environments using consensus mechanisms where forks occur. + + + +[highly available and run in a separate instance to Besu]: ../../how-to/use-privacy/tessera.md +[pruning]: ../../../public-networks/concepts/data-storage-formats.md#pruning diff --git a/versioned_docs/version-23.10.2/private-networks/concepts/privacy/multi-tenancy.md b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/multi-tenancy.md new file mode 100644 index 00000000000..d04b9606eb8 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/multi-tenancy.md @@ -0,0 +1,33 @@ +--- +title: Multi-tenancy +sidebar_position: 4 +description: Multi-tenancy +--- + +# Multi-tenancy + +By default, each participant in a privacy network uses its own Besu and Tessera node. + +Multi-tenancy allows multiple participants to use the same Besu and Tessera node. Each participant is a _tenant_, and the operator is the _owner_ of the Besu and Tessera node. + +:::info + +The operator is responsible for [configuring multi-tenancy](../../tutorials/privacy/multi-tenancy.md), and has access to all tenant data. + +::: + + + +:::tip + +Ensure the multi-tenant Tessera node client API is configured to allow access only by the multi-tenant Besu node. Access to your data is secured through Besu using multi-tenancy mode. + +If not configured to allow access only by the multi-tenant Besu node, other Tessera clients, including other Besu nodes, might be able to access tenant data. + +To secure access, you can [configure TLS between Besu and Tessera](../../how-to/configure/tls/client-and-server.md) with the [`WHITELIST`](https://docs.tessera.consensys.net/en/stable/HowTo/Configure/TLS/#whitelist) trust mode. + +::: + +Multi-tenancy validates that tenants have permission to use the specified HTTP or WebSocket JSON-RPC requests, and the tenant has access to the requested privacy data. Private data is isolated and each tenant uses a JSON Web Token (JWT) for authentication. + +You can [create the JWT either externally or internally](../../../public-networks/how-to/use-besu-api/authenticate.md). diff --git a/versioned_docs/version-23.10.2/private-networks/concepts/privacy/plugin.md b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/plugin.md new file mode 100644 index 00000000000..fbc34fb440f --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/plugin.md @@ -0,0 +1,111 @@ +--- +title: Privacy plugin +description: Privacy plugin +sidebar_position: 5 +--- + +# Privacy plugin + +You can define your own strategy for private transactions by building a plugin that extends Hyperledger Besu functionality. + +The plugin can take many forms, but it must provide Besu with a private transaction when required. + +:::danger + +The privacy plugin is an early access feature and plugin interfaces are subject to change between releases. + +::: + +## Configuration + +Enable the privacy plugin by starting Besu and including the `--Xprivacy-plugin-enabled` command line option. The registered plugin must implement the `PrivacyPluginPayloadProvider` interface. + +## Use the payload provider interface + +The privacy plugin must define the [privacy marker transaction (PMT)] payload. Use the payload to retrieve the contents of the private transaction which could be a link to a location in an enclave, or an encrypted form of the private payload itself. + +Besu doesn't need to know how the private transaction is distributed, it just needs to know what the private transaction for the PMT is. + +### Send transactions + +When submitting a private transaction using [`eea_sendRawTransaction`](../../../public-networks/reference/api/index.md#eea_sendrawtransaction), the signed transaction must be sent to `0x000000000000000000000000000000000000007a` to indicate which [privacy precompiled contract](private-transactions/processing.md) is being used. + +The transaction flow is as follows: + +1. The JSON-RPC endpoint passes the private transaction to the private transaction manager (for example Tessera). +2. The private transaction manager sends the private transaction to the privacy plugin. +3. The plugin decides what data to store onchain in the payload, for example the encrypted and serialized private transaction. +4. The plugin returns what needs to be stored in the payload for the PMT. +5. The private transaction handler creates a PMT for the private transaction, and propagates the PMT using devP2P in the same way as a public Ethereum transaction. + +### Mine transactions + +The process of mining transactions happens in reverse to sending transactions. + +1. The Mainnet transaction processor processes the PMT in the same way as any other public transaction. On nodes containing the [privacy precompile contract](../../../public-networks/reference/api/index.md#priv_getprivacyprecompileaddress) specified in the `to` attribute of the PMT, the Mainnet transaction processor passes the PMT to the privacy precompile contract. + + :::note + + Nodes receiving the PMT that do not contain the specified privacy precompile contract will ignore the PMT. + + ::: + +1. The privacy precompile contract queries the plugin for the private transaction using the PMT. +1. The privacy precompile contract passes the private transaction to the private transaction manager. The privacy group ID specifies the private world state to use. +1. The private transaction manager executes the transaction. The private transaction manager can read and write to the private world state, and read from the public world state. + +## Transaction factory + +An additional extension is available to help you define how PMTs are signed. Currently, Besu supports fixed or random key signing for PMTs. + +The extension allows you to use a more dynamic approach, for example different keys for different groups. + +Your plugin needs to register the `PrivateMarkerTransactionFactory` interface which is called before submitting a PMT to the transaction pool. The responsibility then lies with the plugin to sign and serialize the PMT. + +[privacy marker transaction (PMT)]: ../../how-to/use-privacy/access-private-transactions.md + +## Register your plugin + +To enable Besu to use your privacy plugin, you must implement the `PrivacyPluginService` interface and you must call `setPayloadProvider`. + +```java + +@AutoService(BesuPlugin.class) +public class TestPrivacyPlugin implements BesuPlugin { + + private PrivacyPluginService service; + + @Override + public void register(BesuContext context) { + service = context.getService(PrivacyPluginService.class).get(); + } + + @Override + public void start() { + service.setPayloadProvider(new PrivacyPluginPayloadProvider() { + @Override + public Bytes generateMarkerPayload(PrivateTransaction privateTransaction, String privacyUserId) { + // perform logic to serialize the payload of the marker transaction + // in this example we are serialising the private transaction using rlp https://ethereum.org/en/developers/docs/data-structures-and-encoding/rlp/ + return org.hyperledger.besu.ethereum.privacy.PrivateTransaction.serialize(privateTransaction).encoded(); + } + + @Override + public Optional+ + + +
+ +:::note + +For clarity, the Tessera nodes are not shown in the previous diagram. To send private transactions, each Besu node must have an associated Tessera node. + +::: + +### Access between states + +A contract in a privacy group: + +- Can read or write to a contract in the same privacy group. +- Can read from the public state including public contracts. +- Cannot access contracts from a different privacy group. + +A public contract cannot access a private contract. + +### Enterprise Ethereum Alliance privacy + +In the privacy implementation complying with the [EEA Client Specification](https://entethalliance.org/technical-documents/) the group of nodes specified by `privateFrom` and `privateFor` form a privacy group with a unique privacy group ID provided by Tessera. + +The previous diagram illustrates two privacy groups enabling: + +- A, B, and C to send transactions that are private from D. +- A, C, and D to send transactions that are private from B. + +Using EEA-compliant privacy, to send private transactions between A, B, and C, A initializes a contract in a private transaction with B and C specified as the `privateFor` and A specified as the `privateFrom`. Initializing the contract creates a privacy group consisting of A, B, and C. For the ABC private state to remain consistent, A, B, and C must be included on transactions (as either `privateFrom` or `privateFor`) even if they are between only two of the three parties. + +To send private transactions between A, C, and D, C initializes a different contract in a private transaction with A and D specified as the `privateFor` and C specified as the `privateFrom`. Initializing the contract creates a privacy group consisting of A, C, and D. For the ACD private state to remain consistent, A, C, and D must be included on transactions (as either `privateFrom` or `privateFor`) even if they are between only two of the three parties. + +### Besu-extended privacy + +The Besu-extended privacy implementation creates a privacy group using [`priv_createPrivacyGroup`](../../../public-networks/reference/api/index.md#priv_createprivacygroup) with private transactions sent to the privacy group ID. + +Using the same privacy groups as in the previous example. + +Using Besu-extended privacy, to send private transactions between A, B, and C, A creates a privacy group consisting of A, B, and C. The privacy group ID is specified when sending private transactions and A, B, and C are recipients of all private transactions sent to the privacy group. + +To send private transactions between A, C, and D, A creates a privacy group consisting of A, C, and D. The privacy group ID of this group is specified when sending private transactions with A, C, and D as recipients. + +## Multi-tenancy + +When using [multi-tenancy](multi-tenancy.md) with privacy groups, each user provides a JSON Web Token (JWT) which allows Besu to check that the user has access to functionality and data associated with a privacy group. diff --git a/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/_category_.json b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/_category_.json new file mode 100644 index 00000000000..465b0be5572 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Private transactions", + "position": 1 +} diff --git a/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/index.md b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/index.md new file mode 100644 index 00000000000..c149089080d --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/index.md @@ -0,0 +1,98 @@ +--- +description: Private transaction overview +--- + +# Private transactions + +Private transactions have the same parameters as public Ethereum transactions, with the following additions: + +- `privateFrom` - The Tessera public key of the transaction sender. +- One of the following: + - `privateFor` - The Tessera public keys of the transaction recipients. + - `privacyGroupId` - [The privacy group to receive the transaction](../privacy-groups.md). +- `restriction` - Whether the private transaction is `restricted` or `unrestricted`: + + - `restricted` - Only the nodes participating in the transaction receive and store the payload of the private transaction. + - `unrestricted` - All nodes in the network receive the payload of the private transaction, but only the nodes participating in the transaction can read the transaction. + + :::info + + Besu implements `restricted` private transactions only. + + ::: + +The `gas` and `gasPrice` are used by the [privacy marker transaction (PMT)](processing.md), not the private transaction itself. + +:::info + +Because gas isn't required in private transactions, inefficient contracts deployed accidentally or deliberately can cause performance issues in privacy-enabled networks. Ensure your network has a mechanism to [establish trust offchain](../index.md#privacy-enabled-networks). + +::: + +You can [create and send private transactions](../../../how-to/send-transactions/private-transactions.md). + +## Besu and Tessera keys + +Besu and Tessera nodes both have public/private key pairs identifying them. A Besu node sending a private transaction to a Tessera node signs the transaction with the Besu node private key. The `privateFrom` and `privateFor` parameters specified in the RLP-encoded transaction string for [`eea_sendRawTransaction`](../../../reference/api/index.md#eea_sendrawtransaction) are the public keys of the Tessera nodes sending and receiving the transaction. + +:::info + +The mapping of Besu node addresses to Tessera node public keys is offchain. That is, the sender of a private transaction must know the Tessera node public key of the recipient. + +::: + +## Nonces + +A nonce is the number of previous transactions made by the sender. + +[Private transaction processing](processing.md) involves two transactions: the private transaction distributed to involved participants, and the privacy marker transaction (PMT) included on the public blockchain. Each of these transactions has its own nonce. Since the PMT is a public transaction, the PMT nonce is the public nonce for the account. + +### Private transaction nonce + +Besu maintains separate private states for each [privacy group](../privacy-groups.md). The private transaction nonce for an account is specific to the privacy group. That is, the nonce for account A for privacy group ABC is different to the nonce for account A for privacy group AB. + +### Private nonce validation + +Unlike public transactions, private transactions are not submitted to the [transaction pool](../../../../public-networks/concepts/transactions/pool.md). The private transaction is distributed directly to the participants in the transaction, and the PMT is submitted to the transaction pool. + +Unlike [public transaction nonces](../../../../public-networks/concepts/transactions/validation.md), private transaction nonces aren't validated when the private transaction is submitted. If a private transaction has an incorrect nonce, the PMT is still valid and is added to a block. However, in this scenario, the private transaction execution fails when [processing the PMT](processing.md) for the private transaction with the incorrect nonce. + +The following private transaction flow illustrates when nonce validation occurs: + +1. Submit a private transaction with a [nonce value](#private-transaction-nonce). +1. The private transaction is distributed to all participants in the privacy group. +1. The PMT is created and submitted to the transaction pool with a nonce of `0` if using one-time accounts. If using a specific account with [`--privacy-marker-transaction-signing-key-file`](../../../reference/cli/options.md#privacy-marker-transaction-signing-key-file), the public nonce for that account is obtained and used for the PMT. +1. The PMT is mined and included in the block. +1. After the block containing the PMT is imported, and the PMT is processed, the private transaction is retrieved from the private transaction manager and executed. + + If the private transaction was submitted with a correct nonce in step 1, the nonce is validated as correct. If an incorrect nonce was submitted, the private transaction execution fails. + +### Private nonce management + +In Besu, you call [`eth_getTransactionCount`](../../../../public-networks/reference/api/index.md#eth_gettransactioncount) to get a nonce, then use that nonce with [`eea_sendRawTransaction`](../../../reference/api/index.md#eea_sendrawtransaction) to send a private transaction. + +However, when you send multiple transactions in row, if a subsequent call to `getTransactionCount` happens before a previous transaction is processed, you can get the same nonce again. + +You can manage private nonces in multiple ways: + +- Let Besu handle it. You just need to wait long enough between calls to `sendRawTransaction` for the transactions to process. The current window is around 1.5 seconds, depending on block time. + + Public transactions deal with this issue, but the window is shorter, since you can use the transaction pool to take into account pending transactions (by using `eth_getTransactionCount("pending")`). + + For private transactions, the window is longer because private transactions aren't submitted to the transaction pool. You must wait until the private transaction's corresponding PMT is included in a block. + +- Manage the nonce yourself, by keeping track of and providing the nonce at each call. We recommend this if you're [sending many transactions that are independent of each other](../../../how-to/send-transactions/concurrent-private-transactions.md). + + :::note + + You can use [`priv_getTransactionCount`](../../../reference/api/index.md#priv_gettransactioncount) or [`priv_getEeaTransactionCount`](../../../reference/api/index.md#priv_geteeatransactioncount) to get the nonce for an account for the specified privacy group or participants. + + ::: + +- Use [Orchestrate](https://docs.orchestrate.consensys.net/en/stable/) for nonce management. We recommend this for enterprise use. + +:::tip + +The [web3js-quorum library includes an example](https://github.com/ConsenSys/web3js-quorum/blob/9a0f9eb1b91a4a0d93801f77782b509ae2e7314c/example/concurrentPrivateTransactions/concurrentPrivateTransactions.js) of nonce management when [sending concurrent private transactions](../../../how-to/send-transactions/concurrent-private-transactions.md). The example calculates the correct nonces for the private transactions and PMTs outside of Besu. + +::: diff --git a/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/processing.md b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/processing.md new file mode 100644 index 00000000000..196f2b9e7e0 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/concepts/privacy/private-transactions/processing.md @@ -0,0 +1,72 @@ +--- +title: Private transaction processing +sidebar_position: 1 +description: Private transaction processing +--- + +# Private transaction processing + +Processing [private transactions](index.md) involves the following: + +- **Precompiled contract**: A smart contract compiled from the source language to EVM bytecode and stored by an Ethereum node for later execution. + +- **Privacy marker transaction (PMT)**: A public Ethereum transaction with a payload of the enclave key. The enclave key is a pointer to the private transaction in Tessera. The `to` attribute of the PMT is the [address of the privacy precompiled contract](../../../reference/api/index.md#priv_getprivacyprecompileaddress). + + The PMT is [signed with a random key or the key specified on the command line]. + +Private transaction processing is illustrated and described in the following diagram. + + + +1. Submit a private transaction using [`eea_sendRawTransaction`](../../../reference/api/index.md#eea_sendrawtransaction). The signed transaction includes transaction parameters specific to private transactions, including: + + - `privateFor` or `privacyGroupId`, which specifies the list of recipients. + - `privateFrom`, which specifies the sender. + - `restriction`, which specifies the transaction is restricted to the transaction participants. + +1. The JSON-RPC endpoint passes the private transaction to the Private Transaction Handler. + +1. The Private Transaction Handler sends the private transaction to Tessera. + +1. Tessera distributes the private transaction directly (that is, point-to-point) to the Tessera nodes specified in `privateFor` or belonging to the privacy group identified by `privacyGroupId`. All recipient Tessera nodes store the transaction. Tessera associates the stored transaction with the transaction hash and privacy group ID. + +1. Tessera returns the transaction hash to the Private Transaction Handler. + +1. The Private Transaction Handler creates a PMT for the private transaction. The Private Transaction Handler propagates the PMT using devP2P in the same way as any other public Ethereum transaction. + + :::tip + + If you want to sign the PMT outside of Besu, use [`priv_distributeRawTransaction`](../../../how-to/send-transactions/private-transactions.md#priv_distributerawtransaction) instead of [`eea_sendRawTransaction`](../../../reference/api/index.md#eea_sendrawtransaction). + + ::: + +1. Besu mines the PMT into a block and the PMT is distributed to all Ethereum nodes in the network. + +1. The Mainnet Transaction Processor processes the PMT in the same way as any other public transaction. On nodes containing the [privacy precompile contract](../../../reference/api/index.md#priv_getprivacyprecompileaddress) specified in the `to` attribute of the PMT, the Mainnet Transaction Processor passes the PMT to the privacy precompile contract. + + :::note + + Nodes receiving the PMT that don't contain the privacy precompile contract ignore the PMT. + + ::: + +1. The privacy precompile contract queries Tessera for the private transaction and privacy group ID using the transaction hash. + +1. The privacy precompile contract passes the private transaction to the Private Transaction Processor. The privacy group ID specifies the private world state to use. + +1. The Private Transaction Processor executes the transaction. The Private Transaction Processor can read and write to the private world state, and read from the public world state. + +:::danger Recommendations + +- We recommend using a network with a consensus mechanism supporting transaction finality. For example, [IBFT 2.0](../../../how-to/configure/consensus/ibft.md). +- Tessera must be [highly available and run in a separate instance to Besu](../../../how-to/use-privacy/tessera.md). + +Using private transactions with [pruning] or [fast sync](../../../../public-networks/reference/cli/options.md#sync-mode) is not supported. + +::: + + + +[signed with a random key or the key specified on the command line]: ../../../how-to/use-privacy/sign-pmts.md +[highly available and run in a separate instance to Besu]: ../../../how-to/use-privacy/tessera.md +[pruning]: ../../../../public-networks/concepts/data-storage-formats.md#pruning diff --git a/versioned_docs/version-23.10.2/private-networks/get-started/_category_.json b/versioned_docs/version-23.10.2/private-networks/get-started/_category_.json new file mode 100644 index 00000000000..3b3ed44eda6 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/get-started/_category_.json @@ -0,0 +1,9 @@ +{ + "label": "Get started", + "position": 2, + "collapsed": false, + "link": { + "type": "generated-index", + "slug": "/private-networks/get-started" + } +} diff --git a/versioned_docs/version-23.10.2/private-networks/get-started/install/_category_.json b/versioned_docs/version-23.10.2/private-networks/get-started/install/_category_.json new file mode 100644 index 00000000000..043580c1474 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/get-started/install/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Install Besu", + "position": 2 +} diff --git a/versioned_docs/version-23.10.2/private-networks/get-started/install/binary-distribution.md b/versioned_docs/version-23.10.2/private-networks/get-started/install/binary-distribution.md new file mode 100644 index 00000000000..74dcba7e6f4 --- /dev/null +++ b/versioned_docs/version-23.10.2/private-networks/get-started/install/binary-distribution.md @@ -0,0 +1,94 @@ +--- +title: Install binary distribution +description: Install or upgrade Hyperledger Besu from binary distribution +sidebar_position: 3 +tags: + - private networks +--- + +# Install binary distribution + +## MacOS with Homebrew + +### Prerequisites + +- [Homebrew](https://brew.sh/) +- Java JDK + +:::caution + +Hyperledger Besu supports: + +- MacOS High Sierra 10.13 or later versions. +- Java 17+. You can install Java using `brew install openjdk`. Alternatively, you can manually install the [Java JDK](https://www.oracle.com/java/technologies/downloads). + +::: + +### Install (or upgrade) using Homebrew + +To install Besu using Homebrew: + +```bash +brew tap hyperledger/besu +brew install hyperledger/besu/besu +``` + +To upgrade an existing Besu installation using Homebrew: + +```bash +brew upgrade hyperledger/besu/besu +``` + +:::note + +If you've upgraded your MacOS version between installing and upgrading Besu, when running `brew upgrade hyperledger/besu/besu` you may be prompted to reinstall command line tools with `xcode-select --install`. + +::: + +:::note + +When upgrading Besu, you might be prompted to fix the remote branch names in Homebrew by using the command `brew tap --repair`. + +::: + +To display the Besu version and confirm installation: + +```bash +besu --version +``` + +To display Besu command line help: + +```bash +besu --help +``` + +## Linux / Unix + +### Prerequisites + +- [Java JDK 17+](https://www.oracle.com/java/technologies/downloads/) + +:::note Linux open file limit + +If synchronizing to Mainnet on Linux or other chains with large data requirements, increase the maximum number of open files allowed using `ulimit`. If the open files limit is not high enough, a `Too many open files` RocksDB exception occurs. + +::: + +:::tip + +We recommend installing [jemalloc](https://jemalloc.net/) to reduce memory usage. If using Ubuntu, you can install it with the command: `apt install libjemalloc-dev`. + +::: + +### Install from packaged binaries + +Download the Besu [packaged binaries](https://github.com/hyperledger/besu/releases). + +Unpack the downloaded files and change into the `besu-+ + + +
+ +### Pruning + +Pruning reduces the storage required by removing state trie nodes unreachable from [recent blocks](../../public-networks/reference/cli/options.md#pruning-blocks-retained). + +Pruning is disabled by default, and can be enabled with the [`--pruning-enabled`](../../public-networks/reference/cli/options.md#pruning-enabled) command line option. + +:::info + +Using pruning with [private transactions](../../private-networks/concepts/privacy/private-transactions)\ +isn't supported. + +::: + +Pruning might increase block import times, but it doesn't affect the ability of nodes to stay in sync. + +:::caution + +Pruning is being deprecated for [Bonsai Tries](#bonsai-tries) and is currently not being updated. + +::: + +## Bonsai Tries + +Bonsai Tries is a data storage layout policy designed to reduce storage requirements and increase read performance. + +Bonsai stores leaf values in a trie log, separate from the branches of the trie. Bonsai stores nodes by the location of the node instead of the hash of the node. Bonsai can access the leaf from the underlying storage directly using the account key. This greatly reduces the disk space needed for storage and allows for less resource-demanding and faster read performance. Bonsai inherently prunes orphaned nodes and old branches. + +To run a node with Bonsai Tries data storage format, use the command line option [`--data-storage-format=BONSAI`](../reference/cli/options.md#data-storage-format). + ++ + + +
+ +## Forest of Tries vs. Bonsai Tries + +### Storage requirements + +Forest mode uses significantly more memory than Bonsai. With an [archive node](../get-started/connect/sync-node.md#run-an-archive-node), forest mode uses an estimated 12 TB of storage, while Bonsai uses an estimated 1100 GB of storage. With a [full node](../get-started/connect/sync-node.md#run-a-full-node), forest mode uses an estimated 750 GB of storage, while Bonsai uses an estimated 650 GB of storage. + +### Accessing data + +Forest mode must go through all the branches by hash to read a leaf value. Bonsai can access the leaf from the underlying storage directly using the account key. Bonsai will generally read faster than forest mode, particularly if the blocks are more recent. + +However, Bonsai becomes increasingly more resource-intensive the further in history you try to read data. To prevent this, you can limit how far Bonsai looks back while reconstructing data. The default limit Bonsai looks back is 512. To change the parameter, use the [`--bonsai-historical-block-limit`](../reference/cli/options.md#bonsai-historical-block-limit) option. + +:::note + +Using `--bonsai-historical-block-limit` doesn't affect the size of the database being stored, only how far back to load. This means there is no "safe minimum" value to use with this option. + +::: + +### Syncing nodes + +The following table shows the ways you can [sync a full node](../get-started/connect/sync-node.md#run-a-full-node) with the different data storage formats using [fast](../get-started/connect/sync-node.md#fast-synchronization) and [snap](../get-started/connect/sync-node.md#snap-synchronization) sync. + +| Data storage format | Sync mode | Storage estimate | Can other nodes sync to your node? | +| --- | --- | --- | --- | +| Bonsai | Fast | 650 GB | No | +| Bonsai | Snap | 650 GB | To be implemented | +| Forest | Fast | 750 GB | Yes | +| Forest | Snap | 750 GB | No | + +:::tip + +We recommend using snap sync with Bonsai for the fastest sync and lowest storage requirements. + +::: diff --git a/versioned_docs/version-23.10.2/public-networks/concepts/events-and-logs.md b/versioned_docs/version-23.10.2/public-networks/concepts/events-and-logs.md new file mode 100644 index 00000000000..c5b78fe2bbb --- /dev/null +++ b/versioned_docs/version-23.10.2/public-networks/concepts/events-and-logs.md @@ -0,0 +1,186 @@ +--- +title: Events and logs +sidebar_position: 6 +description: Learn about events and logs in Besu. +tags: + - public networks + - private networks +--- + +# Events and logs + +Transaction mining causes smart contracts to emit events and write logs to the blockchain. + +The smart contract address is the link to the logs and the blockchain includes the logs, but contracts cannot access logs. Log storage is cheaper than contract storage (that is, it costs less gas) so storing and accessing the required data in logs reduces the cost. For example, use logs to display all transfers made using a specific contract, but not the current state of the contract. + +A Dapp front end can either access logs using the [JSON-RPC API filter methods](../how-to/use-besu-api/access-logs.md) or subscribe to logs using the [RPC Pub/Sub API](../how-to/use-besu-api/rpc-pubsub.md#logs). + +Use [`admin_generateLogBloomCache`](../reference/api/index.md#admin_generatelogbloomcache) to improve log retrieval performance. + +## Topics + +Log entries contain up to four topics. The first topic is the [event signature hash](#event-signature-hash) and up to three topics are the indexed [event parameters](#event-parameters). + +```json title="A log entry for an event with one indexed parameter" +{ + "logIndex": "0x0", + "removed": false, + "blockNumber": "0x84", + "blockHash": "0x5fc573d76ec48ec80cbc43f299ebc306a8168112e3a4485c23e84e9a40f5d336", + "transactionHash": "0xcb52f02342c2498df82c49ac26b2e91e182155c8b2a2add5b6dc4c249511f85a", + "transactionIndex": "0x0", + "address": "0x42699a7612a82f1d9c36148af9c77354759b210b", + "data": "0x", + "topics": [ + "0x04474795f5b996ff80cb47c148d4c5ccdbe09ef27551820caa9c2f8ed149cce3", + "0x0000000000000000000000000000000000000000000000000000000000000001" + ] +} +``` + +## Event parameters + +Up to three event parameters can have the `indexed` attribute. Logs store these indexed parameters as `topics`. Indexed parameters are searchable and filterable. + +Topics are 32 bytes. If an indexed argument is an array (including `string` and `byte` datatypes), the log stores the keccak-256 hash of the parameter as a topic. + +Log `data` includes non-indexed parameters but is difficult to search or filter. + +A Solidity contract storing one indexed and one non-indexed parameter and has an event emitting the value of each parameter: + +```solidity +pragma solidity ^0.5.1; +contract Storage { + uint256 public valueIndexed; + uint256 public valueNotIndexed; + + event Event1(uint256 indexed valueIndexed, uint256 valueNotIndexed); + + function setValue(uint256 _valueIndexed, uint256 _valueNotIndexed) public { + valueIndexed = _valueIndexed; + valueNotIndexed = _valueNotIndexed; + emit Event1(_valueIndexed, _valueNotIndexed); + } +} +``` + +A log entry created by invoking the contract in the previous example with `valueIndexed` set to 5 and `valueNotIndexed` set to 7: + +```json +{ + "logIndex": "0x0", + "removed": false, + "blockNumber": "0x4d6", + "blockHash": "0x7d0ac7c12ac9f622d346d444c7e0fa4dda8d4ed90de80d6a28814613a4884a67", + "transactionHash": "0xe994022ada94371ace00c4e1e20663a01437846ced02f18b3f3afec827002781", + "transactionIndex": "0x0", + "address": "0x43d1f9096674b5722d359b6402381816d5b22f28", + "data": "0x0000000000000000000000000000000000000000000000000000000000000007", + "topics": [ + "0xd3610b1c54575b7f4f0dc03d210b8ac55624ae007679b7a928a4f25a709331a8", + "0x0000000000000000000000000000000000000000000000000000000000000005" + ] +} +``` + +## Event signature hash + +The first topic in a log entry is always the event signature hash. The event signature hash is a keccak-256 hash of the event name and input argument types, with argument names ignored. For example, the event `Hello(uint256 worldId)` has the signature hash `keccak('Hello(uint256)')`. The signature identifies to which event log topics belong. + +A Solidity contract with two different events: + +```solidity +pragma solidity ^0.5.1; +contract Storage { + uint256 public valueA; + uint256 public valueB; + + event Event1(uint256 indexed valueA); + event Event2(uint256 indexed valueB); + + function setValue(uint256 _valueA) public { + valueA = _valueA; + emit Event1(_valueA); + } + + function setValueAgain(uint256 _valueB) public { + valueB = _valueB; + emit Event2(_valueB); + } +} +``` + +The event signature hash for event 1 is `keccak('Event1(uint256)')` and the event signature hash for event 2 is `keccak('Event2(uint256)')`. The hashes are: + +- `04474795f5b996ff80cb47c148d4c5ccdbe09ef27551820caa9c2f8ed149cce3` for event 1 +- `06df6fb2d6d0b17a870decb858cc46bf7b69142ab7b9318f7603ed3fd4ad240e` for event 2. + +:::tip + +You can use a library keccak (sha3) hash function, such as provided in [Web3.js](https://web3js.readthedocs.io/en/v1.2.11/web3-utils.html?highlight=sha3#sha3), or an online tool, such as https://emn178.github.io/online-tools/keccak_256.html, to generate event signature hashes. + +::: + +Log entries from invoking the Solidity contract in the previous example: + +```json +[ + { + "logIndex": "0x0", + "removed": false, + "blockNumber": "0x84", + "blockHash": "0x5fc573d76ec48ec80cbc43f299ebc306a8168112e3a4485c23e84e9a40f5d336", + "transactionHash": "0xcb52f02342c2498df82c49ac26b2e91e182155c8b2a2add5b6dc4c249511f85a", + "transactionIndex": "0x0", + "address": "0x42699a7612a82f1d9c36148af9c77354759b210b", + "data": "0x", + "topics": [ + "0x04474795f5b996ff80cb47c148d4c5ccdbe09ef27551820caa9c2f8ed149cce3", + "0x0000000000000000000000000000000000000000000000000000000000000001" + ] + }, + { + "logIndex": "0x0", + "removed": false, + "blockNumber": "0x87", + "blockHash": "0x6643a1e58ad857f727552e4572b837a85b3ca64c4799d085170c707e4dad5255", + "transactionHash": "0xa95295fcea7df3b9e47ab95d2dadeb868145719ed9cc0e6c757c8a174e1fcb11", + "transactionIndex": "0x0", + "address": "0x42699a7612a82f1d9c36148af9c77354759b210b", + "data": "0x", + "topics": [ + "0x06df6fb2d6d0b17a870decb858cc46bf7b69142ab7b9318f7603ed3fd4ad240e", + "0x0000000000000000000000000000000000000000000000000000000000000002" + ] + } +] +``` + +## Topic filters + +[Filter options objects](../reference/api/objects.md#filter-options-object) have a `topics` key to filter logs by topics. + +Topics are order-dependent. A transaction with a log containing topics `[A, B]` matches with the following topic filters: + +- `[]` - Match any topic +- `[A]` - Match A in first position +- `[[null], [B]]` - Match any topic in first position AND B in second position +- `[[A],[B]]` - Match A in first position AND B in second position +- `[[A, C], [B, D]]` - Match (A OR C) in first position AND (B OR D) in second position. + +The following filter option object returns log entries for the [Event Parameters example contract](#event-parameters) with `valueIndexed` set to 5 or 9: + +```json +{ + "fromBlock": "earliest", + "toBlock": "latest", + "address": "0x43d1f9096674b5722d359b6402381816d5b22f28", + "topics": [ + ["0xd3610b1c54575b7f4f0dc03d210b8ac55624ae007679b7a928a4f25a709331a8"], + [ + "0x0000000000000000000000000000000000000000000000000000000000000005", + "0x0000000000000000000000000000000000000000000000000000000000000009" + ] + ] +} +``` diff --git a/versioned_docs/version-23.10.2/public-networks/concepts/genesis-file.md b/versioned_docs/version-23.10.2/public-networks/concepts/genesis-file.md new file mode 100644 index 00000000000..d91a237e096 --- /dev/null +++ b/versioned_docs/version-23.10.2/public-networks/concepts/genesis-file.md @@ -0,0 +1,50 @@ +--- +title: Genesis file +sidebar_position: 7 +description: Learn about configuring a network using the genesis file. +tags: + - public networks + - private networks +--- + +# Genesis file + +The genesis file defines the first block in the chain, and the first block defines which chain you want to join. + +For Ethereum Mainnet and public testnets (for example, Goerli) the genesis configuration definition is in Besu and used when specifying a public network using the [`--network`](../reference/cli/options.md#network) command line option. + +For private networks, [create a JSON genesis file](https://consensys.net/blog/quorum/hyperledger-besu-how-to-create-an-ethereum-genesis-file/), then specify the genesis file using the [`--genesis-file`](../reference/cli/options.md#genesis-file) command line option. + +The genesis file specifies the [network-wide settings](../reference/genesis-items.md), such as those for a [free gas network](../../private-networks/how-to/configure/free-gas.md), so all nodes in a network must use the same genesis file. + +:::note + +You can specify node-level settings on the command line or in the [node configuration file](../how-to/configuration-file.md). + +::: + +```json title="Example IBFT 2.0 genesis file" +{ + "config": { + "chainId": 2018, + "berlinBlock": 0, + "ibft2": { + "blockperiodseconds": 2, + "epochlength": 30000, + "requesttimeoutseconds": 4 + } + }, + "nonce": "0x0", + "timestamp": "0x58ee40ba", + "extraData": "0xf83ea00000000000000000000000000000000000000000000000000000000000000000d5949811ebc35d7b06b3fa8dc5809a1f9c52751e1deb808400000000c0", + "gasLimit": "0x1fffffffffffff", + "difficulty": "0x1", + "mixHash": "0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365", + "coinbase": "0x0000000000000000000000000000000000000000", + "alloc": { + "9811ebc35d7b06b3fa8dc5809a1f9c52751e1deb": { + "balance": "0xad78ebc5ac6200000" + } + } +} +``` diff --git a/versioned_docs/version-23.10.2/public-networks/concepts/network-and-chain-id.md b/versioned_docs/version-23.10.2/public-networks/concepts/network-and-chain-id.md new file mode 100644 index 00000000000..a72133d3a5e --- /dev/null +++ b/versioned_docs/version-23.10.2/public-networks/concepts/network-and-chain-id.md @@ -0,0 +1,74 @@ +--- +title: Network ID and chain ID +sidebar_position: 5 +description: Learn about network ID and chain ID in Besu. +tags: + - public networks + - private networks +--- + +# Network ID and chain ID + +Ethereum networks have two identifiers, a network ID and a chain ID. Although they often have the same value, they have different uses. + +Peer-to-peer communication between nodes uses the _network ID_, while the transaction signature process uses the _chain ID_. + +:::note + +[EIP-155](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-155.md) introduced using the chain ID as part of the transaction signing process to protect against transaction replay attacks. + +::: + +For most networks, including Mainnet and the public testnets, the network ID and the chain ID are the same, with the network ID defaulting to the chain ID, as specified in the genesis file. + +```json title="Chain ID in the genesis file" +{ + "config": { + "ethash": { + }, + "chainID": 1981 + }, + ... +} +``` + +Besu sets the chain ID (and by default the network ID) automatically, using either the [`--genesis-file`](../reference/cli/options.md#genesis-file) option or when specifying a network using the [`--network`](../reference/cli/options.md#network) option. The following table lists the available networks and their chain and network IDs. + +| Network | Chain | Chain ID | Network ID | Type | +| --------- | ----- | -------- | ---------- | ----------- | +| `mainnet` | ETH | 1 | 1 | Production | +| `goerli` | ETH | 5 | 5 | Test | +| `sepolia` | ETH | 11155111 | 11155111 | Test | +| `dev` | ETH | 2018 | 2018 | Development | +| `classic` | ETC | 61 | 1 | Production | +| `mordor` | ETC | 63 | 7 | Test | + +:::info + +The Ropsten, Rinkeby, and Kiln testnets are deprecated. + +::: + +## Specify a different network ID + +Usually the network ID is the same as the chain ID, but if you want to separate specific nodes from the rest of the network so they can't connect or synchronize with other nodes, you can override the default network ID for those nodes using the [`--network-id`](../reference/cli/options.md#network-id) option. + +## Start a new chain with a new chain ID + +If you update the chain ID (or network ID) of existing nodes, they can no longer peer with other nodes in the network. Nodes need to have a matching [genesis file](genesis-file.md), including the chain ID, in order to peer. In this case, you're effectively running two chains that can't communicate with each other. + +To change a chain ID and start a new chain: + +1. Stop all your nodes using ctrl+c in each terminal window. +2. Update the [genesis file](genesis-file.md) with the new chain ID. +3. Make sure all nodes have the same genesis file. +4. Delete the old data directory or point to a new location for each node. +5. [Restart the nodes](../../private-networks/tutorials/ibft/index.md#6-start-the-first-node-as-the-bootnode). + +:::danger Warning + +Starting a new chain is starting from block zero. + +This means when you start a new chain with a new chain ID, you lose all previous data. + +::: diff --git a/versioned_docs/version-23.10.2/public-networks/concepts/node-keys.md b/versioned_docs/version-23.10.2/public-networks/concepts/node-keys.md new file mode 100644 index 00000000000..01f1c98a31a --- /dev/null +++ b/versioned_docs/version-23.10.2/public-networks/concepts/node-keys.md @@ -0,0 +1,104 @@ +--- +title: Node keys +sidebar_position: 8 +description: Learn about node public and private keys, and the node address. +tags: + - public networks + - private networks +--- + +# Node keys and node address + +Each node has a private and public key pair, and a node address. Hyperledger Besu uses the private and public key pair to sign and verify transactions, and the node address as an identifier for the node. + +## Node private key + +When starting Hyperledger Besu, if the [`--node-private-key-file`](../reference/cli/options.md#node-private-key-file) option is not specified and a `key` file does not exist in the data directory for the node, Besu generates a node private key and writes it to the `key` file. + +If a `key` file does exist in the data directory when starting Besu, the node starts using the private key in the `key` file. + +:::info + +The private key is not encrypted. + +::: + +## Node public key + +The node public key displays in the log after starting Besu. Also referred to as the node ID, the node public key forms part of the enode URL of a node. + +You can export the node public key, either to standard output or to a specified file, using the [`public-key export`](../reference/cli/subcommands.md#public-key) subcommand. + +## Node address + +Besu generates the node address by creating a hash of the node public key and using the last 20 bytes of the hash as the node address. It is also displayed in the logs after starting Besu. + +You can export the node address, either to standard output or to a specified file, using the [`public-key export-address`](../reference/cli/subcommands.md#public-key) subcommand. + +## Specify a custom node private key file + +Use the [`--node-private-key-file`](../reference/cli/options.md#node-private-key-file) option to specify a custom `key` file in any location. + +If the `key` file exists, the node starts with the private key in the `key` file. If the `key` file does not exist, Besu generates a node private key and writes it to the `key` file. + +For example, the following command either reads the node private key from `privatekeyfile` or writes a generated private key to `privatekeyfile`. + +```bash +besu --node-private-key-file="/Users/username/privatekeyfile" +``` + +## Enode URL + +The enode URL identifies a node. For example, the [`--bootnodes`](../reference/cli/options.md#bootnodes) option and the [`perm_addNodesToAllowlist`](../reference/api/index.md#perm_addnodestoallowlist) method specify nodes by enode URL. + +The enode URL format is `enode://