From cfad9707a34165de46a8f46642dced90f3a53a5a Mon Sep 17 00:00:00 2001 From: Alexandra Carrillo <12214231+alexandratran@users.noreply.github.com> Date: Mon, 21 Oct 2024 10:30:40 -0700 Subject: [PATCH] Update syncing docs (#1717) * Update syncing docs Signed-off-by: Alexandra Tran * Update options.md Co-authored-by: Joan E <153745173+joaniefromtheblock@users.noreply.github.com> Signed-off-by: Alexandra Carrillo <12214231+alexandratran@users.noreply.github.com> * address reviewer feedback + edits Signed-off-by: Alexandra Tran --------- Signed-off-by: Alexandra Tran Signed-off-by: Alexandra Carrillo <12214231+alexandratran@users.noreply.github.com> Co-authored-by: Alexandra Tran Co-authored-by: Joan E <153745173+joaniefromtheblock@users.noreply.github.com> --- .../concepts/node-sync-private.md | 22 +- .../get-started/start-node.md | 2 +- .../how-to/configure/consensus/ibft.md | 11 +- .../how-to/configure/consensus/qbft.md | 9 +- .../concepts/data-storage-formats.md | 10 +- docs/public-networks/concepts/node-sync.md | 245 +++++++++--------- .../how-to/troubleshoot/trace-transactions.md | 2 +- docs/public-networks/reference/api/index.md | 8 +- docs/public-networks/reference/cli/options.md | 39 +-- 9 files changed, 172 insertions(+), 176 deletions(-) diff --git a/docs/private-networks/concepts/node-sync-private.md b/docs/private-networks/concepts/node-sync-private.md index 67007927d6f..169456484a1 100644 --- a/docs/private-networks/concepts/node-sync-private.md +++ b/docs/private-networks/concepts/node-sync-private.md @@ -19,14 +19,15 @@ To sync Besu on a private network: - Set the network ID and bootnodes specific to your private network. - Implement permissioning features to control network access. -Choose the appropriate sync mode based on your private network's requirements and node purposes. +The following is an overview of the private network sync modes. +Select the sync mode based on your network's requirements and node purposes. -| Sync mode | Description | Requirements | Limitations | -|-----------|-------------|--------------|-------------| -| [Snap](../../public-networks/concepts/node-sync.md#snap-synchronization) | Recommended for fastest sync and lowest storage requirements on Mainnet. Downloads as many leaves of the trie as possible and reconstructs the trie locally. Faster than fast sync. | Available as an _early access feature_ in Besu version 24.7.1 or later | Cannot switch from fast sync to snap sync mid-process. | -| [Checkpoint](../../public-networks/concepts/node-sync.md#checkpoint-synchronization) | Syncs from a specific checkpoint block configured in the genesis file. Fastest sync mode with lowest storage requirements. | Besu version 22.4.3 or later | Not supported for QBFT or IBFT 2.0 networks without a checkpoint configuration. | -| [Fast](../../public-networks/concepts/node-sync.md#fast-synchronization) |Default for named networks except `dev`. Downloads block headers and transaction receipts, verifies chain from genesis block. | None | Not supported with private transactions. | -| [Full](../../public-networks/concepts/node-sync.md#run-an-archive-node) | Downloads and verifies the entire blockchain and state from genesis block, building an archive node with full state history. | None | Slowest sync mode, requires the most disk space. | +| Sync mode | Description | Requirements | Limitations | +|--------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|-------------| +| [Snap](../../public-networks/concepts/node-sync.md#snap-synchronization) | Recommended for fastest sync and lowest storage requirements on Mainnet. Downloads as many leaves of the trie as possible and reconstructs the trie locally. Faster than fast sync. | Available as an _early access feature_ in Besu version 24.7.1 or later | Cannot switch from fast sync to snap sync mid-process. | +| [Checkpoint](../../public-networks/concepts/node-sync.md#checkpoint-synchronization) | Syncs from a specific checkpoint block configured in the genesis file. Fastest sync mode with lowest storage requirements. | Besu version 22.4.3 or later | Not supported for QBFT or IBFT 2.0 networks without a checkpoint configuration. | +| [Fast](../../public-networks/concepts/node-sync.md#fast-synchronization) | Downloads block headers and transaction receipts, verifies chain from genesis block. | None | Not supported with private transactions. | +| [Full](../../public-networks/concepts/node-sync.md#full-synchronization) | Default for all private networks. Downloads and verifies the entire blockchain and state from genesis block, building an archive node with full state history. | None | Slowest sync mode, requires the most disk space. | :::warning Early access feature @@ -36,10 +37,3 @@ It is not stable, so use this option with caution. Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable snap sync in QBFT and IBFT 2.0 private networks. ::: - -:::tip - -We recommend snap sync because it follows the Ethereum specification and enables you to serve full historical data. - -::: - diff --git a/docs/private-networks/get-started/start-node.md b/docs/private-networks/get-started/start-node.md index 0a7559a38fd..48ee42032a2 100644 --- a/docs/private-networks/get-started/start-node.md +++ b/docs/private-networks/get-started/start-node.md @@ -107,7 +107,7 @@ You might need to set [`--tx-pool-limit-by-account-percentage`](../../public-net :::note Sync nodes for BFT -If you're running a node on a [QBFT](../how-to/configure/consensus/qbft.md) or [IBFT 2.0](../how-to/configure/consensus/ibft.md) network, your node must use [fast sync](../../public-networks/concepts/node-sync.md#fast-synchronization) or [full sync](../../public-networks/concepts/node-sync.md#run-an-archive-node). +If you're running a node on a [QBFT](../how-to/configure/consensus/qbft.md) or [IBFT 2.0](../how-to/configure/consensus/ibft.md) network, your node must use [fast sync](../../public-networks/concepts/node-sync.md#fast-synchronization) or [full sync](../../public-networks/concepts/node-sync.md#full-synchronization). Full sync is set by default. diff --git a/docs/private-networks/how-to/configure/consensus/ibft.md b/docs/private-networks/how-to/configure/consensus/ibft.md index f39a2220d5b..5dc12fe5bee 100644 --- a/docs/private-networks/how-to/configure/consensus/ibft.md +++ b/docs/private-networks/how-to/configure/consensus/ibft.md @@ -19,9 +19,11 @@ Existing validators propose and vote to [add or remove validators](#add-and-remo You can [create a private network using IBFT](../../../tutorials/ibft/index.md). -:::danger +:::caution -Configure your network to ensure you never lose more than 1/3 of your validators. If more than 1/3 of validators stop participating, new blocks are no longer created, and the network stalls. It may take significant time to recover once nodes are restarted. +Configure your network to ensure you never lose more than 1/3 of your validators. +If more than 1/3 of validators stop participating, the network stops creating new blocks and stalls. +It might take significant time to recover after nodes are restarted. ::: @@ -31,12 +33,13 @@ You can use a plugin to securely store a validator's key using the [`--security- ::: -:::warning Early access feature +:::info Early access feature `--Xsnapsync-bft-enabled` is an early access feature available in Besu version 24.7.1 and later. It is not stable, so use this option with caution. -Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable snap sync in IBFT 2.0 networks. +Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable +[snap sync](../../../concepts/node-sync-private.md) in IBFT 2.0 networks. ::: diff --git a/docs/private-networks/how-to/configure/consensus/qbft.md b/docs/private-networks/how-to/configure/consensus/qbft.md index 205dde8c246..71b1a25e31c 100644 --- a/docs/private-networks/how-to/configure/consensus/qbft.md +++ b/docs/private-networks/how-to/configure/consensus/qbft.md @@ -21,7 +21,9 @@ You can [create a private network using QBFT](../../../tutorials/qbft.md). :::caution -Configure your network to ensure you never lose more than 1/3 your validators. If more than 1/3 of validators stop participating, new blocks are no longer created, and the network stalls. It may take significant time to recover once nodes are restarted. +Configure your network to ensure you never lose more than 1/3 of your validators. +If more than 1/3 of validators stop participating, the network stops creating new blocks and stalls. +It might take significant time to recover after nodes are restarted. ::: @@ -31,12 +33,13 @@ You can use a plugin to securely store a validator's key using the [`--security- ::: -:::warning Early access feature +:::info Early access feature `--Xsnapsync-bft-enabled` is an early access feature available in Besu version 24.7.1 and later. It is not stable, so use this option with caution. -Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable snap sync in QBFT networks. +Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable +[snap sync](../../../concepts/node-sync-private.md) in QBFT networks. ::: diff --git a/docs/public-networks/concepts/data-storage-formats.md b/docs/public-networks/concepts/data-storage-formats.md index 8331b28c9e9..a45b0759100 100644 --- a/docs/public-networks/concepts/data-storage-formats.md +++ b/docs/public-networks/concepts/data-storage-formats.md @@ -26,7 +26,7 @@ To run a node with Bonsai Tries data storage format, use the command line option :::caution important -Do not run an [archive node](node-sync.md#run-an-archive-node) with Bonsai Tries. +Do not run an [archive node](node-sync.md#archive-nodes) with Bonsai Tries. Bonsai is designed for retrieving recent data only. ::: @@ -39,7 +39,7 @@ You can read more about Bonsai in [Consensys' Guide to Bonsai Tries](https://con ## Forest of Tries -Forest of Tries, also called forest mode, is another method of representing the world state, and is more suitable for [archive nodes](node-sync.md#run-an-archive-node). +Forest of Tries, also called forest mode, is another method of representing the world state, and is more suitable for [archive nodes](node-sync.md#archive-nodes). In forest mode, each node in the trie is saved in a key-value store by hash. For each block, the world state is updated with new nodes, leaf nodes, and a new state root. Old leaf nodes remain in the underlying data store. Data is accessed and stored by hash, which increases the size of the database and increases the resources and time needed to access account data. @@ -61,9 +61,9 @@ We recommend using [Bonsai Tries](#bonsai-tries) to save disk space. ### Storage requirements Forest mode uses significantly more memory than Bonsai. -With a [full node](node-sync.md#run-a-full-node), forest mode uses an +With a [full node](node-sync.md#full-nodes), forest mode uses an estimated 750 GB of storage, while Bonsai uses an estimated 650 GB of storage. -[Archive nodes](node-sync.md#run-an-archive-node) must use forest mode, which +[Archive nodes](node-sync.md#archive-nodes) must use forest mode, which uses an estimated 12 TB of storage. ### Accessing data @@ -80,7 +80,7 @@ Using `--bonsai-historical-block-limit` doesn't affect the size of the database ### Syncing nodes -The following table shows the ways you can [sync a full node](node-sync.md#run-a-full-node) with the different data storage formats using [fast](node-sync.md#fast-synchronization) and [snap](node-sync.md#snap-synchronization) sync. +The following table shows the ways you can sync a [full node](node-sync.md#full-nodes) with the different data storage formats using [fast](node-sync.md#fast-synchronization) and [snap](node-sync.md#snap-synchronization) sync. | Data storage format | Sync mode | Storage estimate | Can other nodes sync to your node? | | --- | --- | --- | --- | diff --git a/docs/public-networks/concepts/node-sync.md b/docs/public-networks/concepts/node-sync.md index 9583aa4de3f..99874a3b628 100644 --- a/docs/public-networks/concepts/node-sync.md +++ b/docs/public-networks/concepts/node-sync.md @@ -8,8 +8,12 @@ tags: # Node synchronization -Besu supports two node types, commonly referred to as [full nodes](#run-a-full-node) and -[archive nodes](#run-an-archive-node). +Besu supports two [node types](#node-types) and several [synchronization modes](#sync-modes) on +public networks. + +## Node types + +### Full nodes A full node consists of an [execution and consensus client](node-clients.md#execution-and-consensus-clients), @@ -21,97 +25,48 @@ Full nodes can guarantee the latest state of the blockchain (and some older stat can't serve the network with all data requests (for example, the balance of an account at an old block). +You can run a full node using [snap synchronization](#snap-synchronization), +[checkpoint synchronization](#checkpoint-synchronization), or +[fast synchronization](#fast-synchronization). + +### Archive nodes + An archive node is a node that also stores the intermediary state of every account and contract for every block since the genesis block. - Archive nodes can do everything full nodes do, and they can also access historical state data. This means that archive nodes require more disk space than full nodes. -Besu must connect with other peers to sync with the network. If your node is having trouble peering, -try [troubleshooting peering](../how-to/troubleshoot/peering.md). - -## Sync modes - -Besu supports several sync modes for different network types and use cases. -For public networks, Besu offers the following sync modes: - -| Sync mode | Description | Requirements | Limitations | -|-----------|-------------|--------------|-------------| -| [Snap](#snap-synchronization) | Default for Mainnet. Efficient sync from genesis block, downloading as many trie leaves as possible and reconstructing locally. Faster than fast sync. | Besu version 22.4.0 or later | Cannot switch from fast sync to snap sync mid-process. | -| [Checkpoint](#checkpoint-synchronization) | Syncs from a specific checkpoint block configured in the genesis file. Fastest sync mode with lowest storage requirements. | Besu version 22.4.3 or later | | -| [Fast](#fast-synchronization) | Default for named networks except `dev`. Downloads block headers and transaction receipts, verifies chain from genesis block. | None | Might become impossible to fast sync Ethereum Mainnet in the future. | -| [Full](#run-an-archive-node) | Downloads and verifies the entire blockchain and state from genesis block, building an archive node with full state history. | None | Slowest sync mode, requires the most disk space. | - -:::tip - -We recommend snap sync because it follows the Ethereum specification and enables you to serve full historical data. +You can only run an archive node using [full synchronization](#full-synchronization). +:::caution important +Do not run an archive node with the [Bonsai Tries](data-storage-formats.md#bonsai-tries) +data storage format. +Bonsai is designed for retrieving recent data only. ::: -:::note +## Sync modes -Ethereum nodes sync two types of data: chain data (blocks) and world state data (account storage). -Snap and checkpoint syncs handle chain data similarly to fast sync, but differ in how they process world state data. +Besu supports several synchronization modes for different network types, node types, and use cases. +The following is an overview of the public network sync modes: -::: +| Sync mode | Description | Requirements | Limitations | +|-------------------------------------------|-------------|--------------|-------------| +| [Snap](#snap-synchronization) | Efficient sync from genesis block, downloading as many trie leaves as possible and reconstructing locally. Faster than fast sync. | Besu version 22.4.0 or later | Cannot switch from fast sync to snap sync mid-process. | +| [Checkpoint](#checkpoint-synchronization) | Syncs from a specific checkpoint block configured in the genesis file. Fastest sync mode with lowest storage requirements. | Besu version 22.4.3 or later | | +| [Fast](#fast-synchronization) | Downloads block headers and transaction receipts, verifies chain from genesis block. | None | Might become impossible to fast sync Ethereum Mainnet in the future. | +| [Full](#full-synchronization) | Downloads and verifies the entire blockchain and state from genesis block, building an archive node with full state history. | None | Slowest sync mode, requires the most disk space. | -:::note Private network syncing +:::info Private network syncing Private networks can use the same sync methods as public networks, but might require different configurations. See [Node synchronization for private networks](../../private-networks/concepts/node-sync-private.md) for more information. ::: -## Sync times - -To sync with a public network, Besu runs two processes in parallel: the world state sync and the -blockchain download. - -While the world state syncs, Besu downloads and imports the blockchain in the background. -The blockchain download time depends on CPU, the network, Besu's peers, and disk speed. -The blockchain download generally takes longer than the world state sync. Besu must catch up to the -current chain head and sync the world state to participate on Mainnet. - -The following table shows the average world state sync time, and blockchain download time, for each -sync mode on Mainnet. - -All times are hardware dependent; this table is based on running AWS instances m6gd.2xlarge. -Each sync mode also has its own world state database size. - -| Sync mode | Time to sync world state | Time to download blockchain | Disk usage | -|------------|--------------------------|-----------------------------|---------------| -| Snap | ~6 hours | ~1.5 days | Average disk | -| Checkpoint | ~5 hours | ~13 hours | Smallest disk | -| Fast | ~1.5 days | ~1.5 days | Average disk | -| Full | ~weeks | ~weeks | Largest disk | - -:::note Notes -- As of late 2023, an average Mainnet snap sync consumes around 1000 GB using Bonsai Tries. - Read more about [storage requirements](data-storage-formats.md#storage-requirements) - across data storage formats and sync modes. -- Testnets take significantly less time and space to sync. -::: - -## Storage - -You can store the world state using [Forest of Tries](data-storage-formats.md#forest-of-tries) -or [Bonsai Tries](data-storage-formats.md#bonsai-tries). - -If you're [running a full node](#run-a-full-node), we recommend using Bonsai Tries for the lowest -storage requirements. - -## Run a full node +:::note Troubleshooting -A full node stores a local copy of the blockchain and the current state of the blockchain. -You can run a full node using [snap synchronization (snap sync)](#snap-synchronization), -[checkpoint synchronization (checkpoint sync)](#checkpoint-synchronization), or -[fast synchronization (fast sync)](#fast-synchronization). - -:::note Sync nodes for BFT - -Snap sync and checkpoint sync are not supported for -[QBFT](../../private-networks/how-to/configure/consensus/qbft.md) or -[IBFT 2.0](../../private-networks/how-to/configure/consensus/ibft.md) networks. +Besu must connect with other peers to sync with the network. +If your node is having trouble peering, try [troubleshooting peering](../how-to/troubleshoot/peering.md). ::: @@ -119,43 +74,44 @@ Snap sync and checkpoint sync are not supported for :::tip -We recommend using snap sync over fast sync because snap sync can be faster than fast sync by +We recommend using snap sync over fast sync because snap sync can be faster than fast sync by several days (for Mainnet). - -We recommend using snap sync with the [Bonsai](data-storage-formats.md#bonsai-tries) +Use snap sync with the [Bonsai](data-storage-formats.md#bonsai-tries) data storage format for the fastest sync and lowest storage requirements. ::: -Enable snap sync using [`--sync-mode=SNAP`](../reference/cli/options.md#sync-mode). You need Besu -version 22.4.0 or later to use snap sync. +Snap sync is the default sync mode for all named [networks](../reference/cli/options.md#network) +except `dev`. +You can enable snap sync using [`--sync-mode=SNAP`](../reference/cli/options.md#sync-mode). +You need Besu version 22.4.0 or later to use snap sync. -Instead of downloading the [state trie](data-storage-formats.md) node by node, snap +Instead of downloading the [state trie](data-storage-formats.md) node by node, snap sync downloads as many leaves of the trie as possible, and reconstructs the trie locally. -You can't switch from fast sync to snap sync. If your node is blocked in the middle of a fast sync, -you can start over using snap sync instead by stopping the node, deleting the data directory, and +You can't switch from fast sync to snap sync. If your node is blocked in the middle of a fast sync, +you can start over using snap sync instead by stopping the node, deleting the data directory, and starting over using `--sync-mode=SNAP`. -You can restart Besu during a snap sync in case of hardware or software problems. The sync resumes -from the last valid world state and continues to download blocks starting from the last downloaded +You can restart Besu during a snap sync in case of hardware or software problems. The sync resumes +from the last valid world state and continues to download blocks starting from the last downloaded block. -See [how to read the Besu metrics charts](../how-to/monitor/understand-metrics.md) when using +See [how to read the Besu metrics charts](../how-to/monitor/understand-metrics.md) when using snap sync. ### Checkpoint synchronization -Enable checkpoint sync using [`--sync-mode=CHECKPOINT`](../reference/cli/options.md#sync-mode). +You can enable checkpoint sync using [`--sync-mode=CHECKPOINT`](../reference/cli/options.md#sync-mode). You need Besu version 22.4.3 or later to use checkpoint sync. -Checkpoint sync behaves like [snap sync](#snap-synchronization), but instead of syncing from the +Checkpoint sync behaves like [snap sync](#snap-synchronization), but instead of syncing from the genesis block, it syncs from a specific checkpoint block configured in the [Besu genesis file](genesis-file.md). -Ethereum Mainnet and the Holesky testnet configurations already define default checkpoints, so you +Ethereum Mainnet and the Holesky testnet configurations already define default checkpoints, so you don't have to add this yourself. -For other networks, you can configure a checkpoint in the genesis file by specifying the block hash, +For other networks, you can configure a checkpoint in the genesis file by specifying the block hash, number, and total difficulty as in the following example. ```json title="Checkpoint configuration example" @@ -168,61 +124,56 @@ number, and total difficulty as in the following example. :::note -If using [Clique](../../private-networks/how-to/configure/consensus/clique.md) consensus, the +If using [Clique](../../private-networks/how-to/configure/consensus/clique.md) consensus, the checkpoint must be the beginning of an epoch. ::: -If you enable checkpoint sync without a checkpoint configuration in the genesis file, Besu snap +If you enable checkpoint sync without a checkpoint configuration in the genesis file, Besu snap syncs from the genesis block. -You can restart Besu during a checkpoint sync in case of hardware or software problems. The sync -resumes from the last valid world state and continues to download blocks starting from the last +You can restart Besu during a checkpoint sync in case of hardware or software problems. The sync +resumes from the last valid world state and continues to download blocks starting from the last downloaded block. ### Fast synchronization :::caution -It might become impossible to sync Ethereum Mainnet using fast sync in the future. If you sync for +It might become impossible to sync Ethereum Mainnet using fast sync in the future. If you sync for the first time or ever need to re-sync, update Besu to a version that supports newer sync methods. ::: -Enable fast sync using [`--sync-mode=FAST`](../reference/cli/options.md#sync-mode). +You can enable fast sync using [`--sync-mode=FAST`](../reference/cli/options.md#sync-mode). -Fast sync downloads the block headers and transaction receipts, and verifies the chain of block +Fast sync downloads the block headers and transaction receipts, and verifies the chain of block headers from the genesis block. -When starting fast sync, Besu first downloads the world state for a recent block verified by its +When starting fast sync, Besu first downloads the world state for a recent block verified by its peers (referred to as a pivot block), and then begins fast sync from the genesis block. -Fast sync is the default for named networks specified using the -[`--network`](../reference/cli/options.md#network) option, except for the `dev` development -network. It's also the default if connecting to Ethereum Mainnet by not specifying the -[`--network`](../reference/cli/options.md#network) or -[`--genesis-file`](../reference/cli/options.md#genesis-file) options. - -Using fast sync with [private transactions](../../private-networks/concepts/privacy/index.md) +Using fast sync with [private transactions](../../private-networks/concepts/privacy/index.md) isn't supported. -You can observe the `besu_synchronizer_fast_sync_*` and `besu_synchronizer_world_state_*` -[metrics](../how-to/monitor/metrics.md#metrics-list) to monitor fast sync. +You can observe the `besu_synchronizer_fast_sync_*` and `besu_synchronizer_world_state_*` +[metrics](../how-to/monitor/metrics.md#view-the-metrics-list) to monitor fast sync. :::note -When fast syncing, block numbers increase until close to the head block, then the process pauses -while the world state download completes. This may take a significant amount of time depending on -world state size, during which the current head block doesn't increase. For example, Mainnet may -take several days or more to fast sync. Fast sync time may increase because Besu picks new pivot +When fast syncing, block numbers increase until close to the head block, then the process pauses +while the world state download completes. This may take a significant amount of time depending on +world state size, during which the current head block doesn't increase. For example, Mainnet may +take several days or more to fast sync. Fast sync time may increase because Besu picks new pivot blocks, or because peers prune the world state before it completes downloading. ::: -:::caution RocksDB error on AWS +
+RocksDB error on AWS -When running Besu on some cloud providers, a known -[RocksDB](https://github.com/facebook/rocksdb/issues/6435) issue causes fast sync to fail occasionally. +When running Besu on some cloud providers, a known +[RocksDB](https://github.com/facebook/rocksdb/issues/6435) issue causes fast sync to fail occasionally. The following error is displayed repeatedly: ``` @@ -231,35 +182,35 @@ Restarting after short delay. java.util.concurrent.CompletionException: org.hyperledger.besu.plugin.services.exception.StorageException: org.rocksdb.RocksDBException: block checksum mismatch: ``` -The failure has been seen on AWS and Digital Ocean. On AWS, A full restart of the VM is required to -restart the fast sync. Fast sync isn't +The failure has been seen on AWS and Digital Ocean. On AWS, A full restart of the VM is required to +restart the fast sync. Fast sync isn't [currently supported on Digital Ocean](https://github.com/hyperledger/besu/blob/750580dcca349d22d024cc14a8171b2fa74b505a/CHANGELOG.md#143). -::: +
-:::caution Pending state nodes stays constant +
+Pending state nodes stays constant -When fast syncing, the pending state nodes count is the number of nodes yet to be downloaded, and it +When fast syncing, the pending state nodes count is the number of nodes yet to be downloaded, and it should change constantly. Pending state nodes trend to 0 during fast sync and then goes to 0. If the number stays constant, this could mean your node isn't syncing against any peers. -In the following example, the pivot block is 0 and the pending state nodes value is constant. This -means the node isn't syncing against any peers. The fact that state nodes have been downloaded means +In the following example, the pivot block is 0 and the pending state nodes value is constant. This +means the node isn't syncing against any peers. The fact that state nodes have been downloaded means at some stage it was syncing. ![Fast synchronization](../../assets/images/fastsync.png) The easiest solution in this scenario is to restart fast sync to obtain a new pivot block. -::: +
-## Run an archive node - -An archive node stores all historical states of the blockchain. -To run an archive node, enable full synchronization (full sync) using -[`--sync-mode=FULL`](../reference/cli/options.md#sync-mode). +### Full synchronization +Full sync is the default sync mode for the [`dev` network](../reference/cli/options.md#network). +You can enable full sync using [`--sync-mode=FULL`](../reference/cli/options.md#sync-mode). +Use full sync to run an [archive node](#archive-nodes). Full sync starts from the genesis block and reprocesses all transactions. :::caution important @@ -267,3 +218,43 @@ Do not run an archive node with the [Bonsai Tries](data-storage-formats.md#bonsa data storage format. Bonsai is designed for retrieving recent data only. ::: + +## Sync times + +To sync with a public network, Besu runs two processes in parallel: the world state sync and the +blockchain download. + +While the world state syncs, Besu downloads and imports the blockchain in the background. +The blockchain download time depends on CPU, the network, Besu's peers, and disk speed. +The blockchain download generally takes longer than the world state sync. Besu must catch up to the +current chain head and sync the world state to participate on Mainnet. + +The following table shows the average world state sync time, and blockchain download time, for each +sync mode on Mainnet. + +All times are hardware dependent; this table is based on running AWS instances m6gd.2xlarge. +Each sync mode also has its own world state database size. + +| Sync mode | Time to sync world state | Time to download blockchain | Disk usage | +|------------|--------------------------|-----------------------------|---------------| +| Snap | ~6 hours | ~1.5 days | Average disk | +| Checkpoint | ~5 hours | ~13 hours | Smallest disk | +| Fast | ~1.5 days | ~1.5 days | Average disk | +| Full | ~weeks | ~weeks | Largest disk | + +:::note Notes +- Snap and checkpoint syncs handle blockchain data similarly to fast sync, but differ in how they + process world state data. +- As of late 2023, an average Mainnet snap sync consumes around 1000 GB using Bonsai Tries. + Read more about [storage requirements](data-storage-formats.md#storage-requirements) + across data storage formats and sync modes. +- Testnets take significantly less time and space to sync. +::: + +## Storage + +You can store the world state using [Forest of Tries](data-storage-formats.md#forest-of-tries) +or [Bonsai Tries](data-storage-formats.md#bonsai-tries). +Besu uses Bonsai by default. + +If you're running a [full node](#full-nodes), we recommend using Bonsai for the lowest storage requirements. diff --git a/docs/public-networks/how-to/troubleshoot/trace-transactions.md b/docs/public-networks/how-to/troubleshoot/trace-transactions.md index 913369eef07..d2273f3fa88 100644 --- a/docs/public-networks/how-to/troubleshoot/trace-transactions.md +++ b/docs/public-networks/how-to/troubleshoot/trace-transactions.md @@ -34,7 +34,7 @@ The ad-hoc tracing APIs are: These APIs allow you to filter and search by specific information such as the block, address, or transaction. These APIs only use the [`trace` type](../../reference/trace-types.md#trace). To use the transaction-trace filtering APIs, your node must be an -[archive node](../../concepts/node-sync.md#run-an-archive-node), or the requested block +[archive node](../../concepts/node-sync.md#archive-nodes), or the requested block or transaction must be within the number of [blocks retained](../../reference/cli/options.md#bonsai-historical-block-limit) when using [Bonsai Tries](../../concepts/data-storage-formats.md#bonsai-tries) (by default, 512 from the head diff --git a/docs/public-networks/reference/api/index.md b/docs/public-networks/reference/api/index.md index 5327d71f4fe..6e6b31122b2 100644 --- a/docs/public-networks/reference/api/index.md +++ b/docs/public-networks/reference/api/index.md @@ -7131,7 +7131,7 @@ The `TRACE` API methods are not enabled by default for JSON-RPC. To enable the ` Provides transaction processing of [type `trace`](../trace-types.md#trace) for the specified block. :::info note -Your node must be an [archive node](../../concepts/node-sync.md#run-an-archive-node), or +Your node must be an [archive node](../../concepts/node-sync.md#archive-nodes), or the requested block must be within the number of [blocks retained](../cli/options.md#bonsai-historical-block-limit) when using [Bonsai](../../concepts/data-storage-formats.md#bonsai-tries) (by default, 512 from the head of the chain). @@ -7441,7 +7441,7 @@ curl -X POST --data '{"jsonrpc":"2.0","method":"trace_callMany","params":[[[{"fr Returns traces matching the specified filter. The maximum number of blocks you can supply to `trace_filter` is 1000 by default. You can adjust this limit using the [`--rpc-max-trace-filter-range`](../cli/options.md#rpc-max-trace-filter-range) option. :::info note -Your node must be an [archive node](../../concepts/node-sync.md#run-an-archive-node), or +Your node must be an [archive node](../../concepts/node-sync.md#archive-nodes), or the requested blocks must be within the number of [blocks retained](../cli/options.md#bonsai-historical-block-limit) when using [Bonsai](../../concepts/data-storage-formats.md#bonsai-tries) (by default, 512 from the head of the chain). @@ -7545,7 +7545,7 @@ curl -X POST --data '{"jsonrpc":"2.0","method":"trace_filter","params":[{"fromBl Returns a trace at the given position. :::info note -Your node must be an [archive node](../../concepts/node-sync.md#run-an-archive-node), or +Your node must be an [archive node](../../concepts/node-sync.md#archive-nodes), or the requested transaction must be contained in a block within the number of [blocks retained](../cli/options.md#bonsai-historical-block-limit) when using [Bonsai](../../concepts/data-storage-formats.md#bonsai-tries) (by default, 512 from the head of the chain). @@ -7822,7 +7822,7 @@ curl -X POST --data '{"jsonrpc": "2.0", "method": "trace_replayBlockTransactions Provides transaction processing of [type `trace`](../trace-types.md#trace) for the specified transaction. :::info note -Your node must be an [archive node](../../concepts/node-sync.md#run-an-archive-node), or +Your node must be an [archive node](../../concepts/node-sync.md#archive-nodes), or the requested transaction must be contained in a block within the number of [blocks retained](../cli/options.md#bonsai-historical-block-limit) when using [Bonsai](../../concepts/data-storage-formats.md#bonsai-tries) (by default, 512 from the head of the chain). diff --git a/docs/public-networks/reference/cli/options.md b/docs/public-networks/reference/cli/options.md index e07a7bc17fd..6469deaa9bb 100644 --- a/docs/public-networks/reference/cli/options.md +++ b/docs/public-networks/reference/cli/options.md @@ -2679,19 +2679,19 @@ The predefined network configuration. The default is `mainnet`. Possible values include the following: -| Network | Chain | Type | Default Sync Mode | Consensus Mechanism | Description | -| :-------- | :---- | :-----------| :----------------- | :----------------------- | :----------------------------------------------------------------------------------- | -| `mainnet` | ETH | Production | [SNAP](#sync-mode) | A PoS network | The main [Ethereum network](https://ethereum.org/en/developers/docs/networks/) | -| `holesky` | ETH | Test | [SNAP](#sync-mode) | A PoS network | Multi-client testnet [Hoelsky](https://holesky.dev) | -| `sepolia` | ETH | Test | [SNAP](#sync-mode) | A PoS network | Multi-client testnet [Sepolia](https://sepolia.dev) | -| `lukso` | ETH | Production | [SNAP](#sync-mode) | A PoS network | Network for the [Lukso chain](https://lukso.network/) | -| `dev` | ETH | Development | [FULL](#sync-mode) | A PoW network | Development network with low difficulty to enable local CPU mining | -| `classic` | ETC | Production | [SNAP](#sync-mode) | A PoW network | The main [Ethereum Classic network](https://ethereumclassic.org) | -| `mordor ` | ETC | Test | [SNAP](#sync-mode) | A PoW network | Testnet for [Ethereum Classic](https://github.com/eth-classic/mordor) | +| Network | Chain | Type | Default sync mode | Consensus mechanism | Description | +| :-------- | :---- | :-----------| :----------------- | :----------------------- |:-------------------------------------------------------------------------------| +| `mainnet` | ETH | Production | [`SNAP`](#sync-mode) | A PoS network | The main [Ethereum network](https://ethereum.org/en/developers/docs/networks/) | +| `holesky` | ETH | Test | [`SNAP`](#sync-mode) | A PoS network | Multi-client testnet [Holesky](https://holesky.dev) | +| `sepolia` | ETH | Test | [`SNAP`](#sync-mode) | A PoS network | Multi-client testnet [Sepolia](https://sepolia.dev) | +| `lukso` | ETH | Production | [`SNAP`](#sync-mode) | A PoS network | Network for the [Lukso chain](https://lukso.network/) | +| `dev` | ETH | Development | [`FULL`](#sync-mode) | A PoW network | Development network with low difficulty to enable local CPU mining | +| `classic` | ETC | Production | [`SNAP`](#sync-mode) | A PoW network | The main [Ethereum Classic network](https://ethereumclassic.org) | +| `mordor ` | ETC | Test | [`SNAP`](#sync-mode) | A PoW network | Testnet for [Ethereum Classic](https://github.com/eth-classic/mordor) | :::tip -Values are case insensitive, so either `mainnet` or `MAINNET` works. +Values are case-insensitive, so either `mainnet` or `MAINNET` works. ::: @@ -5089,22 +5089,27 @@ sync-mode="SNAP" -The synchronization mode. Use `SNAP` for [snap sync](../../concepts/node-sync.md#snap-synchronization), `CHECKPOINT` for [checkpoint sync](../../concepts/node-sync.md#checkpoint-synchronization), `FAST` for [fast sync](../../concepts/node-sync.md#fast-synchronization), and `FULL` for [full sync](../../concepts/node-sync.md#run-an-archive-node). +The synchronization mode. Use `SNAP` for [snap sync](../../concepts/node-sync.md#snap-synchronization), `CHECKPOINT` for [checkpoint sync](../../concepts/node-sync.md#checkpoint-synchronization), `FAST` for [fast sync](../../concepts/node-sync.md#fast-synchronization), and `FULL` for [full sync](../../concepts/node-sync.md#full-synchronization). - The default is `FULL` when connecting to a private network by not using the [`--network`](#network) option and specifying the [`--genesis-file`](#genesis-file) option. - The default is `SNAP` when using the [`--network`](#network) option with named networks, except for the `dev` development network. `SNAP` is also the default if running Besu on the default network (Ethereum Mainnet) by specifying neither [network](#network) nor [genesis file](#genesis-file). -:::note Sync nodes for BFT +:::note Notes -If you're running a node on a [QBFT](../../../private-networks/how-to/configure/consensus/qbft.md) or [IBFT 2.0](../../../private-networks/how-to/configure/consensus/ibft.md) network, your node must use fast sync or full sync. +- We recommend using snap sync over fast sync because snap sync can be faster by several days. +- It might become impossible to sync Mainnet using fast sync in the future, as clients drop support for fast sync. + We recommend updating Besu to a version that supports other sync methods. +- When using a mode other than `FULL`, most historical world state data is unavailable. + Any methods attempting to access unavailable world state data return `null`. ::: -:::tip +:::warning Early access feature -- We recommend using snap sync over fast sync because snap sync can be faster by several days. -- It might become impossible to sync Ethereum Mainnet using fast sync in the future, as clients drop support for fast sync. We recommend you update Besu to a version that supports newer sync methods. -- When synchronizing in a mode other than `FULL`, most historical world state data is unavailable. Any methods attempting to access unavailable world state data return `null`. +`--Xsnapsync-bft-enabled` is an early access feature available in Besu version 24.7.1 and later. +It is not stable, so use this option with caution. + +Use `--Xsnapsync-bft-enabled` with `--sync-mode=SNAP` to enable snap sync in QBFT and IBFT 2.0 private networks. :::