From 3340441a9d4164042465cebdf068dc8c3d9cc1a7 Mon Sep 17 00:00:00 2001 From: vishal <1117327+vishalchangrani@users.noreply.github.com> Date: Wed, 10 May 2023 17:30:55 -0700 Subject: [PATCH 1/3] updating spork dates --- docs/concepts/nodes/node-operation/upcoming-sporks.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/concepts/nodes/node-operation/upcoming-sporks.mdx b/docs/concepts/nodes/node-operation/upcoming-sporks.mdx index 0d5f1f61be..33d8612e2f 100644 --- a/docs/concepts/nodes/node-operation/upcoming-sporks.mdx +++ b/docs/concepts/nodes/node-operation/upcoming-sporks.mdx @@ -9,7 +9,9 @@ The following are the upcoming Spork dates. These dates indicate the intention t | Mainnet Spork Date | Spork Info | Testnet Spork Date | Spork Info | | :----------------- | :---------------------------------------------------| ---------------------------------------------- | ---------- | -| June, 2023 | Mainnet 23 | Apr 12, 2023 β€”> June, 2023 | Devnet43 | +| June, 2023 (exact date tbd) | Mainnet 23 | June, 2023 (exact date tbd) | | +| | | Apr 24, 2023 | Devnet44 | +| | | Apr 12, 2023 | Devnet43 | | | | Apr 12, 2023 | Devnet42 | | Feb 22, 2023 | Mainnet 22 | Jan 30, 2023 β€”> Feb 22, 2023 | Devnet41 | | | | Jan 23, 2023 | Devnet40 | From f304f099ebec3d230bf54c67c27daef1b36b2513 Mon Sep 17 00:00:00 2001 From: vishal <1117327+vishalchangrani@users.noreply.github.com> Date: Wed, 10 May 2023 17:31:22 -0700 Subject: [PATCH 2/3] updating .gitignore with .idea/ for IDE --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 64978a7d34..6c81732c21 100644 --- a/.gitignore +++ b/.gitignore @@ -24,4 +24,5 @@ yarn-error.log* /temp /src/data/doc-collections /docs/unsorted +.idea/ From 4570bc751ebe5785242c375cc4a61ce120ed3bb4 Mon Sep 17 00:00:00 2001 From: vishal <1117327+vishalchangrani@users.noreply.github.com> Date: Fri, 12 May 2023 14:06:12 -0700 Subject: [PATCH 3/3] revising the node operator docs --- docs/concepts/nodes/node-operation/index.md | 33 ++++++++-- .../nodes/node-operation/observer-node.mdx | 66 +++++++++---------- 2 files changed, 59 insertions(+), 40 deletions(-) diff --git a/docs/concepts/nodes/node-operation/index.md b/docs/concepts/nodes/node-operation/index.md index d5e30db280..e3086cbf2c 100644 --- a/docs/concepts/nodes/node-operation/index.md +++ b/docs/concepts/nodes/node-operation/index.md @@ -15,18 +15,37 @@ By running a node, you also directly contribute to the security and decentraliza Flow multirole architecture makes it more scalable and provides several node types that you as a node operator can pick and choose from. - ## Which node should you run? -The different types of nodes are described [here](./node-roles.mdx). As node operator, you can choose to run any of the different types of node that best fits your needs. +There are different types of node that you can choose from based on your use case. The nodes differ in terms of the data that they sync and APIs that they provide. +There are three main types of nodes: +1. Full node +2. Light node +3. Archive node +Apart from this, there is a staked node that you can run if you want to become a node validator. + + +### Full node + +### Light node (a.k.a observer node) + +The light node is one of the easiest node to spin up and can be run by Dapp developers who need the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. +A light node downloads every block and locally verifies that the blocks that are received are the correct extension of the chain. +Light node have very low hardware and bandwidth requirements and eventually might be able to run on mobile phones or embdedded devices as well. +They do not participate in block production but provide access to chain data by serving the Access API. + +To run a light node, follow this [guide](./observer-node.mdx). -### Observer node -The observer node is one of the easiest node to spin up and can be run by Dapp developers who need the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. -In addition to supporting dapps, an observer node can also be run by access node operators who want to scale their access nodes' endpoints. Access node operators can spin up geographically dispersed observer nodes which can talk to their staked access nodes and to each other. +### Archive node -The observer node is not staked but still provides the same API as the access node. +The archive node provides a scalable and efficient way to access historical states. +It follows the chain and locally stores and indexes all the chain data - blocks, transaction, collections, state register values etc. +It serves the chain data using an API interface. +Additionally, it also allows read-only queries such as script execution that require the execution state register values. +It can be used to answer any queries from past data e.g. β€œwhat was the Flow account balance at height X?” where X is several thousand blocks in the past. +Since it stores all the historic data, it requires a large amount of disk space and is not suitable to be run on simple commodity hardware. -To run an observer node, follow this [guide](./observer-node.mdx). +To run an archive node, follow this [guide](./archive-node.mdx). ### Access node If you want local access to the protocol state data (blocks, collections, transactions) and do not want to use one of the community access nodes you can run an access node. diff --git a/docs/concepts/nodes/node-operation/observer-node.mdx b/docs/concepts/nodes/node-operation/observer-node.mdx index e228b02517..5e3ce933b0 100644 --- a/docs/concepts/nodes/node-operation/observer-node.mdx +++ b/docs/concepts/nodes/node-operation/observer-node.mdx @@ -1,25 +1,25 @@ --- -title: Observer Node -sidebar_title: Observer Node +title: Light node (a.k.a Light node Node) +sidebar_title: Light Node --- -An observer node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](https://developers.flow.com/nodes/access-api.md) but unlike an access node, an observer node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes. +A light node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](https://developers.flow.com/nodes/access-api.md) but unlike an access node, a light node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes. -The observer node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other observer nodes. It then continuously receives blocks that are being added to the chain either directly from the access node or from other observer nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the observer node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to an observer node are also forwarded to the upstream access node. Future versions of the observer node will be able to serve this data locally as well. +The light node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other light nodes. It then continuously receives blocks that are being added to the chain either directly from the access node or from other light nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the light node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to a light node are also forwarded to the upstream access node. Future versions of the light node will be able to serve this data locally as well. -Since the observer node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space. +Since the light node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space. -![Observer nodes](observer.png) +![Light nodes](observer.png) -## Who should run an observer node? +## Who should run a light node? -The observer node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. In addition to that, access node operators who want to scale their access node endpoints geographically can spin up geographically dispersed observer nodes which can talk to their staked access node and to each other. +The light node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. In addition to that, access node operators who want to scale their access node endpoints geographically can spin up geographically dispersed light nodes which can talk to their staked access node and to each other. -## Running an observer node +## Running a light node ### Hardware -In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run an observer node. +In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run a light node. Minimum requirements @@ -28,12 +28,12 @@ Minimum requirements - 300 GB SSD disk - 10Mbps network connection -### Steps to run an observer node +### Steps to run a light node > [Here](https://www.loom.com/share/990a725531754106b91d8ccec6244219) is video walk-though of these πŸ‘‡ steps. #### Step 1 - Generate the node directory structure -The observer node requires the following directory structure, +The light node requires the following directory structure, ```shell $ tree flow_observer flow_observer/ @@ -41,7 +41,7 @@ flow_observer/ β”‚ β”œβ”€β”€ network.key (file containing the node private network key) β”‚ └── public-root-information β”‚ └── root-protocol-state-snapshot.json (the genesis data of the current spork) -└── data (directory used by the observer node to store block data) +└── data (directory used by the light node to store block data) ``` Create the parent and the sub-directories @@ -95,7 +95,7 @@ Add the `flow-signer@onflow.org` public key gpg --keyserver keys.openpgp.org --search-keys flow-signer@onflow.org gpg: data source: http://keys.openpgp.org:11371 -(1) Flow Team (Flow Full Observer node snapshot verification master key) < +(1) Flow Team (Flow Full Light node snapshot verification master key) < 256 bit ECDSA key CB5264F7FD4CDD27, created: 2021-09-15 Keys 1-1 of 1 for "flow-signer@onflow.org". Enter number(s), N)ext, or Q)uit > 1 ``` @@ -107,7 +107,7 @@ gpg --verify ./flow_observer/bootstrap/public-root-information/root-protocol-sta gpg: assuming signed data in 'bootstrap/public-root-information/root-protocol-state-snapshot.json' gpg: Signature made Wed Sep 15 11:34:33 2021 PDT gpg: using ECDSA key 40CD95717AC463E61EE3B285B718CA310EDB542F -gpg: Good signature from "Flow Team (Flow Full Observer node snapshot verification master key) " [unknown] +gpg: Good signature from "Flow Team (Flow Full Light node snapshot verification master key) " [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 7D23 8D1A E6D3 2A71 8ECD 8611 CB52 64F7 FD4C DD27 @@ -128,9 +128,9 @@ flow snapshot save ./flow_observer/bootstrap/public-root-information/root-protoc #### Step 4 - Start the node -The observer node can be run as a docker container +The light node can be run as a docker container -##### Observer for Flow Mainnet +##### Light node for Flow Mainnet ```shell docker run --rm \ @@ -154,7 +154,7 @@ docker run --rm \ --observer-networking-key-path=/bootstrap/network.key ``` -##### Observer for Flow Testnet +##### Light node for Flow Testnet ```shell docker run --rm \ @@ -178,15 +178,15 @@ docker run --rm \ --observer-networking-key-path=/bootstrap/network.key ``` -The observer node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server. +The light node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server. The upstream bootstrap server is specified using the `bootstrap-node-addresses` which is the comma-separated list of hostnames of the access nodes. The `bootstrap-node-public-keys` is the list of the corresponding networking public key of those nodes. -The observer node delegates many of the API calls to the upstream access nodes. -The `upstream-node-addresses` is the list of access node hostnames to which this observer node can delegate to. The list can be different from the bootstrap node list. +The light node delegates many of the API calls to the upstream access nodes. +The `upstream-node-addresses` is the list of access node hostnames to which this light node can delegate to. The list can be different from the bootstrap node list. The `bootstrap-node-public-key` is the list of the corresponding networking public key of those nodes. -> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports an observer node can be used +> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports a light node can be used _All parameters and their explanation can be found [here](https://github.com/onflow/flow-go/blob/82da35141ff095fbf75ce2c950efec240ad38565/cmd/access/node_builder/access_node_builder.go#L523-L558)_ @@ -201,37 +201,37 @@ flow blocks get latest --host localhost:9000 > REST API is currently not support (see [issue](https://github.com/onflow/flow-go/issues/3138)) -The observer node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics) +The light node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics) ## FAQs -### Does the observer node need to be staked? +### Does the light node need to be staked? -No, the observer node is not a staked node. +No, the light node is not a staked node. -### Can any access be used to bootstrap an observer node? +### Can any access be used to bootstrap a light node? -No, only Access nodes which have explicitly turned ON support for observer nodes can be used to bootstrap an observer node. +No, only Access nodes which have explicitly turned ON support for light nodes can be used to bootstrap a light node. -The public access nodes that support observer nodes are listed below. Apart from these, other public access nodes run by node operators other than DapperLabs may choose to support observer nodes. +The public access nodes that support light nodes are listed below. Apart from these, other public access nodes run by node operators other than DapperLabs may choose to support light nodes. -### How can an access node turn ON support for observer node? +### How can an access node turn ON support for light node? -An access node can support an observer node by passing in the following two parameters when starting the access node +An access node can support a light node by passing in the following two parameters when starting the access node ```shell --supports-observer=true --public-network-address=0.0.0.0:3570 ``` -`public-network-address` is the address the observer nodes will connect to. +`public-network-address` is the address the light nodes will connect to. -### Are observer nodes subject to rate limits? +### Are light nodes subject to rate limits? The observer serves all the [Block related queries](https://github.com/onflow/flow/blob/master/protobuf/flow/access/access.proto#L24-L42) from is local database. These are not subjected to any rate limits. However, it proxies all the other requests to the access node and those will be rate limited as per the rate limits defined on that access node. -### Flow community access nodes that support connections from observer nodes +### Flow community access nodes that support connections from light nodes #### For mainnet