Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vishal/node operator docs #65

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Vishal/node operator docs #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3340441
updating spork dates
vishalchangrani May 11, 2023
f304f09
updating .gitignore with .idea/ for IDE
vishalchangrani May 11, 2023
4570bc7
revising the node operator docs
vishalchangrani May 12, 2023
e081506
Merge branch 'main' into vishal/node_operator_docs
vishalchangrani May 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 26 additions & 7 deletions docs/concepts/nodes/node-operation/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,37 @@ By running a node, you also directly contribute to the security and decentraliza

Flow multirole architecture makes it more scalable and provides several node types that you as a node operator can pick and choose from.


## Which node should you run?

The different types of nodes are described [here](./node-roles.mdx). As node operator, you can choose to run any of the different types of node that best fits your needs.
There are different types of node that you can choose from based on your use case. The nodes differ in terms of the data that they sync and APIs that they provide.
There are three main types of nodes:
1. Full node
2. Light node
3. Archive node
Apart from this, there is a staked node that you can run if you want to become a node validator.


### Full node

### Light node (a.k.a observer node)

The light node is one of the easiest node to spin up and can be run by Dapp developers who need the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height.
A light node downloads every block and locally verifies that the blocks that are received are the correct extension of the chain.
Light node have very low hardware and bandwidth requirements and eventually might be able to run on mobile phones or embdedded devices as well.
They do not participate in block production but provide access to chain data by serving the Access API.

To run a light node, follow this [guide](./observer-node.mdx).

### Observer node
The observer node is one of the easiest node to spin up and can be run by Dapp developers who need the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height.
In addition to supporting dapps, an observer node can also be run by access node operators who want to scale their access nodes' endpoints. Access node operators can spin up geographically dispersed observer nodes which can talk to their staked access nodes and to each other.
### Archive node

The observer node is not staked but still provides the same API as the access node.
The archive node provides a scalable and efficient way to access historical states.
It follows the chain and locally stores and indexes all the chain data - blocks, transaction, collections, state register values etc.
It serves the chain data using an API interface.
Additionally, it also allows read-only queries such as script execution that require the execution state register values.
It can be used to answer any queries from past data e.g. “what was the Flow account balance at height X?” where X is several thousand blocks in the past.
Since it stores all the historic data, it requires a large amount of disk space and is not suitable to be run on simple commodity hardware.

To run an observer node, follow this [guide](./observer-node.mdx).
To run an archive node, follow this [guide](./archive-node.mdx).

### Access node
If you want local access to the protocol state data (blocks, collections, transactions) and do not want to use one of the community access nodes you can run an access node.
Expand Down
66 changes: 33 additions & 33 deletions docs/concepts/nodes/node-operation/observer-node.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
---
title: Observer Node
sidebar_title: Observer Node
title: Light node (a.k.a Light node Node)
sidebar_title: Light Node
---

An observer node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](../access-api.mdx) but unlike an access node, an observer node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes.
A light node is similar to an access node and provides a locally accessible, continuously updated, verified copy of the block data. It serves the [gRPC Access API](https://developers.flow.com/nodes/access-api.md) but unlike an access node, a light node does not need to be staked, and **anyone** can run it without being added to the approved list of nodes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to use relative links instead of absolute links. ../access-api.md should work fine.


The observer node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other observer nodes. It then continuously receives blocks that are being added to the chain either directly from the access node or from other observer nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the observer node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to an observer node are also forwarded to the upstream access node. Future versions of the observer node will be able to serve this data locally as well.
The light node bootstraps by connecting to an access node and becoming part of the public network comprised of access nodes and other light nodes. It then continuously receives blocks that are being added to the chain either directly from the access node or from other light nodes that are part of the public network. However, it makes no trust assumption of the upstream access node or the light node which is providing the block and locally verifies that the blocks that are received are the correct extension of the chain e.g. after receiving valid blocks A, B and C when it receives block D, it verifies that block D is indeed signed by the consensus nodes and is a valid next block. The received block data is indexed and made available via the Access API. For Collection, Transactions and Account queries, it delegates those requests to the upstream access node. Similarly, transactions and scripts sent to a light node are also forwarded to the upstream access node. Future versions of the light node will be able to serve this data locally as well.

Since the observer node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space.
Since the light node is not staked, it does not produce or execute blocks but instead serves as an _unstaked access node_ that can be easily run on any consumer-grade computer which has enough disk space.

![Observer nodes](observer.png)
![Light nodes](observer.png)

## Who should run an observer node?
## Who should run a light node?

The observer node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. In addition to that, access node operators who want to scale their access node endpoints geographically can spin up geographically dispersed observer nodes which can talk to their staked access node and to each other.
The light node provides an alternative to running an access node. Hence, it is ideal for Dapps that need access to the latest block data available locally e.g. a wallet application that needs to track the latest block ID and height. In addition to that, access node operators who want to scale their access node endpoints geographically can spin up geographically dispersed light nodes which can talk to their staked access node and to each other.

## Running an observer node
## Running a light node

### Hardware

In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run an observer node.
In general, any consumer-grade computer with a decent network connection and sufficient disk space should be able to run a light node.

Minimum requirements

Expand All @@ -28,20 +28,20 @@ Minimum requirements
- 300 GB SSD disk
- 10Mbps network connection

### Steps to run an observer node
### Steps to run a light node

> [Here](https://www.loom.com/share/990a725531754106b91d8ccec6244219) is video walk-though of these 👇 steps.

#### Step 1 - Generate the node directory structure
The observer node requires the following directory structure,
The light node requires the following directory structure,
```shell
$ tree flow_observer
flow_observer/
├── bootstrap
│ ├── network.key (file containing the node private network key)
│ └── public-root-information
│ └── root-protocol-state-snapshot.json (the genesis data of the current spork)
└── data (directory used by the observer node to store block data)
└── data (directory used by the light node to store block data)
```

Create the parent and the sub-directories
Expand Down Expand Up @@ -95,7 +95,7 @@ Add the `[email protected]` public key
gpg --keyserver keys.openpgp.org --search-keys [email protected]

gpg: data source: http://keys.openpgp.org:11371
(1) Flow Team (Flow Full Observer node snapshot verification master key) <
(1) Flow Team (Flow Full Light node snapshot verification master key) <
256 bit ECDSA key CB5264F7FD4CDD27, created: 2021-09-15
Keys 1-1 of 1 for "[email protected]". Enter number(s), N)ext, or Q)uit > 1
```
Expand All @@ -107,7 +107,7 @@ gpg --verify ./flow_observer/bootstrap/public-root-information/root-protocol-sta
gpg: assuming signed data in 'bootstrap/public-root-information/root-protocol-state-snapshot.json'
gpg: Signature made Wed Sep 15 11:34:33 2021 PDT
gpg: using ECDSA key 40CD95717AC463E61EE3B285B718CA310EDB542F
gpg: Good signature from "Flow Team (Flow Full Observer node snapshot verification master key) <[email protected]>" [unknown]
gpg: Good signature from "Flow Team (Flow Full Light node snapshot verification master key) <[email protected]>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7D23 8D1A E6D3 2A71 8ECD 8611 CB52 64F7 FD4C DD27
Expand All @@ -128,9 +128,9 @@ flow snapshot save ./flow_observer/bootstrap/public-root-information/root-protoc

#### Step 4 - Start the node

The observer node can be run as a docker container
The light node can be run as a docker container

##### Observer for Flow Mainnet
##### Light node for Flow Mainnet

```shell
docker run --rm \
Expand All @@ -154,7 +154,7 @@ docker run --rm \
--observer-networking-key-path=/bootstrap/network.key
```

##### Observer for Flow Testnet
##### Light node for Flow Testnet

```shell
docker run --rm \
Expand All @@ -178,15 +178,15 @@ docker run --rm \
--observer-networking-key-path=/bootstrap/network.key
```

The observer node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server.
The light node acts as a DHT client and bootstraps from upstream access nodes which run the DHT server.
The upstream bootstrap server is specified using the `bootstrap-node-addresses` which is the comma-separated list of hostnames of the access nodes.
The `bootstrap-node-public-keys` is the list of the corresponding networking public key of those nodes.

The observer node delegates many of the API calls to the upstream access nodes.
The `upstream-node-addresses` is the list of access node hostnames to which this observer node can delegate to. The list can be different from the bootstrap node list.
The light node delegates many of the API calls to the upstream access nodes.
The `upstream-node-addresses` is the list of access node hostnames to which this light node can delegate to. The list can be different from the bootstrap node list.
The `bootstrap-node-public-key` is the list of the corresponding networking public key of those nodes.

> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports an observer node can be used
> In the above docker commands, the Flow community access nodes are being used as the upstream access nodes. However, any other Flow access node that supports a light node can be used

_All parameters and their explanation can be found [here](https://github.com/onflow/flow-go/blob/82da35141ff095fbf75ce2c950efec240ad38565/cmd/access/node_builder/access_node_builder.go#L523-L558)_

Expand All @@ -201,37 +201,37 @@ flow blocks get latest --host localhost:9000

> REST API is currently not support (see [issue](https://github.com/onflow/flow-go/issues/3138))

The observer node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics)
The light node, like the other type of Flow nodes, also produces Prometheus metrics that can be used to monitor node health. More on that [here](./node-setup.mdx#monitoring-and-metrics)


## FAQs

### Does the observer node need to be staked?
### Does the light node need to be staked?

No, the observer node is not a staked node.
No, the light node is not a staked node.

### Can any access be used to bootstrap an observer node?
### Can any access be used to bootstrap a light node?

No, only Access nodes which have explicitly turned ON support for observer nodes can be used to bootstrap an observer node.
No, only Access nodes which have explicitly turned ON support for light nodes can be used to bootstrap a light node.

The public access nodes that support observer nodes are listed below. Apart from these, other public access nodes run by node operators other than DapperLabs may choose to support observer nodes.
The public access nodes that support light nodes are listed below. Apart from these, other public access nodes run by node operators other than DapperLabs may choose to support light nodes.

### How can an access node turn ON support for observer node?
### How can an access node turn ON support for light node?

An access node can support an observer node by passing in the following two parameters when starting the access node
An access node can support a light node by passing in the following two parameters when starting the access node
```shell
--supports-observer=true --public-network-address=0.0.0.0:3570
```
`public-network-address` is the address the observer nodes will connect to.
`public-network-address` is the address the light nodes will connect to.

### Are observer nodes subject to rate limits?
### Are light nodes subject to rate limits?

The observer serves all the [Block related queries](https://github.com/onflow/flow/blob/master/protobuf/flow/access/access.proto#L24-L42) from is local database. These are not subjected to any rate limits.

However, it proxies all the other requests to the access node and those will be rate limited as per the rate limits defined on that access node.


### Flow community access nodes that support connections from observer nodes
### Flow community access nodes that support connections from light nodes

#### For mainnet

Expand Down