From 9a23aa635cfc1923655d48aab2591858da0808c5 Mon Sep 17 00:00:00 2001 From: Alexandra Tran Date: Fri, 15 Nov 2024 17:39:26 -0800 Subject: [PATCH] Edit content Signed-off-by: Alexandra Tran --- .../images/aws-node-runners-1.png} | Bin .../images/aws-node-runners-2.png} | Bin .../how-to/bonsai-limit-trie-logs.md | 2 +- .../how-to/deploy-aws-node-runners.md | 424 ++++++++++++++++++ docs/public-networks/how-to/deploy-besu-teku | 273 ----------- .../how-to/develop/_category_.json | 2 +- .../how-to/troubleshoot/_category_.json | 2 +- docs/public-networks/how-to/upgrade-node.md | 2 +- 8 files changed, 428 insertions(+), 277 deletions(-) rename docs/{public-networks/how-to/4F4D60BC-25D0-4BB1-AF24-DCC4BE139F15.png => assets/images/aws-node-runners-1.png} (100%) rename docs/{public-networks/how-to/89571298-F580-487E-BAEF-1CAC3B44545B.png => assets/images/aws-node-runners-2.png} (100%) create mode 100644 docs/public-networks/how-to/deploy-aws-node-runners.md delete mode 100644 docs/public-networks/how-to/deploy-besu-teku diff --git a/docs/public-networks/how-to/4F4D60BC-25D0-4BB1-AF24-DCC4BE139F15.png b/docs/assets/images/aws-node-runners-1.png similarity index 100% rename from docs/public-networks/how-to/4F4D60BC-25D0-4BB1-AF24-DCC4BE139F15.png rename to docs/assets/images/aws-node-runners-1.png diff --git a/docs/public-networks/how-to/89571298-F580-487E-BAEF-1CAC3B44545B.png b/docs/assets/images/aws-node-runners-2.png similarity index 100% rename from docs/public-networks/how-to/89571298-F580-487E-BAEF-1CAC3B44545B.png rename to docs/assets/images/aws-node-runners-2.png diff --git a/docs/public-networks/how-to/bonsai-limit-trie-logs.md b/docs/public-networks/how-to/bonsai-limit-trie-logs.md index eb73f85b36b..f3a10e33cfe 100644 --- a/docs/public-networks/how-to/bonsai-limit-trie-logs.md +++ b/docs/public-networks/how-to/bonsai-limit-trie-logs.md @@ -1,6 +1,6 @@ --- title: Reduce storage for Bonsai Tries -sidebar_position: 12 +sidebar_position: 9 description: Reduce the size of your database when using Bonsai Tries tags: - public networks diff --git a/docs/public-networks/how-to/deploy-aws-node-runners.md b/docs/public-networks/how-to/deploy-aws-node-runners.md new file mode 100644 index 00000000000..7ceaa1c1cfd --- /dev/null +++ b/docs/public-networks/how-to/deploy-aws-node-runners.md @@ -0,0 +1,424 @@ +--- +sidebar_position: 11 +description: Configure Ethereum nodes using AWS Blockchain Node Runners. +toc_max_heading_level: 3 +tags: + - Public networks +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Deploy AWS Node Runners + +[AWS Blockchain Node Runners](https://aws-samples.github.io/aws-blockchain-node-runners/docs/intro) +is an open-source initiative aimed at simplifying the deployment of self-managed blockchain nodes +on AWS using vetted deployment blueprints and infrastructure configurations. +AWS Node Runners solves common challenges in architecting and deploying blockchain nodes on AWS, +helping users identify optimal configurations for specific protocol clients. + +This page walks you through the AWS Node Runners [architecture](#architecture), and how to +[deploy Besu and Teku on AWS](#deploy-aws-node-runners). + +## Architecture + +AWS Blockchain Node Runners supports several Ethereum client combinations and offers two +configuration options: a single node setup for development environments, and a highly available +multi-node setup for production environments. +The following diagrams illustrate the high level architecture of these setups. + +### Single RPC node setup + +
+

+ +![Architecture-PoC](../../assets/images/aws-node-runners-1.png) + +

+
+ +This single node setup is for small-scale development environments. +It deploys a single EC2 instance with both consensus and execution clients. +The RPC port is exposed only to internal IP range of the VPC, while P2P ports allow external access to keep the clients synced. + +### Highly available setup + +
+

+ +![Architecture](../../assets/images/aws-node-runners-2.png) + +

+
+ +In this highly available, multiple node setup: + +1. The sync node synchronizes data continuously with the Ethereum network. +1. The sync node copies node state data to an Amazon S3 bucket. +1. New RPC nodes copy state data from the Amazon S3 bucket to accelerate their initial sync. +1. The Application Load Balancer routes application and smart contract development tool requests to available RPC nodes. + +### Architecture checklist + +The following is a checklist for an implementation of the AWS Blockchain Node Runners. +This checklist takes into account questions from the [AWS Well-Architected framework](https://aws.amazon.com/architecture/well-architected/) +that are relevant to this workload. +You can add more checks from the framework if required for your workload. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PillarControlQuestion/CheckNotes
SecurityNetwork protectionAre there unnecessary open ports in security groups?The Erigon snap sync port (`42069`) remains open for non-Erigon clients.
Traffic inspectionAWS WAF can be implemented for traffic inspection. Additional charges will apply.
Compute protectionReduce attack surfaceThis solution uses Amazon Linux 2 AMI. You can run hardening scripts on it.
Enable users to perform actions at a distanceThis solution uses AWS Systems Manager for terminal sessions, not SSH ports.
Data protection at restUse encrypted Amazon Elastic Block Store (Amazon EBS) volumesThis solution uses encrypted Amazon EBS volumes.
Use encrypted Amazon Simple Storage Service (Amazon S3) bucketsThis solution uses Amazon S3 managed keys (SSE-S3) encryption.
Data protection in transitUse TLSThe AWS Application Load Balancer currently uses an HTTP listener. To use TLS, create an HTTPS listener with a self-signed certificate.
Authorization and access controlUse instance profile with Amazon Elastic Compute Cloud (Amazon EC2) instancesThis solution uses AWS Identity and Access Management (AWS IAM) role instead of IAM user.
Follow the principle of least privilege accessIn the sync node, the root user is not used (it uses the special user `ethereum` instead).
Application securitySecurity-focused development practicescdk-nag is used with appropriate suppressions.
Cost optimizationUse cost-effective resourcesAWS Graviton-based Amazon EC2 instances are used, which are cost-effective compared to Intel/AMD instances.
Estimate costsOne sync node with m7g.2xlarge for geth-Lighthouse configuration (2048 GB SSD) will cost around $430 per month in the US East region. Additional charges apply if you deploy RPC nodes with a load balancer.
ReliabilityWithstand component failuresThis solution uses AWS Application Load Balancer with RPC nodes for high availability. If the sync node fails, Amazon S3 backup can be used to reinstate the nodes.
How is data backed up?Data is backed up to Amazon S3 using the s5cmd tool.
How are workload resources monitored?Resources are monitored using Amazon CloudWatch dashboards. Amazon CloudWatch custom metrics are pushed via CloudWatch Agent.
Performance efficiencyHow is the compute solution selected?The solution is selected based on best price-performance, that is, AWS Graviton-based Amazon EC2 instances.
How is the storage solution selected?The solution is selected based on best price-performance, that is, gp3 Amazon EBS volumes with optimal IOPS and throughput.
How is the architecture selected?The s5cmd tool is used for Amazon S3 uploads/downloads because it gives better price-performance compared to Amazon EBS snapshots.
Operational excellenceHow is health of the workload determined?Workload health is determined via AWS Application Load Balancer Target Group Health Checks, on port `8545`.
SustainabilitySelect the most efficient hardware for your workloadThis solution uses AWS Graviton-based Amazon EC2 instances, which offer the best performance per watt of energy use in Amazon EC2.
+ +## Deploy Besu and Teku on AWS + +:::note +In this guide, you'll set all major configuration through environment variables, but you can also +modify parameters in the `config/config.ts` file. +::: + +### 1. Configure the AWS CloudShell + +#### 1.1. Log into AWS + +Log in to your [AWS account](https://aws.amazon.com/) with permissions to create and modify +resources in IAM, EC2, EBS, VPC, S3, KMS, and Secrets Manager. +From the AWS Management Console, open the [AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html), +a web-based shell environment. +For more information, see [this demo](https://youtu.be/fz4rbjRaiQM) on +[CloudShell with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html), +which you'll use to test APIs from an internal IP address space. + +#### 1.2. Install dependencies + +To deploy and test blueprints in the CloudShell, clone the following repository and install dependencies: + +```bash +git clone https://github.com/aws-samples/aws-blockchain-node-runners.git +cd aws-blockchain-node-runners +npm install +``` + +### 2. Prepare to deploy nodes + +In the root directory of your project: + +1. If you have deleted or don't have the default VPC, create a default VPC: + + ```bash + aws ec2 create-default-vpc + ``` + + :::note + You might see the following error if the default VPC already exists: + + ```bash + An error occurred (DefaultVpcAlreadyExists) when calling the CreateDefaultVpc operation: A Default VPC already exists for this account in this region. + ``` + + This means that he default VPC must have at least two public subnets in different Availability + Zones, and public subnet must set `Auto-assign public IPv4 address` to `YES`. + ::: + +1. Configure your Node Runners Ethereum blueprint deployment. + To specify the Ethereum client combination you want to deploy, create your own copy of the `.env` + file and edit it using your preferred text editor. + The following example uses a sample configuration from the repository for a Besu and Teku node deployment: + + ```bash + # Make sure you are in aws-blockchain-node-runners/lib/ethereum + cd lib/ethereum + pwd + cp ./sample-configs/.env-besu-teku .env + nano .env + ``` + + :::note + You can find more examples for other Ethereum client combinations in the `sample-configs` directory. + ::: + +1. Deploy common components, such as IAM role and Amazon S3 bucket to store data snapshots: + + ```bash + pwd + # Make sure you are in aws-blockchain-node-runners/lib/ethereum + npx cdk deploy eth-common + ``` + +### 3. Deploy nodes + +#### 3.1. (Option 1) Single RPC node + +In a single RPC node setup: + +1. Deploy the node: + + ```bash + pwd + # Make sure you are in aws-blockchain-node-runners/lib/ethereum + npx cdk deploy eth-single-node --json --outputs-file single-node-deploy.json + ``` + + :::note + The default VPC must have at least two public subnets in different Availability Zones, and the + public subnets must set `Auto-assign public IPv4 address` to `YES`. + ::: + +1. After starting the node, wait for the initial synchronization process to finish. + It can take from half a day to approximately 6-10 days, depending on the client combination and + the network state. + You can use Amazon CloudWatch to track the progress, which publishes metrics every five minutes. + Watch `sync distance` for the consensus client, and `blocks behind` for the execution client. + When the node is fully synced, those two metrics should be `0`. + To see them: + + - Navigate to [CloudWatch service](https://console.aws.amazon.com/cloudwatch/) (make sure you are + in the region you have specified for `AWS_REGION`). + - Open `Dashboards` and select `eth-sync-node-` from the list of dashboards. + +1. Once the initial synchronization is done, you can access the RPC API of that node from within the + same VPC. + The RPC port is not exposed to the Internet. + Turn the following query against the private IP of the single RPC node you deployed: + + ```bash + INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') + NODE_INTERNAL_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[*].Instances[*].PrivateIpAddress' --output text) + echo "NODE_INTERNAL_IP=$NODE_INTERNAL_IP" + ``` + + Copy the output from the last `echo` command with `NODE_INTERNAL_IP=` and open + [CloudShell tab with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html) + to access the internal IP address space. + Paste `NODE_INTERNAL_IP=` into the new CloudShell tab. + Then, query the API: + + ``` bash + # IMPORTANT: Run from CloudShell VPC environment tab + # This queries the token balance of a Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa + curl http://$NODE_INTERNAL_IP:8545 -X POST -H "Content-Type: application/json" \ + --data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}' + ``` + + The result should look like the following (the actual balance might change): + + ```javascript + {"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"} + ``` + +#### 3.2. (Option 2) Highly available RPC nodes + +In a highly available multi-node setup: + +1. Deploy the sync node: + + ```bash + pwd + # Make sure you are in aws-blockchain-node-runners/lib/ethereum + npx cdk deploy eth-sync-node --json --outputs-file sync-node-deploy.json + ``` + + :::note + The default VPC must have at least two public subnets in different Availability Zones, and the + public subnets must set `Auto-assign public IPv4 address` to `YES`. + ::: + +1. After starting the node, wait for the initial synchronization process to finish. + It can take from half a day to approximately 6-10 days, depending on the client combination and + the network state. + You can use Amazon CloudWatch to track the progress, which publishes metrics every five minutes. + Watch `sync distance` for the consensus client, and `blocks behind` for the execution client. + When the node is fully synced, those two metrics should be `0`. + To see them: + + - Navigate to [CloudWatch service](https://console.aws.amazon.com/cloudwatch/) (make sure you are + in the region you have specified for `AWS_REGION`). + - Open `Dashboards` and select `eth-sync-node-` from the list of dashboards. + + Once the synchronization process is over, the script automatically stops both clients and copies + all the contents of the `/data` directory to your snapshot S3 bucket. + That can take from 30 minutes to approximately 2 hours. + During the process, you will see lower CPU and RAM usage, but high data disc throughput and + outbound network traffic. + The script automatically starts the clients after the process is done. + + :::note + The snapshot backup process automatically runs every day at midnight of the time zone were the + sync node runs. + To change the schedule, modify `crontab` of the root user on the node's EC2 instance. + ::: + +1. Configure and deploy two RPC nodes: + + ```bash + pwd + # Make sure you are in aws-blockchain-node-runners/lib/ethereum + npx cdk deploy eth-rpc-nodes --json --outputs-file rpc-node-deploy.json + ``` + +1. Give the new RPC nodes approximately 30 minutes to initialize, then run the following query + against the load balancer behind the RPC node created: + + ```bash + export ETH_RPC_ABL_URL=$(cat rpc-node-deploy.json | jq -r '..|.alburl? | select(. != null)') + echo ETH_RPC_ABL_URL=$ETH_RPC_ABL_URL + ``` + + ```bash + # IMPORTANT: Run from CloudShell VPC environment tab + # We query token balance of Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa + curl http://$ETH_RPC_ABL_URL:8545 -X POST -H "Content-Type: application/json" \ + --data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}' + ``` + + The result should look like the following (the actual balance might change): + + ```javascript + {"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"} + ``` + + If the nodes are still starting and catching up with the chain, you will see the following response: + + ```HTML + + 503 Service Temporarily Unavailable + +

503 Service Temporarily Unavailable

+ + ``` + + :::note + By default and for security reasons, the load balancer is available only from within the default + VPC in the region where it is deployed. + It is not available from the Internet and is not open for external connections. + Before opening it up, protect your RPC APIs. + ::: + +### 4. Clear and undeploy nodes + +To clear and undeploy the RPC nodes, sync nodes, and common components, use the following commands: + +```bash +# Set the AWS account ID and region in case the local .env file is lost. +export AWS_ACCOUNT_ID= +export AWS_REGION= + +pwd +# Make sure you are in aws-blockchain-node-runners/lib/ethereum. + +# Destroy the single RPC node. +cdk destroy eth-single-node + +# Destroy multiple RPC nodes. +cdk destroy eth-rpc-nodes + +# Destroy the sync node. +cdk destroy eth-sync-node + +# You need to manually delete an s3 bucket with a name similar to 'eth-snapshots-$accountid-eth-nodes-common' +# on the console: +# 1. Empty the bucket +# 2. Delete the bucket +# 3. Execute and delete all common components like IAM role and Security Group +cdk destroy eth-common +``` diff --git a/docs/public-networks/how-to/deploy-besu-teku b/docs/public-networks/how-to/deploy-besu-teku deleted file mode 100644 index ef4e2c90e18..00000000000 --- a/docs/public-networks/how-to/deploy-besu-teku +++ /dev/null @@ -1,273 +0,0 @@ ---- -title: AWS Node Runners -description: Configure Ethereum nodes with the execution/consensus layer client combination using AWS blockchain node runners. ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -# AWS Ethereum node deployment - -[AWS Blockchain Node Runners (Node Runners)](https://aws-samples.github.io/aws-blockchain-node-runners/docs/intro) are an open-source initiative aimed at simplifying the deployment of self-managed blockchain nodes on AWS using vetted deployment blueprints and infrastructure configurations. -AWS Blockchain Node Runners solves common challenges in architecting and deploying blockchain nodes on AWS. -It helps users identify optimal configurations for specific protocol clients - -This supports several Ethereum client combinations and offers two configuration options: - -- [Single JSON-RPC node setup for development and testing](#single-rpc-node-setup) -- [Multiple nodes in a highly-available setup for production environments](#highly-available-setup) - -## Architecture - -### Single RPC node setup -![Architecture-PoC](../docs/public-networks/how-to/89571298-F580-487E-BAEF-1CAC3B44545B.png) - -This setup is for small scale proof of concept (PoC) or development environments. -It deploys a single EC2 instance with both consensus and execution clients. -The RPC port is exposed only to internal IP range of the VPC, while P2P ports allow external access to keep the clients synced. - -### Highly available setup -![Architecture](../docs/public-networks/how-to/4F4D60BC-25D0-4BB1-AF24-DCC4BE139F15.png) - -1. The sync node synchronizes data continuously with the Ethereum network. -1. The sync node copies node state data to an Amazon S3 bucket. -1. New RPC nodes copy state data from the Amazon S3 bucket to accelerate their initial sync. -1. The Application Load Balancer routes application and smart contract development tool requests to available RPC nodes. - -### Architecture checklist - -The following is a checklist for Ethereum nodes implementation of the AWS Blockchain Node Runner. -This checklist takes into account questions from the [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) which are relevant to this workload. -You can add more checks from the framework if required for your workload. - -| Pillar | Control | Question/Check | Remarks | -|:------------------------|:----------------------------------|:---------------------------------------------------------------------------------|:-----------------| -| Security | Network protection | Are there unnecessary open ports in security groups? | Please note that Erigon snap sync port remains open for non-erigon clients, i.e. Port 42069 (TCP/UDP). | -| | | Traffic inspection | AWS WAF could be implemented for traffic inspection. Additional charges will apply. | -| | Compute protection | Reduce attack surface | This solution uses Amazon Linux 2 AMI. You may choose to run hardening scripts on it. | -| | | Enable people to perform actions at a distance | This solution uses AWS Systems Manager for terminal session, not ssh ports. | -| | Data protection at rest | Use encrypted Amazon Elastic Block Store (Amazon EBS) volumes | This solution uses encrypted Amazon EBS volumes. | -| | | Use encrypted Amazon Simple Storage Service (Amazon S3) buckets | This solution uses Amazon S3 managed keys (SSE-S3) encryption. | -| | Data protection in transit | Use TLS | The AWS Application Load balancer currently uses HTTP listener. Create HTTPS listener with self signed certificate if TLS is desired. | -| | Authorization and access control | Use instance profile with Amazon Elastic Compute Cloud (Amazon EC2) instances | This solution uses AWS Identity and Access Management (AWS IAM) role instead of IAM user. | -| | | Following principle of least privilege access | In sync node, root user is not used (using special user "ethereum" instead"). | -| | Application security | Security focused development practices | cdk-nag is being used with appropriate suppressions. | -| Cost optimization | Service selection | Use cost effective resources | AWS Graviton-based Amazon EC2 instances are being used, which are cost effective compared to Intel/AMD instances. | -| | Cost awareness | Estimate costs | One sync node with m7g.2xlarge for geth-lighthouse configuration (2048GB ssd) will cost around US$430 per month in the US East (N. Virginia) region. Additional charges will apply if you choose to deploy RPC nodes with load balancer. | -| Reliability | Resiliency implementation | Withstand component failures | This solution uses AWS Application Load Balancer with RPC nodes for high availability. If sync node fails, Amazon S3 backup can be used to reinstate the nodes. | -| | Data backup | How is data backed up? | Data is backed up to Amazon S3 using [s5cmd](https://github.com/peak/s5cmd) tool. | -| | Resource monitoring | How are workload resources monitored? | Resources are being monitored using Amazon CloudWatch dashboards. Amazon CloudWatch custom metrics are being pushed via CloudWatch Agent. | -| Performance efficiency | Compute selection | How is compute solution selected? | Compute solution is selected based on best price-performance, i.e. AWS Graviton-based Amazon EC2 instances. | -| | Storage selection | How is storage solution selected? | Storage solution is selected based on best price-performance, i.e. gp3 Amazon EBS volumes with optimal IOPS and throughput. | -| | Architecture selection | How is the best performance architecture selected? | s5cmd tool has been chosen for Amazon S3 uploads/downloads because it gives better price-performance compared to Amazon EBS snapshots (including Fast Snapshot Restore, which can be expensive). | -| Operational excellence | Workload health | How is health of workload determined? | Health of workload is determined via AWS Application Load Balancer Target Group Health Checks, on port 8545. | -| Sustainability | Hardware & services | Select most efficient hardware for your workload | This solution uses AWS Graviton-based Amazon EC2 instances which offer the best performance per watt of energy use in Amazon EC2. | - - -## Deploy and manage Besu and Teku Ethereum nodes on AWS - -### 1. Configure the AWS CloudShell - -::: note - -In this tutorial you'll set all major configuration through environment variables, but you can also modify parameters in `config/config.ts` file. - -::: - -#### 1.1 Log into AWS - -Log in to your AWS account with permissions to create and modify resources in IAM, EC2, EBS, VPC, S3, KMS, and Secrets Manager. -From the AWS Management Console, open the [AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html), a web-based shell environment. \ -For more information, see the [overview](https://youtu.be/fz4rbjRaiQM) on [CloudShell with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html), which you'll use to test nodes API from internal IP address space. - -#### 1.2 Install and configure dependencies - -To deploy and test blueprints in the CloudShell, clone the following repository and install dependencies: - -```bash -git clone https://github.com/aws-samples/aws-blockchain-node-runners.git -cd aws-blockchain-node-runners -npm install -``` - -### 2. Prepare to deploy nodes - -1. Ensure you are in the root directory of the cloned repository. - -2. If you have deleted or don't have the default VPC, create default VPC: - - ```bash - aws ec2 create-default-vpc - ``` - - ::: note - - You might see the following error if the default VPC already exists: - - ```bash - An error occurred (DefaultVpcAlreadyExists) when calling the CreateDefaultVpc operation: A Default VPC already exists for this account in this region. - ``` - This means that he default VPC must have at least two public subnets in different Availability Zones, and public subnet must set `Auto-assign public IPv4 address` to `YES`. - - You can continue with the following steps. - - ::: - -3. Configure your Node Runners Ethereum blueprint deployment. -To specify the Ethereum client combination you want to deploy, create your own copy of `.env` file and edit it using your preferred text editor. -The contents of your file for a Besu or Teku node deployment are the following, which uses a sample config from the repository: - - ```bash - # Make sure you are in aws-blockchain-node-runners/lib/ethereum - cd lib/ethereum - pwd - cp ./sample-configs/.env-besu-teku .env - nano .env - ``` - - :::note - - You can find more examples inside the `sample-configs` directory, which illustrates other Ethereum client combinations. - - ::: - - -4. Deploy common components such as IAM role, and Amazon S3 bucket to store data snapshots - - ```bash - pwd - # Ensure you are in aws-blockchain-node-runners/lib/ethereum - npx cdk deploy eth-common - ``` - -#### Single RPC Node - -1. Deploy Single RPC Node - -```bash -pwd -# Make sure you are in aws-blockchain-node-runners/lib/ethereum -npx cdk deploy eth-single-node --json --outputs-file single-node-deploy.json -``` - -:::note - -The default VPC must have at least two public subnets in different Availability Zones, and public subnet must set `Auto-assign public IPv4 address` to `YES` - -::: - -2. After starting the node you need to wait for the inital syncronization process to finish. It may take from half a day to about 6-10 days depending on the client combination and the state of the network. You can use Amazon CloudWatch to track the progress. There is a script that publishes CloudWatch metrics every 5 minutes, where you can watch `sync distance` for consensus client and `blocks behind` for execution client. When the node is fully synced those two metrics shold show 0. To see them: - - - Navigate to [CloudWatch service](https://console.aws.amazon.com/cloudwatch/) (make sure you are in the region you have specified for `AWS_REGION`) - - Open `Dashboards` and select `eth-sync-node-` from the list of dashboards. - -4. Once the initial synchronization is done, you should be able to access the RPC API of that node from within the same VPC. The RPC port is not exposed to the Internet. Turn the following query against the private IP of the single RPC node you deployed: - -```bash -INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') -NODE_INTERNAL_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[*].Instances[*].PrivateIpAddress' --output text) -echo "NODE_INTERNAL_IP=$NODE_INTERNAL_IP" -``` - -Copy output from the last `echo` command with `NODE_INTERNAL_IP=` and open [CloudShell tab with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html) to access internal IP address space. Paste `NODE_INTERNAL_IP=` into the new CloudShell tab. Then query the API: - -``` bash -# IMPORTANT: Run from CloudShell VPC environment tab -# We query token balance of Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa -curl http://$NODE_INTERNAL_IP:8545 -X POST -H "Content-Type: application/json" \ ---data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}' -``` - -The result should be like this (the actual balance might change): - -```javascript -{"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"} -``` - -### Option 2: Highly Available RPC Nodes - -1. Deploy Sync Node - -```bash -pwd -# Make sure you are in aws-blockchain-node-runners/lib/ethereum -npx cdk deploy eth-sync-node --json --outputs-file sync-node-deploy.json -``` -**NOTE:** *The default VPC must have at least two public subnets in different Availability Zones, and public subnet must set `Auto-assign public IPv4 address` to `YES`* - -2. After starting the node you need to wait for the inital syncronization process to finish. It may take from half a day to about 6-10 days depending on the client combination and the state of the network. You can use Amazon CloudWatch to track the progress. There is a script that publishes CloudWatch metrics every 5 minutes, where you can watch `sync distance` for consensus client and `blocks behind` for execution client. When the node is fully synced those two metrics shold show 0. To see them: - - - Navigate to [CloudWatch service](https://console.aws.amazon.com/cloudwatch/) (make sure you are in the region you have specified for `AWS_REGION`) - - Open `Dashboards` and select `eth-sync-node-` from the list of dashboards. - -Once synchronization process is over, the script will automatically stop both clients and copy all the contents of the `/data` directory to your snapshot S3 bucket. That may take from 30 minutes to about 2 hours. During the process on the dashboard you will see lower CPU and RAM utilization but high data disc throughput and outbound network traffic. The script will automatically start the clients after the process is done. - -> **Note:** *The snapshot backup process will automatically run every day at midnight of the time zone were the sync node runs. To change the schedule, modify `crontab` of the root user on the node's EC2 instance.* - -3. Configure and deploy 2 RPC Nodes - -```bash -pwd -# Make sure you are in aws-blockchain-node-runners/lib/ethereum -npx cdk deploy eth-rpc-nodes --json --outputs-file rpc-node-deploy.json -``` - -4. Give the new RPC nodes about 30 minutes (up to 2 hours for Erigon(???)) to initialize and then run the following query against the load balancer behind the RPC node created - -```bash -export ETH_RPC_ABL_URL=$(cat rpc-node-deploy.json | jq -r '..|.alburl? | select(. != null)') -echo ETH_RPC_ABL_URL=$ETH_RPC_ABL_URL -``` - -```bash -# IMPORTANT: Run from CloudShell VPC environment tab -# We query token balance of Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa -curl http://$ETH_RPC_ABL_URL:8545 -X POST -H "Content-Type: application/json" \ - --data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}' -``` - -The result should be like this (the actual balance might change): - -```javascript -{"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"} -``` - - If the nodes are still starting and catching up with the chain, you will see the following repsonse: - -```HTML - - 503 Service Temporarily Unavailable - -

503 Service Temporarily Unavailable

- -``` - -> **NOTE:** By default and for security reasons the load balancer is available only from within the default VPC in the region where it is deployed. It is not available from the Internet and is not open for external connections. Before opening it up please make sure you protect your RPC APIs. - -### Clearing up and undeploying everything - -Destroy RPC Nodes, Sync Nodes and Comon components - -```bash -# Setting the AWS account id and region in case local .env file is lost - export AWS_ACCOUNT_ID= - export AWS_REGION= - -pwd -# Make sure you are in aws-blockchain-node-runners/lib/ethereum - -# Destroy Single RPC Node -cdk destroy eth-single-node - -# Destroy RPC Nodes -cdk destroy eth-rpc-nodes - -# Destroy Sync Node -cdk destroy eth-sync-node - -# You need to manually delete an s3 bucket with a name similar to 'eth-snapshots-$accountid-eth-nodes-common' on the console,firstly empty the bucket,secondly delete the bucket,and then execute -# Delete all common components like IAM role and Security Group -cdk destroy eth-common -``` - -docs/public-networks/how-to/4F4D60BC-25D0-4BB1-AF24-DCC4BE139F15.png diff --git a/docs/public-networks/how-to/develop/_category_.json b/docs/public-networks/how-to/develop/_category_.json index 8c8a280b930..2d22b35f5bc 100644 --- a/docs/public-networks/how-to/develop/_category_.json +++ b/docs/public-networks/how-to/develop/_category_.json @@ -1,4 +1,4 @@ { "label": "Develop dapps", - "position": 9 + "position": 11 } diff --git a/docs/public-networks/how-to/troubleshoot/_category_.json b/docs/public-networks/how-to/troubleshoot/_category_.json index cf5f9653813..e5ec35957e9 100644 --- a/docs/public-networks/how-to/troubleshoot/_category_.json +++ b/docs/public-networks/how-to/troubleshoot/_category_.json @@ -1,4 +1,4 @@ { "label": "Troubleshoot", - "position": 12 + "position": 13 } diff --git a/docs/public-networks/how-to/upgrade-node.md b/docs/public-networks/how-to/upgrade-node.md index 4e33d3dce08..fb092180334 100644 --- a/docs/public-networks/how-to/upgrade-node.md +++ b/docs/public-networks/how-to/upgrade-node.md @@ -1,6 +1,6 @@ --- title: Upgrade Besu -sidebar_position: 11 +sidebar_position: 12 description: Upgrade your Besu node to a new version. tags: - public networks