diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..bfa5d73 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,35 @@ +# Architecture Layout + +This document explains the architecture of the library components. The low-level networking and threading infrastructure makes use of async [Tokio runtime](https://docs.rs/tokio). In the future, the code will be made modular so to allow the use of [different runtimes](https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html). + +The architecture follow the diagram below. For concrete usage examples please refer to the [crate's documentation](https://crates.io/crates/kadcast) + +![architecture](architecture.jpg) + +## Kadcast Peer +The `kadcast::peer` is the entry point of the library and the only public API. It implements the Kadcast Network Peer. + +## Message Handler +The `Message Handler` represents the core component of the library. It handles the incoming messages and replies according to the specification. + +## Mantainer +The `Maintainer` performs the [initial bootstrap](../bootstrapping) as well as the `k-table` [maintenance](../periodic-network-manteinance). +It keeps track of idle buckets and is in charge of triggering the routing nodes lookup. + +## Wire Network +It is responsible to instantiate the UDP server that receives the messages from the network. It's also in charge of outgoing communication (UDP client). +It performs message serialization/deserialization and applies the FEC encoding/decoding where appropriate. + +### RaptorQ Encoder +The `RaptorQ Encoder` splits a single broadcast message in multiple chunks according to the RaptorQ FEC algorithm. + +### RaptorQ Decoder +The `RaptorQ Encoder` joins the encoded chunks in a single decoded broadcast message. It keeps a cache to avoid processing brodcasted identical messages multiple times. + +## Channels +### Inbound Channel +Provides message passing between incoming UDP packets (deserialized by the WireNetwork) and Message Handler +### Outbound Channel +Channel dedicated to any message which needs to be transmitted to the network. +### Notification Channel +Successfully received broadcast messages are dispatched to the `Notifier` through the `Notification Channel`. This is done in order to safely forward messages to the user's callback without any risk for the latter to block any other thread. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..c9adf22 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,14 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [0.1.0] - 29-10-21 + +### Added + +- Kadcast Network protocol implementation +- RaptorQ as Forward Error Correction. +- Examples in `example` dir diff --git a/README.md b/README.md index f76552b..6bdd4ae 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,63 @@ -# kadcast -Dusk-blockchain version of Kadcast +# Kadcast +Implementation of the Kadcast Network layer according to [the latest version of the paper to date (2021)](https://eprint.iacr.org/2021/996.pdf) + +## Overlay Construction +Kadcast is an UDP-based peer-to-peer protocol in which peers form a structured overlay. + +> 💡 The specifications suggest that the protocol can be transport-agnostic, however this implementation uses the canonical choice of UDP. + +All peers have an unique ID generated upon joining the network, which is used as an identifier unique to them. + +> 💡 This implementation uses _128 bits identifiers_ generated from hashing the ``|`` concatenation truncated to the agreed 16 bytes. The hashing algorithm used is `Blake2`. + +Following the [Kademlia's specifications](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) for a structured network, the ID determines the peer position in a binary routing tree. + +Each peer mantains the routing state in a set of `k-buckets` containing the location of the location of each peer. The location is fully determined by the tuple `(ip_addr, port, ID)`. + +Each bucket comprises of a list of the `K` last recently seen peers, located at a given distance in relation to the peer. The distance is calculated as the `non-euclidean XOR metric` which represents: `d(x,y) = (x ^ y)`. + +This implies that `i` buckets contain `K` Peers which distance `d` follow the formula: `2^i <= d < 2^(i +1)`. + +> 💡 The factor `K` is a system-wide parameter which determines the routing state and the lookup complexity. + +- In case a bucket's peer list is full (i.e. `k` entries are already stored), new entries to that list may be inserted by dropping `LRU` (Least Recently Used) peers. + +- Before dropping an entry from the list, the peer will check whether the peer to be dropped is still reachable, by sending a `PING` message and waiting for a `PONG` response. In case the `PONG` won't be collected within pre-determined timeout, the check fails and the peer is dropped from the list. + +> 💡 By following a `LRU` policy, the protocol inherently is biased toward the older and more stable peers on the network. + +### Bootstrapping +Upon joining the Kadcast network, a peer retrieves from the so-called `bootstrap nodes` the locations of other neighbour peers, so to fill its buckets. + +> 💡 From the specification: _When a peer first joins the network, it has to know the address of at least one bootstrapping node. It therefore sends `PING` messages to known peers to check whether they are actually online. Additionally, `PING` transmits the sending peer’s routing information to the recipient, thereby distributing its existence in the network._ + +The bootstrap node will reply with a `PONG` message which also contains its node info. + +### Network discovery. +Starts just after the [bootstrapping](#bootstrapping) process. +The main goal of this phase is to fill up the `k-buckets` with `k` different peers. +- The process starts by sending a `FIND_NODE` message to the bootstraping nodes that have been added to the `k-bucket`s with the `PONG` messages received. +- Then the *lookup process* starts: + 1) The node looks up the 𝛼 closest peers regarding the XOR-metric in its own buckets. + 2) It queries this 𝛼 peers for the ID by sending `FIND_NODE` messages and wait for the `NODES` reply. + 3) The queried peers respond with a `NODES` reply containing the set of `k` peers they believe are the closest to `ID`. + 4) Based on the info acquired, the node builds a new set of closest peers and repeats steps *1* to *3* until it no longer gets peers closer to those it got from previous iterations. + +> ⚠️ Note that like the bucket size, `𝛼` and `k` are global constants that will determine the redundancy and overhead of the network. The typical values are: `k = [20, 100]` & `𝛼 = 3`. + +#### Periodic Network Manteinance +Each peer periodically refreshes every bucket with no activity. For each such bucket, it picks a random `ID` with appropriate distance, performs a look up, and populates its buckets with fresh routing information. + +### Improving Broadcast Reliability And Performance +Instead of delegating the broadcast to one peer per bucket, we select `β` delegates. This is done to increase the probability of at least one honest peer to receive the message broadcasted. This is a measure intended to help with the inherently lossy nature of the UDP protocol without forcing the use of slow CRC checking procedures. + +Since each broadcast process is repeated on every hop (decreasing the height), it is imperative that the peers ignore duplicate `CHUNK` messages. Moreover, transmission failures have to be considered. Because of these factors, this implementation includes the use of *forward error connection schemes* based on [RaptorQ](https://tools.ietf.org/pdf/rfc6330.pdf) specifications and [related rust library](https://github.com/cberner/raptorq). +The `FEC` overhead factor can be adjusted through the paramenter `f = (n-s)/s`. + +As shown in the benchmarks, we can get full network coverage even assuming a 12% packet-loss ratio with a `β = 3` & `f = 0,15` (Please refer to the benchmarks included with the library documentation). + +### Security +The specification provides solutions for DOS, Sybil and Eclipse attacks, as well as obstruction of block delivery. All advices have been taken into consideration during the development. + +## Internal Architecture +For more information related to the internal architecture please check [the architecture diagram](ARCHITECTURE.md). diff --git a/architecture.jpg b/architecture.jpg new file mode 100644 index 0000000..8209fd3 Binary files /dev/null and b/architecture.jpg differ diff --git a/examples/main.rs b/examples/main.rs index a34009b..2f04e0a 100644 --- a/examples/main.rs +++ b/examples/main.rs @@ -5,7 +5,7 @@ // Copyright (c) DUSK NETWORK. All rights reserved. use clap::{App, Arg}; -use kadcast::Server; +use kadcast::Peer; use rustc_tools_util::{get_version_info, VersionInfo}; use std::io::{self, BufRead}; @@ -75,7 +75,7 @@ pub async fn main() { tracing::subscriber::set_global_default(subscriber) .expect("Failed on subscribe tracing"); - let server = Server::builder( + let peer = Peer::builder( public_ip.to_string(), bootstrapping_nodes, crate::on_message, @@ -86,9 +86,9 @@ pub async fn main() { for message in stdin.lock().lines().flatten() { match &message[..] { "report" => { - server.report().await; + peer.report().await; } - v => server.broadcast(v.as_bytes()).await, + v => peer.broadcast(v.as_bytes()).await, } } } diff --git a/src/lib.rs b/src/lib.rs index 5cf982c..6716947 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -50,13 +50,13 @@ pub const BUCKET_DEFAULT_NODE_EVICT_AFTER_MILLIS: u64 = 5000; /// Default value after which a bucket is considered idle pub const BUCKET_DEFAULT_TTL_SECS: u64 = 60 * 60; -/// Struct representing the Kadcast Network -pub struct Server { +/// Struct representing the Kadcast Network Peer +pub struct Peer { outbound_sender: Sender, ktable: Arc>>, } -impl Server { +impl Peer { fn new( public_ip: String, bootstrapping_nodes: Vec, @@ -68,7 +68,7 @@ impl Server { let (notification_channel_tx, listener_channel_rx) = mpsc::channel(32); let table = Arc::new(RwLock::new(tree)); - let server = Server { + let peer = Peer { outbound_sender: outbound_channel_tx.clone(), ktable: table.clone(), }; @@ -88,8 +88,8 @@ impl Server { table, outbound_channel_tx, )); - task::spawn(Server::notifier(listener_channel_rx, on_message)); - server + task::spawn(Peer::notifier(listener_channel_rx, on_message)); + peer } async fn notifier( @@ -136,9 +136,9 @@ impl Server { } } - /// Instantiate a [ServerBuilder]. + /// Instantiate a [PeerBuilder]. /// - /// * `public_ip` - public `SocketAddress` of the `Server`. No domain name + /// * `public_ip` - public `SocketAddress` of the [Peer]. No domain name /// allowed /// * `bootstrapping_nodes` - List of known bootstrapping kadcast nodes. It /// accepts the same representation of `public_ip` but with domain names @@ -149,13 +149,13 @@ impl Server { public_ip: String, bootstrapping_nodes: Vec, on_message: fn(Vec), - ) -> ServerBuilder { - ServerBuilder::new(public_ip, bootstrapping_nodes, on_message) + ) -> PeerBuilder { + PeerBuilder::new(public_ip, bootstrapping_nodes, on_message) } } -/// ServerBuilder instantiates a Kadcast [Server]. -pub struct ServerBuilder { +/// PeerBuilder instantiates a Kadcast [Peer]. +pub struct PeerBuilder { node_ttl: Duration, node_evict_after: Duration, bucket_ttl: Duration, @@ -164,12 +164,12 @@ pub struct ServerBuilder { on_message: fn(Vec), } -impl ServerBuilder { +impl PeerBuilder { /// Sets the maximum duration for a node to be considered alive (no eviction /// will be requested). /// /// Default value [BUCKET_DEFAULT_NODE_TTL_MILLIS] - pub fn with_node_ttl(mut self, node_ttl: Duration) -> ServerBuilder { + pub fn with_node_ttl(mut self, node_ttl: Duration) -> PeerBuilder { self.node_ttl = node_ttl; self } @@ -177,7 +177,7 @@ impl ServerBuilder { /// Set duration after which a bucket is considered idle /// /// Default value [BUCKET_DEFAULT_TTL_SECS] - pub fn with_bucket_ttl(mut self, bucket_ttl: Duration) -> ServerBuilder { + pub fn with_bucket_ttl(mut self, bucket_ttl: Duration) -> PeerBuilder { self.bucket_ttl = bucket_ttl; self } @@ -188,7 +188,7 @@ impl ServerBuilder { pub fn with_node_evict_after( mut self, node_evict_after: Duration, - ) -> ServerBuilder { + ) -> PeerBuilder { self.node_evict_after = node_evict_after; self } @@ -197,8 +197,8 @@ impl ServerBuilder { public_ip: String, bootstrapping_nodes: Vec, on_message: fn(Vec), - ) -> ServerBuilder { - ServerBuilder { + ) -> PeerBuilder { + PeerBuilder { public_ip, bootstrapping_nodes, on_message, @@ -211,15 +211,15 @@ impl ServerBuilder { } } - /// Builds the [Server] - pub fn build(self) -> Server { + /// Builds the [Peer] + pub fn build(self) -> Peer { let tree = TreeBuilder::new(PeerNode::from_address(&self.public_ip[..])) .with_node_evict_after(self.node_evict_after) .with_node_ttl(self.node_ttl) .with_bucket_ttl(self.bucket_ttl) .build(); - Server::new( + Peer::new( self.public_ip, self.bootstrapping_nodes, self.on_message,