Protocol Design 🚀 #262
Merged
Protocol Design 🚀 #262
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…es into feat/protocol-overview
| The Ethereum protocol evolves and changes over time but it always follow certain principles. These principles reflect values of the whole community and are reflected in some of the main design decisions of Ethereum. | ||
|
|
||
| ### Sandwich complexity model |
There was a problem hiding this comment.
I would change this to 'Managing complexity' and explain both sandwich model and encapsulated complexity
There was a problem hiding this comment.
Maybe also add another section part about modularity
There was a problem hiding this comment.
understood based Vitalik's blog. I see the wiki needs to around the current state of the protocol with no to very little background
|
|
||
| ### Accounts over UTXOs | ||
|
|
||
| ### Merkle patricia trie |
There was a problem hiding this comment.
The MPT and RLP part should be simple, just high level explaining the decision. Both are being replaced, by verkles and ssz, so that could be mentioned as well
There was a problem hiding this comment.
will keep the wiki as close to the current state of the protocol design 🚀
|
|
||
| ### Trie trinity | ||
|
|
||
| ### Uncle Incentivization |
There was a problem hiding this comment.
Uncles and difficulty are now obsolete, maybe skip it all together. But on the other hand, you can add a rationale for Gasper PoS
|
|
||
| ### Difficulty Update Algorithm | ||
|
|
||
| ### Gas and Fee |
There was a problem hiding this comment.
Part on gas and evm is quite elaborate in the original doc, maybe it could be part of EVM docs or its own page. I would first contribute what's missing in the EVM page and avoid it here
|
|
||
| ### Merkle patricia trie | ||
|
|
||
| ### RLP () |
There was a problem hiding this comment.
Similarly with compression and trie trinity, maybe it could be part of p2p docs #255
shyam-patel-kira
requested changes
May 19, 2024
|
|
||
| Further implementation and details about Simple Serialize can be found [**here**](#) | ||
|
|
||
| ### **Tryst with Finality** |
There was a problem hiding this comment.
Did you mean trust here?
| Further implementation and details about Simple Serialize can be found [**here**](#) | ||
|
|
||
| ### **Tryst with Finality** | ||
| In Ethereum's proof-of-stake based consensus mechanisms, finality refers to the guarantee that a block cannot be altered or removed from the blockchain without burning at least 33% of the total stacked ETH. The underlying algorithm to achieve this is called **Gasper** |
There was a problem hiding this comment.
Suggested change
| In Ethereum's proof-of-stake based consensus mechanisms, finality refers to the guarantee that a block cannot be altered or removed from the blockchain without burning at least 33% of the total stacked ETH. The underlying algorithm to achieve this is called **Gasper** | |
| In Ethereum's proof-of-stake based consensus mechanisms, finality refers to the guarantee that a block cannot be altered or removed from the blockchain without burning at least 33% of the total stacked ETH. The underlying algorithm to achieve this is called **Gasper** |
Suggested change
| In Ethereum's proof-of-stake based consensus mechanisms, finality refers to the guarantee that a block cannot be altered or removed from the blockchain without burning at least 33% of the total stacked ETH. The underlying algorithm to achieve this is called **Gasper** | |
| In Ethereum's proof-of-stake based consensus mechanisms, finality refers to the guarantee that a block cannot be altered or removed from the blockchain without burning at least 33% of the total staked ETH. The underlying consensus protocol to achieve this is called **Casper FFG**, more details on this type of attacks can be found [here](https://blog.ethereum.org/2016/05/09/on-settlement-finality) |
|
|
||
| Serialization is the process of converting data structures into format that can be transmuted, transmitted and reconstructed later. SSZ is a serialization format that is used in Ethereum 2.0 Beacon chain. Designed to be serialization scheme that is not self-describing -- rather it relies on a schema that must be known in advances. SSZ has a bunch of advantages over RLP, like efficient re-hashing of objects and fast indexing which RLP lacks resulting in $O(N)$ complexity. | ||
|
|
||
| Based on [V's comment](https://ethresear.ch/t/replacing-ssz-with-rlp-zip-and-sha256/5706/12), one of the major problem SSZ tries to solve is RLP doesn't allow Merkelization, and this would mean disqualifying any possibility of succinct light client proofs of anything. Thus, leaving no scope of achieving statelessness -- while stateleness remains a crucial objective of current Ethereum's R&D. |
There was a problem hiding this comment.
Suggested change
| Based on [V's comment](https://ethresear.ch/t/replacing-ssz-with-rlp-zip-and-sha256/5706/12), one of the major problem SSZ tries to solve is RLP doesn't allow Merkelization, and this would mean disqualifying any possibility of succinct light client proofs of anything. Thus, leaving no scope of achieving statelessness -- while stateleness remains a crucial objective of current Ethereum's R&D. | |
| Based on [Vitaliks's comment](https://ethresear.ch/t/replacing-ssz-with-rlp-zip-and-sha256/5706/12), one of the major problem SSZ tries to solve is RLP doesn't allow Merkelization, and this would mean disqualifying any possibility of succinct light client proofs of anything. Thus, leaving no scope of achieving statelessness -- while stateleness remains a crucial objective of current Ethereum's R&D. |
|
|
||
| Based on [V's comment](https://ethresear.ch/t/replacing-ssz-with-rlp-zip-and-sha256/5706/12), one of the major problem SSZ tries to solve is RLP doesn't allow Merkelization, and this would mean disqualifying any possibility of succinct light client proofs of anything. Thus, leaving no scope of achieving statelessness -- while stateleness remains a crucial objective of current Ethereum's R&D. | ||
|
|
||
| Further implementation and details about Simple Serialize can be found [**here**](#) |
There was a problem hiding this comment.
Suggested change
| Further implementation and details about Simple Serialize can be found [**here**](#) | |
| Further implementation and details about Simple Serialize can be found [**here**](/wiki/CL/ssz.md) |
raxhvl
requested changes
May 22, 2024
|
|
||
| > :warning: This article is a [stub](https://en.wikipedia.org/wiki/Wikipedia:Stub), help the wiki by [contributing](/contributing.md) and expanding it. | ||
| These are the core tenets that sparked the work on Ethereum's architeture and imlementation. |
There was a problem hiding this comment.
Suggested change
| These are the core tenets that sparked the work on Ethereum's architeture and imlementation. | |
| These are the core tenets that sparked the work on Ethereum's architecture and implementation: |
| These are the core tenets that sparked the work on Ethereum's architeture and imlementation. | ||
|
|
||
| - **Simplicity**: | ||
| Since its inception, the Ethereum protocol was designed in a way that reduced complexity and made it simpler, even at the cost of data storage inefficiency and time inefficiency. This stemmed from the idea that any average programmer should ideally be able to understand and implement the entire specification -- primarily to minimize the influence on the protocol by an individual or an elite group of developers. While this narrative has transmuted largely due to major changes that have been made to the protocol. Further |
There was a problem hiding this comment.
Suggested change
| Since its inception, the Ethereum protocol was designed in a way that reduced complexity and made it simpler, even at the cost of data storage inefficiency and time inefficiency. This stemmed from the idea that any average programmer should ideally be able to understand and implement the entire specification -- primarily to minimize the influence on the protocol by an individual or an elite group of developers. While this narrative has transmuted largely due to major changes that have been made to the protocol. Further | |
| Since its inception, the Ethereum protocol was designed with simplicity in mind and ambitious roadmap to add features along the way. This stemmed from the idea that any average programmer should ideally be able to understand and implement the entire specification -- primarily to minimize the influence on the protocol by an individual or an elite group of developers. While this narrative has transmuted largely due to major changes that have been made to the protocol further. |
|
|
||
| - **Universality**: | ||
| One of the fundamental doctrine of Ethereum's design philosophy is that Ethereum has no ***features***. | ||
| Instead, Ethereum provides an internal Turing-complete scripting language [**solidity**](https://docs.soliditylang.org/en/v0.8.25/), which you can use to construct any smart contract or transaction type that can be mathematically defined. Ethereum tries to become a platform where new age developers can build decentralised and truly trustless applications without being prudent about the underlying complexity. |
There was a problem hiding this comment.
Suggested change
| Instead, Ethereum provides an internal Turing-complete scripting language [**solidity**](https://docs.soliditylang.org/en/v0.8.25/), which you can use to construct any smart contract or transaction type that can be mathematically defined. Ethereum tries to become a platform where new age developers can build decentralised and truly trustless applications without being prudent about the underlying complexity. | |
| Ethereum provides an internal Turing-complete virtual machine, called the [EVM](/wiki/EL/evm.md), which you can use to construct any smart contract or transaction type that can be mathematically defined. Ethereum aims to become a platform where new age developers can build decentralised and truly trustless applications without being prudent about the underlying complexity. |
Ethereum protocol per say doesn't concern with Solidity or any high level programming language. Which is why we (or the yellow paper) does not cover it.
There was a problem hiding this comment.
oh yeah just realised it. will make the changes
| Since its inception, the Ethereum protocol was designed in a way that reduced complexity and made it simpler, even at the cost of data storage inefficiency and time inefficiency. This stemmed from the idea that any average programmer should ideally be able to understand and implement the entire specification -- primarily to minimize the influence on the protocol by an individual or an elite group of developers. While this narrative has transmuted largely due to major changes that have been made to the protocol. Further | ||
|
|
||
| - **Universality**: | ||
| One of the fundamental doctrine of Ethereum's design philosophy is that Ethereum has no ***features***. |
There was a problem hiding this comment.
Suggested change
| One of the fundamental doctrine of Ethereum's design philosophy is that Ethereum has no ***features***. | |
| Fundamentally, Ethereum attempts to build an extensible, generalised, end-to-end decentralised compute technology for developers to build trustless applications. |
Saying Ethereum has no feature is quite radical.
There was a problem hiding this comment.
I think no features is indeed the right term here but it needs to be explained correctly. The idea is that ethereum is generalized platform and doesn't provide features for specific use cases.
| Instead, Ethereum provides an internal Turing-complete scripting language [**solidity**](https://docs.soliditylang.org/en/v0.8.25/), which you can use to construct any smart contract or transaction type that can be mathematically defined. Ethereum tries to become a platform where new age developers can build decentralised and truly trustless applications without being prudent about the underlying complexity. | ||
|
|
||
| - **Modularity**: | ||
| Making the Ethereum protocol modular is crucial to it being **future-proof**. While Ethereum is far from being perfect, there is a continuous and rigourous research and engineering effort that runs parallel to the existence of the protocol. Over the course of development, it should be easy to make a small protocol modification in one place and have the application stack continue to function without any further modification. Innovations such as Dagger, Patricia trees and RLP should be implemented as separate libraries and made to be feature-complete even if Ethereum does not require certain features so as to make them usable in other protocols as well. Ethereum development should be maximally done so as to benefit the entire cryptocurrency ecosystem, not just itself. |
There was a problem hiding this comment.
Suggested change
| Making the Ethereum protocol modular is crucial to it being **future-proof**. While Ethereum is far from being perfect, there is a continuous and rigourous research and engineering effort that runs parallel to the existence of the protocol. Over the course of development, it should be easy to make a small protocol modification in one place and have the application stack continue to function without any further modification. Innovations such as Dagger, Patricia trees and RLP should be implemented as separate libraries and made to be feature-complete even if Ethereum does not require certain features so as to make them usable in other protocols as well. Ethereum development should be maximally done so as to benefit the entire cryptocurrency ecosystem, not just itself. | |
| Making the Ethereum protocol modular is crucial to it being **future-proof**. While Ethereum is far from being perfect, there is a continuous and rigorous research and engineering effort that runs parallel to the existence of the protocol. Over the course of development, it should be easy to make a small protocol modification in one place and have the application stack continue to function without any further modification. Innovations such as Dagger, Patricia trees and RLP has been implemented as separate libraries and made to be feature-complete even if Ethereum does not require certain features so as to make them usable in other protocols as well. With features such as [Proto-Danksharding](/wiki/research/scaling/core-changes/eip-4844.md), Ethereum provides building blocks for scaling of Layer 2 chains. |
There was a problem hiding this comment.
The modularity should also explain another important benefits. It talks only about modifying individual parts but it should also mention the encapsulated complexity, easier debugging of individual pieces, open plugable designs for different implementations
|
|
||
| A user's "balance" in the system is thus the total value of the set of coins for which the user has a private key capable of producing a valid signature. The account based model is more flexible and allows for more complex transactions. | ||
|
|
||
| #### Benefits of UTXOs |
There was a problem hiding this comment.
Not sure if covering UTXO is worthwhile. cc: @taxmeifyoucan
There was a problem hiding this comment.
The general reason for account model should be explained but I think we just could sum up pros and cons in few sentences without its own section on Benefits of utxo.
Utxo means more complexity for system like Eth, more privacy in certain use cases but less fungibility, etc..
|
|
||
| - **Simiplicity**: Accounts are simpler to implement and reason about than UTXOs. UTXOs require a more complex transaction validation algorithm, and the UTXO model is less flexible and less powerful than the account model. For example, it is impossible to implement a decentralized exchange in the UTXO model, as the UTXO model does not allow for the existence of a "sell" order that is not tied to a specific UTXO. | ||
|
|
||
| One weakness of the account paradigm is that in order to prevent replay attacks, every transaction must have a [**nounce**](https://ethereum.stackexchange.com/questions/27432/what-is-nonce-in-ethereum-how-does-it-prevent-double-spending), such that the account keeps track of the nonces used and only accepts a transaction if its nonce is 1 after the last nonce used. This means that even no-longer-used accounts can never be pruned from the account state. A simple solution to this problem is to require transactions to contain a block number, making them un-replayable after some period of time, and reset nonces once every period. Miners or other users will need to "ping" unused accounts in order to delete them from the state, as it would be too expensive to do a full sweep as part of the blockchain protocol itself. |
There was a problem hiding this comment.
Suggested change
| One weakness of the account paradigm is that in order to prevent replay attacks, every transaction must have a [**nounce**](https://ethereum.stackexchange.com/questions/27432/what-is-nonce-in-ethereum-how-does-it-prevent-double-spending), such that the account keeps track of the nonces used and only accepts a transaction if its nonce is 1 after the last nonce used. This means that even no-longer-used accounts can never be pruned from the account state. A simple solution to this problem is to require transactions to contain a block number, making them un-replayable after some period of time, and reset nonces once every period. Miners or other users will need to "ping" unused accounts in order to delete them from the state, as it would be too expensive to do a full sweep as part of the blockchain protocol itself. | |
| One weakness of the account paradigm is that in order to prevent replay attacks, every transaction must have a [**nonce**](https://ethereum.stackexchange.com/questions/27432/what-is-nonce-in-ethereum-how-does-it-prevent-double-spending), such that the account keeps track of the nonces used and only accepts a transaction if its nonce is 1 after the last nonce used. This means that even no-longer-used accounts can never be pruned from the account state. A simple solution to this problem is to require transactions to contain a block number, making them un-replayable after some period of time, and reset nonces once every period. Miners or other users will need to "ping" unused accounts in order to delete them from the state, as it would be too expensive to do a full sweep as part of the blockchain protocol itself. |
This means that even no-longer-used accounts can never be pruned from the account state.
Nonce is simply the number of transaction sent from an account. I don't think its much of a storage overhead.
cc: @taxmeifyoucan
There was a problem hiding this comment.
The nonce has to be mentioned here, it's necessary for account model and it should be clear it is certain complexity that protocol needs to deal with. But the section could be shorter, the pruning of old nonce transactions etc is irrelevant old idea, nodes don't prune txs now
|
|
||
| ### **Recursive Length Prefix (RLP)** | ||
| The rationale begin creating a new serialization scheme, lies in the | ||
| probablistic nature of other schemes. RLP solves this problem by being highly minimalistic serialization; and guarantees absolute byte-perfect consistency. RLP does not attempt to define any specific data type such as boolean, floats, doubles or even integers -- instead, it simply exists to store structure, in the form of nested arrays. Key/value maps are also not explicitly supported; the semi-official suggestion for supporting key/value maps is to represent such maps as``` [[k1, v1], [k2, v2], ...]``` where ```k1, k2...``` are sorted using the standard ordering for strings. |
There was a problem hiding this comment.
Suggested change
| probablistic nature of other schemes. RLP solves this problem by being highly minimalistic serialization; and guarantees absolute byte-perfect consistency. RLP does not attempt to define any specific data type such as boolean, floats, doubles or even integers -- instead, it simply exists to store structure, in the form of nested arrays. Key/value maps are also not explicitly supported; the semi-official suggestion for supporting key/value maps is to represent such maps as``` [[k1, v1], [k2, v2], ...]``` where ```k1, k2...``` are sorted using the standard ordering for strings. | |
| probabilistic nature of other schemes. RLP solves this problem by being a simple yet deterministic serialization technique; and guarantees absolute byte-perfect consistency. RLP does not attempt to define any specific data type such as boolean, floats, doubles or even integers -- instead, it simply exists to store structure, in the form of nested arrays. Key/value maps are also not explicitly supported; the semi-official suggestion for supporting key/value maps is to represent such maps as``` [[k1, v1], [k2, v2], ...]``` where ```k1, k2...``` are sorted using the standard ordering for strings. |
|
|
||
| One of the pressing problems of Ethereum is the current state size. Estimate at around 1-2TB(at the time of writing this article). It is impractical for nodes to hold in working memory or even in slower permanent storage per se, thus, the need for statelessness becomes crucial to growth of the network. Verkle trees with it's vector commitments allow for much smaller proofs (**called witnesses**). Instead of needing to provide hashes of all "sibling nodes" at each level, Merkle Trees, the prover needs only to provide all parent nodes(plus an extra proof, called an optional) along the path from each each leaf to the root. | ||
|
|
||
| ### **Recursive Length Prefix (RLP)** |
There was a problem hiding this comment.
RLP is covered in the wiki maybe give a one line intro and link to the RLP page
|
|
||
| The notion of complete anonymity of the data structure to the serizalation algorithm over the course of time has turn out to ineffinient in case of fixed length data types like boolean, integers. SimpleSerialize(SSZ) was introduced in Ethereum 2.0 which supported both variable sized and fixed sized data types with additional features like Merkleization. | ||
|
|
||
| ### **Simple serialize (SSZ)** |
There was a problem hiding this comment.
Like RLP, SSZ is covered in the wiki. Consider linking the page without going into depth here.
|
@hopinheimer Thanks for tackling this. Consider keeping this article accessible to non technical folks. @taxmeifyoucan This is a good start. I think RLP, SSZ, and other technical bit can be left out. We can start from a bare bones article and keep building it up. |
|
@raxhvl I have included all the necessary changes as mentioned above |
|
hey @hopinheimer
ℹ️ Here's how to fix them:
|
raxhvl
requested changes
May 23, 2024
taxmeifyoucan
approved these changes
May 23, 2024
|
Thanks a lot @hopinheimer! I think it looks good now. Feel free to open subsequent PR if there are any changes or comments to address |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Protocol Design
solves #154
This a WIP PR. I have added the topic considered under the protocol design wiki and will be adding the content by EOD or in a few days time 😄
please do review the topic if you feel something might be off.