feat: add rfc 321 #112
feat: add rfc 321 #112
Conversation
| should complete or be aborted in a timely manner to release the resources. | ||
|
|
||
| Ordering of transactions is also important across all shards. While the different | ||
| shards form a Direct Acyclic Graph (DAG), there should be a consistent ordering |
There was a problem hiding this comment.
You can never achieve ordering, only the partial ordering (which is result of DAG). Or do you mean ordering of concurrent transactions (e.g. two transaction from the same account).
There was a problem hiding this comment.
Given the later comments, perhaps I should remove this requirement and add it back later if we need it
| If a node receives a foreign proposal (not the command), and it has not received | ||
| the previous foreign proposal, then it should ask the committee to provide it to them. | ||
|
|
||
| If the LOCAL_PREPARE command appears in a local proposal for a transaction and is the last such LOCAL_PREPARE and all shards have voted to accept, an ALL_PREPARED(tx) command **must** also be present in the local proposal. |
There was a problem hiding this comment.
So if we have LOCAL_PREPARE(tx) and we have all foreign LOCAL_PREPARE already, we should put ALL_PREPARED(tx) instead? Or do you mean both will be present? What will happen when the block is too big and it's not enough space to add both?
There was a problem hiding this comment.
Maybe we can just put the ALL_PREPARED in then, and same for SOME_PREPARED and ALREADY_ABORTED
| To ensure this, each node in the local committee will forward this committed block, along with a 3 chain of QC's proving it was committeed. | ||
|
|
||
| As a local committee member, when I receive a foreign proposal, if it is valid I will queue up a special command ForeignProposal(number, QC_Hash) that I must propose | ||
| when I am next leader (if it has not been proposed already). I also **should** request all transaction hashes that I have not seen from involved_shards for each transaction in the proposal, and add them to my mempool for execution. |
There was a problem hiding this comment.
This is just for notifying the rest of the committee that they should be aware of the foreign proposal? They will need it anyway when they will try to validate TX going from local_prepared to accept. Voting on this special command is the same as voting on individual transactions that includes foreign shards. Or am I missing something?
There was a problem hiding this comment.
The main reason is to set a starting point for a timeout. If the ForeignProposal command is committed, then 2f+1 nodes have agreed that this data has arrived and all of them will try to retrieve the transaction. If the transaction does exist, then at least one of them should find it and be able to propose it, if none of the nodes can find it in that time, the transaction must timeout.
There was a problem hiding this comment.
I was expecting that the timer starts once I send something to the foreign committees and I expect an answer. E.g. I send Prepare(tx)/LocalPrepare to foreign committee, I start a timer that I'm expecting to receive something.
Maybe I'm misunderstanding what is the timer waiting for.
If I understand that correctly. The current approach is that I send something to foreign committee and I expect that committee to start a timer and if they are not able to process the Tx, after the timeout they will send a timeout proposal to me. That sounds more complicated than starting a timer once I send to foreign committee something and I wait for response. Because I'll not rely on the foreign committee to process it. I will just time it out myself.
There was a problem hiding this comment.
Clarify: if a node receives ForeignProposal(tx_1) and doesn't know tx_1, it will request it from the local nodes?
It may not be necessary to have an explicit command to indicate receipt of foreign LocalPrepared because All/SomePrepared (a.k.a Accept?) already implies that. But we may need a way to indicate that the foreign LocalPrepared at the All/SomePrepared level was never received to get finality on all transactions.
Let's say we have these commands:
Prepare(P) -> LocalPrepare(LP) -> AllPrepare(AP)|SomePrepare(SP)|Timeout(T) ->Finalize(F)
and these statuses:
COMMIT, ABORT, TIMEOUT
Formatting "tx atoms" as {Command}({tx_id}, {status:=COMMIT|ABORT|TIMEOUT) e.g. LocalPrepare(tx_1, COMMIT). fp(LP, tx_1, COMMIT) = foreign proposal for LocalPrepare to COMMIT tx_1
| Case | Local knows tx_1 | Received foreign LP |
|---|---|---|
| A | Yes | Yes |
| B | Not yet | Yes |
| C | Never | Yes |
| D | Yes | Not yet |
| E | Yes | Never |
Case A: The "normal" case. Prepare -> LocalPrepare ->(send) -> AllPrepare->(send) ->Finalize
Case B: Request tx_1 on receipt of fp(LP, tx_1, COMMIT), (start timer),..., RECV tx_1, fLP exists (timer stop),
Prepare->LocalPrepare->(send ourfp(LP, tx_1, COMMIT|ABORT)) -> AllPrepare ->Finalize`
Case C: Request tx_1 on receipt of LP(tx_1, COMMIT), (start timer),..., (timer expires), should the local committee even vote on anything involving tx_1 here? We don't know tx_1 or if it event exists so it likely is equivocation. Maybe Prepare(tx_1, TIMEOUT) -> LocalPrepare(tx_1, TIMEOUT) ->(send our fp(LP, tx_1, TIMEOUT))-> SomePrepare(tx_1, TIMEOUT) ->Finalize(tX_1, TIMEOUT)
Case D: We have LocalPrepare(tx_1, COMMIT), Waiting for fp(LP, tx_1, *), (start timer),..., tx_1 exists, RECV fp(LP, tx_1, COMMIT) (timer stop),
Prepare->LocalPrepare->(send ourfLP(tx_1, COMMIT|ABORT)) -> AllPrepare ->Finalize`
Case E: We have LocalPrepare(tx_1, COMMIT), Waiting for fp(LP,tx_1, *) (start timer),..., (timer expires), propose SomePrepare(tx_1, TIMEOUT) -> (send fp(SP, tx_1, TIMEOUT)) -> Finalize(tx_1, TIMEOUT)
Rather than local round that all local nodes did receive foreign data, which I think is covered by votes on AllPrepared/SomePrepared proposals, we may need to have a LocalPrepared(tx_1, TIMEOUT)' and a final foreign round sending the next LocalPrepared' to ensure that if shards didnt get the foreign data, no shard commits, I suspect this is needed anyway.
| As a local committee member, when I receive a foreign proposal, if it is valid I will queue up a special command ForeignProposal(number, QC_Hash) that I must propose | ||
| when I am next leader (if it has not been proposed already). I also **should** request all transaction hashes that I have not seen from involved_shards for each transaction in the proposal, and add them to my mempool for execution. | ||
|
|
||
| In the same order of transactions in the proposal, the transactions must be sequenced into the local chain as either a TIMEOUT or a LOCALPREPARE(TxId, ForeignShardId). |
There was a problem hiding this comment.
I think the ordering of transactions within a proposal will cause some inconsistency.
E.g. I have Shard Sa and Sb, and Sa receive Ta (transaction over both shards) and Sb receive Tb (also over both shards). Then Sa will prepare a block with ordering Ta,Tb and Sb will prepare a block with order Tb,Ta. Then the foreign proposal will have wrong order compared to local proposal.
Why does the order in the proposal matters? It's not like I can have concurrent transactions in the proposal, right?
There was a problem hiding this comment.
I suppose the order inside the proposal does not matter, so long as the proposals themselves are processed in entirety before any transactions for a new proposal can be sequenced. The rule should then be updated that any transactions that have not been processed within the time limit must be timed out
There was a problem hiding this comment.
I'm curious when can it happen that I receive a foreign proposal with some Tx and I'm not able to do localprepare(tx). That means that even the committee from which I got the foreign proposal don't have the Tx.
Now I understand why you want timeout in foreign committee to send you that it timed out.
There was a problem hiding this comment.
Yeah, the only case I can think of is that the foreign committee is malicious and has made up a transaction. There may need to be some further action, but in this model, the local committee can now process further proposals
|
@Cifko made some changes, please have a look |
| When a LOCALPREPARE command is found in a local proposal, there **must** be a | ||
| Prepare(tx) command for that transaction in or before the current proposal. | ||
|
|
||
| In the `TIMEOUT_TIME`` block (e.g. 1000 blocks, but should probably be equal to committee_size * X rounds) after the ForeignProposal command has been proposed, |
There was a problem hiding this comment.
Add "Dummy blocks should be excluded from this count"
There was a problem hiding this comment.
Thanks for writing this up. It's a complex subject.
I agree that we need a way to ABORT/TIMEOUT a transaction if a LocalPrepare is never received. If a node receives AllPrepared/SomePrepared and does not have all foreign shard QCs listed in the evidence (or perhaps entire the 3-chain evidence, I think the current Evidence struct is too simplistic), it could request them from the proposer before voting. I think this is equivalent to agreeing on "having the foreign proposal".
As always, there's a ton of ways to approach this and maybe the best way is to attempt an implementation and see what realisations we have.
| To ensure this, each node in the local committee will forward this committed block, along with a 3 chain of QC's proving it was committeed. | ||
|
|
||
| As a local committee member, when I receive a foreign proposal, if it is valid I will queue up a special command ForeignProposal(number, QC_Hash) that I must propose | ||
| when I am next leader (if it has not been proposed already). I also **should** request all transaction hashes that I have not seen from involved_shards for each transaction in the proposal, and add them to my mempool for execution. |
There was a problem hiding this comment.
Clarify: if a node receives ForeignProposal(tx_1) and doesn't know tx_1, it will request it from the local nodes?
It may not be necessary to have an explicit command to indicate receipt of foreign LocalPrepared because All/SomePrepared (a.k.a Accept?) already implies that. But we may need a way to indicate that the foreign LocalPrepared at the All/SomePrepared level was never received to get finality on all transactions.
Let's say we have these commands:
Prepare(P) -> LocalPrepare(LP) -> AllPrepare(AP)|SomePrepare(SP)|Timeout(T) ->Finalize(F)
and these statuses:
COMMIT, ABORT, TIMEOUT
Formatting "tx atoms" as {Command}({tx_id}, {status:=COMMIT|ABORT|TIMEOUT) e.g. LocalPrepare(tx_1, COMMIT). fp(LP, tx_1, COMMIT) = foreign proposal for LocalPrepare to COMMIT tx_1
| Case | Local knows tx_1 | Received foreign LP |
|---|---|---|
| A | Yes | Yes |
| B | Not yet | Yes |
| C | Never | Yes |
| D | Yes | Not yet |
| E | Yes | Never |
Case A: The "normal" case. Prepare -> LocalPrepare ->(send) -> AllPrepare->(send) ->Finalize
Case B: Request tx_1 on receipt of fp(LP, tx_1, COMMIT), (start timer),..., RECV tx_1, fLP exists (timer stop),
Prepare->LocalPrepare->(send ourfp(LP, tx_1, COMMIT|ABORT)) -> AllPrepare ->Finalize`
Case C: Request tx_1 on receipt of LP(tx_1, COMMIT), (start timer),..., (timer expires), should the local committee even vote on anything involving tx_1 here? We don't know tx_1 or if it event exists so it likely is equivocation. Maybe Prepare(tx_1, TIMEOUT) -> LocalPrepare(tx_1, TIMEOUT) ->(send our fp(LP, tx_1, TIMEOUT))-> SomePrepare(tx_1, TIMEOUT) ->Finalize(tX_1, TIMEOUT)
Case D: We have LocalPrepare(tx_1, COMMIT), Waiting for fp(LP, tx_1, *), (start timer),..., tx_1 exists, RECV fp(LP, tx_1, COMMIT) (timer stop),
Prepare->LocalPrepare->(send ourfLP(tx_1, COMMIT|ABORT)) -> AllPrepare ->Finalize`
Case E: We have LocalPrepare(tx_1, COMMIT), Waiting for fp(LP,tx_1, *) (start timer),..., (timer expires), propose SomePrepare(tx_1, TIMEOUT) -> (send fp(SP, tx_1, TIMEOUT)) -> Finalize(tx_1, TIMEOUT)
Rather than local round that all local nodes did receive foreign data, which I think is covered by votes on AllPrepared/SomePrepared proposals, we may need to have a LocalPrepared(tx_1, TIMEOUT)' and a final foreign round sending the next LocalPrepared' to ensure that if shards didnt get the foreign data, no shard commits, I suspect this is needed anyway.
| Before transactions in the `N+1`th foreign proposal for a shard, all transactions in the `N`th foreign proposal for that shard must be sequenced into the local chain as either a TIMEOUT or a LOCALPREPARE(TxId, ForeignShardId). | ||
|
|
||
| When a LOCALPREPARE command is found in a local proposal, there **must** be a | ||
| Prepare(tx) command for that transaction in or before the current proposal. |
There was a problem hiding this comment.
I think the Prepare MUST come strictly before the LocalPrepare in a previous block. A LocalPrepare means that all nodes are aware that n-f others have prepared tx_1 . If Prepare(tx_1) is in the same block as LocalPrepare(tx_1) then we cannot be sure that we are ready to send the LocalPrepare(tx_1) to foreign shards until that block is in the commit phase.
There was a problem hiding this comment.
The main difference here is to make sure we start the transaction locally before committing to other shards
|
|
||
| In the `TIMEOUT_TIME`` block (e.g. 1000 blocks, but should probably be equal to committee_size * X rounds) after the ForeignProposal command has been proposed, | ||
| all transactions in the proposal must be sequenced, either as LOCALPREPAREs or TIMEOUTs. This is to cater for the case where a transaction was prepared by a foreign committee, but was never found on the network. This is unlikely to happen | ||
| in a non-malicious scenario. The more common case is that all transactions will be found and have a LOCALPREPARE, but a block at `TIMEOUT_TIME` **must** be considered invalid if there are any transactions that are not LOCALPREPARED or TIMEOUT from the foreign proposal that started the timeout. |
There was a problem hiding this comment.
Is this (invalid block) necessary? I worry about stoppages because of timer and state mismatches across local nodes if a majority are not aware of the original foreign proposal and the complexity dealing with that introduces. A non-byzantine node will include a timeout in the block eventually.
There was a problem hiding this comment.
Because of the new Command ForeignProposal (or perhaps ForeignProposalReceived is a better name), the nodes in the local committee have all agreed that they have received the foreign proposal, so if this command is in the local chain, the majority acknowledge that they have the ForeignProposal in hand
|
|
||
| When the proposed block becomes committed locally, the block **must** be broadcast to each involved shard that was incremented, along with evidence of being committeed (The chain of QCs must be included). | ||
|
|
||
| To ensure this, each node in the local committee will forward this committed block, along with a 3 chain of QC's proving it was committeed. |
There was a problem hiding this comment.
Well, if you send only 3 QCs you can't prove they are actually from one 3 chain. You can prove that you have a QC1 on top of the fLP block. But you can't prove that you have a QC2 on top of the block with QC1.
Maybe we should add block.justify() not just block_id to the QC. But I'm hope that we can come up with a better solution.
Description Update pr_title workflow. Motivation and Context Prevent code injection.
A high-level description of the DAN, introducing its component parts, and some placeholder RFCs to come.
RFC-304 was already taken. Renaming to RFC-305. We use P-cerb, not O-Cerb. Other editorial changes.
Description --- Updates the documentation for the `ToRistrettoPoint` opcode. Motivation and Context --- This opcode now requires a canonical scalar parsing. Further, the existing documentation was somewhat unclear. This PR updates the documentation for clarity and correctness. How Has This Been Tested? --- It renders as expected.
First draft of PCerb description with transaction flow, and accompanying mermaid diagram. The way that substate addresses (currently called ShardIds in the code) are derived is fairly elaborate. This commit elucidates the concept. It largely interprets how the code has been written.
- Added RFC-235 to summary, so it appears in the list. - Editing for grammar, concision and clarity.
Remove/replace anyone marked as a maintainer that is no longer maintaining RFCs.
This Request for Comment (RFC) describes a possible manifestation of a privacy-preserving stablecoin on the Tari Digital Assets Network (DAN). The stablecoin design broadly follows that of Tether and Circle's ERC-20 stabecoin, with the exception that amounts and transacting parties are confidential. The latest draft represents a more fully-fledged design for a privacy-enabled stablecoin and one that approaches feasibility. It is still likely insecure in places and there are still a few holes to plug in some of the operations, but the broad brushstrokes are certainly there. The specification given here focuses on a surveillance-enabled stablecoin (i.e. CBDC) model. The specification is presented neutrally as pure technology and should not be construed as an endorsement of centralised surveillance. In fact, the author abhors surveillance of individuals' sovereign wealth, but since this is actually the more difficult case to implement in a zero-knowledge manner, it is fairly straightforward to modify the specification to implement a free (as in speech) stablecoin (i.e. digital cash) model. Some hints are given at the end on how one might actually achieve this. There are also some tweaks that could allow/deny the central issuer to arbitrarily seize funds from accounts. Obviously a free model would disallow this feature.
The title was wrong and inconsistent in the ToC
Description --- Corrects typos and makes small updates to stablecoin RFC-0385. Supersedes #117. Motivation and Context --- Stablecoin RFC-0385 had several typos, as well as outdated content from previous design iterations. This PR makes the necessary fixes. How Has This Been Tested? --- The RFC builds and renders correctly.
Locally, mermaid works fine on v0.4.34. The workflow still has 0.4.10, so I'm assuming upgrading will fix the mermaid not displaying properly issue.
Typo in tar cmd
Description ---
Description --- Motivation and Context --- How Has This Been Tested? ---
GA can't find the preprocessor binary for some reason. Description --- Motivation and Context --- How Has This Been Tested? ---
RFC-331 was just a stub. This provides the content for it.
A draft of the Tari network logic layer description. The logic layer has many moving parts and I'm trying to thread the needle of making it a useful roadmap for the code without simply rewriting the code in English, or being too vague as to make it useless.
The CI has trouble finding the mermaid pre-processor. Fixing the call to the local folder works, but this breaks local builds. Adding the PWD to the path lets the CI find the binaries its looking for.
…-project/rfcs into st-rfc-processing-foreign-proposals
This reverts commit b150f58.
Adds an RFC for handling foreign DAN proposals