Merge branch 'develop' of https://github.com/dashpay/dash into develop

dashpay · Dec 30, 2024 · ee0a9a8 · ee0a9a8
2 parents 733eda9 + a5787c9
commit ee0a9a8
Show file tree

Hide file tree

Showing 55 changed files with 2,132 additions and 1,224 deletions.
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
@@ -3,6 +3,7 @@ image: "ubuntu:jammy"
 variables:
   DOCKER_DRIVER: overlay2
   FAST_MODE: "false" # when "true", only run linter on arm and unit/functional tests on linux64, skip everything else
+  CI_FAILFAST_TEST_LEAVE_DANGLING: "1"  # Gitlab CI does not care about dangling process and setting this variable avoids killing the CI script itself on error
 
 workflow:
   rules:

diff --git a/doc/README.md b/doc/README.md
@@ -80,6 +80,7 @@ The Dash Core repo's [root README](/README.md) contains relevant information on
 - [Reduce Memory](reduce-memory.md)
 - [Reduce Traffic](reduce-traffic.md)
 - [Tor Support](tor.md)
+- [Transaction Relay Policy](policy/README.md)
 - [ZMQ](zmq.md)
 
 License

diff --git a/doc/policy/README.md b/doc/policy/README.md
@@ -0,0 +1,10 @@
+# Transaction Relay Policy
+
+Policy is a set of validation rules, in addition to consensus, enforced for unconfirmed
+transactions.
+
+This documentation is not an exhaustive list of all policy rules.
+
+- [Packages](packages.md)
+
+
diff --git a/doc/policy/packages.md b/doc/policy/packages.md
@@ -0,0 +1,62 @@
+# Package Mempool Accept
+
+## Definitions
+
+A **package** is an ordered list of transactions, representable by a connected Directed Acyclic
+Graph (a directed edge exists between a transaction that spends the output of another transaction).
+
+For every transaction `t` in a **topologically sorted** package, if any of its parents are present
+in the package, they appear somewhere in the list before `t`.
+
+A **child-with-unconfirmed-parents** package is a topologically sorted package that consists of
+exactly one child and all of its unconfirmed parents (no other transactions may be present).
+The last transaction in the package is the child, and its package can be canonically defined based
+on the current state: each of its inputs must be available in the UTXO set as of the current chain
+tip or some preceding transaction in the package.
+
+## Package Mempool Acceptance Rules
+
+The following rules are enforced for all packages:
+
+* Packages cannot exceed `MAX_PACKAGE_COUNT=25` count and `MAX_PACKAGE_SIZE=101KvB` total size
+   (#20833)
+
+   - *Rationale*: This is already enforced as mempool ancestor/descendant limits. If
+     transactions in a package are all related, exceeding this limit would mean that the package
+     can either be split up or it wouldn't pass individual mempool policy.
+
+   - Note that, if these mempool limits change, package limits should be reconsidered. Users may
+     also configure their mempool limits differently.
+
+* Packages must be topologically sorted. (#20833)
+
+* Packages cannot have conflicting transactions, i.e. no two transactions in a package can spend
+   the same inputs. Packages cannot have duplicate transactions. (#20833)
+
+* No transaction in a package can conflict with a mempool transaction.
+
+* When packages are evaluated against ancestor/descendant limits, the union of all transactions'
+  descendants and ancestors is considered. (#21800)
+
+   - *Rationale*: This is essentially a "worst case" heuristic intended for packages that are
+     heavily connected, i.e. some transaction in the package is the ancestor or descendant of all
+     the other transactions.
+
+The following rules are only enforced for packages to be submitted to the mempool (not enforced for
+test accepts):
+
+* Packages must be child-with-unconfirmed-parents packages. This also means packages must contain at
+  least 2 transactions. (#22674)
+
+* Transactions in the package that have the same txid as another transaction already in the mempool
+  will be removed from the package prior to submission ("deduplication").
+
+   - *Rationale*: Node operators are free to set their mempool policies however they please, nodes
+     may receive transactions in different orders, and malicious counterparties may try to take
+     advantage of policy differences to pin or delay propagation of transactions. As such, it's
+     possible for some package transaction(s) to already be in the mempool, and there is no need to
+     repeat validation for those transactions or double-count them in fees.
+
+   - *Rationale*: We want to prevent potential censorship vectors. We should not reject entire
+     packages because we already have one of the transactions. Also, if an attacker first broadcasts
+     a competing package, the honest package should still be considered for acceptance.
diff --git a/doc/release-notes-6499.md b/doc/release-notes-6499.md
@@ -0,0 +1,11 @@
+Updated RPCs
+------------
+
+- The top-level fee fields `fee`, `modifiedfee`, `ancestorfees` and `descendantfees`
+  returned by RPCs `getmempoolentry`,`getrawmempool(verbose=true)`,
+  `getmempoolancestors(verbose=true)` and `getmempooldescendants(verbose=true)`
+  are deprecated and will be removed in the next major version (use
+  `-deprecated=fees` if needed in this version). The same fee fields can be accessed
+  through the `fees` object in the result. WARNING: deprecated
+  fields `ancestorfees` and `descendantfees` are denominated in sats, whereas all
+  fields in the `fees` object are denominated in DASH.
diff --git a/src/Makefile.am b/src/Makefile.am
@@ -300,6 +300,7 @@ BITCOIN_CORE_H = \
   rpc/blockchain.h \
   rpc/client.h \
   rpc/index_util.h \
+  rpc/mempool.h \
   rpc/mining.h \
   rpc/protocol.h \
   rpc/rawtransaction_util.h \
@@ -528,8 +529,10 @@ libbitcoin_server_a_SOURCES = \
   rpc/blockchain.cpp \
   rpc/coinjoin.cpp \
   rpc/evo.cpp \
+  rpc/fees.cpp \
   rpc/index_util.cpp \
   rpc/masternode.cpp \
+  rpc/mempool.cpp \
   rpc/governance.cpp \
   rpc/mining.cpp \
   rpc/misc.cpp \
@@ -538,6 +541,8 @@ libbitcoin_server_a_SOURCES = \
   rpc/rawtransaction.cpp \
   rpc/server.cpp \
   rpc/server_util.cpp \
+  rpc/signmessage.cpp \
+  rpc/txoutproof.cpp \
   script/sigcache.cpp \
   shutdown.cpp \
   spork.cpp \

diff --git a/src/Makefile.test.include b/src/Makefile.test.include
@@ -174,6 +174,7 @@ BITCOIN_TESTS =\
   test/torcontrol_tests.cpp \
   test/transaction_tests.cpp \
   test/txindex_tests.cpp \
+  test/txpackage_tests.cpp \
   test/txreconciliation_tests.cpp \
   test/txvalidation_tests.cpp \
   test/txvalidationcache_tests.cpp \

diff --git a/src/bench/rpc_mempool.cpp b/src/bench/rpc_mempool.cpp
@@ -3,7 +3,7 @@
 // file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
 #include <bench/bench.h>
-#include <rpc/blockchain.h>
+#include <rpc/mempool.h>
 #include <txmempool.h>
 
 #include <univalue.h>

diff --git a/src/llmq/dkgsession.h b/src/llmq/dkgsession.h
@@ -11,10 +11,12 @@
 #include <bls/bls.h>
 #include <bls/bls_ies.h>
 #include <bls/bls_worker.h>
+#include <saltedhasher.h>
 #include <sync.h>
 #include <util/underlying.h>
 
 #include <optional>
+#include <unordered_set>
 
 class CActiveMasternodeManager;
 class CInv;
@@ -293,7 +295,7 @@ class CDKGSession
 private:
     std::vector<std::unique_ptr<CDKGMember>> members;
     std::map<uint256, size_t> membersMap;
-    std::set<uint256> relayMembers;
+    std::unordered_set<uint256, StaticSaltedHasher> relayMembers;
     BLSVerificationVectorPtr vvecContribution;
     std::vector<CBLSSecretKey> m_sk_contributions;
 
@@ -382,7 +384,7 @@ class CDKGSession
 
 public:
     [[nodiscard]] CDKGMember* GetMember(const uint256& proTxHash) const;
-    [[nodiscard]] const std::set<uint256>& RelayMembers() const { return relayMembers; }
+    [[nodiscard]] const std::unordered_set<uint256, StaticSaltedHasher>& RelayMembers() const { return relayMembers; }
     [[nodiscard]] const CBlockIndex* BlockIndex() const { return m_quorum_base_block_index; }
     [[nodiscard]] const uint256& ProTx() const { return myProTxHash; }
     [[nodiscard]] const Consensus::LLMQParams GetParams() const { return params; }

diff --git a/src/llmq/dkgsessionhandler.cpp b/src/llmq/dkgsessionhandler.cpp
@@ -367,24 +367,22 @@ std::set<NodeId> BatchVerifyMessageSigs(CDKGSession& session, const std::vector<
     pubKeys.reserve(messages.size());
     messageHashes.reserve(messages.size());
     bool first = true;
-    for (const auto& p : messages ) {
-        const auto& msg = *p.second;
-
-        auto member = session.GetMember(msg.proTxHash);
+    for (const auto& [nodeId, msg] : messages) {
+        auto member = session.GetMember(msg->proTxHash);
         if (!member) {
             // should not happen as it was verified before
-            ret.emplace(p.first);
+            ret.emplace(nodeId);
             continue;
         }
 
         if (first) {
-            aggSig = msg.sig;
+            aggSig = msg->sig;
         } else {
-            aggSig.AggregateInsecure(msg.sig);
+            aggSig.AggregateInsecure(msg->sig);
         }
         first = false;
 
-        auto msgHash = msg.GetSignHash();
+        auto msgHash = msg->GetSignHash();
         if (!messageHashesSet.emplace(msgHash).second) {
             // can only happen in 2 cases:
             // 1. Someone sent us the same message twice but with differing signature, meaning that at least one of them
@@ -418,16 +416,15 @@ std::set<NodeId> BatchVerifyMessageSigs(CDKGSession& session, const std::vector<
         // different nodes, let's figure out who are the bad ones
     }
 
-    for (const auto& p : messages) {
-        if (ret.count(p.first)) {
+    for (const auto& [nodeId, msg] : messages) {
+        if (ret.count(nodeId)) {
             continue;
         }
 
-        const auto& msg = *p.second;
-        auto member = session.GetMember(msg.proTxHash);
-        bool valid = msg.sig.VerifyInsecure(member->dmn->pdmnState->pubKeyOperator.Get(), msg.GetSignHash());
+        auto member = session.GetMember(msg->proTxHash);
+        bool valid = msg->sig.VerifyInsecure(member->dmn->pdmnState->pubKeyOperator.Get(), msg->GetSignHash());
         if (!valid) {
-            ret.emplace(p.first);
+            ret.emplace(nodeId);
         }
     }
     return ret;
@@ -506,7 +503,6 @@ bool ProcessPendingMessageBatch(const CConnman& connman, CDKGSession& session, C
 
     auto badNodes = BatchVerifyMessageSigs(session, preverifiedMessages);
     if (!badNodes.empty()) {
-        LOCK(cs_main);
         for (auto nodeId : badNodes) {
             LogPrint(BCLog::LLMQ_DKG, "%s -- failed to verify signature, peer=%d\n", __func__, nodeId);
             pendingMessages.Misbehaving(nodeId, 100, peerman);

diff --git a/src/llmq/quorums.cpp b/src/llmq/quorums.cpp
@@ -370,7 +370,7 @@ void CQuorumManager::CheckQuorumConnections(CConnman& connman, const Consensus::
                 LogPrint(BCLog::LLMQ, "CQuorumManager::%s -- llmqType[%d] h[%d] keeping mn quorum connections for quorum: [%d:%s]\n", __func__, ToUnderlying(llmqParams.type), pindexNew->nHeight, quorum->m_quorum_base_block_index->nHeight, quorum->m_quorum_base_block_index->GetBlockHash().ToString());
             }
         } else if (watchOtherISQuorums && !quorum->IsMember(myProTxHash)) {
-            std::set<uint256> connections;
+            std::unordered_set<uint256, StaticSaltedHasher> connections;
             const auto& cindexes = utils::CalcDeterministicWatchConnections(llmqParams.type, quorum->m_quorum_base_block_index, quorum->members.size(), 1);
             for (auto idx : cindexes) {
                 connections.emplace(quorum->members[idx]->proTxHash);

diff --git a/src/llmq/signing_shares.cpp b/src/llmq/signing_shares.cpp
@@ -717,7 +717,7 @@ void CSigSharesManager::ProcessSigShare(PeerManager& peerman, const CSigShare& s
     bool canTryRecovery = false;
 
     // prepare node set for direct-push in case this is our sig share
-    std::set<NodeId> quorumNodes;
+    std::unordered_set<NodeId> quorumNodes;
     if (!IsAllMembersConnectedEnabled(llmqType, m_sporkman) && sigShare.getQuorumMember() == quorum->GetMemberIndex(m_mn_activeman->GetProTxHash())) {
         quorumNodes = connman.GetMasternodeQuorumNodes(sigShare.getLlmqType(), sigShare.getQuorumHash());
     }
@@ -739,16 +739,14 @@ void CSigSharesManager::ProcessSigShare(PeerManager& peerman, const CSigShare& s
         // Update the time we've seen the last sigShare
         timeSeenForSessions[sigShare.GetSignHash()] = GetTime<std::chrono::seconds>().count();
 
-        if (!quorumNodes.empty()) {
-            // don't announce and wait for other nodes to request this share and directly send it to them
-            // there is no way the other nodes know about this share as this is the one created on this node
-            for (auto otherNodeId : quorumNodes) {
-                auto& nodeState = nodeStates[otherNodeId];
-                auto& session = nodeState.GetOrCreateSessionFromShare(sigShare);
-                session.quorum = quorum;
-                session.requested.Set(sigShare.getQuorumMember(), true);
-                session.knows.Set(sigShare.getQuorumMember(), true);
-            }
+        // don't announce and wait for other nodes to request this share and directly send it to them
+        // there is no way the other nodes know about this share as this is the one created on this node
+        for (auto otherNodeId : quorumNodes) {
+            auto& nodeState = nodeStates[otherNodeId];
+            auto& session = nodeState.GetOrCreateSessionFromShare(sigShare);
+            session.quorum = quorum;
+            session.requested.Set(sigShare.getQuorumMember(), true);
+            session.knows.Set(sigShare.getQuorumMember(), true);
         }
 
         size_t sigShareCount = sigShares.CountForSignHash(sigShare.GetSignHash());

diff --git a/src/llmq/utils.cpp b/src/llmq/utils.cpp
@@ -653,12 +653,13 @@ uint256 DeterministicOutboundConnection(const uint256& proTxHash1, const uint256
     return proTxHash2;
 }
 
-std::set<uint256> GetQuorumConnections(const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, const CSporkManager& sporkman,
-                                       gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, const uint256& forMember, bool onlyOutbound)
+std::unordered_set<uint256, StaticSaltedHasher> GetQuorumConnections(
+    const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, const CSporkManager& sporkman,
+    gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, const uint256& forMember, bool onlyOutbound)
 {
     if (IsAllMembersConnectedEnabled(llmqParams.type, sporkman)) {
         auto mns = GetAllQuorumMembers(llmqParams.type, dmnman, pQuorumBaseBlockIndex);
-        std::set<uint256> result;
+        std::unordered_set<uint256, StaticSaltedHasher> result;
 
         for (const auto& dmn : mns) {
             if (dmn->proTxHash == forMember) {
@@ -677,23 +678,25 @@ std::set<uint256> GetQuorumConnections(const Consensus::LLMQParams& llmqParams,
     return GetQuorumRelayMembers(llmqParams, dmnman, pQuorumBaseBlockIndex, forMember, onlyOutbound);
 }
 
-std::set<uint256> GetQuorumRelayMembers(const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex,
-                                        const uint256& forMember, bool onlyOutbound)
+std::unordered_set<uint256, StaticSaltedHasher> GetQuorumRelayMembers(const Consensus::LLMQParams& llmqParams,
+                                                                      CDeterministicMNManager& dmnman,
+                                                                      gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex,
+                                                                      const uint256& forMember, bool onlyOutbound)
 {
     auto mns = GetAllQuorumMembers(llmqParams.type, dmnman, pQuorumBaseBlockIndex);
-    std::set<uint256> result;
+    std::unordered_set<uint256, StaticSaltedHasher> result;
 
     auto calcOutbound = [&](size_t i, const uint256& proTxHash) {
+        // Relay to nodes at indexes (i+2^k)%n, where
+        //   k: 0..max(1, floor(log2(n-1))-1)
+        //   n: size of the quorum/ring
+        std::unordered_set<uint256, StaticSaltedHasher> r{};
         if (mns.size() == 1) {
             // No outbound connections are needed when there is one MN only.
             // Also note that trying to calculate results via the algorithm below
             // would result in an endless loop.
-            return std::set<uint256>();
+            return r;
         }
-        // Relay to nodes at indexes (i+2^k)%n, where
-        //   k: 0..max(1, floor(log2(n-1))-1)
-        //   n: size of the quorum/ring
-        std::set<uint256> r;
         int gap = 1;
         int gap_max = (int)mns.size() - 1;
         int k = 0;
@@ -775,8 +778,8 @@ bool EnsureQuorumConnections(const Consensus::LLMQParams& llmqParams, CConnman&
     LogPrint(BCLog::NET_NETCONN, "%s -- isMember=%d for quorum %s:\n",
             __func__, isMember, pQuorumBaseBlockIndex->GetBlockHash().ToString());
 
-    std::set<uint256> connections;
-    std::set<uint256> relayMembers;
+    std::unordered_set<uint256, StaticSaltedHasher> connections;
+    std::unordered_set<uint256, StaticSaltedHasher> relayMembers;
     if (isMember) {
         connections = GetQuorumConnections(llmqParams, dmnman, sporkman, pQuorumBaseBlockIndex, myProTxHash, true);
         relayMembers = GetQuorumRelayMembers(llmqParams, dmnman, pQuorumBaseBlockIndex, myProTxHash, true);

diff --git a/src/llmq/utils.h b/src/llmq/utils.h
@@ -5,13 +5,15 @@
 #ifndef BITCOIN_LLMQ_UTILS_H
 #define BITCOIN_LLMQ_UTILS_H
 
+#include <gsl/pointers.h>
 #include <llmq/params.h>
+#include <saltedhasher.h>
 #include <sync.h>
-#include <gsl/pointers.h>
 #include <uint256.h>
 
 #include <map>
 #include <set>
+#include <unordered_set>
 #include <vector>
 
 class CConnman;
@@ -34,8 +36,13 @@ namespace utils
 std::vector<CDeterministicMNCPtr> GetAllQuorumMembers(Consensus::LLMQType llmqType, CDeterministicMNManager& dmnman, gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, bool reset_cache = false);
 
 uint256 DeterministicOutboundConnection(const uint256& proTxHash1, const uint256& proTxHash2);
-std::set<uint256> GetQuorumConnections(const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, const CSporkManager& sporkman, gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, const uint256& forMember, bool onlyOutbound);
-std::set<uint256> GetQuorumRelayMembers(const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, const uint256& forMember, bool onlyOutbound);
+std::unordered_set<uint256, StaticSaltedHasher> GetQuorumConnections(
+    const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman, const CSporkManager& sporkman,
+    gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, const uint256& forMember, bool onlyOutbound);
+std::unordered_set<uint256, StaticSaltedHasher> GetQuorumRelayMembers(const Consensus::LLMQParams& llmqParams,
+                                                                      CDeterministicMNManager& dmnman,
+                                                                      gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex,
+                                                                      const uint256& forMember, bool onlyOutbound);
 std::set<size_t> CalcDeterministicWatchConnections(Consensus::LLMQType llmqType, gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex, size_t memberCount, size_t connectionCount);
 
 bool EnsureQuorumConnections(const Consensus::LLMQParams& llmqParams, CConnman& connman, CDeterministicMNManager& dmnman, const CSporkManager& sporkman,