Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bump to v0.24.0 #5

Merged
merged 243 commits into from
Jan 9, 2024
Merged

bump to v0.24.0 #5

merged 243 commits into from
Jan 9, 2024

Conversation

ygaberman-px
Copy link

@ygaberman-px ygaberman-px commented Jan 9, 2024

Overview

New release of btcd. This PR updates our fork. Specifically by merging in the tag git merge v0.24.0

Changes

  • wire: add string parser to outpoint
  • wire: add outpoint string parser unit tests
  • txscript: modify TweakTaprootPrivKey to operate on private key copy
  • btcec/schnorr/musig2: add pk option to NonceGen
  • btcec/schnorr/musig2: add public key to secnonce
  • btcec/schnorr/musig2: add pubkey check to Sign
  • btcutil/psbt: add method Packet.GetTxFee
  • btcec/schnorr: use private key copy for BIP-340 signatures
  • btcec/schnorr/musig2: fix BenchmarkPartialVerify
  • base58: fix decoding issue
  • Update Dockerfile to Alpine 3.16
  • Update Docker documentation towards building your own image
  • chainhash: JSON marshal hash as string
  • [connmgr] remove pending entry on failed connection
  • Update mining.md (Update mining.md btcsuite/btcd#1938)
  • txscript: fix sighash bug in RawTxInTaprootSignature
  • txscript: add new PayToTaprootScript function
  • txscript: add tests for RawTxInTaprootSignature and RawTxInTapscriptSignature
  • psbt: export Bip32 encode/decode functions
  • psbt: fix formatting and typos
  • psbt: allow Unknowns in outputs
  • psbt: encode global unknowns
  • psbt: use pointer slice for global unknowns
  • psbt: add unit test for unknowns
  • docs: Update Go version as per the Readme.
  • txscript: Fix typo in IsUnspendable() comment
  • txscript: allow script builder capacity to be specified
  • Export MakeScritpNum, AsSmallInt, and IsSmallInt
  • btcd: Add memory profiling flag
  • psbt: add verification method for psbt input data
  • chainhash: JSON Unmarshal hash from appropriate string.
  • Supplementary chainhash test cases
  • txscript: fix script typos
  • chaincfg: Update checkpoints
  • blockchain: Use slices when fetching utxos
  • blockchain: Add benchmark for using a map vs a slice
  • main: Update README.md's minimum go version
  • docs: Update minimum Go version
  • rpc: Add ScriptPubKeyResult address field
  • multi: Run gofmt on the entire repository
  • blockchain+chaincfg: disable retargeting for regtest
  • multi: remove repetitive the
  • blockchain: intro HeaderCtx, ChainCtx + refactor CheckBlockHeaderContext
  • blockchain: export CheckBlockHeaderSanity as a library function
  • btcutil: format BTC amounts with trailing zeroes
  • musig2: fix early nonce gen option
  • musig2: add early nonce gen KnownSigners test
  • blockchain: Add InactiveTips() to blockindex
  • blockchain: Add ChainTips() method to BlockChain
  • btcjson, main: Implement the getchaintips call
  • docs: Update json_rpc_api.md for getchaintips rpc
  • rpcclient, integration: Add test for getchaintips call
  • psbt: finalizer add proper sighash flag
  • blockchain, btcutil/bloom: BuildMerkleTreeStore returns chainhash.Hash
  • blockchain: Add RollingMerkleTree
  • blockchain: Add benchmarks for Merkle root calculation
  • blockchain: Add CalcMerkleRoot
  • blockchain, rpctest, mining, main: replace usage of BuildMerkleTreeStore with CalcMerkleRoot
  • schnorr: correct rfc reference in docs
  • database/ffldb: Change scanBlockFiles behavior
  • txscript/engine: add execution stepCallback
  • txscript/engine_debug_test: add TestDebugEngine
  • schnorr: bip-340 compliant verification
  • schnorr: bip-340 compliant signing
  • database/ffldb: Add PruneBlocks to db interface
  • blockchain: Add pruning support
  • wire, main: Add SFNodeNetworkLimited flag
  • main: Add prune flag
  • main: Disable enabling both --prune and --txindex
  • main: Disable enabling both --prune and --addrindex
  • database, database/ffldb: add BeenPruned() method
  • main: force user to enable pruning if database is already pruned
  • main: fetch prune status from db for handleGetBlockChainInfo
  • blockchain/indexers: add functions to report init status
  • main: force the user to drop tx and addr indexes for pruning
  • main: return error if user requests addr or tx index while pruned
  • main: cfindex related sanity checks when enabled with pruning
  • chainhash: add support to legacy-marshaled hashes
  • gitignore: ignore vim files
  • gomod: clean go mod files
  • rpcclient: remove redundant params used in handleSendPostMessage
  • rpcclient: catch shutdown signal when sending requests
  • wire: add NodeNetworkLimitedBlockThreshold const
  • integration: add test to check prune status
  • fixup! integration: add test to check prune status
  • fix typos in sighash.go
  • Update LICENSE
  • blockchain: export CheckSerializedHeight, reduce allocs for cb height parsing
  • fix: default forward port
  • Added tests for FutureGetBestBlockHashResult.Receive
  • Added test for client.GetChainTxStatsAsync() in rpcclient. This sets up a test websocket server to run the tests. Also, ensure these are run within a timeout, since they rely on concurrency
  • added tests for GetBestBlockHashAsync
  • chainhash: Add DoubleHashRaw
  • wire: add HasFlag method
  • netsync: change isSyncCandidate behavior to include pruned nodes
  • blockchain: better Ancestor with skiplists
  • Fix typos
  • Fix typos on signature.go
  • schnorr: simplify some signing math
  • Correct comments in singature.go
  • Update petertodd seed DNS to .net
  • multi: remove use of GO111MODULE
  • git: add binaries to .gitignore file
  • make: add install and release-install goals
  • blockchain: Add sizehelper
  • btcutil: update to chaincfg/chainhash/v1.1.0
  • btcutl/gcs: update filter logic to use new DoubleHashRaw
  • btcutil/psbt: update to chaincfg/chainhash/v1.1.0 + btcutil/v1.1.3
  • wire/bench_test: report allocs in benchmarks
  • wire/bench: add witness block
  • wire/common: optimize Read/WriteVarInt
  • wire: introduce Read/WriteVarIntBuf to reuse buffers between invocations
  • wire/msgtx: use Read/WriteVarIntBuf in tx serialization
  • wire/msgtx: reuse tx-level buffer for version and locktime
  • wire/common: add optimized Read/WriteVarBytesBuf
  • wire/msgtx: introduce optimized read/writeOutPointBuf
  • wire/msgtx: introduce optimized writeTxInBuf
  • wire/msgtx: use writeTxInBuf in txn encoding
  • wire/msgtx: introduce optimized readScriptBuf
  • wire/msgtx: introduce optimized readTxInBuf
  • wire/msgtx: use readTxInBuf in txn serialization
  • wire/msgtx: introduce optimized WriteTxOutBuf
  • wire/msgtx: use WriteTxOutBuf in txn serialization
  • wire/msgtx: introduce optimized readTxOutBuf
  • wire/msgtx: use readTxOutBuf in txn serialization
  • wire/msgtx: introduce optimized writeTxWitnessBuf
  • wire/msgtx: use writeTxWitnessBuf in txn serialization
  • wire/msgtx: use readScriptBuf in txn serialization
  • wire/bench_test: introduce optimized readBlockHeaderBuf
  • wire/blockheader: introduce optimized writeBlockHeaderBuf
  • wire/invvect: add optimized readInvVectBuf and writeInvVectBuf
  • wire/msggetblocks: optimize by reusing small buffer
  • wire/msgblock: use only one small buffer per block encode/decode
  • wire/msgblock: optimize DeserializeTxLoc by reusing small buffers
  • wire/msggetheaders: optimize serialization by reusing small buffers
  • wire/msgheaders: optimize serialization by reusing small buffers
  • wire/msggetcfheaders: use single small buffer for encode/decode
  • wire/msgcfheaders: optimize encode/decode by using one small buffer
  • wire/msggetcfcheckpt: optimize by removing read/writeElement
  • wire/msgcfcheckpt: optimize serialization by reusing small buffers
  • wire/msginv: optimize by reusing small buffers
  • wire/msggetdata: optimize serialization by reusing small buffers
  • wiree/msggetcfilters: optimize serialization by reusing small buffers
  • wire/msgcfilter: optimize serialization by reusing small buffers
  • wire/msgnotfound: optimize serialization by reusing small buffers
  • wire/invvect: remove unused readInvVect and writeInvVect
  • wire/common: add optimized writeVarStrBuf an readVarStrBuf
  • wire/msgreject: optimize serialization by reusing small buffers
  • wire/netaddress: add optimiezed read/writeNetAddressBuf
  • wire/msgmerkleblock: optimize serialization by reusing small buffers
  • wire/msgping: remove usage for read/writeElement
  • wire/msgpong: remove usage of read/writeElement
  • wire/msgtx: remove unused writeTxWitness
  • wire/msgtx: remove unused writeTxIn
  • wire/msgtx: remove unused readTxIn
  • wire/msgtx: remove unused readScript
  • wire/msgtx: remove unused read/writeOutPoint
  • wire/msgtx: use tx-level script slab
  • wire/msgblock+msgtx: user block-level script slab
  • blockchain: Add mapslice
  • blockchain: Add tfFresh to txoFlags
  • blockchain: Add memoryUsage() method on UtxoEntry
  • blockchain: Add utxoStateConsistency read and write functions
  • blockchain: Refactor dbPutUtxoView
  • blockchain: Return early on nil utxo view in dbPutUtxoView
  • blockchain: Require utxoBucket in dbFetchUtxoEntry
  • blockchain: Refactor fetchInputUtxos
  • blockchain: Add utxocache
  • blockchain, netsync, main, cmd/addblock: Use utxocache
  • database/ffldb: make PruneBlocks atomic
  • blockchain: add flushNeededAfterPrune
  • blockchain: flush the utxo cache on prune if needed
  • ffldb: change export_test.go to export.go
  • blockchain: add test for InitConsistentState
  • blockchain: address nit comments
  • fix: enhance code comments (fix: enhance code comments btcsuite/btcd#2074)
  • btcutil/psbt: update to btcutil btcutil/v1.1.4
  • multi: update main package to chainhash/v1.1.0, use optimized dsha256
  • txscript: use DoubleHashRaw to write directly crypto.Hash for segwit sighash
  • txscript: use DoubleHashRaw for segwit sighash single calc
  • txscript: use a single shared scratch buffer in segwit sighash calc
  • doc: correct comments
  • btcutil: reuse serialized tx during TxHash
  • rpctest: fix formatting
  • rpctest: fix test node directory
  • rpctest: add more context to errors
  • rpctest: make test TCP ports unique per process
  • blockchain: fix compilation issues with 32bit systems
  • btcutil: align new serialization caching logic w/ codebase style
  • btcutil: add benchmarks for Hash + WitnessHash
  • wire: consistently use defer for returning scratch buffers
  • build: update to btcutil v1.1.5
  • btcd: add SECURITY.md
  • chaincfg: update mainnet block hashes
  • build: bump version to btcd v0.24
  • rpcclient: fix race condition in doDisconnect

GeorgeTsagk and others added 30 commits August 17, 2022 18:25
In this commit, we fix an inadvertent mutation bug that would at times
cause the private key passed into the tweak function to actually be
*modified* in place.

We fix this by accepting the value instead of a pointer. The actual
private key struct itself contains no pointer fields, so this is
effectively a deep copy via dereference.

We also add a new unit test that fails w/o this change, to show that the
private key was indeed being modified.
This commit adds the pk option to NonceGen and makes it mandatory.

Reference: jonasnick/bips@a89f857
This commit adds the public key to the sec nonce and ensures that we're
signing with the right key.

Reference: jonasnick/bips#74
This commit adds a check that the public key of the private key
that is passed to the Sign function is included in the slice of public
keys.

Reference jonasnick/bips@ea47d52
This commit adds a GetTxFee method to the psbt.Packet structure. The method returns the PSBT's fee.
btcutil/psbt: add method Packet.GetTxFee
txscript: modify TweakTaprootPrivKey to operate on private key copy
This is a fix similar to btcsuite#1905.
We'll always make a copy of the key in the local scope before passing it
around elsewhere. Depending on the parity of the public key, the private
key itself might need to be negated.

A similar test is added here that fails without the patch to the
signature.go file.
btcec/schnorr: use private key copy for BIP-340 signatures
In this commit, we fix the `BenchmarkPartialVerify` test. When we moved
to musig 1.0, we stopped requiring the input as an x-only key. So we
need to remove the round trip serialization to force the key to be
x-only.
btcec/schnorr/musig2: fix BenchmarkPartialVerify
Alpine 3.12 is EOL and contains multiple vulnerabilites.
Deploying on Alpine 3.16 works fine.
As DockerHub image non existant, update the documentation for Compose to build your own image instead.
chainhash: JSON marshal hash as string
Update Dockerfile to Alpine 3.16
fix memory leak in connmanager caused by:

* pending entries are not removed on error
* new connection always allocates new struct
* capture id value for callback

Fix by calling Remove in both handleFailedConn callbacks to delete the
item from the pending hash map

Signed-off-by: Christopher Hall <[email protected]>
minor changes to increase readability.
In this commit, we fix a bug in RawTxInTaprootSignature that would cause
the function to not properly apply the sighash flag for non-default
sighash signatures. The logic would end up applying `0x00` as a mask,
which will always be `0x00` on the other end.

The RawTxInTapscriptSignature function was correct, though it had the
ordering switched as it applies the sighash if the type doesn't equal
default.
…ignature

In this commit, we add tests for the public functions used to generate
keyspend and tapscript signatures. Without the prior commit, these tests
will fail as the keyspend function won't properly add the sighash bytes
for things that aren't sighash default.
…nd-bug-fix

Sighash taproot keyspend bug fix
Roasbeef and others added 28 commits December 18, 2023 16:36
btcutil: update to chaincfg/chainhash/v1.1.0
* fix: enhance code comments
In this commit, we update the top-level btcd package to use the latest
version of btcutil and also the chainhash package. With this version
bump, we can now use the new optimized dsha256 routine where applicable.

With this commit, I've covered most of the areas we'll hash an entire
transaction/block/header, but we may want to optimize some other areas
further, in particular, the witness sighash calc.
…sighash

In this commit, we optimize the sighash calc further by writing directly
into the buffer used for serialization by the sha256.New() instance
rather than to an intermediate buffer, which is then write to the hash
buffer.
We can write direly into the hash writer vs serializing into a buffer,
then writing that into the hash writer.
We used to use a lot of small buffers for serialization, but now we'll
use one buffer large enough, and slice into it when needed.

``
name                  old time/op    new time/op    delta
CalcWitnessSigHash-8    31.5µs ± 0%    29.2µs ± 0%   -7.05%  (p=0.000 n=10+10)

name                  old alloc/op   new alloc/op   delta
CalcWitnessSigHash-8    19.9kB ± 0%    18.5kB ± 0%   -7.14%  (p=0.000 n=10+10)

name                  old allocs/op  new allocs/op  delta
CalcWitnessSigHash-8       801 ± 0%       445 ± 0%  -44.44%  (p=0.000 n=10+10)
```
…c-2023

multi: update main package and btcutil to chainhash/v1.1.0, use optimized dsha256
btcutil.Block caches the serialized raw bytes of the block during ibd.
This serialized block bytes includes the serialized tx. The current tx
hash generation will re-serialized the de-serialized tx to create the
raw bytes and it'll only then hash that.

This commit changes the code so that the re-serialization never happens,
saving tons of cpu and memory overhead.
This commit adds a new NextAvailablePortForProcess function that takes a
process ID and then assures unique (non-occupied) port numbers are
returned per process.
This uses a temporary file that contains the latest used port and a
secondary temporary lock file to assure only a single goroutine can
request a new port at a time.

The GenerateProcessUniqueListenerAddresses is intened to be used as a
package-level override for the ListenAddressGenerator variable. We don't
use it by default to make sure we don't break any existing assumptions.
…txhash

btcutil: reuse serialized tx during TxHash
wire: only borrow/return binaryFreeList buffers at the message level
In this commit, we fix the following race condition:
```
==================
WARNING: DATA RACE
Write at 0x00c000216018 by goroutine 31:
  github.com/btcsuite/btcd/rpcclient.(*Client).wsReconnectHandler()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:736 +0x2aa
  github.com/btcsuite/btcd/rpcclient.New·dwrap·13()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1496 +0x39

Previous read at 0x00c000216018 by goroutine 29:
  github.com/btcsuite/btcd/rpcclient.(*Client).doDisconnect()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1079 +0x247
  github.com/btcsuite/btcd/rpcclient.(*Client).Disconnect()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1111 +0x47
  github.com/btcsuite/btcd/rpcclient.(*Client).wsInHandler()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:491 +0x1eb
  github.com/btcsuite/btcd/rpcclient.(*Client).start·dwrap·11()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1181 +0x39

Goroutine 31 (running) created at:
  github.com/btcsuite/btcd/rpcclient.New()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1496 +0xd77
  github.com/btcsuite/btcd/rpcclient.makeClient()
      /home/runner/work/btcd/btcd/rpcclient/chain_test.go:268 +0x1f9
  github.com/btcsuite/btcd/rpcclient.TestClientConnectedToWSServerRunner.func2()
      /home/runner/work/btcd/btcd/rpcclient/chain_test.go:164 +0x47
  testing.tRunner()
      /opt/hostedtoolcache/go/1.17.5/x64/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /opt/hostedtoolcache/go/1.17.5/x64/src/testing/testing.go:1306 +0x47

Goroutine 29 (finished) created at:
  github.com/btcsuite/btcd/rpcclient.(*Client).start()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1181 +0x2e4
  github.com/btcsuite/btcd/rpcclient.New()
      /home/runner/work/btcd/btcd/rpcclient/infrastructure.go:1493 +0xc51
  github.com/btcsuite/btcd/rpcclient.makeClient()
      /home/runner/work/btcd/btcd/rpcclient/chain_test.go:268 +0x1f9
  github.com/btcsuite/btcd/rpcclient.TestClientConnectedToWSServerRunner.func2()
      /home/runner/work/btcd/btcd/rpcclient/chain_test.go:164 +0x47
  testing.tRunner()
      /opt/hostedtoolcache/go/1.17.5/x64/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /opt/hostedtoolcache/go/1.17.5/x64/src/testing/testing.go:1306 +0x47
```

This arises as in `wsReconnectHandler`, the mutex isn't held while the
`conn` pointer is modified.
build: create btcd v0.24 release branch
@ygaberman-px ygaberman-px merged commit 38285ce into master Jan 9, 2024
6 checks passed
@ygaberman-px ygaberman-px deleted the bump-to-v0.24.0 branch January 9, 2024 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.