BIP346: OP_TXHASH and OP_CHECKTXHASHVERIFY #1500

stevenroose · 2023-09-30T11:40:55Z

Semantic changes

I thought it might be valuable to keep track of actual semantic changes being made since the initial out-of-draft version.

2023-12-19: Added relative indices for individual mode.

Implementations

A proposed implementation for Bitcoin Core is available here:
Add OP_TXHASH and OP_CHECKTXHASHVERIFY opcodes bitcoin#29050
A proposed implementation for rust-bitcoin is available here:
txhash: Implement TxHashCache for OP_TXHASH and OP_CHECKTXHASHVERIFY rust-bitcoin/rust-bitcoin#2275

bip-txhash.mediawiki

rustyrussell · 2023-10-23T06:31:09Z

bip-txhash.mediawiki

+* If the first byte is exactly 0x00, the Script execution succeeds immediately.
+  //TODO(stevenroose) is this valuable? it's the only "exception case" that
+  could potentially be hooked for some future upgrade


Why not allow extra bytes at the end to mean OP_SUCCESS?

@roconnor-blockstream has previously warned about non-trivial OP_SUCCESS semantics. Though the current SUCCESS semantics are "any OP_SUCCESS opcode occurring in the script means SUCCESS", but we could have different semantics that allow any opcode internally to trigger "instant success", but (1) that are very different semantics and will require entirely different code and (2) it becomes way harder to reason about.

IIRC, @sanket1729 also noted that such SUCCESS semantics make reasoning about scripts for things like miniscript way harder.

Actually this BIP seems outdated, I have to push a small update. I decided to propose to make the 0x00 special case mean "ALL" to make this more ergonomic to use as a sighash together with CSFS. ("ALL" isn't valuable as a template check because it contains the prevout scriptPubkey which should contain the hash) Other suggestions welcome.

stevenroose · 2023-10-23T16:11:38Z

Alternatively, but slightly even more complicating the cases, since the first two fields (version, locktime), are not very valuable without anything else (especially since we have OP_CLTV), we could introduce four special-cased bytes to mimick other popular SIGHASH modes: 0x00, 0x01, 0x02 and 0x03. Though locktime might be useful with OP_TX at some point. So I would argue against that. Mimicking "regular" sighashes isn't super useful in the first place because any system that expects to use regular sighashes can use current regular schnorr keys.

stevenroose · 2023-12-11T08:44:00Z

I just pushed an updated version of this BIP. It has a reference implementation that produces test vectors that are tested against an implementation for Bitcoin Core and for rust-bitcoin.

I think it should be ready for review. I have one small last TODO in the specification related to txfs malleability.

luke-jr

Missing a section on backward compatibility

rustyrussell

Sorry for the delay! I've finally found a round tuit, and have performed a more detailed review.

bip-txhash.md

rustyrussell · 2024-01-01T22:47:50Z

bip-txhash.md

+    * &nbsp;3. `TXFS_INOUT_NUMBER | TXFS_INOUT_SELECTION_ALL`
+    * &nbsp;4. `TXFS_INOUT_NUMBER | TXFS_INOUT_SELECTION_ALL`
+  * the `0x00` byte: it is set equal to `TXFS_SPECIAL_ALL`, which means "ALL" and is primarily
+    useful to emulate `SIGHASH_ALL` when `OP_TXHASH` is used in combination


Hmm, "would be useful if that were proposed which it isn't". I am skeptical of this magic value.

While I understand Russell O'Connor's dislike of runtime OP_SUCCESS, it is a lesser evil here than this kind of guessing of future utility which will no doubt prove suboptimal when we get there.

And for miniscript: sure, it will only generate and decode a push followed by TXHASH. But there are other things it can't decode too, and that's OK.

I think the SUCCESS argument has merit, though. Also IMO it's not too much of a pain to pick one of the many SUCCESS opcodes tapscript still has to make a OP_TXHASH2 if really needed. I also don't like that witness input can turn an opcode into a SUCCESS operation for the entire script. This can be tricky when collaboratively constructing scripts.

rustyrussell · 2024-01-01T22:49:51Z

bip-txhash.md

+summary, followed by a reference implementation of the CalculateTxHash function.
+
+* There are two special cases for the TxFieldSelector:
+  * the empty value, zero bytes long: it is set equal to `TXFS_SPECIAL_TEMPLATE`,


You re-use this term TXFS_SPECIAL_TEMPLATE twice for different things, which is confusing.

Ah sorry, one of them is a typo and should be TXFS_SPECIAL_ALL. Fixing.

bip-txhash.md

rustyrussell · 2024-01-01T22:52:44Z

bip-txhash.md

+* The last (highest) bit of the first byte (`TXFS_CONTROL`), we will call the
+  "control bit", and it can be used to control the behavior of the opcode. For
+  `OP_TXHASH` and `OP_CHECKTXHASHVERIFY`, the control bit is used to determine
+  whether the TxFieldSelector itself has to be included in the resulting hash.
+  (For potential other uses of the TxFieldSelector (like a hypothetical
+  `OP_TX`), this bit can be repurposed.)


That's a footnote, at best, mentioning how this could be expanded for a new OP_TX. But there's no reason to design for it now that I can see, except to leave a clear carve-out for future expansion.

So TXFS_CONTROL is a terrible name. TXFS_FIELD_SELECTOR perhaps?

rustyrussell · 2024-01-01T23:00:57Z

bip-txhash.md

+
+For both inputs and then outputs, do the following:
+
+* If the "in/outputs" field is set to 1, another additional byte is expected:


If I'm following this correctly, the (non-special) TxFieldSelector format is, in bytes:

CORE_SELECTOR [INOUT_SELECTOR] [IN_SELECTOR] [OUT_SELECTOR]

If TXFS_INPUTS is set in the CORE_SELECTOR, then INOUT_SELECTOR and IN_SELECTOR are present. If TXFS_OUTPUTS is set in CORE_SELECTOR, then INOUT_SELECTOR and OUT_SELECTOR are present?

Exactly. I'm thinking of changing this as follows:

remove TXFS_INPUTS and TXFS_OUTPUTS bits

reader will know the entire size of the txfs, so when a second byte is present, look at the bits present in the INOUT_SELECTOR byte to know whether to expect IN_SELECTOR and/or OUT_SELECTOR.

this frees up two bits in the CORE_SELECTOR, one of which I'm thinking to repurpose for SPEND_SCRIPT (i.e. scriptCode for segwit v0 inputs and tapscript for v1 inputs, scriptPubkey for non-segwit)

bip-txhash.md

rustyrussell · 2024-01-01T23:17:19Z

bip-txhash.md

+* the leading in/outputs up to 8192
+* up to 64 individually selected in/outputs
+** using absolute indices up to 16384
+** using indices relative to the current input index from -64 to +64.


This is incredibly complex, and seems to mismatch what I can see covenants being used for in practice. I anticipate fees being high in future, such that people will do a reasonable amount of engineering to minimize their total footprint. In particular, they will want to add fees after commitment, and want to batch transactions using stacking.

The first case implies you want to exclude a specific input and output, to allow for fees, or at least allow binding not to cover the final input/output. The second case implies you want to mul/divide an input number to get the corresponding range of outputs.

The simplest case is a single input and output pair: a-la SIGHASH_SINGLE. This both allows almost arbitrary fee inputs/outputs, and stacking.

But what if you want to bind a pair of inputs to one output? Or a pair of outputs to one input? Both seem reasonably common things to want to do (e.g. opening a dual-funded lightning channel, and closing a channel).

That means you need to be able to select outputs as "current input index / 2" or "current input index * 2 and current input index * 2 + 1". Numbers other than 2 are possible but this is the most likely case (since, in order to stack, all txs must be of same input-output number form, and I consider 1 and 2 by far the most likely numbers here).

Yeah this is true. Initially I didn't have relative indices. I'm still not entirely convinced they are useful. Precisely for the 1-in 2-out case which seems super common to me. I heard "you'd be surprised how easy it is to add an extra input".

My initial thought was that private aggregation (i.e. not through public broadcast media like mempools) would be easily possible as a user can just create/sign a thousand variants of their txs, for each possible input index. This works with absolute indices and doesn't need relative indices. It might even work with public broadcast.

The problem is that doing this with absolute indices only works if everyone in the protocol has the same in-out ratio. (Everyone needs 1-to-2 so you can sign 1in1,2out, 2in3,4out, 3in5,6out, etc). Otherwise you get a quadratic amount of data. With relative indices, you can sign XinX,1out, XinX,2out, XinX,3out,.. and this way the coordinator can put you in any place and put your second output in some arbitraty place and pick your signatures based on where your second output is placed.

Ok this doesn't really require relative indices, but it requires the ability to mix "current" with absolute.

This leaves some protocols vulnerable to the partial signature attacks. Say the covenant requires your outputs go half to pubkeyA and half to pubkeyB. Now I have two identical 1BTC covenant UTXO inputs, but re-use the same outputs to satisfy both, and steal the other 1BTC.

The same problem applies to "tell me the outputs in the witness data".

rustyrussell · 2024-01-01T23:21:25Z

bip-txhash.md

+  future addition of byte manipulation opcodes like `OP_CAT`, an additional
+  cost is specified per TransactionHash execution. Using the same validation
+  budget ("sigops budget") introduced in BIP-0342, each TransactionHash
+  decreases the validation budget by 10. If this brings the budget below zero,


This needs much more justification. Why 10? It has an implied cost of 2 already, since you have to use the opcode and a selector. If it has to hash a lot, hasn't someone already paid that to make such a large transaction?

Yeah it's tricky. In practice, it has a similar amortized per-tx hash cost that sighashes have. It's hard to count those to the budget because they are amortized, it's basically hashing all the large tx fields once so that if they are repeatedly requested their hash can be used.

After the amortized hash cost, it's just a finite series of ~32-byte chunks with maximally 64 in/out which in total can have 8 fields that are each ~32 bytes. This is ~16,384 bytes max.

Then, another consideration is that it would be nice and reasonable if TXHASH+CSFS would not have a higher cost than what naturally would be placed in the witness, the 64-byte signature.

I see it like this: we have a 64-byte budget to divide over TXHASH+CSFS as I think it's reasonable that this combination doesn't cost more than 28% more than a CHECKSIG (which is 50).

So maybe it's right that TXHASH can actually cost more, something like 25 if CSFS would be priced at 35 or 40.

This is not a signature budget, it's a hashing budget. Perhaps we should make this a first-class citizen then?

See https://rusty.ozlabs.org/2023/12/22/script-limits-opcat.html#my-proposal-a-dynamic-limit-for-hashing

rustyrussell · 2024-01-01T23:29:48Z

bip-txhash.md

+
+# Detailed Specification
+
+A reference implementation in Rust is provided attached as part of this BIP


OK, I would really appreciate a table of all the bits and exactly what and how they encode them. It's particularly nasty because some values are little-endian 32 bit encoded, not CScriptNum encoded, and others are varint encoded?

But it's nice to be explicit in each case, for people like me who are not deep in the weeds of bitcoin's onchain representation, since it helps when considering how to use this alongside things like OP_CAT and extended arithmetic opcodes.

Ok, I agree. I think I tried to encode values the way they are consistently encoded in other contexts like sighashes and p2p. But I will go over them and list them in the BIP as well. It's true that I didn't consider the interactions between regular LE encoding and CScriptNum encoding which is what will be used when math is done in Script for things like values.

halseth · 2024-01-04T12:17:44Z

bip-txhash.md

+* The element on the stack is at least 32 bytes long, fail otherwise.
+* The first 32 bytes are interpreted as the TxHash and the remaining suffix bytes specify the TxFieldSelector.
+* If the TxFieldSelector is invalid, fail.
+* The actual TxHash of the transaction at the current input index, calculated


Should maybe specify that the element is not popped off the stack, or is that implicit?

Hmm, it might be worth mentioning yeah, but I thought it was implicit as the other opcode explicitly mentions that it takes the items from the stack. It's kinda characteristic of a -VERIFY opcode to not touch the stack.

bip-txhash.md

Roasbeef · 2024-04-24T17:54:11Z

Assigned BIP 346.

murchandamus

There seem to be a lot of unresolved comments in this PR, and the document appears to still be missing the Backwards Compatibility section. Please resolve existing review comments and let us know when this is ready for an editor review

bip-txhash.md

ProofOfKeags

Did a first pass. Big Concept ACK from me. I have some notes about doc organization and plan to do another pass really drilling into the details soon.

ProofOfKeags · 2024-05-22T20:06:06Z

bip-txhash.md

+The TxFieldSelector has the following semantics. We will give a brief conceptual
+summary, followed by a reference implementation of the CalculateTxHash function.
+
+* There are two special cases for the TxFieldSelector:


Can I suggest that we move the section on the special cases to be after the fundamental atomic selector flags? It's much easier to read this document by building up from the building blocks rather than starting with the template and having to squint to find where all the sub-properties are defined later.

I think the discussion for the optimized case follows very naturally once we understand the components here.

bip-txhash.md

murchandamus · 2024-11-14T20:29:32Z

Hey @stevenroose, this pull request has had unaddressed review for over six months. Are you still working on this? If not, is there someone else that wants to pick this project up?

jonatack · 2024-12-17T22:14:32Z

Hey @stevenroose, this pull request has had unaddressed review for over six months. Are you still working on this? If not, is there someone else that wants to pick this project up?

pinging @stevenroose

stevenroose · 2025-01-14T15:50:07Z

Hey all, sorry for the long delay. I addressed some of the above comments, and also made some changes to the BIP, together with @reardencode. Changes are the following:

slightly modified the TEMPLATE special case to more correctly be equivalent to CTV
removed the 0x00 ALL special case, instead
add a special 1-byte version of the TxFieldSelector, that maps directly into a full TxFieldSelector
- this 1-byte txfs can directly emulate APO, APOAS and all currently existing sighashes, as outlined in the bip text
added wording for the bip to be an upgrade path of an existing CTV deployment

The requested Backward Compatibility section has been added

murchandamus

This seems to be pretty far along.
There are a few minor formatting issues which are described in line.

Please add the required Copyright section and consider renaming the "Summary" section "Specification".

murchandamus · 2025-01-15T15:34:00Z

bip-0346.md

+  BIP: 346
+  Layer: Consensus (soft fork)
+  Title: OP_TXHASH and OP_CHECKTXHASHVERIFY
+  Author: Steven Roose <[email protected]>, Brandon Black <[email protected]>


Suggested change

Author: Steven Roose <[email protected]>, Brandon Black <[email protected]>

Author: Steven Roose <[email protected]>

Brandon Black <[email protected]>

murchandamus · 2025-01-15T15:35:23Z

bip-0346.md

+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0346
+  Status: Draft
+  Type: Standards Track
+  Created: 2023-09-03


Per BIP 2, the "Created" header field uses the date a BIP was assigned a number.

Suggested change

Created: 2023-09-03

Created: 2024-04-24

murchandamus · 2025-01-16T16:04:05Z

bip-0346.md

+
+This BIP proposes two new opcodes, `OP_CHECKTXHASHVERIFY`, to be activated
+as a change to the semantics of `OP_NOP4` in legacy script, segwit and tapscript;
+and `OP_TXHASH`, to be activated as a change to the semantics of `OP_SUCCESS189`


I double checked, OP_SUCCESS appears to be unclaimed, OP_VAULT used OP_SUCCESS187 and OP_SUCCESS188.

murchandamus · 2025-01-16T16:07:24Z

bip-0346.md

+(`0xb3`) as a soft fork upgrade. This opcode is active in both legacy script,
+segwitv0 p2wsh and tapscript contexts.


I count three different contexts, so maybe:

Suggested change

(`0xb3`) as a soft fork upgrade. This opcode is active in both legacy script,

segwitv0 p2wsh and tapscript contexts.

(`0xb3`) as a soft fork upgrade. This opcode is active in legacy script,

segwitv0 p2wsh, and tapscript contexts.

murchandamus · 2025-01-16T16:08:22Z

bip-0346.md

+* The element on the stack is at least 32 bytes long, NOP otherwise.
+* The first 32 bytes are interpreted as the StackTxHash and the remaining
+  suffix bytes specify the TxFieldSelector.
+* The TxFieldSelector valid, fail otherwise.


Suggested change

* The TxFieldSelector valid, fail otherwise.

* The TxFieldSelector is valid, fail otherwise.

murchandamus · 2025-01-16T16:36:40Z

bip-0346.md

@@ -0,0 +1,438 @@
+```


Suggested change

```

<pre>

murchandamus · 2025-01-16T16:36:56Z

bip-0346.md

+  Type: Standards Track
+  Created: 2023-09-03
+  License: BSD-3-Clause
+```


Suggested change

```

</pre>

murchandamus · 2025-01-16T16:39:35Z

bip-0346.md

+  `<txfs> OP_TXHASH <pubkey> OP_CHECKSIGFROMSTACK` effectively emulates `SIGHASH_ANYPREVOUT`.
+
+
+# Detailed Specification


This seems to be talking about the Reference Implementation, not a Specification?

murchandamus · 2025-01-16T16:40:09Z

bip-0346.md

+potentially be renamed to `OP_TEMPLATEHASH`.
+
+
+# Summary


Should this section perhaps be called "Specification"?

murchandamus · 2025-01-16T16:43:15Z

bip-0346.md

+implementation.
+
+
+# Backwards Compatibility


This might also be a good place to mention the interactions with BIP 118, BIP 119, BIP 347, and BIP 348.

stevenroose force-pushed the txhash branch from 6af28b9 to 5b4092a Compare September 30, 2023 11:44

rustyrussell reviewed Oct 23, 2023

View reviewed changes

sennhann29 approved these changes Oct 24, 2023

View reviewed changes

stevenroose force-pushed the txhash branch from ec98a85 to 71073ff Compare December 11, 2023 08:42

stevenroose marked this pull request as ready for review December 11, 2023 08:42

stevenroose mentioned this pull request Dec 11, 2023

Add OP_TXHASH and OP_CHECKTXHASHVERIFY opcodes bitcoin/bitcoin#29050

Draft

stevenroose force-pushed the txhash branch 8 times, most recently from 330d9c1 to 16dd455 Compare December 16, 2023 09:51

stevenroose force-pushed the txhash branch from 16dd455 to 245c0b9 Compare December 19, 2023 13:16

sputn1ck mentioned this pull request Dec 24, 2023

Add OP_TXHASH and OP_CHECKTXHASHVERIFY opcodes btcsuite/btcd#2078

Open

luke-jr previously requested changes Dec 26, 2023

View reviewed changes

rustyrussell reviewed Jan 1, 2024

View reviewed changes

halseth reviewed Jan 4, 2024

View reviewed changes

bitcoin deleted a comment from mahmoudfranc Feb 28, 2024

starius reviewed Mar 10, 2024

View reviewed changes

bip-txhash.md Outdated Show resolved Hide resolved

stevenroose force-pushed the txhash branch from 245c0b9 to 52df2b8 Compare March 27, 2024 23:45

stevenroose mentioned this pull request Mar 28, 2024

Add OP_TXHASH and OP_CHECKTXHASHVERIFY bitcoin-inquisition/bitcoin#51

Closed

jonatack added the New BIP label Apr 23, 2024

murchandamus added the PR Author action required Needs updates, has unaddressed review comments, or is otherwise waiting for PR author label May 8, 2024

murchandamus reviewed May 8, 2024

View reviewed changes

bip-txhash.md Outdated Show resolved Hide resolved

0xBEEFCAF3 reviewed May 21, 2024

View reviewed changes

bip-txhash.md Outdated Show resolved Hide resolved

ProofOfKeags reviewed May 22, 2024

View reviewed changes

jonatack changed the title ~~Add BIP for OP_TXHASH and OP_CHECKTXHASHVERIFY~~ BIP draft: OP_TXHASH and OP_CHECKTXHASHVERIFY Dec 17, 2024

murchandamus changed the title ~~BIP draft: OP_TXHASH and OP_CHECKTXHASHVERIFY~~ BIP346: OP_TXHASH and OP_CHECKTXHASHVERIFY Dec 26, 2024

stevenroose force-pushed the txhash branch 2 times, most recently from da2154e to fd6b0d2 Compare January 14, 2025 15:46

jonatack removed the PR Author action required Needs updates, has unaddressed review comments, or is otherwise waiting for PR author label Jan 14, 2025

Add BIP for OP_TXHASH and OP_CHECKTXHASHVERIFY

debd349

stevenroose force-pushed the txhash branch from fd6b0d2 to debd349 Compare January 14, 2025 15:57

bucko13 mentioned this pull request Jan 15, 2025

January Topics austinbitdevs/austinbitdevs.com#48

Open

john-light mentioned this pull request Jan 15, 2025

Public launch TODO john-light/softforks.org#7

Open

12 tasks

murchandamus reviewed Jan 16, 2025

View reviewed changes


		For both inputs and then outputs, do the following:

		* If the "in/outputs" field is set to 1, another additional byte is expected:


		# Detailed Specification

		A reference implementation in Rust is provided attached as part of this BIP

	Author: Steven Roose <[email protected]>, Brandon Black <[email protected]>
	Author: Steven Roose <[email protected]>
	Brandon Black <[email protected]>

		(`0xb3`) as a soft fork upgrade. This opcode is active in both legacy script,
		segwitv0 p2wsh and tapscript contexts.

	* The TxFieldSelector valid, fail otherwise.
	* The TxFieldSelector is valid, fail otherwise.

		`<txfs> OP_TXHASH <pubkey> OP_CHECKSIGFROMSTACK` effectively emulates `SIGHASH_ANYPREVOUT`.


		# Detailed Specification

BIP346: OP_TXHASH and OP_CHECKTXHASHVERIFY #1500

Are you sure you want to change the base?

BIP346: OP_TXHASH and OP_CHECKTXHASHVERIFY #1500

Conversation

stevenroose commented Sep 30, 2023 • edited Loading

Semantic changes

Implementations

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenroose commented Oct 23, 2023

stevenroose commented Dec 11, 2023

luke-jr left a comment

Choose a reason for hiding this comment

rustyrussell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenroose Jan 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenroose Jan 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenroose Jan 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef commented Apr 24, 2024

murchandamus left a comment

Choose a reason for hiding this comment

ProofOfKeags left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

murchandamus commented Nov 14, 2024

jonatack commented Dec 17, 2024

stevenroose commented Jan 14, 2025

murchandamus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenroose commented Sep 30, 2023 •

edited

Loading

stevenroose Jan 2, 2024 •

edited

Loading

stevenroose Jan 2, 2024 •

edited

Loading

stevenroose Jan 2, 2024 •

edited

Loading