Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Ysc7W-0001A1-Rp for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 19:15:06 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.215.50 as permitted sender) client-ip=209.85.215.50; envelope-from=decker.christian@gmail.com; helo=mail-la0-f50.google.com; Received: from mail-la0-f50.google.com ([209.85.215.50]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Ysc7U-00051g-VK for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 19:15:06 +0000 Received: by lagv1 with SMTP id v1so36980957lag.3 for ; Wed, 13 May 2015 12:14:58 -0700 (PDT) X-Received: by 10.152.29.198 with SMTP id m6mr300443lah.11.1431544498460; Wed, 13 May 2015 12:14:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Christian Decker Date: Wed, 13 May 2015 19:14:57 +0000 Message-ID: To: Pieter Wuille Content-Type: multipart/alternative; boundary=089e0158c7dc4fa0c70515fb6d15 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (decker.christian[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Ysc7U-00051g-VK Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] [BIP] Normalized Transaction IDs X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 19:15:06 -0000 --089e0158c7dc4fa0c70515fb6d15 Content-Type: text/plain; charset=UTF-8 On Wed, May 13, 2015 at 8:40 PM Pieter Wuille wrote: > On Wed, May 13, 2015 at 11:04 AM, Christian Decker < > decker.christian@gmail.com> wrote: > >> If the inputs to my transaction have been long confirmed I can be >> reasonably safe in assuming that the transaction hash does not change >> anymore. It's true that I have to be careful not to build on top of >> transactions that use legacy references to transactions that are >> unconfirmed or have few confirmations, however that does not invalidate the >> utility of the normalized transaction IDs. >> > > Sufficient confirmations help of course, but make systems like this less > useful for more complex interactions where you have multiple unconfirmed > transactions waiting on each other. I think being able to rely on this > problem being solved unconditionally is what makes the proposal attractive. > For the simple cases, see BIP62. > If we are building a long running contract using a complex chain of transactions, or multiple transactions that depend on each other, there is no point in ever using any malleable legacy transaction IDs and I would simply stop cooperating if you tried. I don't think your argument applies. If we build our contract using only normalized transaction IDs there is no way of suffering any losses due to malleability. The reason I mentioned the confirmation is that all protocols I can think of start by collaboratively creating a transaction that locks in funds into a multisig output, that is committed to the blockchain. Starting from this initial setup transaction would be using normalized transaction IDs, therefore not be susceptible to malleability. > > I remember reading about the SIGHASH proposal somewhere. It feels really >> hackish to me: It is a substantial change to the way signatures are >> verified, I cannot really see how this is a softfork if clients that did >> not update are unable to verify transactions using that SIGHASH Flag and it >> is adding more data (the normalized hash) to the script, which has to be >> stored as part of the transaction. It may be true that a node observing >> changes in the input transactions of a transaction using this flag could >> fix the problem, however it requires the node's intervention. >> > > I think you misunderstand the idea. This is related, but orthogonal to the > ideas about extended the sighash flags that have been discussed here before. > > All it's doing is adding a new CHECKSIG operator to script, which, in its > internally used signature hash, 1) removes the scriptSigs from transactions > before hashing 2) replaces the txids in txins by their ntxid. It does not > add any data to transactions, and it is a softfork, because it only impacts > scripts which actually use the new CHECKSIG operator. Wallets that don't > support signing with this new operator would not give out addresses that > use it. > In that case I don't think I heard this proposal before, and I might be missing out :-) So if transaction B spends an output from A, then the input from B contains the CHECKSIG operator telling the validating client to do what exactly? It appears that it wants us to go and fetch A, normalize it, put the normalized hash in the txIn of B and then continue the validation? Wouldn't that also need a mapping from the normalized transaction ID to the legacy transaction ID that was confirmed? A client that did not update still would have no clue on how to handle these transactions, since it simply does not understand the CHECKSIG operator. If such a transaction ends up in a block I cannot even catch up with the network since the transaction does not validate for me. Could you provide an example of how this works? > >> Compare that to the simple and clean solution in the proposal, which does >> not add extra data to be stored, keeps the OP_*SIG* semantics as they are >> and where once you sign a transaction it does not have to be monitored or >> changed in order to be valid. >> > > OP_*SIG* semantics don't change here either, we're just adding a superior > opcode (which in most ways behaves the same as the existing operators). I > agree with the advantage of not needing to monitor transactions afterwards > for malleated inputs, but I think you underestimate the deployment costs. > If you want to upgrade the world (eventually, after the old index is > dropped, which is IMHO the only point where this proposal becomes superior > to the alternatives) to this, you're changing *every single piece of > Bitcoin software on the planet*. This is not just changing some validation > rules that are opt-in to use, you're fundamentally changing how > transactions refer to each other. > As I mentioned before, this is a really long term strategy, hoping to get the cleanest and easiest solution, so that we do not further complicate the inner workings of Bitcoin. I don't think that it is completely out of question to eventually upgrade to use normalized transactions, after all the average lifespan of hardware is a few years tops. > > Also, what do blocks commit to? Do you keep using the old transaction ids > for this? Because if you don't, any relayer on the network can invalidate a > block (and have the receiver mark it as invalid) by changing the txids. You > need to somehow commit to the scriptSig data in blocks still so the POW of > a block is invalidated by changing a scriptSig. > How could I change the transaction IDs if I am a relayer? The miner decides which flavor of IDs it is adding into its merkle tree, the block hash locks in the choice. If we saw a transaction having a valid sigScript, it does not matter how we reference it in the block. > > There certainly are merits using the SIGHASH approach in the short term >> (it does not require a hard fork), however I think the normalized >> transaction ID is a cleaner and simpler long-term solution, even though it >> requires a hard-fork. >> > > It requires a hard fork, but more importantly, it requires the whole world > to change their software (not just validation code) to effectively use it. > That, plus large up-front deployment costs (doubling the cache size for > every full node for the same propagation speed is not a small thing) which > may not end up being effective. > Yes, hard forks are hard, I'm under no illusion that pushing such a change through takes time, but in the end the advantages will prevail. I didn't want to put it in the initial proposal, but we could also increase the transaction version which signals to the client that the transaction may only be referenced by the normalized transaction ID. So every transaction would be either in one index or the other, reducing the deployment cost to almost nothing. > > -- > Pieter > --089e0158c7dc4fa0c70515fb6d15 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, May 13, 2015 at 8:40 PM Pieter Wuille <pieter.wuille@gmail.com> wrote:
On Wed, May 13, 2015 at 11:04 AM, Ch= ristian Decker <decker.christian@gmail.com> wrote:<= br>
If the inputs to my tr= ansaction have been long confirmed I can be reasonably safe in assuming tha= t the transaction hash does not change anymore. It's true that I have t= o be careful not to build on top of transactions that use legacy references= to transactions that are unconfirmed or have few confirmations, however th= at does not invalidate the utility of the normalized transaction IDs.=C2=A0=

Sufficient confirmatio= ns help of course, but make systems like this less useful for more complex = interactions where you have multiple unconfirmed transactions waiting on ea= ch other. I think being able to rely on this problem being solved unconditi= onally is what makes the proposal attractive. For the simple cases, see BIP= 62.

If we are b= uilding a long running contract using a complex chain of transactions, or m= ultiple transactions that depend on each other, there is no point in ever u= sing any malleable legacy transaction IDs and I would simply stop cooperati= ng if you tried. I don't think your argument applies. If we build our c= ontract using only normalized transaction IDs there is no way of suffering = any losses due to malleability.

The reason I menti= oned the confirmation is that all protocols I can think of start by collabo= ratively creating a transaction that locks in funds into a multisig output,= that is committed to the blockchain. Starting from this initial setup tran= saction would be using normalized transaction IDs, therefore not be suscept= ible to malleability.
=C2=A0
=

I remember reading about the SIGHASH proposal somewhere. It feels really= hackish to me: It is a substantial change to the way signatures are verifi= ed, I cannot really see how this is a softfork if clients that did not upda= te are unable to verify transactions using that SIGHASH Flag and it is addi= ng more data (the normalized hash) to the script, which has to be stored as= part of the transaction. It may be true that a node observing changes in t= he input transactions of a transaction using this flag could fix the proble= m, however it requires the node's intervention.

I think you misunderstand the idea. This= is related, but orthogonal to the ideas about extended the sighash flags t= hat have been discussed here before.

All it's doing i= s adding a new CHECKSIG operator to script, which, in its internally used s= ignature hash, 1) removes the scriptSigs from transactions before hashing 2= ) replaces the txids in txins by their ntxid. It does not add any data to t= ransactions, and it is a softfork, because it only impacts scripts which ac= tually use the new CHECKSIG operator. Wallets that don't support signin= g with this new operator would not give out addresses that use it.

In that case I don'= t think I heard this proposal before, and I might be missing out :-)
<= div>So if transaction B spends an output from A, then the input from B cont= ains the CHECKSIG operator telling the validating client to do what exactly= ? It appears that it wants us to go and fetch A, normalize it, put the norm= alized hash in the txIn of B and then continue the validation? Wouldn't= that also need a mapping from the normalized transaction ID to the legacy = transaction ID that was confirmed?

A client that d= id not update still would have no clue on how to handle these transactions,= since it simply does not understand the CHECKSIG operator. If such a trans= action ends up in a block I cannot even catch up with the network since the= transaction does not validate for me.

Could you p= rovide an example of how this works?=C2=A0
=C2=A0
=

Compare that to the simple and clean s= olution in the proposal, which does not add extra data to be stored, keeps = the OP_*SIG* semantics as they are and where once you sign a transaction it= does not have to be monitored or changed in order to be valid.
=

OP_*SIG* semantics don'= ;t change here either, we're just adding a superior opcode (which in mo= st ways behaves the same as the existing operators). I agree with the advan= tage of not needing to monitor transactions afterwards for malleated inputs= , but I think you underestimate the deployment costs. If you want to upgrad= e the world (eventually, after the old index is dropped, which is IMHO the = only point where this proposal becomes superior to the alternatives) to thi= s, you're changing *every single piece of Bitcoin software on the plane= t*. This is not just changing some validation rules that are opt-in to use,= you're fundamentally changing how transactions refer to each other.

As I mentioned be= fore, this is a really long term strategy, hoping to get the cleanest and e= asiest solution, so that we do not further complicate the inner workings of= Bitcoin. I don't think that it is completely out of question to eventu= ally upgrade to use normalized transactions, after all the average lifespan= of hardware is a few years tops.
=C2=A0

Also, what do blocks commit to? Do you keep u= sing the old transaction ids for this? Because if you don't, any relaye= r on the network can invalidate a block (and have the receiver mark it as i= nvalid) by changing the txids. You need to somehow commit to the scriptSig = data in blocks still so the POW of a block is invalidated by changing a scr= iptSig.

How cou= ld I change the transaction IDs if I am a relayer? The miner decides which = flavor of IDs it is adding into its merkle tree, the block hash locks in th= e choice. If we saw a transaction having a valid sigScript, it does not mat= ter how we reference it in the block.
=C2=A0

There certainly are merits using the SIGHASH approach in= the short term (it does not require a hard fork), however I think the norm= alized transaction ID is a cleaner and simpler long-term solution, even tho= ugh it requires a hard-fork.

It requires a hard fork, but more importantly, it requires = the whole world to change their software (not just validation code) to effe= ctively use it. That, plus large up-front deployment costs (doubling the ca= che size for every full node for the same propagation speed is not a small = thing) which may not end up being effective.

Yes, hard forks are hard, I'm under no il= lusion that pushing such a change through takes time, but in the end the ad= vantages will prevail.

I didn't want to put it= in the initial proposal, but we could also increase the transaction versio= n which signals to the client that the transaction may only be referenced b= y the normalized transaction ID. So every transaction would be either in on= e index or the other, reducing the deployment cost to almost nothing.
=
=C2=A0

--
Pieter
--089e0158c7dc4fa0c70515fb6d15--