Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Yuctj-0001PC-1C for bitcoin-development@lists.sourceforge.net; Tue, 19 May 2015 08:29:11 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.215.43 as permitted sender) client-ip=209.85.215.43; envelope-from=decker.christian@gmail.com; helo=mail-la0-f43.google.com; Received: from mail-la0-f43.google.com ([209.85.215.43]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Yucte-00046R-0q for bitcoin-development@lists.sourceforge.net; Tue, 19 May 2015 08:29:11 +0000 Received: by lagr1 with SMTP id r1so11810731lag.0 for ; Tue, 19 May 2015 01:28:59 -0700 (PDT) X-Received: by 10.112.139.198 with SMTP id ra6mr3258249lbb.15.1432024139618; Tue, 19 May 2015 01:28:59 -0700 (PDT) MIME-Version: 1.0 References: <5555C26F.7080706@sky-ip.org> In-Reply-To: From: Christian Decker Date: Tue, 19 May 2015 08:28:58 +0000 Message-ID: To: Stephen , "s7r@sky-ip.org" Content-Type: multipart/alternative; boundary=001a11c3474e26df3405166b1a4b X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (decker.christian[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Yucte-00046R-0q Cc: Bitcoin Development Subject: Re: [Bitcoin-development] [BIP] Normalized Transaction IDs X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 08:29:11 -0000 --001a11c3474e26df3405166b1a4b Content-Type: text/plain; charset=UTF-8 Thanks Stephen, I hadn't thought about BIP 34 and we need to address this in both proposals. If we can avoid it I'd like not to have one transaction hashed one way and other transactions in another way. Since BIP 34 explicitly uses the scriptSig to make the coinbase transaction unique, simply removing the scriptSig is not an option as it would potentially cause collisions. I don't remember why the scriptSig was chosen, but we also have the option of putting the blockchain height in the sequence number of the coinbase input or the locktime of the transaction, restoring the uniqueness constraint in normalized transaction IDs (for both proposals). Is there a specific reason why that was not chosen at the time? On Sat, May 16, 2015 at 5:58 AM Stephen wrote: > We should make sure to consider how BIP34 affects normalized transaction > ids, since the height of the block is included in the scriptSig ensuring > that the txid will be different. We wouldn't want to enable replay attacks > in the form of spending coinbase outputs in the same way they were spent > from a previous block. > > So maybe normalized txids should strip the scriptSigs of all transactions > except for coinbase transactions? This seems to make sense, since coinbase > transactions are inherently not malleable anyway. > > Also, s7r linked to my 'Build your own nHashType' proposal (although V2 is > here: > https://github.com/scmorse/bitcoin-misc/blob/master/sighash_proposal_v2.md). > I just wanted to add that I think even with normalized ids, it could still > be useful to be able to apply these flags to choose which parts of the > transaction become signed. I've also seen vague references to some kind of > a merklized abstract syntax tree, but am not fully sure how that would > work. Maybe someone on here could explain it? > > Best, > Stephen > > > > > On May 15, 2015, at 5:54 AM, s7r wrote: > > > > Hello, > > > > How will this exactly be safe against: > > a) the malleability of the parent tx (2nd level malleability) > > b) replays > > > > If you strip just the scriptSig of the input(s), the txid(s) can still > > be mutated (with higher probability before it gets confirmed). > > > > If you strip both the scriptSig of the parent and the txid, nothing can > > any longer be mutated but this is not safe against replays. This could > > work if we were using only one scriptPubKey per tx. But this is not > > enforced, and I don't think it's the proper way to do it. > > > > Something similar can be achieved if you would use a combination of > > flags from here: > > > > https://github.com/scmorse/bitcoin-misc/blob/master/sighash_proposal.md > > > > But this has some issues too. > > > > I've read your draft but didn't understand how exactly will this prevent > > normal malleability as we know it, second level malleability and replays > > as well as how will we do the transition into mapping the txes in the > > blockchain to normalized txids. Looking forward to read more on this > > topic. Thanks for the brainstorming ;) > > > > > >> On 5/13/2015 3:48 PM, Christian Decker wrote: > >> Hi All, > >> > >> I'd like to propose a BIP to normalize transaction IDs in order to > >> address transaction malleability and facilitate higher level protocols. > >> > >> The normalized transaction ID is an alias used in parallel to the > >> current (legacy) transaction IDs to address outputs in transactions. It > >> is calculated by removing (zeroing) the scriptSig before computing the > >> hash, which ensures that only data whose integrity is also guaranteed by > >> the signatures influences the hash. Thus if anything causes the > >> normalized ID to change it automatically invalidates the signature. When > >> validating a client supporting this BIP would use both the normalized tx > >> ID as well as the legacy tx ID when validating transactions. > >> > >> The detailed writeup can be found > >> here: > https://github.com/cdecker/bips/blob/normalized-txid/bip-00nn.mediawiki. > >> > >> @gmaxwell: I'd like to request a BIP number, unless there is something > >> really wrong with the proposal. > >> > >> In addition to being a simple alternative that solves transaction > >> malleability it also hugely simplifies higher level protocols. We can > >> now use template transactions upon which sequences of transactions can > >> be built before signing them. > >> > >> I hesitated quite a while to propose it since it does require a hardfork > >> (old clients would not find the prevTx identified by the normalized > >> transaction ID and deem the spending transaction invalid), but it seems > >> that hardforks are no longer the dreaded boogeyman nobody talks about. > >> I left out the details of how the hardfork is to be done, as it does not > >> really matter and we may have a good mechanism to apply a bunch of > >> hardforks concurrently in the future. > >> > >> I'm sure it'll take time to implement and upgrade, but I think it would > >> be a nice addition to the functionality and would solve a long standing > >> problem :-) > >> > >> Please let me know what you think, the proposal is definitely not set in > >> stone at this point and I'm sure we can improve it further. > >> > >> Regards, > >> Christian > >> > >> > >> > ------------------------------------------------------------------------------ > >> One dashboard for servers and applications across Physical-Virtual-Cloud > >> Widest out-of-the-box monitoring support with 50+ applications > >> Performance metrics, stats and reports that give you Actionable Insights > >> Deep dive visibility with transaction tracing using APM Insight. > >> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > >> > >> > >> > >> _______________________________________________ > >> Bitcoin-development mailing list > >> Bitcoin-development@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > > One dashboard for servers and applications across Physical-Virtual-Cloud > > Widest out-of-the-box monitoring support with 50+ applications > > Performance metrics, stats and reports that give you Actionable Insights > > Deep dive visibility with transaction tracing using APM Insight. > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > --001a11c3474e26df3405166b1a4b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Stephen, I hadn't thought about BIP 34 and we = need to address this in both proposals.=C2=A0If we can avoid it I&#= 39;d like not to have one transaction hashed one way and other transactions= in another way.

Since BIP 34 explicitly= uses the scriptSig to make the coinbase transaction unique, simply removin= g the scriptSig is not an option as it would potentially cause collisions. = I don't remember why the scriptSig was chosen, but we also have the opt= ion of putting the blockchain height in the sequence number of the coinbase= input or the locktime of the transaction, restoring the uniqueness constra= int in normalized transaction IDs (for both proposals). Is there a specific= reason why that was not chosen at the time?

On Sat, May 16, 2015 at 5:58 AM Stephen <stephencalebmorse@gmail.com> wrote:
We should make sure to consider how BIP34 affe= cts normalized transaction ids, since the height of the block is included i= n the scriptSig ensuring that the txid will be different. We wouldn't w= ant to enable replay attacks in the form of spending coinbase outputs in th= e same way they were spent from a previous block.

So maybe normalized txids should strip the scriptSigs of all transactions e= xcept for coinbase transactions? This seems to make sense, since coinbase t= ransactions are inherently not malleable anyway.

Also, s7r linked to my 'Build your own nHashType' proposal (althoug= h V2 is here: https://github.com/scmorse/bitc= oin-misc/blob/master/sighash_proposal_v2.md). I just wanted to add that= I think even with normalized ids, it could still be useful to be able to a= pply these flags to choose which parts of the transaction become signed. I&= #39;ve also seen vague references to some kind of a merklized abstract synt= ax tree, but am not fully sure how that would work. Maybe someone on here c= ould explain it?

Best,
Stephen



> On May 15, 2015, at 5:54 AM, s7r <s7r@sky-ip.org> wrote:
>
> Hello,
>
> How will this exactly be safe against:
> a) the malleability of the parent tx (2nd level malleability)
> b) replays
>
> If you strip just the scriptSig of the input(s), the txid(s) can still=
> be mutated (with higher probability before it gets confirmed).
>
> If you strip both the scriptSig of the parent and the txid, nothing ca= n
> any longer be mutated but this is not safe against replays. This could=
> work if we were using only one scriptPubKey per tx. But this is not > enforced, and I don't think it's the proper way to do it.
>
> Something similar can be achieved if you would use a combination of > flags from here:
>
> https://github.com/scmorse/bitcoin-misc/blo= b/master/sighash_proposal.md
>
> But this has some issues too.
>
> I've read your draft but didn't understand how exactly will th= is prevent
> normal malleability as we know it, second level malleability and repla= ys
> as well as how will we do the transition into mapping the txes in the<= br> > blockchain to normalized txids. Looking forward to read more on this > topic. Thanks for the brainstorming ;)
>
>
>> On 5/13/2015 3:48 PM, Christian Decker wrote:
>> Hi All,
>>
>> I'd like to propose a BIP to normalize transaction IDs in orde= r to
>> address transaction malleability and facilitate higher level proto= cols.
>>
>> The normalized transaction ID is an alias used in parallel to the<= br> >> current (legacy) transaction IDs to address outputs in transaction= s. It
>> is calculated by removing (zeroing) the scriptSig before computing= the
>> hash, which ensures that only data whose integrity is also guarant= eed by
>> the signatures influences the hash. Thus if anything causes the >> normalized ID to change it automatically invalidates the signature= . When
>> validating a client supporting this BIP would use both the normali= zed tx
>> ID as well as the legacy tx ID when validating transactions.
>>
>> The detailed writeup can be found
>> here: https://github.com/cdecker/bips/b= lob/normalized-txid/bip-00nn.mediawiki.
>>
>> @gmaxwell: I'd like to request a BIP number, unless there is s= omething
>> really wrong with the proposal.
>>
>> In addition to being a simple alternative that solves transaction<= br> >> malleability it also hugely simplifies higher level protocols. We = can
>> now use template transactions upon which sequences of transactions= can
>> be built before signing them.
>>
>> I hesitated quite a while to propose it since it does require a ha= rdfork
>> (old clients would not find the prevTx identified by the normalize= d
>> transaction ID and deem the spending transaction invalid), but it = seems
>> that hardforks are no longer the dreaded boogeyman nobody talks ab= out.
>> I left out the details of how the hardfork is to be done, as it do= es not
>> really matter and we may have a good mechanism to apply a bunch of=
>> hardforks concurrently in the future.
>>
>> I'm sure it'll take time to implement and upgrade, but I t= hink it would
>> be a nice addition to the functionality and would solve a long sta= nding
>> problem :-)
>>
>> Please let me know what you think, the proposal is definitely not = set in
>> stone at this point and I'm sure we can improve it further. >>
>> Regards,
>> Christian
>>
>>
>> ------------------------------------------------------------------= ------------
>> One dashboard for servers and applications across Physical-Virtual= -Cloud
>> Widest out-of-the-box monitoring support with 50+ applications
>> Performance metrics, stats and reports that give you Actionable In= sights
>> Deep dive visibility with transaction tracing using APM Insight. >> http://ad.doubleclick.= net/ddm/clk/290420510;117567292;y
>>
>>
>>
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/b= itcoin-development
>
> ----------------------------------------------------------------------= --------
> One dashboard for servers and applications across Physical-Virtual-Clo= ud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insigh= ts
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/= ddm/clk/290420510;117567292;y
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitco= in-development
--001a11c3474e26df3405166b1a4b--