path: root/40/9e40b42e43cafdb943bad3a87caafbcd55578b

Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
	helo=mx.sourceforge.net)
	by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <decker.christian@gmail.com>) id 1Ysb1o-0008LG-91
	for bitcoin-development@lists.sourceforge.net;
	Wed, 13 May 2015 18:05:08 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.215.51 as permitted sender)
	client-ip=209.85.215.51;
	envelope-from=decker.christian@gmail.com;
	helo=mail-la0-f51.google.com; 
Received: from mail-la0-f51.google.com ([209.85.215.51])
	by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1Ysb1h-0007pM-Vq
	for bitcoin-development@lists.sourceforge.net;
	Wed, 13 May 2015 18:05:08 +0000
Received: by lagv1 with SMTP id v1so35265938lag.3
	for <bitcoin-development@lists.sourceforge.net>;
	Wed, 13 May 2015 11:04:55 -0700 (PDT)
X-Received: by 10.152.29.198 with SMTP id m6mr98670lah.11.1431540295600; Wed,
	13 May 2015 11:04:55 -0700 (PDT)
MIME-Version: 1.0
References: <CALxbBHUnt7ToVK9reH6W6uT4HV=7NbxGHyNWWa-OEHg+Z1+qOg@mail.gmail.com>
	<CAPg+sBggj382me1ATDx4SS9KHVfvX5KH7ZhLHN6B+2_a+Emw1Q@mail.gmail.com>
In-Reply-To: <CAPg+sBggj382me1ATDx4SS9KHVfvX5KH7ZhLHN6B+2_a+Emw1Q@mail.gmail.com>
From: Christian Decker <decker.christian@gmail.com>
Date: Wed, 13 May 2015 18:04:54 +0000
Message-ID: <CALxbBHU-0huAs_y3cZCfmKKAAq3LHut8DwdSGm+1Rym3pb9j2A@mail.gmail.com>
To: Pieter Wuille <pieter.wuille@gmail.com>
Content-Type: multipart/alternative; boundary=089e0158c7dccd11350515fa7223
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	(decker.christian[at]gmail.com)
	-0.0 SPF_PASS               SPF: sender matches SPF record
	1.0 HTML_MESSAGE           BODY: HTML included in message
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Ysb1h-0007pM-Vq
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] [BIP] Normalized Transaction IDs
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 13 May 2015 18:05:08 -0000

--089e0158c7dccd11350515fa7223
Content-Type: text/plain; charset=UTF-8

If the inputs to my transaction have been long confirmed I can be
reasonably safe in assuming that the transaction hash does not change
anymore. It's true that I have to be careful not to build on top of
transactions that use legacy references to transactions that are
unconfirmed or have few confirmations, however that does not invalidate the
utility of the normalized transaction IDs.

The resource doubling is not optimal, I agree, but compare that to dragging
around malleability and subsequent hacks to sort-of fix it forever.
Additionally if we were to decide to abandon legacy transaction IDs we
could eventually drop the legacy index after a sufficient transition period.

I remember reading about the SIGHASH proposal somewhere. It feels really
hackish to me: It is a substantial change to the way signatures are
verified, I cannot really see how this is a softfork if clients that did
not update are unable to verify transactions using that SIGHASH Flag and it
is adding more data (the normalized hash) to the script, which has to be
stored as part of the transaction. It may be true that a node observing
changes in the input transactions of a transaction using this flag could
fix the problem, however it requires the node's intervention.

Compare that to the simple and clean solution in the proposal, which does
not add extra data to be stored, keeps the OP_*SIG* semantics as they are
and where once you sign a transaction it does not have to be monitored or
changed in order to be valid.

There certainly are merits using the SIGHASH approach in the short term (it
does not require a hard fork), however I think the normalized transaction
ID is a cleaner and simpler long-term solution, even though it requires a
hard-fork.

Regards,
Christian

On Wed, May 13, 2015 at 7:14 PM Pieter Wuille <pieter.wuille@gmail.com>
wrote:

> Normalized transaction ids are only effectively non-malleable when all
> inputs they refer to are also non-malleable (or you can have malleability
> in 2nd level dependencies), so I do not believe it makes sense to allow
> mixed usage of the txids at all. They do not provide the actual benefit of
> guaranteed non-malleability before it becomes disallowed to use the old
> mechanism. That, together with the +- resource doubling needed for the UTXO
> set (as earlier mentioned) and the fact that an alternative which is only a
> softfork are available, makes this a bad idea IMHO.
>
> Unsure to what extent this has been presented on the mailinglist, but the
> softfork idea is this:
> * Transactions get 2 txids, one used to reference them (computed as
> before), and one used in an (extended) sighash.
> * The txins keep using the normal txid, so not structural changes to
> Bitcoin.
> * The ntxid is computed by replacing the scriptSigs in inputs by the empty
> string, and by replacing the txids in txins by their corresponding ntxids.
> * A new checksig operator is softforked in, which uses the ntxids in its
> sighashes rather than the full txid.
> * To support efficiently computing ntxids, every tx in the utxo set
> (currently around 6M) stores the ntxid, but only supports lookup bu txid
> still.
>
> This does result in a system where a changed dependency indeed invalidates
> the spending transaction, but the fix is trivial and can be done without
> access to the private key.
> On May 13, 2015 5:50 AM, "Christian Decker" <decker.christian@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I'd like to propose a BIP to normalize transaction IDs in order to
>> address transaction malleability and facilitate higher level protocols.
>>
>> The normalized transaction ID is an alias used in parallel to the current
>> (legacy) transaction IDs to address outputs in transactions. It is
>> calculated by removing (zeroing) the scriptSig before computing the hash,
>> which ensures that only data whose integrity is also guaranteed by the
>> signatures influences the hash. Thus if anything causes the normalized ID
>> to change it automatically invalidates the signature. When validating a
>> client supporting this BIP would use both the normalized tx ID as well as
>> the legacy tx ID when validating transactions.
>>
>> The detailed writeup can be found here:
>> https://github.com/cdecker/bips/blob/normalized-txid/bip-00nn.mediawiki.
>>
>> @gmaxwell: I'd like to request a BIP number, unless there is something
>> really wrong with the proposal.
>>
>> In addition to being a simple alternative that solves transaction
>> malleability it also hugely simplifies higher level protocols. We can now
>> use template transactions upon which sequences of transactions can be built
>> before signing them.
>>
>> I hesitated quite a while to propose it since it does require a hardfork
>> (old clients would not find the prevTx identified by the normalized
>> transaction ID and deem the spending transaction invalid), but it seems
>> that hardforks are no longer the dreaded boogeyman nobody talks about.
>> I left out the details of how the hardfork is to be done, as it does not
>> really matter and we may have a good mechanism to apply a bunch of
>> hardforks concurrently in the future.
>>
>> I'm sure it'll take time to implement and upgrade, but I think it would
>> be a nice addition to the functionality and would solve a long standing
>> problem :-)
>>
>> Please let me know what you think, the proposal is definitely not set in
>> stone at this point and I'm sure we can improve it further.
>>
>> Regards,
>> Christian
>>
>>
>> ------------------------------------------------------------------------------
>> One dashboard for servers and applications across Physical-Virtual-Cloud
>> Widest out-of-the-box monitoring support with 50+ applications
>> Performance metrics, stats and reports that give you Actionable Insights
>> Deep dive visibility with transaction tracing using APM Insight.
>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>

--089e0158c7dccd11350515fa7223
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">If the inputs to my transaction have been long confirmed I=
 can be reasonably safe in assuming that the transaction hash does not chan=
ge anymore. It&#39;s true that I have to be careful not to build on top of =
transactions that use legacy references to transactions that are unconfirme=
d or have few confirmations, however that does not invalidate the utility o=
f the normalized transaction IDs.=C2=A0<div><br></div><div>The resource dou=
bling is not optimal, I agree, but compare that to dragging around malleabi=
lity and subsequent hacks to sort-of fix it forever. Additionally if we wer=
e to decide to abandon legacy transaction IDs we could eventually drop the =
legacy index after a sufficient transition period.</div><div><br></div><div=
>I remember reading about the SIGHASH proposal somewhere. It feels really h=
ackish to me: It is a substantial change to the way signatures are verified=
, I cannot really see how this is a softfork if clients that did not update=
 are unable to verify transactions using that SIGHASH Flag and it is adding=
 more data (the normalized hash) to the script, which has to be stored as p=
art of the transaction. It may be true that a node observing changes in the=
 input transactions of a transaction using this flag could fix the problem,=
 however it requires the node&#39;s intervention.</div><div><br></div><div>=
Compare that to the simple and clean solution in the proposal, which does n=
ot add extra data to be stored, keeps the OP_*SIG* semantics as they are an=
d where once you sign a transaction it does not have to be monitored or cha=
nged in order to be valid.</div><div><br></div><div>There certainly are mer=
its using the SIGHASH approach in the short term (it does not require a har=
d fork), however I think the normalized transaction ID is a cleaner and sim=
pler long-term solution, even though it requires a hard-fork.</div><div><br=
></div><div>Regards,</div><div>Christian</div><div dir=3D"ltr"><div><br><di=
v class=3D"gmail_quote">On Wed, May 13, 2015 at 7:14 PM Pieter Wuille &lt;<=
a href=3D"mailto:pieter.wuille@gmail.com" target=3D"_blank">pieter.wuille@g=
mail.com</a>&gt; wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir=3D"ltr">No=
rmalized transaction ids are only effectively non-malleable when all inputs=
 they refer to are also non-malleable (or you can have malleability in 2nd =
level dependencies), so I do not believe it makes sense to allow mixed usag=
e of the txids at all. They do not provide the actual benefit of guaranteed=
 non-malleability before it becomes disallowed to use the old mechanism. Th=
at, together with the +- resource doubling needed for the UTXO set (as earl=
ier mentioned) and the fact that an alternative which is only a softfork ar=
e available, makes this a bad idea IMHO.</p>
<p dir=3D"ltr">Unsure to what extent this has been presented on the mailing=
list, but the softfork idea is this:<br>
* Transactions get 2 txids, one used to reference them (computed as before)=
, and one used in an (extended) sighash.<br>
* The txins keep using the normal txid, so not structural changes to Bitcoi=
n.<br>
* The ntxid is computed by replacing the scriptSigs in inputs by the empty =
string, and by replacing the txids in txins by their corresponding ntxids.<=
br>
* A new checksig operator is softforked in, which uses the ntxids in its si=
ghashes rather than the full txid.<br>
* To support efficiently computing ntxids, every tx in the utxo set (curren=
tly around 6M) stores the ntxid, but only supports lookup bu txid still.</p=
>
<p dir=3D"ltr">This does result in a system where a changed dependency inde=
ed invalidates the spending transaction, but the fix is trivial and can be =
done without access to the private key.</p>
<div class=3D"gmail_quote"></div><div class=3D"gmail_quote">On May 13, 2015=
 5:50 AM, &quot;Christian Decker&quot; &lt;<a href=3D"mailto:decker.christi=
an@gmail.com" target=3D"_blank">decker.christian@gmail.com</a>&gt; wrote:<b=
r type=3D"attribution"></div><div class=3D"gmail_quote"><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex"><div dir=3D"ltr"><div>Hi All,</div><div><br></div>I&#39;d lik=
e to propose a BIP to normalize transaction IDs in order to address transac=
tion malleability and facilitate higher level protocols.<div><br></div><div=
>The normalized transaction ID is an alias used in parallel to the current =
(legacy) transaction IDs to address outputs in transactions. It is calculat=
ed by removing (zeroing) the scriptSig before computing the hash, which ens=
ures that only data whose integrity is also guaranteed by the signatures in=
fluences the hash. Thus if anything causes the normalized ID to change it a=
utomatically invalidates the signature. When validating a client supporting=
 this BIP would use both the normalized tx ID as well as the legacy tx ID w=
hen validating transactions.</div><div><br></div><div>The detailed writeup =
can be found here:=C2=A0<a href=3D"https://github.com/cdecker/bips/blob/nor=
malized-txid/bip-00nn.mediawiki" target=3D"_blank">https://github.com/cdeck=
er/bips/blob/normalized-txid/bip-00nn.mediawiki</a>.</div><div><br></div><d=
iv>@gmaxwell: I&#39;d like to request a BIP number, unless there is somethi=
ng really wrong with the proposal.</div><div><br></div><div>In addition to =
being a simple alternative that solves transaction malleability it also hug=
ely simplifies higher level protocols. We can now use template transactions=
 upon which sequences of transactions can be built before signing them.</di=
v><div><br></div><div>I hesitated quite a while to propose it since it does=
 require a hardfork (old clients would not find the prevTx identified by th=
e normalized transaction ID and deem the spending transaction invalid), but=
 it seems that hardforks are no longer the dreaded boogeyman nobody talks a=
bout.</div><div>I left out the details of how the hardfork is to be done, a=
s it does not really matter and we may have a good mechanism to apply a bun=
ch of hardforks concurrently in the future.</div><div><br></div><div>I&#39;=
m sure it&#39;ll take time to implement and upgrade, but I think it would b=
e a nice addition to the functionality and would solve a long standing prob=
lem :-)</div><div><br></div><div>Please let me know what you think, the pro=
posal is definitely not set in stone at this point and I&#39;m sure we can =
improve it further.</div><div><br></div><div>Regards,</div><div>Christian</=
div></div>
<br></blockquote></div><div class=3D"gmail_quote"><blockquote class=3D"gmai=
l_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left=
:1ex">---------------------------------------------------------------------=
---------<br>
One dashboard for servers and applications across Physical-Virtual-Cloud<br=
>
Widest out-of-the-box monitoring support with 50+ applications<br>
Performance metrics, stats and reports that give you Actionable Insights<br=
>
Deep dive visibility with transaction tracing using APM Insight.<br>
<a href=3D"http://ad.doubleclick.net/ddm/clk/290420510;117567292;y" style=
=3D"display:none!important" target=3D"_blank">http://ad.doubleclick.net/ddm=
/clk/290420510;117567292;y</a><br>_________________________________________=
______<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net" target=3D"_bla=
nk">Bitcoin-development@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div></blockquote></div></div></div></div>

--089e0158c7dccd11350515fa7223--