Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1YsZMd-0001NL-1D for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 16:18:31 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.216.175 as permitted sender) client-ip=209.85.216.175; envelope-from=tier.nolan@gmail.com; helo=mail-qc0-f175.google.com; Received: from mail-qc0-f175.google.com ([209.85.216.175]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1YsZMb-00087O-QY for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 16:18:31 +0000 Received: by qcyk17 with SMTP id k17so25210818qcy.1 for ; Wed, 13 May 2015 09:18:24 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.140.150.209 with SMTP id 200mr29047295qhw.9.1431533904334; Wed, 13 May 2015 09:18:24 -0700 (PDT) Received: by 10.140.85.241 with HTTP; Wed, 13 May 2015 09:18:24 -0700 (PDT) In-Reply-To: References: Date: Wed, 13 May 2015 17:18:24 +0100 Message-ID: From: Tier Nolan Cc: Bitcoin Development Content-Type: multipart/alternative; boundary=001a11356f76da44e50515f8f57b X-Spam-Score: 2.3 (++) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (tier.nolan[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.2 MISSING_HEADERS Missing To: header 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 1.9 MALFORMED_FREEMAIL Bad headers on message from free email service -0.2 AWL AWL: Adjusted score from AWL reputation of From: address X-Headers-End: 1YsZMb-00087O-QY Subject: Re: [Bitcoin-development] [BIP] Normalized Transaction IDs X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 16:18:31 -0000 --001a11356f76da44e50515f8f57b Content-Type: text/plain; charset=UTF-8 On Wed, May 13, 2015 at 4:24 PM, Christian Decker < decker.christian@gmail.com> wrote > It does and I should have mentioned it in the draft, according to my > calculations a mapping legacy ID -> normalized ID is about 256 MB in size, > or at least it was at height 330'000, things might have changed a bit and > I'll recompute that. I omitted the deprecation of legacy IDs on purpose > since we don't know whether we will migrate completely or leave keep both > options viable. > There are around 20 million UTXOs. At 2*32 bytes per entry, that is more than 1GB. There are more UTXOs than transactions, but 256MB seems a little low. I think both IDs can be used in the merkle tree, since we lookup an ID in > both indices we can use both to address them and we will find them either > way. > The id that is used to sign should be used in the merkle tree. The hard fork should simply be to allow transactions that use the normalized transaction hash. > As for the opcodes I'll have to check, but I currently don't see how they > could be affected. > Agreed, the transaction is simply changed and all the standard rules apply. > We can certainly split the proposal should it get too large, for now it > seems manageable, since opcodes are not affected. > Right it is just a database update. The undo info also needs to be changed so that both txids are included. > Bloom-filtering is resolved by adding the normalized transaction IDs and > checking for both IDs in the filter. > Yeah, if a transaction spends with a legacy txid, it should still match if the normalized txid is included in the filter. > Since you mention bundling the change with other changes that require a hard-fork it might be a good idea to build a separate proposal for a generic hard-fork rollout mechanism. That would be useful. On the other hand, we don't want to make them to easy. I think this is a good choice for a hard fork test, since it is uncontroversial. With a time machine, it would have been done this way at the start. What about the following: The reference client is updated so that it uses version 2 transactions by default (but it can be changed by user). A pop-up could appear for the GUI. There is no other change. All transactions in blocks 375000 to 385000 are considered votes and weighted by bitcoin days destroyed (max 60 days). If > 75% of the transactions by weight are version 2, then the community are considered to support the hard fork. There would need to be a way to protect against miners censoring transactions/votes. Users could submit their transactions directly to a p2p tallying system. The coin would be aged based on the age in block 375000 unless included in the blockchain. These votes don't need to be ordered and multiple votes for the same coin would only count once. In fact, votes could just be based on holding in block X. This is an opinion poll rather than a referendum though. Assuming support of the community, the hard fork can then proceed in a similar way to the way a soft fork does. Devs update the reference client to produce version 4 blocks and version 3 transactions. Miners could watch version 3 transactions to gauge user interest and use that to help decide if they should update. If 750 of the last 1000 blocks are version 4 or higher, reject blocks with transactions of less than version 3 in version 4 blocks This means that legacy clients will be slow to confirm their transactions, since their transactions cannot go into version 4 blocks. This is encouragement to upgrade. If 950 of the last 1000 blocks are version 4 or higher, reject blocks with transactions of less than version 3 in all blocks This means that legacy nodes can no longer send transactions but can still receive. Transactions received from other legacy nodes would remain unconfirmed. If 990 of the last 1000 blocks are version 4 or higher, reject version 3 or lower blocks This is the point of no return. Rejecting version 3 blocks means that the next rule is guaranteed to activate within the next 2016 blocks. Legacy nodes remain on the main chain, but cannot send. Miners mining with legacy clients are (soft) forked off the chain. If 1000 of the last 1000 blocks are version 4 or higher and the difficulty retarget has just happened, activate hard fork rule This hard forks legacy nodes off the chain. 99% of miners support this change and users have been encouraged to update. The block rate for the non-forked chain is ast most 1% of normal. Blocks happen every 16 hours. By timing activation after a difficulty retarget, it makes it harder for the other fork to adapt to the reduced hash rate. > --001a11356f76da44e50515f8f57b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Wed, May 13, 2015 at 4:24 PM= , Christian Decker <decker.christian@gmail.com> wro= te
It does and I should have mentioned it in the draft, according to my=20 calculations a mapping legacy ID -> normalized ID is about 256 MB in=20 size, or at least it was at height 330'000, things might have changed a= =20 bit and I'll recompute that. I omitted the deprecation of legacy IDs on= =20 purpose since we don't know whether we will migrate completely or leave= =20 keep both options viable.

There are around 20 million UTXOs.=C2=A0 At 2*32 bytes per entry, that is more= =20 than 1GB.=C2=A0 There are more UTXOs than transactions, but 256MB seems a= =20 little low.

I think both IDs can be used in the merkle tree, since we lookup an ID in both indices we can use both to address them and we will find them=20 either way.

The id that is used to sign should be used in the merkle tree.=C2=A0 The hard= =20 fork should simply be to allow transactions that use the normalized=20 transaction hash.
=C2=A0
As for the = opcodes I'll have to check, but I currently don't see how they coul= d be affected.

Agr= eed, the transaction is simply changed and all the standard rules apply.
=C2=A0
We can certainly split = the proposal should it get too large, for now it seems manageable, since op= codes are not affected.

= Right it is just a database update.=C2=A0 The undo info also needs to be ch= anged so that both txids are included.
=C2=A0
Bloom-filtering is resolved by adding the normalized transaction IDs = and checking for both IDs in the filter.
=
Yeah, if a transaction spends with a legacy txid, it = should still match if the normalized txid is included in the filter.<= br>
> Since you mention bundling the change with other changes that r= equire a=20 hard-fork it might be a good idea to build a separate proposal for a=20 generic hard-fork rollout mechanism.

That would be= useful.=C2=A0 On the other hand, we don't want to make them to easy.
I think this is a good choice for a hard fork test, since it is=20 uncontroversial.=C2=A0 With a time machine, it would have been done this wa= y=20 at the start.

What about the following:
The reference client is updated so that it uses version 2=20 transactions by default (but it can be changed by user).=C2=A0 A pop-up cou= ld appear for the GUI.

There is no other change.

All transactions in blocks 375000 to 385000 are consid= ered votes and weighted by bitcoin days destroyed (max 60 days).

If > 75% of the transactions by weight are version 2, then the c= ommunity are considered to support the hard fork.

There would need to be a way to protect against miners censoring=20 transactions/votes.=C2=A0

Users could submit their transactions=20 directly to a p2p tallying system.=C2=A0 The coin would be aged based on th= e=20 age in block 375000 unless included in the blockchain.=C2=A0 These votes=20 don't need to be ordered and multiple votes for the same coin would onl= y count once.

In fact, votes could just be based on holdin= g in block X.=C2=A0

This is an opinion poll rather than = a referendum though.

Assuming support of the c= ommunity, the hard fork can then proceed in a similar way to the way a soft= fork does.

Devs update the reference client to produce version 4 blocks and version 3=20 transactions.=C2=A0 Miners could watch version 3 transactions to gauge user= =20 interest and use that to help decide if they should update.

If 750 of the last 1000 blocks are version 4 or higher, reject blocks with transactions of less than version 3 in version 4 blocks

=C2=A0=C2=A0=C2=A0 This means that legacy clients will be slow to confirm their=20 transactions, since their transactions cannot go into version 4=20 blocks.=C2=A0=C2=A0=C2=A0 This is encouragement to upgrade.

If 950 of the last 1000 blocks are version 4 or hi= gher, reject blocks with transactions of less than version 3 in all blocks<= br>
=C2=A0=C2=A0=C2=A0 This means that legacy nodes can no longer send transactions but can=20 still receive.=C2=A0 Transactions received from other legacy nodes would=20 remain unconfirmed.

If 990 of the l= ast 1000 blocks are version 4 or higher, reject version 3 or lower blocks
=C2=A0=C2=A0=C2=A0 This is the point of no return.=C2=A0 Rejecting version 3 blocks means tha= t=20 the next rule is guaranteed to activate within the next 2016 blocks.=C2=A0= =20 Legacy nodes remain on the main chain, but cannot send.=C2=A0 Miners mining= =20 with legacy clients are (soft) forked off the chain.

If 1000 of the last 1000= blocks are version 4 or higher and the difficulty retarget has just happen= ed, activate hard fork rule

=C2=A0=C2=A0=C2=A0 This hard forks legacy nodes off the chain.=C2=A0 99% of miners support th= is change and users have been encouraged to update.=C2=A0 The block rate for= =20 the non-forked chain is ast most 1% of normal.=C2=A0 Blocks happen every 16= =20 hours.=C2=A0 By timing activation after a difficulty retarget, it makes it= =20 harder for the other fork to adapt to the reduced hash rate.

--001a11356f76da44e50515f8f57b--