Return-Path: <morcos@gmail.com> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A06891674 for <bitcoin-dev@lists.linuxfoundation.org>; Fri, 18 Sep 2015 20:07:26 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f177.google.com (mail-ig0-f177.google.com [209.85.213.177]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D87C82E6 for <bitcoin-dev@lists.linuxfoundation.org>; Fri, 18 Sep 2015 20:07:25 +0000 (UTC) Received: by igxx6 with SMTP id x6so24919122igx.1 for <bitcoin-dev@lists.linuxfoundation.org>; Fri, 18 Sep 2015 13:07:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ooq3CPnfqLan5zhnh+h+l0dPLUdEmvgCNw6ws3vDV2A=; b=qjoe+J0y/n/ff+XpzVlJrCb4hTS+ou6jQAqCYzr6nUmHmF0zKL0OKwbqSNz3TTM4Rz m7Zj3eMi4Z9TrupwutSyp9LXfwJ6oakf7JE9L3Wtg3aripQqeOtKDfzYC2vbGXlALTlc sq59Ip1eL6TV3OlhNhVsNmYof55qMRMMzeGBgCM3QZjpzxGDkfn+8yIFiUEI+mm156Bk VpWkZ5OkXNPjoFakPvxi8jbn0ciL4F8Mhaoe5uBOAOFpUCr/w0T8hl0Tg6iWpF0RbJsE 0+m29q5pK4rFlBCRD0K8zQKbXo93eE76mL7shtm4JIDHoORV8md4IQKfv0KQwrLN7RIm zjTg== MIME-Version: 1.0 X-Received: by 10.50.107.68 with SMTP id ha4mr85110igb.35.1442606845249; Fri, 18 Sep 2015 13:07:25 -0700 (PDT) Received: by 10.64.106.103 with HTTP; Fri, 18 Sep 2015 13:07:25 -0700 (PDT) In-Reply-To: <55FC6951.9010704@gmail.com> References: <5D55F6EC-801B-4607-882F-B76CF57298B1@gmail.com> <55FC6951.9010704@gmail.com> Date: Fri, 18 Sep 2015 16:07:25 -0400 Message-ID: <CAPWm=eWrnA9em725uR-56+r7uc752+C-6Ke-UcRXj__DBbwqYw@mail.gmail.com> From: Alex Morcos <morcos@gmail.com> To: Patrick Strateman <patrick.strateman@gmail.com> Content-Type: multipart/alternative; boundary=047d7b10ca758fe36405200b14a7 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org> Subject: Re: [bitcoin-dev] Hash of UTXO set as consensus-critical X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Fri, 18 Sep 2015 20:07:26 -0000 --047d7b10ca758fe36405200b14a7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I guess I always assumed that UTXO set commitments were an alternative security model (between SPV and full-node), not that they would cause the existing security model to be deprecated. On Fri, Sep 18, 2015 at 3:43 PM, Patrick Strateman via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Full nodes using UTXO set commitments is a change to the bitcoin > security model. > > Currently an attacker with >50% of the network hashrate can rewrite > history. > > If full nodes rely on UTXO set commitments such an attacker could create > an infinite number of bitcoins (as in many times more than the current > 21 million bitcoin limit). > > Before we consider mechanisms for UTXO set commitments, we should > seriously discuss whether the security model reduction is reasonable. > > On 09/18/2015 12:05 PM, Rune Kj=C3=A6r Svendsen via bitcoin-dev wrote: > > Currently, when a new node wants to join the network, it needs to > retrieve the entire blockchain history, starting from January 2009 and up > until now, in order to derive a UTXO set that it can verify new > blocks/transactions against. With a blockchain size of 40GB and a UTXO si= ze > of around 1GB, the extra bandwidth required is significant, and will keep > increasing indefinitely. If a newly mined block were to include the UTXO > set hash of the chain up until the previous block =E2=80=94 the hash of t= he UTXO > set on top of which this block builds =E2=80=94 then new nodes, who want = to know > whether a transaction is valid, would be able to acquire the UTXO set in = a > trustless manner, by only verifying proof-of-work headers, and knowing th= at > a block with an invalid UTXO set hash would be rejected. > > > > I=E2=80=99m not talking about calculating a complicated tree structure = from the > UTXO set, which would put further burden on already burdened Bitcoin Core > nodes. We simply include the hash of the current UTXO set in a newly > created block, such that the transactions in the new block build *on top* > of the UTXO set whose hash is specified. This actually alleviates Bitcoin > Core nodes, as it will now become possible for nodes without the entire > blockchain to answer SPV queries (by retrieving the UTXO set trustlessly > and using this to answer queries). It also saves bandwidth for Bitcore Co= re > nodes, who only need to send roughly 1GB of data, in order to synchronise= a > node, rather than 40GB+. I will continue to run a full Bitcoin Core node, > saving the entire blockchain history, but it shouldn=E2=80=99t be a requi= rement to > hold the entire transaction history in order to start verifying new > transactions. > > > > As far as I can see, this also forces miners to actually maintain an > UTXO set, rather than just build on top of the chain with the most > proof-of-work. Producing a UTXO set and verifying a block against a chain > is the same thing, so by including the hash of the UTXO set we force mine= rs > to verify the block that they want to build on top of. > > > > Am I missing something obvious, because as far as I can see, this solve= s > the problem of quadratic time complexity for initial sync: > http://www.youtube.com/watch?v=3DTgjrS-BPWDQ&t=3D2h02m12s > > > > The only added step to verifying a block is to hash the UTXO set. So it > does require additional computation, but most modern CPUs have a SHA256 > throughput of around 500 MB/s, which means it takes only two seconds to > hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s= ). > A small sacrifice for the added ease of initial syncing, in my opinion. > > > > /Rune > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists.linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --047d7b10ca758fe36405200b14a7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">I guess I always assumed that UTXO set commitments were an= alternative security model (between SPV and full-node), not that they woul= d cause the existing security model to be deprecated.<div><br></div></div><= div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Fri, Sep 18, 20= 15 at 3:43 PM, Patrick Strateman via bitcoin-dev <span dir=3D"ltr"><<a h= ref=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitc= oin-dev@lists.linuxfoundation.org</a>></span> wrote:<br><blockquote clas= s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad= ding-left:1ex">Full nodes using UTXO set commitments is a change to the bit= coin<br> security model.<br> <br> Currently an attacker with >50% of the network hashrate can rewrite hist= ory.<br> <br> If full nodes rely on UTXO set commitments such an attacker could create<br= > an infinite number of bitcoins (as in many times more than the current<br> 21 million bitcoin limit).<br> <br> Before we consider mechanisms for UTXO set commitments, we should<br> seriously discuss whether the security model reduction is reasonable.<br> <div class=3D"HOEnZb"><div class=3D"h5"><br> On 09/18/2015 12:05 PM, Rune Kj=C3=A6r Svendsen via bitcoin-dev wrote:<br> > Currently, when a new node wants to join the network, it needs to retr= ieve the entire blockchain history, starting from January 2009 and up until= now, in order to derive a UTXO set that it can verify new blocks/transacti= ons against. With a blockchain size of 40GB and a UTXO size of around 1GB, = the extra bandwidth required is significant, and will keep increasing indef= initely. If a newly mined block were to include the UTXO set hash of the ch= ain up until the previous block =E2=80=94 the hash of the UTXO set on top o= f which this block builds =E2=80=94 then new nodes, who want to know whethe= r a transaction is valid, would be able to acquire the UTXO set in a trustl= ess manner, by only verifying proof-of-work headers, and knowing that a blo= ck with an invalid UTXO set hash would be rejected.<br> ><br> > I=E2=80=99m not talking about calculating a complicated tree structure= from the UTXO set, which would put further burden on already burdened Bitc= oin Core nodes. We simply include the hash of the current UTXO set in a new= ly created block, such that the transactions in the new block build *on top= * of the UTXO set whose hash is specified. This actually alleviates Bitcoin= Core nodes, as it will now become possible for nodes without the entire bl= ockchain to answer SPV queries (by retrieving the UTXO set trustlessly and = using this to answer queries). It also saves bandwidth for Bitcore Core nod= es, who only need to send roughly 1GB of data, in order to synchronise a no= de, rather than 40GB+. I will continue to run a full Bitcoin Core node, sav= ing the entire blockchain history, but it shouldn=E2=80=99t be a requiremen= t to hold the entire transaction history in order to start verifying new tr= ansactions.<br> ><br> > As far as I can see, this also forces miners to actually maintain an U= TXO set, rather than just build on top of the chain with the most proof-of-= work. Producing a UTXO set and verifying a block against a chain is the sam= e thing, so by including the hash of the UTXO set we force miners to verify= the block that they want to build on top of.<br> ><br> > Am I missing something obvious, because as far as I can see, this solv= es the problem of quadratic time complexity for initial sync: <a href=3D"ht= tp://www.youtube.com/watch?v=3DTgjrS-BPWDQ&t=3D2h02m12s" rel=3D"norefer= rer" target=3D"_blank">http://www.youtube.com/watch?v=3DTgjrS-BPWDQ&t= =3D2h02m12s</a><br> ><br> > The only added step to verifying a block is to hash the UTXO set. So i= t does require additional computation, but most modern CPUs have a SHA256 t= hroughput of around 500 MB/s, which means it takes only two seconds to hash= the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A s= mall sacrifice for the added ease of initial syncing, in my opinion.<br> ><br> > /Rune<br> > _______________________________________________<br> > bitcoin-dev mailing list<br> > <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@l= ists.linuxfoundation.org</a><br> > <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-= dev" rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev</a><br> <br> <br> _______________________________________________<br> bitcoin-dev mailing list<br> <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.= linuxfoundation.org</a><br> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev</a><br> </div></div></blockquote></div><br></div> --047d7b10ca758fe36405200b14a7--