Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1WYIQn-0007Vf-Io for bitcoin-development@lists.sourceforge.net; Thu, 10 Apr 2014 17:06:29 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.160.53 as permitted sender) client-ip=209.85.160.53; envelope-from=brianchoffman@gmail.com; helo=mail-pb0-f53.google.com; Received: from mail-pb0-f53.google.com ([209.85.160.53]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1WYIQm-0007d7-LR for bitcoin-development@lists.sourceforge.net; Thu, 10 Apr 2014 17:06:29 +0000 Received: by mail-pb0-f53.google.com with SMTP id rp16so4211796pbb.12 for ; Thu, 10 Apr 2014 10:06:22 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.66.161.69 with SMTP id xq5mr20997360pab.62.1397149582791; Thu, 10 Apr 2014 10:06:22 -0700 (PDT) Received: by 10.70.89.237 with HTTP; Thu, 10 Apr 2014 10:06:22 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Apr 2014 13:06:22 -0400 Message-ID: From: Brian Hoffman To: Pieter Wuille Content-Type: multipart/alternative; boundary=047d7b6dd002949a1c04f6b33c04 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (brianchoffman[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1WYIQm-0007d7-LR Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Chain pruning X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 17:06:29 -0000 --047d7b6dd002949a1c04f6b33c04 Content-Type: text/plain; charset=UTF-8 Ok I think I've got a good understanding of where we're at now. I can promise that the next person to waste your time in 30 days will not be me. I'm pleasantly surprised to see a community that doesn't kickban newcomers and takes the time to explain (re-explain) concepts. Hoping to add *beneficial* thoughts in the future! On Thu, Apr 10, 2014 at 12:59 PM, Pieter Wuille wrote: > On Thu, Apr 10, 2014 at 6:47 PM, Brian Hoffman > wrote: > > Looks like only about ~30% disk space savings so I see your point. Is > there > > a critical reason why blocks couldn't be formed into "superblocks" that > are > > chained together and nodes could serve a specific superblock, which > could be > > pieced together from different nodes to get the full blockchain? This > would > > allow participants with limited resources to serve full portions of the > > blockchain rather than limited pieces of the entire blockchain. > > As this is a suggestion that I think I've seen come up once a month > for the past 3 years, let's try to answer it thoroughly. > > The actual "state" of the blockchain is the UTXO set (stored in > chainstate/ by the reference client). It's the set of all unspent > transaction outputs at the currently active point in the block chain. > It is all you need for validating future blocks. > > The problem is, you can't just give someone the UTXO set and expect > them to trust it, as there is no way to prove that it was the result > of processing the actual blocks. > > As Bitcoin's full node uses a "zero trust" model, where (apart from > one detail: the order of otherwise valid transactions) it never > assumes any data received from the outside it valid, it HAS to see the > previous blocks in order to establish the validity of the current UTXO > set. This is what initial block syncing does. Nothing but the actual > blocks can provide this data, and it is why the actual blocks need to > be available. It does not require everyone to have all blocks, though > - they just need to have seen them during processing. > > A related, but not identical evolution is merkle UTXO commitments. > This means that we shape the UTXO set as a merkle tree, compute its > root after every block, and require that the block commits to this > root hash (by putting it in the coinbase, for example). This means a > full node can copy the chain state from someone else, and check that > its hash matches what the block chain commits to. It's important to > note that this is a strict reduction in security: we're now trusting > that the longest chain (with most proof of work) commits to a valid > UTXO set (at some point in the past). > > In essence, combining both ideas means you get "superblocks" (the UTXO > set is essentially the summary of the result of all past blocks), in a > way that is less-than-currently-but-perhaps-still-acceptably-validated. > > -- > Pieter > --047d7b6dd002949a1c04f6b33c04 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Ok I think I've got a good understanding of where we&#= 39;re at now. I can promise that the next person to waste your time in 30 d= ays will not be me. I'm pleasantly surprised to see a community that do= esn't kickban newcomers and takes the time to explain (re-explain) conc= epts.=C2=A0

Hoping to add *beneficial* thoughts in the future!


On Thu, Apr= 10, 2014 at 12:59 PM, Pieter Wuille <pieter.wuille@gmail.com>= ; wrote:
On Thu, Apr 10, 2014 at 6:47 PM, Brian Hoffm= an <brianchoffman@gmail.com> wrote:
> Looks like only about ~30% disk space savings so I see your point. Is = there
> a critical reason why blocks couldn't be formed into "superbl= ocks" that are
> chained together and nodes could serve a specific superblock, which co= uld be
> pieced together from different nodes to get the full blockchain? This = would
> allow participants with limited resources to serve full portions of th= e
> blockchain rather than limited pieces of the entire blockchain.

As this is a suggestion that I think I've seen come up once a month
for the past 3 years, let's try to answer it thoroughly.

The actual "state" of the blockchain is the UTXO set (stored in chainstate/ by the reference client). It's the set of all unspent
transaction outputs at the currently active point in the block chain.
It is all you need for validating future blocks.

The problem is, you can't just give someone the UTXO set and expect
them to trust it, as there is no way to prove that it was the result
of processing the actual blocks.

As Bitcoin's full node uses a "zero trust" model, where (apar= t from
one detail: the order of otherwise valid transactions) it never
assumes any data received from the outside it valid, it HAS to see the
previous blocks in order to establish the validity of the current UTXO
set. This is what initial block syncing does. Nothing but the actual
blocks can provide this data, and it is why the actual blocks need to
be available. It does not require everyone to have all blocks, though
- they just need to have seen them during processing.

A related, but not identical evolution is merkle UTXO commitments.
This means that we shape the UTXO set as a merkle tree, compute its
root after every block, and require that the block commits to this
root hash (by putting it in the coinbase, for example). This means a
full node can copy the chain state from someone else, and check that
its hash matches what the block chain commits to. It's important to
note that this is a strict reduction in security: we're now trusting that the longest chain (with most proof of work) commits to a valid
UTXO set (at some point in the past).

In essence, combining both ideas means you get "superblocks" (the= UTXO
set is essentially the summary of the result of all past blocks), in a
way that is less-than-currently-but-perhaps-still-acceptably-validated.

--
Pieter

--047d7b6dd002949a1c04f6b33c04--