Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.213.173 as permitted sender)
	client-ip=209.85.213.173; envelope-from=pieter.wuille@gmail.com;
	helo=mail-ig0-f173.google.com; 
MIME-Version: 1.0
In-Reply-To: <CAADm4BB8y=k_f7CG3tyX6ruWF0w3+hU2Szv7ajLp1x7KhS56GA@mail.gmail.com>
References: <CANEZrP04O7EqB=TqyTiC7O1K2A9R0nKJ_ssANHKg=Byum8-LtA@mail.gmail.com>
	<CA+s+GJDbYjwhpsV15a+7kCO_vTstEewVrwvqbnB=a5zOSwFC6Q@mail.gmail.com>
	<CAAS2fgStmEpiUV4Yh-qqu6sZ+VZ5SiQPwp+QA=3X5zR52ia3OA@mail.gmail.com>
	<CA+s+GJBxEC2MifJQY5-vn2zSOHo-UOm8B1vYHHOfuxq26=VscQ@mail.gmail.com>
	<CAADm4BDDJkS_xdjUn=2Yzs4B0RXTvpzpd5Z_kDRorzrn1HWSng@mail.gmail.com>
	<CANEZrP1rPZYkTLmx5GOdj67oQAgFjeaF-LCKAXpg5XsEhXYFuQ@mail.gmail.com>
	<CAADm4BB8y=k_f7CG3tyX6ruWF0w3+hU2Szv7ajLp1x7KhS56GA@mail.gmail.com>
Date: Thu, 10 Apr 2014 18:59:54 +0200
Message-ID: <CAPg+sBg88Q1Ddwsvuk3-17wO=0DF7L1wtxx4mWUoiV1=cWKhEQ@mail.gmail.com>
From: Pieter Wuille <pieter.wuille@gmail.com>
To: Brian Hoffman <brianchoffman@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Chain pruning
Precedence: list

On Thu, Apr 10, 2014 at 6:47 PM, Brian Hoffman <brianchoffman@gmail.com> wrote:
> Looks like only about ~30% disk space savings so I see your point. Is there
> a critical reason why blocks couldn't be formed into "superblocks" that are
> chained together and nodes could serve a specific superblock, which could be
> pieced together from different nodes to get the full blockchain? This would
> allow participants with limited resources to serve full portions of the
> blockchain rather than limited pieces of the entire blockchain.

As this is a suggestion that I think I've seen come up once a month
for the past 3 years, let's try to answer it thoroughly.

The actual "state" of the blockchain is the UTXO set (stored in
chainstate/ by the reference client). It's the set of all unspent
transaction outputs at the currently active point in the block chain.
It is all you need for validating future blocks.

The problem is, you can't just give someone the UTXO set and expect
them to trust it, as there is no way to prove that it was the result
of processing the actual blocks.

As Bitcoin's full node uses a "zero trust" model, where (apart from
one detail: the order of otherwise valid transactions) it never
assumes any data received from the outside it valid, it HAS to see the
previous blocks in order to establish the validity of the current UTXO
set. This is what initial block syncing does. Nothing but the actual
blocks can provide this data, and it is why the actual blocks need to
be available. It does not require everyone to have all blocks, though
- they just need to have seen them during processing.

A related, but not identical evolution is merkle UTXO commitments.
This means that we shape the UTXO set as a merkle tree, compute its
root after every block, and require that the block commits to this
root hash (by putting it in the coinbase, for example). This means a
full node can copy the chain state from someone else, and check that
its hash matches what the block chain commits to. It's important to
note that this is a strict reduction in security: we're now trusting
that the longest chain (with most proof of work) commits to a valid
UTXO set (at some point in the past).

In essence, combining both ideas means you get "superblocks" (the UTXO
set is essentially the summary of the result of all past blocks), in a
way that is less-than-currently-but-perhaps-still-acceptably-validated.

-- 
Pieter