Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
	helo=mx.sourceforge.net)
	by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <bitcoin-list@bluematt.me>) id 1SfXxP-0003yK-KI
	for bitcoin-development@lists.sourceforge.net;
	Fri, 15 Jun 2012 14:57:03 +0000
Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of bluematt.me
	designates 173.246.101.161 as permitted sender)
	client-ip=173.246.101.161;
	envelope-from=bitcoin-list@bluematt.me; helo=mail.bluematt.me; 
Received: from vps.bluematt.me ([173.246.101.161] helo=mail.bluematt.me)
	by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76)
	id 1SfXxM-0006tW-2A for bitcoin-development@lists.sourceforge.net;
	Fri, 15 Jun 2012 14:57:03 +0000
Received: from [IPv6:2001:470:9ff2:1:ee55:f9ff:fec6:e666] (unknown
	[IPv6:2001:470:9ff2:1:ee55:f9ff:fec6:e666])
	by mail.bluematt.me (Postfix) with ESMTPSA id 2454F398A;
	Fri, 15 Jun 2012 14:56:54 +0000 (UTC)
Message-ID: <1339772212.31489.62.camel@bmthinkpad>
From: Matt Corallo <bitcoin-list@bluematt.me>
To: Mike Hearn <mike@plan99.net>
Date: Fri, 15 Jun 2012 16:56:52 +0200
In-Reply-To: <CANEZrP39RHfCDX-x4ARo+oPphLv-70RxuMh3+AJzsNPxzOd=bA@mail.gmail.com>
References: <CA+8xBpecVQcTTbPxUm_3_GWC99dEd4=-VFWb+QT6jUy4rg8U4w@mail.gmail.com>
	<CANEZrP0kNZDByHpK2=UjP+ag0X1KmqHxnJdm=e_pWMitP4QvvA@mail.gmail.com>
	<CA+8xBpcwhQPQRe=stYb9xksLsTbiABKLS7PZnRtvPga6AmSg4Q@mail.gmail.com>
	<CANEZrP39RHfCDX-x4ARo+oPphLv-70RxuMh3+AJzsNPxzOd=bA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.2.2-1 
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0
X-Spam-Score: -1.5 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
	domain
	-0.0 SPF_PASS               SPF: sender matches SPF record
X-Headers-End: 1SfXxM-0006tW-2A
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] New P2P commands for diagnostics,
 SPV clients
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Fri, 15 Jun 2012 14:57:03 -0000

On Fri, 2012-06-15 at 15:43 +0200, Mike Hearn wrote:
> > Yes, the format is something that must be hashed out (no pun
> > intended).  Need input from potential users about what information
> > they might need.
> 
> Matts point that a branch-per-transaction may duplicate data is well
> made, that said, I suspect a format that tries to fix this would be
> much more complicated.
> 
> How about see this project as a three part change?
> 
> First step - add the mempool command and make nodes sync up their
> mempools on startup.
ACK
> 
> Second step - if protocol version >= X, the "block" message consists
> of a header + num transactions + vector<hash>  instead of the full
> transactions themselves.
If vector<hash> is sorted in the order of the merkle tree, you dont need
to forward the merkle tree to non-filtered nodes, further saving some
small amount of bandwidth.  For filtered nodes, you would still need to
forward merkle branches anyway.
> 
> On receiving such a block, we go look to see which transactions we're
> missing from the mempool and request them with getdata. Each time we
> receive a tx message we check to see if it was one we were missing
> from a block. Once all transactions in the block message are in
> memory, we go ahead and assemble the block, then verify as per normal.
> This should speed up block propagation. Miners have an incentive to
> upgrade because it should reduce wasted work.
ACK
> 
> Third step - new message, getmerkletx takes a vector<hash> and returns
> a merkletx message: "merkle branch missing the root + transaction data
> itself" for each requested transaction. The filtering commands are
> added, so the block message now only lists transaction hashes that
> match the filter which can then be requested with getmerkletx.
I really dont think it would be /that/ difficult to make it getmerkletxs
vector<hashes>. And then respond with a partial merkle tree to those
transactions.

Matt