Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1SfXxP-0003yK-KI for bitcoin-development@lists.sourceforge.net; Fri, 15 Jun 2012 14:57:03 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of bluematt.me designates 173.246.101.161 as permitted sender) client-ip=173.246.101.161; envelope-from=bitcoin-list@bluematt.me; helo=mail.bluematt.me; Received: from vps.bluematt.me ([173.246.101.161] helo=mail.bluematt.me) by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1SfXxM-0006tW-2A for bitcoin-development@lists.sourceforge.net; Fri, 15 Jun 2012 14:57:03 +0000 Received: from [IPv6:2001:470:9ff2:1:ee55:f9ff:fec6:e666] (unknown [IPv6:2001:470:9ff2:1:ee55:f9ff:fec6:e666]) by mail.bluematt.me (Postfix) with ESMTPSA id 2454F398A; Fri, 15 Jun 2012 14:56:54 +0000 (UTC) Message-ID: <1339772212.31489.62.camel@bmthinkpad> From: Matt Corallo To: Mike Hearn Date: Fri, 15 Jun 2012 16:56:52 +0200 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.2-1 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 SPF_PASS SPF: sender matches SPF record X-Headers-End: 1SfXxM-0006tW-2A Cc: Bitcoin Development Subject: Re: [Bitcoin-development] New P2P commands for diagnostics, SPV clients X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jun 2012 14:57:03 -0000 On Fri, 2012-06-15 at 15:43 +0200, Mike Hearn wrote: > > Yes, the format is something that must be hashed out (no pun > > intended). Need input from potential users about what information > > they might need. > > Matts point that a branch-per-transaction may duplicate data is well > made, that said, I suspect a format that tries to fix this would be > much more complicated. > > How about see this project as a three part change? > > First step - add the mempool command and make nodes sync up their > mempools on startup. ACK > > Second step - if protocol version >= X, the "block" message consists > of a header + num transactions + vector instead of the full > transactions themselves. If vector is sorted in the order of the merkle tree, you dont need to forward the merkle tree to non-filtered nodes, further saving some small amount of bandwidth. For filtered nodes, you would still need to forward merkle branches anyway. > > On receiving such a block, we go look to see which transactions we're > missing from the mempool and request them with getdata. Each time we > receive a tx message we check to see if it was one we were missing > from a block. Once all transactions in the block message are in > memory, we go ahead and assemble the block, then verify as per normal. > This should speed up block propagation. Miners have an incentive to > upgrade because it should reduce wasted work. ACK > > Third step - new message, getmerkletx takes a vector and returns > a merkletx message: "merkle branch missing the root + transaction data > itself" for each requested transaction. The filtering commands are > added, so the block message now only lists transaction hashes that > match the filter which can then be requested with getmerkletx. I really dont think it would be /that/ difficult to make it getmerkletxs vector. And then respond with a partial merkle tree to those transactions. Matt