Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 18AC77E for ; Wed, 5 Aug 2015 21:19:21 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail.affil.co (mail.affil.co [75.101.133.5]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F2BC0241 for ; Wed, 5 Aug 2015 21:19:18 +0000 (UTC) Received: (qmail 29573 invoked by uid 1008); 5 Aug 2015 21:18:08 +0000 Received: from mail-ob0-f181.google.com (arnoud@pop.affil.co@209.85.214.181) by mail.affil.co with RC4-SHA encrypted SMTP; 5 Aug 2015 21:18:08 +0000 Received: by obnw1 with SMTP id w1so41465877obn.3 for ; Wed, 05 Aug 2015 14:19:17 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.246.202 with SMTP id xy10mr9913744obc.64.1438809557211; Wed, 05 Aug 2015 14:19:17 -0700 (PDT) Received: by 10.76.83.136 with HTTP; Wed, 5 Aug 2015 14:19:17 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Aug 2015 15:19:17 -0600 Message-ID: From: Arnoud Kouwenhoven - Pukaki Corp To: Gregory Maxwell Content-Type: multipart/alternative; boundary=089e01634d6a8ea04f051c96f431 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Arnoud Kouwenhoven - Pukaki Corp via bitcoin-dev Subject: Re: [bitcoin-dev] Idea: Efficient bitcoin block propagation X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2015 21:19:21 -0000 --089e01634d6a8ea04f051c96f431 Content-Type: text/plain; charset=UTF-8 Thanks for this (direct) feedback. It would make sense that if blocks can be submitted using ~5kb packets, that no further optimizations would be needed at this point. I will look into the relay network transmission protocol to understand how it works! I hear that you are saying that this network solves speed of transmission and thereby (technical) block size issues. Presumably it would solve speed of block validation too by prevalidating transactions. Assuming this is all true, and I have no reason to doubt that at this point, I do not understand why there is any discussion at all about the (technical) impact of large blocks, or why there are large numbers of miners building on invalid blocks (SPV mining, https://bitcoin.org/en/alert/2015-07-04-spv-mining), or why there is any discussion about the speed of block validation (cpu processing time to verify blocks and transactions in blocks being a limitation). Our proposal aims at solving all three issues. Now I would be glad if the suggestions we made are already implemented, especially if that is in a more elegant approach. Great! Yet we still see all three discussions, which is a surprise if they have been solved. On Wed, Aug 5, 2015 at 2:16 PM, Gregory Maxwell wrote: > On Wed, Aug 5, 2015 at 7:53 PM, Arnoud Kouwenhoven - Pukaki Corp via > bitcoin-dev wrote: > > Thanks for the reply. My understanding is that the bitcoin relay network > is > > a backbone of connected high speed servers to increase the rate at which > > transactions and new blocks propagate - and remove a number of delays in > > processing. But it would still require the miners to download the entire > > block before building on top of it with any degree of confidence. > > Your understanding is outdated. > > The relay network includes an optimized transmission protocol which > enables sending the "entire" block typically in just a smal number of > bytes (much smaller than the summaries you suggest, which still leave > the participants needing to send the block). > > E.g. block 000ce90846 was 999950 bytes and the relay network protocol > sent it using at most 4906 bytes. > > No trust is required in this scheme because the entire block is > communicated using only a couple packets. > > The current scheme is highly simplified and its efficiency could be > increased greatly with small improvements, or if miners created blocks > in an aware manner.... but with a maximum size blocks turning into 5kb > with the current setup, there hardly appears to be a reason to do so > right now. > > Ultimately there is no need for information communicated with a block > at discovery time proportional to the size of the block; with the > right affordances it can be accomplished with a small constant amount > of data. > > If not for this already being deployed I personally believe the > network would have already fallen into complete centeralization as a > response to larger blocks: this was constructed and deployed in order > to pull the network back from having a single pool with more than half > the hashrate. > --089e01634d6a8ea04f051c96f431 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for this (direct) feedback. It would make sense tha= t if blocks can be submitted using ~5kb packets, that no further optimizati= ons would be needed at this point. I will look into the relay network trans= mission protocol to understand how it works!

I hear that= you are saying that this network solves speed of transmission and thereby = (technical) block size issues. Presumably it would solve speed of block val= idation too by prevalidating transactions. Assuming this is all true, and I= have no reason to doubt that at this point, I do not understand why there = is any discussion at all about the (technical) impact of large blocks, or w= hy there are large numbers of miners building on invalid blocks (SPV mining= , https://bi= tcoin.org/en/alert/2015-07-04-spv-mining), or why there is any discussi= on about the speed of block validation (cpu processing time to verify block= s and transactions in blocks being a limitation).=C2=A0

Our proposal aims at solving all three issues.

Now I would be glad if the suggestions we made are already implemented, = especially if that is in a more elegant approach. Great! Yet we still see a= ll three discussions, which is a surprise if they have been solved.

On Wed, A= ug 5, 2015 at 2:16 PM, Gregory Maxwell <gmaxwell@gmail.com>= wrote:
On Wed, Aug 5, 2= 015 at 7:53 PM, Arnoud Kouwenhoven - Pukaki Corp via
bitcoin-dev <bi= tcoin-dev@lists.linuxfoundation.org> wrote:
> Thanks for the reply. My understanding is that the bitcoin relay netwo= rk is
> a backbone of connected high speed servers to increase the rate at whi= ch
> transactions and new blocks propagate - and remove a number of delays = in
> processing. But it would still require the miners to download the enti= re
> block before building on top of it with any degree of confidence.

Your understanding is outdated.

The relay network includes an optimized transmission protocol which
enables sending the "entire" block typically in just a smal numbe= r of
bytes (much smaller than the summaries you suggest, which still leave
the participants needing to send the block).

E.g. block 000ce90846 was 999950 bytes and the relay network protocol
sent it using at most 4906 bytes.

No trust is required in this scheme because the entire block is
communicated using only a couple packets.

The current scheme is highly simplified and its efficiency could be
increased greatly with small improvements, or if miners created blocks
in an aware manner.... but with a maximum size blocks turning into 5kb
with the current setup, there hardly appears to be a reason to do so
right now.

Ultimately there is no need for information communicated with a block
at discovery time proportional to the size of the block; with the
right affordances it can be accomplished with a small constant amount
of data.

If not for this already being deployed I personally believe the
network would have already fallen into complete centeralization as a
response to larger blocks: this was constructed and deployed in order
to pull the network back from having a single pool with more than half
the hashrate.

--089e01634d6a8ea04f051c96f431--