Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1WoLBL-0007S6-Rh for bitcoin-development@lists.sourceforge.net; Sat, 24 May 2014 23:16:51 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.215.42 as permitted sender) client-ip=209.85.215.42; envelope-from=gmaxwell@gmail.com; helo=mail-la0-f42.google.com; Received: from mail-la0-f42.google.com ([209.85.215.42]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1WoLBK-0006H3-LD for bitcoin-development@lists.sourceforge.net; Sat, 24 May 2014 23:16:51 +0000 Received: by mail-la0-f42.google.com with SMTP id el20so5080052lab.15 for ; Sat, 24 May 2014 16:16:43 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.152.5.135 with SMTP id s7mr3391068las.55.1400973403783; Sat, 24 May 2014 16:16:43 -0700 (PDT) Received: by 10.112.89.68 with HTTP; Sat, 24 May 2014 16:16:43 -0700 (PDT) In-Reply-To: References: Date: Sat, 24 May 2014 16:16:43 -0700 Message-ID: From: Gregory Maxwell To: Ashley Holman Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gmaxwell[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1WoLBK-0006H3-LD Cc: Bitcoin Development Subject: Re: [Bitcoin-development] Cut-through propagation of blocks X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 23:16:52 -0000 On Fri, May 23, 2014 at 8:57 PM, Ashley Holman wrote: > Hi, > On this list there has been some discussion around techniques to speed up > block propagation, with a particular focus on reducing the extra orphan risk > carried by larger blocks. > The current store-and-forward method means that larger blocks will propagate > with higher latency. FWIW, there are a lot of improvements which can be made before more complex changes like cut-through-forwarding that change the protocol. For example, the reference software has a 100ms sleep in p2p message processing which could be replaced with a semaphore, this would dramatically lower latency for block relaying. Likewise nodes which are becoming bandwidth overloaded could adapt their concurrent connection counts down (and ones that are underloaded could accept more connections). Relaying to multiple peers could be done in parallel instead of serialized, and the order in which peers are relayed to could be adapted to place more apparently useful and faster peers first, e.g. every time a peer is the first to tell you about a block or transaction you accept they move up the list, every time their socket send queue fills they move down. Luke-Jr had implemented cut through behavior previously and had posted a patch, but absent those other network processing improvements it didn't appear to help. If you want to go full out crazy in optimizing in this space, there are fancier things that can be done to further reduce latency and increase efficiency: https://en.bitcoin.it/wiki/User:Gmaxwell/block_network_coding ... but some of this stuff really should be done as a seperate protocol. There is no need to have Bitcoin transport all using a single protocol, and we can get better robustness and feature velocity if there are a couple protocols in use (you could just run a block-transport-protocol daemon that connects to your local node via the classic protocol).