Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 6EEC8129E for ; Thu, 24 Sep 2015 08:52:36 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-yk0-f174.google.com (mail-yk0-f174.google.com [209.85.160.174]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B1D7521A for ; Thu, 24 Sep 2015 08:52:35 +0000 (UTC) Received: by ykdt18 with SMTP id t18so69428905ykd.3 for ; Thu, 24 Sep 2015 01:52:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=lVJLG6jq7wX/6WkDgaAsrYuMITvytGdCzJhP+TLEQjs=; b=IZp1+Y7uJxTyR/PvnwK45wZf1xJQi7cB56ZtxjroJwGoJLCn7t39j3qJ0j2Jq2tIrL 9bwm6sCA9Kgw9O2NnQsVaeBUto3/G6ZDKqgQWmhVSNWzAU8CzeWqNEYXjjvaU1AgYuYK j4NFFS9llMdCtRFDcqUe4CE0BzdqbJpEr37a8LhwR/AHQvdpIG8SDZckwCevltVZ/jTf 6ADH7dQHYuFCVexGyBfKcBI8+2DxC62NNPH9BK6duQbY8CugOicg0YpYVldfKrVpKlQ0 NQ8rv5HefWU4xZQbDZezTMHCcM0Zxy9YJQAaoZOxn/Jv2Ji8bB1d/C/ewMUVYG90IEOz 4SGA== MIME-Version: 1.0 X-Received: by 10.31.8.131 with SMTP id 125mr17315239vki.45.1443084754738; Thu, 24 Sep 2015 01:52:34 -0700 (PDT) Received: by 10.103.65.204 with HTTP; Thu, 24 Sep 2015 01:52:34 -0700 (PDT) In-Reply-To: <36CE1B0F-3BE5-4DC3-8488-A57667256059@toom.im> References: <36CE1B0F-3BE5-4DC3-8488-A57667256059@toom.im> Date: Thu, 24 Sep 2015 09:52:34 +0100 Message-ID: From: Tier Nolan Cc: Bitcoin Dev Content-Type: multipart/alternative; boundary=001a11440fa22fe4a005207a5a5b X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MISSING_HEADERS, RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Torrent-style new-block propagation on Merkle trees X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 08:52:36 -0000 --001a11440fa22fe4a005207a5a5b Content-Type: text/plain; charset=UTF-8 On Thu, Sep 24, 2015 at 12:12 AM, Jonathan Toomim (Toomim Bros) via bitcoin-dev wrote: > > > As I understand it, the current block propagation algorithm is this: > > 1. A node mines a block. > 2. It notifies its peers that it has a new block with an *inv*. Typical > nodes have 8 peers. > 3. The peers respond that they have not seen it, and request the block > with *getdata* [hash]. > 4. The node sends out the block in parallel to all 8 peers simultaneously. > If the node's upstream bandwidth is limiting, then all peers will receive > most of the block before any peer receives all of the block. The block is > sent out as the small header followed by a list of transactions. > 5. Once a peer completes the download, it verifies the block, then enters > step 2. > Mining pools currently connect to the "fast relay network". This is optimised for fast block distribution. It does no validation and is only for low latency propagation. The normal network is used as a fallback. My understanding is that it works as follows: Each miner runs a normal full node and a relay node on the same computer. The full node tells the relay node whenever it receives a new transaction via the inv message and the node requests the full transaction. The relay node tells its relay peers that it knows about the transaction (hash only) and its 4 byte key. This is not forwarded onwards, since the relay peer only gets the hash of the transaction and doesn't do validation anyway. The key is just a 4 byte counter. Each relay node keeps a mapping of txid to key for each of its peer. There is some garbage collection and entries are removed once the transaction is included in a block (there might be a confirm threshold). When a block is found, the local node sends it to the relay node. The relay node then forwards it to all of its peers in a compact form. The block is sent as a list of keys for that peer and full transactions are only sent for unknown transactions. When a relay node receives a block, it just verifies the POW, checks that it is new and recent. It does not do tx validation. It forwards the block to its local full node, which does the validation. Since the relay node is on localhost, it never gets kicked due to sending invalid blocks. This prevents a DOS attack where you could send invalid blocks to the relay node and cause the local full node to kick it. If all the transactions are already known, then it can forward a block for only 4 bytes per transactions. I think it has an optimisation, so that is compressed to 1 byte per tx. --001a11440fa22fe4a005207a5a5b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On T= hu, Sep 24, 2015 at 12:12 AM, Jonathan Toomim (Toomim Bros) via bitcoin-dev= <bitcoin-dev@lists.linuxfoundation.org>= wrote:


As I understand it, the cu= rrent block propagation algorithm is this:

1. A no= de mines a block.
2. It notifies its peers that it has a new bloc= k with an=C2=A0inv. Typical nodes have 8 peers.=C2=A0
3. T= he peers respond that they have not seen it, and request the block with=C2= =A0getdata=C2=A0[hash].=C2=A0
4. The node sends out the bl= ock in parallel to all 8 peers simultaneously. If the node's upstream b= andwidth is limiting, then all peers will receive most of the block before = any peer receives all of the block. The block is sent out as the small head= er followed by a list of transactions.
5. Once a peer completes t= he download, it verifies the block, then enters step 2.

Mining pools currently connect to the = "fast relay network".=C2=A0 This is optimised for fast block dist= ribution.=C2=A0 It does no validation and is only for low latency propagati= on.=C2=A0 The normal network is used as a fallback.

My un= derstanding is that it works as follows:

Each miner runs = a normal full node and a relay node on the same computer.

The full node tells the relay node whenever it receives a new transaction = via the inv message and the node requests the full transaction.

The relay node tells its relay peers that it knows about the transac= tion (hash only) and its 4 byte key. This is not forwarded onwards, since t= he relay peer only gets the hash of the transaction and doesn't do vali= dation anyway.=C2=A0 The key is just a 4 byte counter.

Each relay no= de keeps a mapping of txid to key for each of its peer.=C2=A0 There is some= garbage collection and entries are removed once the transaction is include= d in a block (there might be a confirm threshold).

=
When a block is found, the local node sends it to the relay node.=C2= =A0 The relay node then forwards it to all of its peers in a compact form.<= br>
The block is sent as a list of keys for that peer and ful= l transactions are only sent for unknown transactions.

Wh= en a relay node receives a block, it just verifies the POW, checks that it = is new and recent.=C2=A0 It does not do tx validation.=C2=A0 It forwards th= e block to its local full node, which does the validation.=C2=A0 Since the = relay node is on localhost, it never gets kicked due to sending invalid blo= cks.=C2=A0 This prevents a DOS attack where you could send invalid blocks t= o the relay node and cause the local full node to kick it.

If all the transactions are already known, then it can forward a block fo= r only 4 bytes per transactions.=C2=A0 I think it has an optimisation, so t= hat is compressed to 1 byte per tx.
--001a11440fa22fe4a005207a5a5b--