1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
|
Return-Path: <tier.nolan@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 6EEC8129E
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 24 Sep 2015 08:52:36 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-yk0-f174.google.com (mail-yk0-f174.google.com
[209.85.160.174])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B1D7521A
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 24 Sep 2015 08:52:35 +0000 (UTC)
Received: by ykdt18 with SMTP id t18so69428905ykd.3
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 24 Sep 2015 01:52:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:cc
:content-type; bh=lVJLG6jq7wX/6WkDgaAsrYuMITvytGdCzJhP+TLEQjs=;
b=IZp1+Y7uJxTyR/PvnwK45wZf1xJQi7cB56ZtxjroJwGoJLCn7t39j3qJ0j2Jq2tIrL
9bwm6sCA9Kgw9O2NnQsVaeBUto3/G6ZDKqgQWmhVSNWzAU8CzeWqNEYXjjvaU1AgYuYK
j4NFFS9llMdCtRFDcqUe4CE0BzdqbJpEr37a8LhwR/AHQvdpIG8SDZckwCevltVZ/jTf
6ADH7dQHYuFCVexGyBfKcBI8+2DxC62NNPH9BK6duQbY8CugOicg0YpYVldfKrVpKlQ0
NQ8rv5HefWU4xZQbDZezTMHCcM0Zxy9YJQAaoZOxn/Jv2Ji8bB1d/C/ewMUVYG90IEOz
4SGA==
MIME-Version: 1.0
X-Received: by 10.31.8.131 with SMTP id 125mr17315239vki.45.1443084754738;
Thu, 24 Sep 2015 01:52:34 -0700 (PDT)
Received: by 10.103.65.204 with HTTP; Thu, 24 Sep 2015 01:52:34 -0700 (PDT)
In-Reply-To: <36CE1B0F-3BE5-4DC3-8488-A57667256059@toom.im>
References: <36CE1B0F-3BE5-4DC3-8488-A57667256059@toom.im>
Date: Thu, 24 Sep 2015 09:52:34 +0100
Message-ID: <CAE-z3OV-1wziASBZ3moE9D2=bVVh7-RBwjMJAV=3LcanCzbN1w@mail.gmail.com>
From: Tier Nolan <tier.nolan@gmail.com>
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a11440fa22fe4a005207a5a5b
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MISSING_HEADERS,
RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] Torrent-style new-block propagation on Merkle
trees
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 24 Sep 2015 08:52:36 -0000
--001a11440fa22fe4a005207a5a5b
Content-Type: text/plain; charset=UTF-8
On Thu, Sep 24, 2015 at 12:12 AM, Jonathan Toomim (Toomim Bros) via
bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>
> As I understand it, the current block propagation algorithm is this:
>
> 1. A node mines a block.
> 2. It notifies its peers that it has a new block with an *inv*. Typical
> nodes have 8 peers.
> 3. The peers respond that they have not seen it, and request the block
> with *getdata* [hash].
> 4. The node sends out the block in parallel to all 8 peers simultaneously.
> If the node's upstream bandwidth is limiting, then all peers will receive
> most of the block before any peer receives all of the block. The block is
> sent out as the small header followed by a list of transactions.
> 5. Once a peer completes the download, it verifies the block, then enters
> step 2.
>
Mining pools currently connect to the "fast relay network". This is
optimised for fast block distribution. It does no validation and is only
for low latency propagation. The normal network is used as a fallback.
My understanding is that it works as follows:
Each miner runs a normal full node and a relay node on the same computer.
The full node tells the relay node whenever it receives a new transaction
via the inv message and the node requests the full transaction.
The relay node tells its relay peers that it knows about the transaction
(hash only) and its 4 byte key. This is not forwarded onwards, since the
relay peer only gets the hash of the transaction and doesn't do validation
anyway. The key is just a 4 byte counter.
Each relay node keeps a mapping of txid to key for each of its peer. There
is some garbage collection and entries are removed once the transaction is
included in a block (there might be a confirm threshold).
When a block is found, the local node sends it to the relay node. The
relay node then forwards it to all of its peers in a compact form.
The block is sent as a list of keys for that peer and full transactions are
only sent for unknown transactions.
When a relay node receives a block, it just verifies the POW, checks that
it is new and recent. It does not do tx validation. It forwards the block
to its local full node, which does the validation. Since the relay node is
on localhost, it never gets kicked due to sending invalid blocks. This
prevents a DOS attack where you could send invalid blocks to the relay node
and cause the local full node to kick it.
If all the transactions are already known, then it can forward a block for
only 4 bytes per transactions. I think it has an optimisation, so that is
compressed to 1 byte per tx.
--001a11440fa22fe4a005207a5a5b
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On T=
hu, Sep 24, 2015 at 12:12 AM, Jonathan Toomim (Toomim Bros) via bitcoin-dev=
<span dir=3D"ltr"><<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.=
org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a>></span>=
wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor=
der-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-wor=
d"><br><div><div dir=3D"ltr"><div><br></div><div>As I understand it, the cu=
rrent block propagation algorithm is this:</div><div><br></div><div>1. A no=
de mines a block.</div><div>2. It notifies its peers that it has a new bloc=
k with an=C2=A0<i>inv</i>. Typical nodes have 8 peers.=C2=A0</div><div>3. T=
he peers respond that they have not seen it, and request the block with=C2=
=A0<i>getdata</i>=C2=A0[hash].=C2=A0</div><div>4. The node sends out the bl=
ock in parallel to all 8 peers simultaneously. If the node's upstream b=
andwidth is limiting, then all peers will receive most of the block before =
any peer receives all of the block. The block is sent out as the small head=
er followed by a list of transactions.</div><div>5. Once a peer completes t=
he download, it verifies the block, then enters step 2.</div></div></div></=
div></blockquote><div><br></div><div>Mining pools currently connect to the =
"fast relay network".=C2=A0 This is optimised for fast block dist=
ribution.=C2=A0 It does no validation and is only for low latency propagati=
on.=C2=A0 The normal network is used as a fallback.<br><br></div><div>My un=
derstanding is that it works as follows:<br><br></div><div>Each miner runs =
a normal full node and a relay node on the same computer.<br><br></div><div=
>The full node tells the relay node whenever it receives a new transaction =
via the inv message and the node requests the full transaction.<br><br></di=
v><div>The relay node tells its relay peers that it knows about the transac=
tion (hash only) and its 4 byte key. This is not forwarded onwards, since t=
he relay peer only gets the hash of the transaction and doesn't do vali=
dation anyway.=C2=A0 The key is just a 4 byte counter.<br><br>Each relay no=
de keeps a mapping of txid to key for each of its peer.=C2=A0 There is some=
garbage collection and entries are removed once the transaction is include=
d in a block (there might be a confirm threshold).<br></div><div><br></div>=
<div>When a block is found, the local node sends it to the relay node.=C2=
=A0 The relay node then forwards it to all of its peers in a compact form.<=
br><br></div><div>The block is sent as a list of keys for that peer and ful=
l transactions are only sent for unknown transactions.<br><br></div><div>Wh=
en a relay node receives a block, it just verifies the POW, checks that it =
is new and recent.=C2=A0 It does not do tx validation.=C2=A0 It forwards th=
e block to its local full node, which does the validation.=C2=A0 Since the =
relay node is on localhost, it never gets kicked due to sending invalid blo=
cks.=C2=A0 This prevents a DOS attack where you could send invalid blocks t=
o the relay node and cause the local full node to kick it.<br><br></div><di=
v>If all the transactions are already known, then it can forward a block fo=
r only 4 bytes per transactions.=C2=A0 I think it has an optimisation, so t=
hat is compressed to 1 byte per tx.<br></div></div></div></div>
--001a11440fa22fe4a005207a5a5b--
|
