1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
|
Return-Path: <lf-lists@mattcorallo.com>
Received: from smtp2.linuxfoundation.org (smtp2.linux-foundation.org
[172.17.192.36])
by mail.linuxfoundation.org (Postfix) with ESMTPS id A5D38899
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 27 Mar 2017 19:43:37 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from mail.bluematt.me (mail.bluematt.me [192.241.179.72])
by smtp2.linuxfoundation.org (Postfix) with ESMTPS id 31B7C1DCA5
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 27 Mar 2017 19:43:36 +0000 (UTC)
Received: from [25.39.208.129] (unknown [172.56.39.39])
by mail.bluematt.me (Postfix) with ESMTPSA id 17186139BF1;
Mon, 27 Mar 2017 19:32:45 +0000 (UTC)
Date: Mon, 27 Mar 2017 19:32:30 +0000
In-Reply-To: <CAFp6fsHhqdL=MNyAAyfwA7qw5-MhW19Whky+kY3n3=+u61bXBg@mail.gmail.com>
References: <CAFp6fsHhqdL=MNyAAyfwA7qw5-MhW19Whky+kY3n3=+u61bXBg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----IG1U3IKD6KVHEP9LKVLCICMDMTV7UF"
Content-Transfer-Encoding: 7bit
To: Suhas Daftuar <sdaftuar@gmail.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>,
Suhas Daftuar via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>,
Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
From: Matt Corallo <lf-lists@mattcorallo.com>
Message-ID: <B12B2CE9-9A7A-4D8A-8393-528942285CCF@mattcorallo.com>
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,HTML_MESSAGE
autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp2.linux-foundation.org
Subject: Re: [bitcoin-dev] Segregated witness p2p layer compatibility
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Mar 2017 19:43:37 -0000
------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Just to expand a tiny bit here, while the testnet setup of only a few nodes=
acting as "bridges", mainnet already has many systems which act as effecti=
ve bridges today - there are several relay networks in use which effectivel=
y bypass the P2P network, including my legacy relay network (which many min=
ers historically have used, and I'd expect those who aren't paying attentio=
n and don't upgrade will not turn off, fixing the issue for them), ViaBTC's=
super aggressive bandwidth-wasting block announcement network which pushes=
blocks from several pools to many nodes globally, and Bitcoin=2Ecom's priv=
ate relay network=2E (Of course many other miners and pools have private re=
lay networks, but the several other such networks I'm aware of are already =
segwit-compatible, even for pools not signaling segwit)=2E
Matt
On March 27, 2017 12:22:43 PM PDT, Suhas Daftuar via bitcoin-dev <bitcoin-=
dev@lists=2Elinuxfoundation=2Eorg> wrote:
>Hi,
>
>There have been two threads recently that have made references to
>peer-to-peer implementation details in Bitcoin Core's Segregated
>Witness
>code that I would like to clarify=2E
>
>In the thread "Issolated Bitcoin Nodes" (
>https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/01=
3765=2Ehtml),
>there was some discussion about how Bitcoin Core's block download logic
>behaves after segwit activation=2E After segwit activation, Bitcoin Core
>nodes will not (currently) attempt to download any blocks from
>non-segwit
>peers (nodes that do not set the NODE WITNESS service bit)=2E This is a
>bandwidth optimization to prevent a node from downloading a block that
>may
>be invalid only because the sender omitted the witness, requiring
>re-download until the block is received with the required witness data=2E
>
>But to be clear, non-segwit blocks -- that is, blocks without a witness
>commitment in the coinbase, and whose transactions are serialized
>without
>witnesses, and whose transactions are not spending segwit outputs which
>require a witness -- are evaluated under the same rules as prior,
>pre-segwit versions of the software=2E So such non-segwit blocks that
>are
>valid to older, pre-segwit nodes are also valid to segwit-nodes=2E
>
>In
>https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/01=
3796=2Ehtml,
>Eric Voskuil wrote:
>
>Given the protocol requirements of the segwit proposal this is not the
>> case=2E A miner running pre-segwit code will produce blocks that no
>> segwit node will ever receive=2E
>
>
>The phrase "protocol requirements of segwit" is confusing here, because
>there are two different layers that need consideration: the consensus
>protocol layer and the peer-to-peer protocol layer=2E But in neither
>layer
>is the behavior of not downloading blocks from non-NODE WITNESS peers a
>"requirement"=2E This is an implementation detail in the Bitcoin Core
>code
>that alternate implementations compliant with BIP 144 could implement
>differently=2E
>
>At the consensus layer, non-segwit blocks (described above) that are
>valid
>to older nodes are also valid to segwit nodes=2E That means that if a
>miner
>was using an older, pre-segwit version of Bitcoin Core to produce
>blocks
>after segwit activates, that blocks they find will be valid to all
>nodes=2E
>
>At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only
>try
>to download those blocks if announced by a segwit-enabled peer=2E But
>this
>is not a protocol requirement; other implementations can remain
>compatible
>even they take different approaches here=2E (As an example, I could
>imagine
>an implementation that downloaded a new block from any peer, but if the
>block has a witness commitment in the coinbase and was received from a
>non-segwit peer, then the node would attempt re-download from a segwit
>peer=2E I'm sure many other reasonable block download strategies could
>be
>devised=2E)
>
>Still, if a miner wants to continue mining post-segwit activation, but
>using pre-segwit software, they would need a way to relay their blocks
>to a
>segwit-enabled peer=2E
>
>There are a few ways to do this that I can think of:
>
>- Use the RPC call "submitblock" on a segwit-enabled node=2E Calling
>"submitblock" on a Bitcoin Core 0=2E13=2E1 (0=2E13=2E0 in the case of tes=
tnet)
>or
>later node works fine as long as the block is valid (whether or not it
>has
>a witness commitment or witness transactions), and once a
>segwit-enabled
>peer has the block it will relay to other segwit peers=2E
>
>- Explicitly deliver the block to a segwit node over the p2p network,
>even
>if unrequested=2E Currently Bitcoin Core at least will process
>unrequested
>blocks, and valid blocks that update the tip will then be relayed to
>other
>peers=2E
>
>- Run a bridge node, which advertises NODE_WITNESS and can serialize
>blocks
>with witness data, which downloads blocks even from non-NODE WITNESS
>peers=2E Anyone can do this to bridge the networks for the benefit of
>the
>whole network (I have personally been running a few nodes that do this,
>for
>several months now), but miners concerned about this issue for their
>own
>blocks could explicitly do this themselves to ensure that their own
>blocks
>propagate to the segwit-enabled network=2E
>
>- Peer directly with other miners, bypassing the p2p network=2E Many
>miners
>do this already, using protocols which may already serve to bridge the
>network=2E
>
>So saying that "A miner running pre-segwit code will produce blocks
>that no
>segwit node will ever receive" is not really correct, in my view=2E If
>the
>whole network were just running Bitcoin Core software releases, and the
>miner was not able/willing to deliver their block to a segwit-enabled
>node
>(eg by using the RPC call "submitblock", or one of the other
>suggestions I
>had above), then I would agree with the statement=2E But given that
>there
>are bridge nodes on the network, and that miners have other options to
>relay their block, I think this is not an accurate portrayal of what
>would
>actually happen on the network -- I would expect that non-segwit
>miners'
>blocks would still get relayed post-segwit activation, even if only by
>the
>handful of bridge nodes that I expect are currently running=2E
>
>All that said, I do think this is an important detail to highlight and
>that
>this behavior should be better documented (I believe it deserves
>specific
>mention in a BIP), as this is an important issue for miners to be aware
>of=2E
------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head></head><body>Just to expand a tiny bit here, while the testnet =
setup of only a few nodes acting as "bridges", mainnet already ha=
s many systems which act as effective bridges today - there are several rel=
ay networks in use which effectively bypass the P2P network, including my l=
egacy relay network (which many miners historically have used, and I'd =
expect those who aren't paying attention and don't upgrade will not=
turn off, fixing the issue for them), ViaBTC's super aggressive bandwi=
dth-wasting block announcement network which pushes blocks from several poo=
ls to many nodes globally, and <a href=3D"http://Bitcoin=2Ecom">Bitcoin=2Ec=
om</a>'s private relay network=2E (Of course many other miners and pool=
s have private relay networks, but the several other such networks I'm =
aware of are already segwit-compatible, even for pools not signaling segwit=
)=2E<br>
<br>
Matt<br><br><div class=3D"gmail_quote">On March 27, 2017 12:22:43 PM PDT, =
Suhas Daftuar via bitcoin-dev <bitcoin-dev@lists=2Elinuxfoundation=2Eorg=
> wrote:<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0=
=2E8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir=3D"ltr">Hi,<div><br /></div><div>There have been two threads rece=
ntly that have made references to peer-to-peer implementation details in Bi=
tcoin Core's Segregated Witness code that I would like to clarify=2E</div><=
div><br /></div><div>In the thread "Issolated Bitcoin Nodes" (<a =
href=3D"https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-Ma=
rch/013765=2Ehtml">https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-=
dev/2017-March/013765=2Ehtml</a>), there was some discussion about how Bitc=
oin Core's block download logic behaves after segwit activation=2E Af=
ter segwit activation, Bitcoin Core nodes will not (currently) attempt to d=
ownload any blocks from non-segwit peers (nodes that do not set the NODE WI=
TNESS service bit)=2E This is a bandwidth optimization to prevent a n=
ode from downloading a block that may be invalid only because the sender om=
itted the witness, requiring re-download until the block is received with t=
he required witness data=2E</div><div><br /></div><div>But to be clear, non=
-segwit blocks -- that is, blocks without a witness commitment in the coinb=
ase, and whose transactions are serialized without witnesses, and whose tra=
nsactions are not spending segwit outputs which require a witness -- are ev=
aluated under the same rules as prior, pre-segwit versions of the software=
=2E So such non-segwit blocks that are valid to older, pre-segwit nod=
es are also valid to segwit-nodes=2E</div><div><br /></div><div>In <a href=
=3D"https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/=
013796=2Ehtml">https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/=
2017-March/013796=2Ehtml</a>, Eric Voskuil wrote:<br /></div><div><br /></d=
iv><div><blockquote style=3D"margin:0px 0px 0px 0=2E8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex" class=3D"gmail_quote">Given the proto=
col requirements of the segwit proposal this is not the<br />case=2E A mine=
r running pre-segwit code will produce blocks that no<br />segwit node will=
ever receive=2E</blockquote><div><br /></div><div>The phrase "protoco=
l requirements of segwit" is confusing here, because there are two dif=
ferent layers that need consideration: the consensus protocol layer and the=
peer-to-peer protocol layer=2E But in neither layer is the behavior =
of not downloading blocks from non-NODE WITNESS peers a "requirement&q=
uot;=2E This is an implementation detail in the Bitcoin Core code tha=
t alternate implementations compliant with BIP 144 could implement differen=
tly=2E</div></div><div><br /></div><div>At the consensus layer, non-segwit =
blocks (described above) that are valid to older nodes are also valid to se=
gwit nodes=2E That means that if a miner was using an older, pre-segw=
it version of Bitcoin Core to produce blocks after segwit activates, that b=
locks they find will be valid to all nodes=2E<br /></div><div><br /></div><=
div>At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only t=
ry to download those blocks if announced by a segwit-enabled peer=2E =
But this is not a protocol requirement; other implementations can remain co=
mpatible even they take different approaches here=2E (As an example, =
I could imagine an implementation that downloaded a new block from any peer=
, but if the block has a witness commitment in the coinbase and was receive=
d from a non-segwit peer, then the node would attempt re-download from a se=
gwit peer=2E I'm sure many other reasonable block download strategies=
could be devised=2E)</div><div><br /></div><div>Still, if a miner wants to=
continue mining post-segwit activation, but using pre-segwit software, the=
y would need a way to relay their blocks to a segwit-enabled peer=2E</div><=
div><br /></div><div>There are a few ways to do this that I can think of:</=
div><div><br /></div><div>- Use the RPC call "submitblock" on a s=
egwit-enabled node=2E Calling "submitblock" on a Bitcoin Co=
re 0=2E13=2E1 (0=2E13=2E0 in the case of testnet) or later node works fine =
as long as the block is valid (whether or not it has a witness commitment o=
r witness transactions), and once a segwit-enabled peer has the block it wi=
ll relay to other segwit peers=2E</div><div><br /></div><div>- Explicitly d=
eliver the block to a segwit node over the p2p network, even if unrequested=
=2E Currently Bitcoin Core at least will process unrequested blocks, =
and valid blocks that update the tip will then be relayed to other peers=2E=
</div><div><br /></div><div>- Run a bridge node, which advertises NODE_WITN=
ESS and can serialize blocks with witness data, which downloads blocks even=
from non-NODE WITNESS peers=2E Anyone can do this to bridge the netw=
orks for the benefit of the whole network (I have personally been running a=
few nodes that do this, for several months now), but miners concerned abou=
t this issue for their own blocks could explicitly do this themselves to en=
sure that their own blocks propagate to the segwit-enabled network=2E</div>=
<div><br /></div><div>- Peer directly with other miners, bypassing the p2p =
network=2E Many miners do this already, using protocols which may alr=
eady serve to bridge the network=2E</div><div><br /></div><div>So saying th=
at "A miner running pre-segwit code will produce blocks that no segwit=
node will ever receive" is not really correct, in my view=2E If=
the whole network were just running Bitcoin Core software releases, and th=
e miner was not able/willing to deliver their block to a segwit-enabled nod=
e (eg by using the RPC call "submitblock", or one of the other su=
ggestions I had above), then I would agree with the statement=2E But =
given that there are bridge nodes on the network, and that miners have othe=
r options to relay their block, I think this is not an accurate portrayal o=
f what would actually happen on the network -- I would expect that non-segw=
it miners' blocks would still get relayed post-segwit activation, even if o=
nly by the handful of bridge nodes that I expect are currently running=2E</=
div><div><br /></div><div><div>All that said, I do think this is an importa=
nt detail to highlight and that this behavior should be better documented (=
I believe it deserves specific mention in a BIP), as this is an important i=
ssue for miners to be aware of=2E</div></div></div>
</blockquote></div></body></html>
------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF--
|