path: root/20/21a79fd9ac6c80cf94f84e66ef332b585a25e2

Return-Path: <lf-lists@mattcorallo.com>
Received: from smtp2.linuxfoundation.org (smtp2.linux-foundation.org
	[172.17.192.36])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id A5D38899
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Mon, 27 Mar 2017 19:43:37 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from mail.bluematt.me (mail.bluematt.me [192.241.179.72])
	by smtp2.linuxfoundation.org (Postfix) with ESMTPS id 31B7C1DCA5
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Mon, 27 Mar 2017 19:43:36 +0000 (UTC)
Received: from [25.39.208.129] (unknown [172.56.39.39])
	by mail.bluematt.me (Postfix) with ESMTPSA id 17186139BF1;
	Mon, 27 Mar 2017 19:32:45 +0000 (UTC)
Date: Mon, 27 Mar 2017 19:32:30 +0000
In-Reply-To: <CAFp6fsHhqdL=MNyAAyfwA7qw5-MhW19Whky+kY3n3=+u61bXBg@mail.gmail.com>
References: <CAFp6fsHhqdL=MNyAAyfwA7qw5-MhW19Whky+kY3n3=+u61bXBg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----IG1U3IKD6KVHEP9LKVLCICMDMTV7UF"
Content-Transfer-Encoding: 7bit
To: Suhas Daftuar <sdaftuar@gmail.com>,
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>,
	Suhas Daftuar via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>,
	Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
From: Matt Corallo <lf-lists@mattcorallo.com>
Message-ID: <B12B2CE9-9A7A-4D8A-8393-528942285CCF@mattcorallo.com>
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,HTML_MESSAGE
	autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp2.linux-foundation.org
Subject: Re: [bitcoin-dev] Segregated witness p2p layer compatibility
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Mar 2017 19:43:37 -0000

------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable

Just to expand a tiny bit here, while the testnet setup of only a few nodes=
 acting as "bridges", mainnet already has many systems which act as effecti=
ve bridges today - there are several relay networks in use which effectivel=
y bypass the P2P network, including my legacy relay network (which many min=
ers historically have used, and I'd expect those who aren't paying attentio=
n and don't upgrade will not turn off, fixing the issue for them), ViaBTC's=
 super aggressive bandwidth-wasting block announcement network which pushes=
 blocks from several pools to many nodes globally, and Bitcoin=2Ecom's priv=
ate relay network=2E (Of course many other miners and pools have private re=
lay networks, but the several other such networks I'm aware of are already =
segwit-compatible, even for pools not signaling segwit)=2E

Matt

On March 27, 2017 12:22:43 PM PDT, Suhas Daftuar via bitcoin-dev <bitcoin-=
dev@lists=2Elinuxfoundation=2Eorg> wrote:
>Hi,
>
>There have been two threads recently that have made references to
>peer-to-peer implementation details in Bitcoin Core's Segregated
>Witness
>code that I would like to clarify=2E
>
>In the thread "Issolated Bitcoin Nodes" (
>https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/01=
3765=2Ehtml),
>there was some discussion about how Bitcoin Core's block download logic
>behaves after segwit activation=2E  After segwit activation, Bitcoin Core
>nodes will not (currently) attempt to download any blocks from
>non-segwit
>peers (nodes that do not set the NODE WITNESS service bit)=2E  This is a
>bandwidth optimization to prevent a node from downloading a block that
>may
>be invalid only because the sender omitted the witness, requiring
>re-download until the block is received with the required witness data=2E
>
>But to be clear, non-segwit blocks -- that is, blocks without a witness
>commitment in the coinbase, and whose transactions are serialized
>without
>witnesses, and whose transactions are not spending segwit outputs which
>require a witness -- are evaluated under the same rules as prior,
>pre-segwit versions of the software=2E  So such non-segwit blocks that
>are
>valid to older, pre-segwit nodes are also valid to segwit-nodes=2E
>
>In
>https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/01=
3796=2Ehtml,
>Eric Voskuil wrote:
>
>Given the protocol requirements of the segwit proposal this is not the
>> case=2E A miner running pre-segwit code will produce blocks that no
>> segwit node will ever receive=2E
>
>
>The phrase "protocol requirements of segwit" is confusing here, because
>there are two different layers that need consideration: the consensus
>protocol layer and the peer-to-peer protocol layer=2E  But in neither
>layer
>is the behavior of not downloading blocks from non-NODE WITNESS peers a
>"requirement"=2E  This is an implementation detail in the Bitcoin Core
>code
>that alternate implementations compliant with BIP 144 could implement
>differently=2E
>
>At the consensus layer, non-segwit blocks (described above) that are
>valid
>to older nodes are also valid to segwit nodes=2E  That means that if a
>miner
>was using an older, pre-segwit version of Bitcoin Core to produce
>blocks
>after segwit activates, that blocks they find will be valid to all
>nodes=2E
>
>At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only
>try
>to download those blocks if announced by a segwit-enabled peer=2E  But
>this
>is not a protocol requirement; other implementations can remain
>compatible
>even they take different approaches here=2E  (As an example, I could
>imagine
>an implementation that downloaded a new block from any peer, but if the
>block has a witness commitment in the coinbase and was received from a
>non-segwit peer, then the node would attempt re-download from a segwit
>peer=2E  I'm sure many other reasonable block download strategies could
>be
>devised=2E)
>
>Still, if a miner wants to continue mining post-segwit activation, but
>using pre-segwit software, they would need a way to relay their blocks
>to a
>segwit-enabled peer=2E
>
>There are a few ways to do this that I can think of:
>
>- Use the RPC call "submitblock" on a segwit-enabled node=2E  Calling
>"submitblock" on a Bitcoin Core 0=2E13=2E1 (0=2E13=2E0 in the case of tes=
tnet)
>or
>later node works fine as long as the block is valid (whether or not it
>has
>a witness commitment or witness transactions), and once a
>segwit-enabled
>peer has the block it will relay to other segwit peers=2E
>
>- Explicitly deliver the block to a segwit node over the p2p network,
>even
>if unrequested=2E  Currently Bitcoin Core at least will process
>unrequested
>blocks, and valid blocks that update the tip will then be relayed to
>other
>peers=2E
>
>- Run a bridge node, which advertises NODE_WITNESS and can serialize
>blocks
>with witness data, which downloads blocks even from non-NODE WITNESS
>peers=2E  Anyone can do this to bridge the networks for the benefit of
>the
>whole network (I have personally been running a few nodes that do this,
>for
>several months now), but miners concerned about this issue for their
>own
>blocks could explicitly do this themselves to ensure that their own
>blocks
>propagate to the segwit-enabled network=2E
>
>- Peer directly with other miners, bypassing the p2p network=2E  Many
>miners
>do this already, using protocols which may already serve to bridge the
>network=2E
>
>So saying that "A miner running pre-segwit code will produce blocks
>that no
>segwit node will ever receive" is not really correct, in my view=2E  If
>the
>whole network were just running Bitcoin Core software releases, and the
>miner was not able/willing to deliver their block to a segwit-enabled
>node
>(eg by using the RPC call "submitblock", or one of the other
>suggestions I
>had above), then I would agree with the statement=2E  But given that
>there
>are bridge nodes on the network, and that miners have other options to
>relay their block, I think this is not an accurate portrayal of what
>would
>actually happen on the network -- I would expect that non-segwit
>miners'
>blocks would still get relayed post-segwit activation, even if only by
>the
>handful of bridge nodes that I expect are currently running=2E
>
>All that said, I do think this is an important detail to highlight and
>that
>this behavior should be better documented (I believe it deserves
>specific
>mention in a BIP), as this is an important issue for miners to be aware
>of=2E

------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF
Content-Type: text/html;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head></head><body>Just to expand a tiny bit here, while the testnet =
setup of only a few nodes acting as &quot;bridges&quot;, mainnet already ha=
s many systems which act as effective bridges today - there are several rel=
ay networks in use which effectively bypass the P2P network, including my l=
egacy relay network (which many miners historically have used, and I&#39;d =
expect those who aren&#39;t paying attention and don&#39;t upgrade will not=
 turn off, fixing the issue for them), ViaBTC&#39;s super aggressive bandwi=
dth-wasting block announcement network which pushes blocks from several poo=
ls to many nodes globally, and <a href=3D"http://Bitcoin=2Ecom">Bitcoin=2Ec=
om</a>&#39;s private relay network=2E (Of course many other miners and pool=
s have private relay networks, but the several other such networks I&#39;m =
aware of are already segwit-compatible, even for pools not signaling segwit=
)=2E<br>
<br>
Matt<br><br><div class=3D"gmail_quote">On March 27, 2017 12:22:43 PM PDT, =
Suhas Daftuar via bitcoin-dev &lt;bitcoin-dev@lists=2Elinuxfoundation=2Eorg=
&gt; wrote:<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0=
=2E8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir=3D"ltr">Hi,<div><br /></div><div>There have been two threads rece=
ntly that have made references to peer-to-peer implementation details in Bi=
tcoin Core's Segregated Witness code that I would like to clarify=2E</div><=
div><br /></div><div>In the thread &quot;Issolated Bitcoin Nodes&quot; (<a =
href=3D"https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-Ma=
rch/013765=2Ehtml">https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-=
dev/2017-March/013765=2Ehtml</a>), there was some discussion about how Bitc=
oin Core's block download logic behaves after segwit activation=2E&nbsp; Af=
ter segwit activation, Bitcoin Core nodes will not (currently) attempt to d=
ownload any blocks from non-segwit peers (nodes that do not set the NODE WI=
TNESS service bit)=2E&nbsp; This is a bandwidth optimization to prevent a n=
ode from downloading a block that may be invalid only because the sender om=
itted the witness, requiring re-download until the block is received with t=
he required witness data=2E</div><div><br /></div><div>But to be clear, non=
-segwit blocks -- that is, blocks without a witness commitment in the coinb=
ase, and whose transactions are serialized without witnesses, and whose tra=
nsactions are not spending segwit outputs which require a witness -- are ev=
aluated under the same rules as prior, pre-segwit versions of the software=
=2E&nbsp; So such non-segwit blocks that are valid to older, pre-segwit nod=
es are also valid to segwit-nodes=2E</div><div><br /></div><div>In <a href=
=3D"https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/2017-March/=
013796=2Ehtml">https://lists=2Elinuxfoundation=2Eorg/pipermail/bitcoin-dev/=
2017-March/013796=2Ehtml</a>, Eric Voskuil wrote:<br /></div><div><br /></d=
iv><div><blockquote style=3D"margin:0px 0px 0px 0=2E8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex" class=3D"gmail_quote">Given the proto=
col requirements of the segwit proposal this is not the<br />case=2E A mine=
r running pre-segwit code will produce blocks that no<br />segwit node will=
 ever receive=2E</blockquote><div><br /></div><div>The phrase &quot;protoco=
l requirements of segwit&quot; is confusing here, because there are two dif=
ferent layers that need consideration: the consensus protocol layer and the=
 peer-to-peer protocol layer=2E&nbsp; But in neither layer is the behavior =
of not downloading blocks from non-NODE WITNESS peers a &quot;requirement&q=
uot;=2E&nbsp; This is an implementation detail in the Bitcoin Core code tha=
t alternate implementations compliant with BIP 144 could implement differen=
tly=2E</div></div><div><br /></div><div>At the consensus layer, non-segwit =
blocks (described above) that are valid to older nodes are also valid to se=
gwit nodes=2E&nbsp; That means that if a miner was using an older, pre-segw=
it version of Bitcoin Core to produce blocks after segwit activates, that b=
locks they find will be valid to all nodes=2E<br /></div><div><br /></div><=
div>At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only t=
ry to download those blocks if announced by a segwit-enabled peer=2E&nbsp; =
But this is not a protocol requirement; other implementations can remain co=
mpatible even they take different approaches here=2E &nbsp;(As an example, =
I could imagine an implementation that downloaded a new block from any peer=
, but if the block has a witness commitment in the coinbase and was receive=
d from a non-segwit peer, then the node would attempt re-download from a se=
gwit peer=2E&nbsp; I'm sure many other reasonable block download strategies=
 could be devised=2E)</div><div><br /></div><div>Still, if a miner wants to=
 continue mining post-segwit activation, but using pre-segwit software, the=
y would need a way to relay their blocks to a segwit-enabled peer=2E</div><=
div><br /></div><div>There are a few ways to do this that I can think of:</=
div><div><br /></div><div>- Use the RPC call &quot;submitblock&quot; on a s=
egwit-enabled node=2E&nbsp; Calling &quot;submitblock&quot; on a Bitcoin Co=
re 0=2E13=2E1 (0=2E13=2E0 in the case of testnet) or later node works fine =
as long as the block is valid (whether or not it has a witness commitment o=
r witness transactions), and once a segwit-enabled peer has the block it wi=
ll relay to other segwit peers=2E</div><div><br /></div><div>- Explicitly d=
eliver the block to a segwit node over the p2p network, even if unrequested=
=2E&nbsp; Currently Bitcoin Core at least will process unrequested blocks, =
and valid blocks that update the tip will then be relayed to other peers=2E=
</div><div><br /></div><div>- Run a bridge node, which advertises NODE_WITN=
ESS and can serialize blocks with witness data, which downloads blocks even=
 from non-NODE WITNESS peers=2E&nbsp; Anyone can do this to bridge the netw=
orks for the benefit of the whole network (I have personally been running a=
 few nodes that do this, for several months now), but miners concerned abou=
t this issue for their own blocks could explicitly do this themselves to en=
sure that their own blocks propagate to the segwit-enabled network=2E</div>=
<div><br /></div><div>- Peer directly with other miners, bypassing the p2p =
network=2E&nbsp; Many miners do this already, using protocols which may alr=
eady serve to bridge the network=2E</div><div><br /></div><div>So saying th=
at &quot;A miner running pre-segwit code will produce blocks that no segwit=
 node will ever receive&quot; is not really correct, in my view=2E&nbsp; If=
 the whole network were just running Bitcoin Core software releases, and th=
e miner was not able/willing to deliver their block to a segwit-enabled nod=
e (eg by using the RPC call &quot;submitblock&quot;, or one of the other su=
ggestions I had above), then I would agree with the statement=2E&nbsp; But =
given that there are bridge nodes on the network, and that miners have othe=
r options to relay their block, I think this is not an accurate portrayal o=
f what would actually happen on the network -- I would expect that non-segw=
it miners' blocks would still get relayed post-segwit activation, even if o=
nly by the handful of bridge nodes that I expect are currently running=2E</=
div><div><br /></div><div><div>All that said, I do think this is an importa=
nt detail to highlight and that this behavior should be better documented (=
I believe it deserves specific mention in a BIP), as this is an important i=
ssue for miners to be aware of=2E</div></div></div>
</blockquote></div></body></html>
------IG1U3IKD6KVHEP9LKVLCICMDMTV7UF--