Content-Type: multipart/alternative;
 boundary=Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
Content-Transfer-Encoding: 7bit
From: Eric Voskuil <eric@voskuil.org>
Mime-Version: 1.0 (1.0)
Date: Mon, 17 Aug 2020 14:21:53 -0700
Message-Id: <6FCCEC96-02F7-48D7-92EC-52234B94805B@voskuil.org>
References: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com>
In-Reply-To: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com>
To: Suhas Daftuar <sdaftuar@gmail.com>
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p
	connections are setup
Precedence: list


--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Suhas,

It seems to me that your first two paragraphs contradict each other, so I=E2=
=80=99m not sure we have understanding. As you say in the first paragraph, a=
 peer would never get messages that it does not understand, so there is no c=
hance that missing a protocol change would matter.

In case it=E2=80=99s not understood, version negotiation provides each peer w=
ith the maximum supported protocol version of the other. Once complete both h=
ave negotiated to the highest common version. No message not supported at th=
at version may be sent by either.

If the protocol was to accept *any* message traffic then it will cease to be=
 a protocol. People will drop in their changes without obtaining broad suppo=
rt, and peers will evolve to the point of no longer being peers. They won=E2=
=80=99t speak the same language and the =E2=80=9Cnetwork=E2=80=9D will be li=
ttle more than a broadcast transport, broadcasting all traffic to all peers,=
 whether it=E2=80=99s for them or not.

People need to either build support or build a distinct network. That=E2=80=99=
s the actual coordination issue, which is inherent to protocol development.

Best,
e

> On Aug 17, 2020, at 13:40, Suhas Daftuar <sdaftuar@gmail.com> wrote:
>=20
> Hi Eric,
>=20
> Thanks for your response.  If I understand correctly, you're suggesting th=
at in the future we do the same as what was done in BIP 339, of accompanying=
 new messages (which are optional) with a protocol version bump, so that net=
work clients are never reading unknown messages from a peer (and can be free=
 to disconnect a peer for sending an unknown message)?
>=20
> I think that works fine, so if indeed there will be software that will exp=
ect things to operate this way then I can withdraw the suggestion I've made i=
n this thread.  However I wanted to clarify that this is what you suggest, b=
ecause there is another downside to this approach (beyond the sequential nat=
ure of sequence numbers that you mention) -- if a software implementation mi=
sses a proposed new protocol upgrade, and thus fails to parse (and ignore) s=
ome proposed new message, the result can be a network split down the road as=
 incompatible clients get slowly upgraded over time.=20
>=20
> I think this coordination cost is something to be concerned about -- for i=
nstance, the lack of response to my wtxid-relay proposal made me wonder if o=
ther software would be implementing something to account for the new message=
 that proposal introduces (for clients with version >=3D 70016).  It's reaso=
nable for people to be busy and miss things like this, and I think it's wort=
h considering whether there's a safer way for us to deploy changes.
>=20
> That said, I don't think this coordination cost is unbearable, so as long a=
s we have a process for making p2p protocol improvements I'm not too worried=
 about what mechanism we use.  So if this concern over coordination of chang=
es doesn't sway you, I think we can continue to just bump protocol version a=
t the same time as deploying new messages, as we have been doing, and hope t=
hat we don't run into problems down the road. =20
>=20
> If I have misunderstood how you think we should be making future protocol c=
hanges, please let me know.
>=20
> Thanks,
> Suhas
>=20
>=20
>=20
>> On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil <eric@voskuil.org> wrote:
>> A requirement to ignore unknown (invalid) messages is not only a protocol=
 breaking change but poor protocol design. The purpose of version negotiatio=
n is to determine the set of valid messages. Changes to version negotiation i=
tself are very problematic.
>>=20
>> The only limitation presented by versioning is that the system is sequent=
ial. As such, clients that do not wish to implement (or operators who do not=
 wish to enable) them are faced with a problem when wanting to support later=
 features. This is resolvable by making such features optional at the new pr=
otocol level. This allows each client to limit its communication to the nego=
tiated protocol, and allows ignoring of known but unsupported/disabled featu=
res.
>>=20
>> Sorry I missed your earlier post. Been distracted for a while.
>>=20
>> e
>>=20
>>=20
>>>> On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev <bitcoin-dev@l=
ists.linuxfoundation.org> wrote:
>>>>=20
>>> =EF=BB=BF
>>> Hi,
>>>=20
>>> Back in February I posted a proposal for WTXID-based transaction relay[1=
] (now known as BIP 339), which included a proposal for feature negotiation t=
o take place prior to the VERACK message being received by each side.  In my=
 email to this list, I had asked for feedback as to whether that proposal wa=
s problematic, and didn't receive any responses.
>>>=20
>>> Since then, the implementation of BIP 339 has been merged into Bitcoin C=
ore, though it has not yet been released.
>>>=20
>>> In thinking about the mechanism used there, I thought it would be helpfu=
l to codify in a BIP the idea that Bitcoin network clients should ignore unk=
nown messages received before a VERACK.  A draft of my proposal is available=
 here[2].
>>>=20
>>> I presume that software upgrading past protocol version 70016 was alread=
y planning to either implement BIP 339, or ignore the wtxidrelay message pro=
posed in BIP 339 (if not, then this would create network split concerns in t=
he future -- so I hope that someone would speak up if this were a problem). =
 When we propose future protocol upgrades that would benefit from feature ne=
gotiation at the time of connection, I think it would be nice to be able to u=
se the same method as proposed in BIP 339, without even needing to bump the p=
rotocol version.  So having an understanding that this is the standard of ho=
w other network clients operate would be helpful.
>>>=20
>>> If, on the other hand, this is problematic for some reason, I look forwa=
rd to hearing that as well, so that we can be careful about how we deploy fu=
ture p2p changes to avoid disruption.
>>>=20
>>> Thanks,
>>> Suhas Daftuar
>>>=20
>>> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Februar=
y/017648.html
>>>=20
>>> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-ne=
gotiation/bip-p2p-feature-negotiation.mediawiki
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div dir=3D"ltr">Hi Suhas,</div><div dir=3D=
"ltr"><br></div><div dir=3D"ltr">It seems to me that your first two paragrap=
hs contradict each other, so I=E2=80=99m not sure we have understanding. As y=
ou say in the first paragraph, a peer would never get messages that it does n=
ot understand, so there is no chance that missing a protocol change would ma=
tter.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">In case it=E2=80=99s n=
ot understood, version negotiation provides each peer with the maximum suppo=
rted protocol version of the other. Once complete both have negotiated to th=
e highest common version. No message not supported at that version may be se=
nt by either.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">If the protoc=
ol was to accept *any* message traffic then it will cease to be a protocol. P=
eople will drop in their changes without obtaining broad support, and peers w=
ill evolve to the point of no longer being peers. They won=E2=80=99t speak t=
he same language and the =E2=80=9Cnetwork=E2=80=9D will be little more than a=
 broadcast transport, broadcasting all traffic to all peers, whether it=E2=80=
=99s for them or not.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Peopl=
e need to either build support or build a distinct network. That=E2=80=99s t=
he actual coordination issue, which is inherent to protocol development.</di=
v><div dir=3D"ltr"><br></div><div dir=3D"ltr">Best,</div><div dir=3D"ltr">e<=
/div><div dir=3D"ltr"><br><blockquote type=3D"cite">On Aug 17, 2020, at 13:4=
0, Suhas Daftuar &lt;sdaftuar@gmail.com&gt; wrote:<br><br></blockquote></div=
><blockquote type=3D"cite"><div dir=3D"ltr"><div dir=3D"ltr">Hi Eric,<div><b=
r></div><div>Thanks for your response.&nbsp; If I understand correctly, you'=
re suggesting that in the future we do the same as what was done in BIP 339,=
 of accompanying new messages (which are optional) with a protocol version b=
ump, so that network clients are never reading unknown messages from a peer (=
and can be free to disconnect a peer for sending an unknown message)?</div><=
div><br></div><div>I think that works fine, so if indeed there will be softw=
are that will expect things to operate this way then I can withdraw the sugg=
estion I've made in this thread.&nbsp; However I wanted to clarify that this=
 is what you suggest, because there is another downside to this approach (be=
yond the sequential nature of sequence numbers that you mention) -- if a sof=
tware implementation misses a proposed new protocol upgrade, and thus fails t=
o parse (and ignore) some proposed new message, the result can be a network s=
plit down the road as incompatible clients get slowly upgraded over time.&nb=
sp;</div><div><br></div><div>I think this coordination cost is something to b=
e concerned about -- for instance, the lack of response to my wtxid-relay pr=
oposal made me wonder if other software would be implementing something to a=
ccount for the new message that proposal introduces (for clients with versio=
n &gt;=3D 70016).&nbsp; It's reasonable for people to be busy and miss thing=
s like this, and I think it's worth considering whether there's a safer way f=
or us to deploy changes.</div><div><br></div><div>That said, I don't think t=
his coordination cost is unbearable, so as long as we have a process for mak=
ing p2p protocol improvements I'm not too worried about what mechanism we us=
e.&nbsp; So if this concern over coordination of changes doesn't sway you, I=
 think we can continue to just bump protocol version at the same time as dep=
loying new messages, as we have been doing, and hope that we don't run into p=
roblems down the road.&nbsp;&nbsp;</div><div><br></div><div>If I have misund=
erstood how you think we should be making future protocol changes, please le=
t me know.</div><div><br></div><div>Thanks,</div><div>Suhas</div><div><br></=
div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla=
ss=3D"gmail_attr">On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil &lt;<a href=3D=
"mailto:eric@voskuil.org" target=3D"_blank">eric@voskuil.org</a>&gt; wrote:<=
br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto">=
<div dir=3D"ltr"><div dir=3D"ltr">A requirement to ignore unknown (invalid) m=
essages is not only a protocol breaking change but poor protocol design. The=
 purpose of version negotiation is to determine the set of valid messages. C=
hanges to version negotiation itself are very problematic.</div><div dir=3D"=
ltr"><br></div><div dir=3D"ltr">The only limitation presented by versioning i=
s that the system is sequential. As such, clients that do not wish to implem=
ent (or operators who do not wish to enable) them are faced with a problem w=
hen wanting to support later features. This is resolvable by making such fea=
tures optional at the new protocol level. This allows each client to limit i=
ts communication to the negotiated protocol, and allows ignoring of known bu=
t unsupported/disabled features.</div><div dir=3D"ltr"><br></div><div dir=3D=
"ltr">Sorry I missed your earlier post. Been distracted for a while.</div><d=
iv dir=3D"ltr"><br></div><div dir=3D"ltr">e</div><div dir=3D"ltr"><br></div>=
<div dir=3D"ltr"><br></div></div><div dir=3D"ltr"><blockquote type=3D"cite">=
On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev &lt;<a href=3D"mail=
to:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@list=
s.linuxfoundation.org</a>&gt; wrote:<br><br></blockquote></div><blockquote t=
ype=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"ltr">Hi,<div><br></div><d=
iv>Back in February I posted a proposal for WTXID-based transaction relay[1]=
 (now known as BIP 339), which included a proposal for feature negotiation t=
o take place prior to the VERACK message being received by each side.&nbsp; I=
n my email to this list, I had asked for feedback as to whether that proposa=
l was problematic, and didn't receive any responses.</div><div><br></div><di=
v>Since then, the implementation&nbsp;of BIP 339 has been merged into Bitcoi=
n Core, though it has not yet been released.</div><div><br></div><div>In thi=
nking about the mechanism used there, I thought it would&nbsp;be helpful to c=
odify in a BIP the idea that Bitcoin network clients should ignore unknown m=
essages received before a VERACK.&nbsp; A draft of my proposal is available h=
ere[2].</div><div><br></div><div>I presume that software upgrading past prot=
ocol version 70016 was already planning to either implement BIP 339, or igno=
re the wtxidrelay message proposed in BIP 339 (if not, then this would creat=
e network split concerns in the future -- so I hope that someone would speak=
 up if this were a problem).&nbsp; When we propose future protocol upgrades t=
hat would benefit from feature negotiation at the time of connection, I thin=
k it would be nice to be able to use the same method as proposed in BIP 339,=
 without even needing to bump the protocol version.&nbsp; So having an under=
standing that this is the standard of how other network clients operate woul=
d be helpful.</div><div><br></div><div>If, on the other hand, this is proble=
matic for some reason, I look forward to hearing that as well, so that we ca=
n be careful about how we deploy future p2p changes to avoid disruption.</di=
v><div><br></div><div>Thanks,</div><div>Suhas Daftuar</div><div><br></div><d=
iv>[1]&nbsp;<a href=3D"https://lists.linuxfoundation.org/pipermail/bitcoin-d=
ev/2020-February/017648.html" target=3D"_blank">https://lists.linuxfoundatio=
n.org/pipermail/bitcoin-dev/2020-February/017648.html</a><br></div><div><br>=
</div><div>[2]&nbsp;<a href=3D"https://github.com/sdaftuar/bips/blob/2020-08=
-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki" targ=
et=3D"_blank">https://github.com/sdaftuar/bips/blob/2020-08-generalized-feat=
ure-negotiation/bip-p2p-feature-negotiation.mediawiki</a></div></div>
<span>_______________________________________________</span><br><span>bitcoi=
n-dev mailing list</span><br><span><a href=3D"mailto:bitcoin-dev@lists.linux=
foundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><=
/span><br><span><a href=3D"https://lists.linuxfoundation.org/mailman/listinf=
o/bitcoin-dev" target=3D"_blank">https://lists.linuxfoundation.org/mailman/l=
istinfo/bitcoin-dev</a></span><br></div></blockquote></div></blockquote></di=
v>
</div></blockquote></body></html>=

--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7--