summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorEric Voskuil <eric@voskuil.org>2020-08-17 14:21:53 -0700
committerbitcoindev <bitcoindev@gnusha.org>2020-08-17 21:51:26 +0000
commit3b29a6562f68d57c2813404e1cbcd899570a9908 (patch)
tree1e6f8f7e4592d69f19851895a8f912ce3360dee5
parent20995872dc2a0ccf1aacf20fbfdd9cba8c3c1255 (diff)
downloadpi-bitcoindev-3b29a6562f68d57c2813404e1cbcd899570a9908.tar.gz
pi-bitcoindev-3b29a6562f68d57c2813404e1cbcd899570a9908.zip
Re: [bitcoin-dev] Generalizing feature negotiation when new p2p connections are setup
-rw-r--r--c3/ab56d3586b358b17963b74301812ce8bd710da343
1 files changed, 343 insertions, 0 deletions
diff --git a/c3/ab56d3586b358b17963b74301812ce8bd710da b/c3/ab56d3586b358b17963b74301812ce8bd710da
new file mode 100644
index 000000000..8d869f59b
--- /dev/null
+++ b/c3/ab56d3586b358b17963b74301812ce8bd710da
@@ -0,0 +1,343 @@
+Return-Path: <eric@voskuil.org>
+Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136])
+ by lists.linuxfoundation.org (Postfix) with ESMTP id 08777C0051
+ for <bitcoin-dev@lists.linuxfoundation.org>;
+ Mon, 17 Aug 2020 21:51:26 +0000 (UTC)
+Received: from localhost (localhost [127.0.0.1])
+ by silver.osuosl.org (Postfix) with ESMTP id BD35F2035C
+ for <bitcoin-dev@lists.linuxfoundation.org>;
+ Mon, 17 Aug 2020 21:51:25 +0000 (UTC)
+X-Virus-Scanned: amavisd-new at osuosl.org
+Received: from silver.osuosl.org ([127.0.0.1])
+ by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
+ with ESMTP id 1GFKIYmVxNqY
+ for <bitcoin-dev@lists.linuxfoundation.org>;
+ Mon, 17 Aug 2020 21:51:21 +0000 (UTC)
+X-Greylist: delayed 00:23:37 by SQLgrey-1.7.6
+Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com
+ [209.85.167.195])
+ by silver.osuosl.org (Postfix) with ESMTPS id 04B5F20450
+ for <bitcoin-dev@lists.linuxfoundation.org>;
+ Mon, 17 Aug 2020 21:51:20 +0000 (UTC)
+Received: by mail-oi1-f195.google.com with SMTP id j7so16140871oij.9
+ for <bitcoin-dev@lists.linuxfoundation.org>;
+ Mon, 17 Aug 2020 14:51:20 -0700 (PDT)
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
+ d=voskuil-org.20150623.gappssmtp.com; s=20150623;
+ h=content-transfer-encoding:from:mime-version:subject:date:message-id
+ :references:cc:in-reply-to:to;
+ bh=F5YBEVlobZzZ6l9qMRq1mzwd98RmqT7VPrHutsyhf5U=;
+ b=iWxPh4DA1jP9EWH2b5Asl2EiDML1tk3LGgBJVwPkPpUqyoze3P5xBTu1gi7ZbculWe
+ i4StOgolAFgU0I1vrZ47mJHkR4vYnlSLfU58MFxntnSz7mba28aZiVh80HF9vn9afUFA
+ qWzBYoDk66QOIfhkcAFf05ug51f0VQME493jPVy4TEE0BEh2mWnQzaNJ70zuajE7jooL
+ J4ho9lsJZvBTLpjiruw10w74Dyw1KAuDrUWk71DAa0bRu3e+ZgT6SUaWqw6dLs/1eZbC
+ DO1vf1/CRYxpG3oQiyHRArLEkUeujLNk1+uN4rlG+PDFgHqVjYO12S3GhnlIFVEFB68n
+ 7AeA==
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
+ d=1e100.net; s=20161025;
+ h=x-gm-message-state:content-transfer-encoding:from:mime-version
+ :subject:date:message-id:references:cc:in-reply-to:to;
+ bh=F5YBEVlobZzZ6l9qMRq1mzwd98RmqT7VPrHutsyhf5U=;
+ b=Cc6+k8aw9TPO/pAsGLaiJD80cjzHAMYNzN/uQsOfRN4DTNQvl3PlBnCXx+sBiCKEbv
+ 6saJOmXg3S0YrbTe8G187hW8yU256NX9eb57hfxICt8v1Wr/DcWrox6Gt2I+OeNYMYUk
+ pFdUc4JLFfKer0zIlLUHsaNCpLnQ3uW8M7K/cQCvM77Tu54iUNvFjBm12z0a/8QR1MSd
+ gOGq0dMfgMBlt3gSKRkZNcm4wFC0fH0576trxKaP1BOIv531ciRGzQNC4a9+tgtHhE9z
+ v/qnWqmNwnw6wBKVJKWB5xRj5YvMJcPa0Lia4PAJr4UY2DYLJkcfjnEPTn9ArP0qcY71
+ 7GcQ==
+X-Gm-Message-State: AOAM532tr4vsIpWGKo8BBgY/IDGQRFI2KQXU6uAA9xlOAQ1KQRnAbUne
+ R4NfTBWNkTG4ZEXBdzdZIvckqKOeRTdBTw==
+X-Google-Smtp-Source: ABdhPJyQuDp36nUf/1q+MpGqFqe3lVdSXwH5wD9QD1fjgdRzBWTVTlZ+e7yR79imwiQ7uRDB8TGxoA==
+X-Received: by 2002:a17:90a:5298:: with SMTP id
+ w24mr13343912pjh.221.1597699316124;
+ Mon, 17 Aug 2020 14:21:56 -0700 (PDT)
+Received: from ?IPv6:2600:380:7066:ce6e:c976:12fb:75f1:c075?
+ ([2600:380:7066:ce6e:c976:12fb:75f1:c075])
+ by smtp.gmail.com with ESMTPSA id x14sm21340292pfj.157.2020.08.17.14.21.54
+ (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
+ Mon, 17 Aug 2020 14:21:55 -0700 (PDT)
+Content-Type: multipart/alternative;
+ boundary=Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
+Content-Transfer-Encoding: 7bit
+From: Eric Voskuil <eric@voskuil.org>
+Mime-Version: 1.0 (1.0)
+Date: Mon, 17 Aug 2020 14:21:53 -0700
+Message-Id: <6FCCEC96-02F7-48D7-92EC-52234B94805B@voskuil.org>
+References: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com>
+In-Reply-To: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com>
+To: Suhas Daftuar <sdaftuar@gmail.com>
+X-Mailer: iPhone Mail (17G68)
+X-Mailman-Approved-At: Mon, 17 Aug 2020 21:53:52 +0000
+Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
+Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p
+ connections are setup
+X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
+X-Mailman-Version: 2.1.15
+Precedence: list
+List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
+List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
+ <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
+List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
+List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
+List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
+List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
+ <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
+X-List-Received-Date: Mon, 17 Aug 2020 21:51:26 -0000
+
+
+--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
+Content-Type: text/plain;
+ charset=utf-8
+Content-Transfer-Encoding: quoted-printable
+
+Hi Suhas,
+
+It seems to me that your first two paragraphs contradict each other, so I=E2=
+=80=99m not sure we have understanding. As you say in the first paragraph, a=
+ peer would never get messages that it does not understand, so there is no c=
+hance that missing a protocol change would matter.
+
+In case it=E2=80=99s not understood, version negotiation provides each peer w=
+ith the maximum supported protocol version of the other. Once complete both h=
+ave negotiated to the highest common version. No message not supported at th=
+at version may be sent by either.
+
+If the protocol was to accept *any* message traffic then it will cease to be=
+ a protocol. People will drop in their changes without obtaining broad suppo=
+rt, and peers will evolve to the point of no longer being peers. They won=E2=
+=80=99t speak the same language and the =E2=80=9Cnetwork=E2=80=9D will be li=
+ttle more than a broadcast transport, broadcasting all traffic to all peers,=
+ whether it=E2=80=99s for them or not.
+
+People need to either build support or build a distinct network. That=E2=80=99=
+s the actual coordination issue, which is inherent to protocol development.
+
+Best,
+e
+
+> On Aug 17, 2020, at 13:40, Suhas Daftuar <sdaftuar@gmail.com> wrote:
+>=20
+> Hi Eric,
+>=20
+> Thanks for your response. If I understand correctly, you're suggesting th=
+at in the future we do the same as what was done in BIP 339, of accompanying=
+ new messages (which are optional) with a protocol version bump, so that net=
+work clients are never reading unknown messages from a peer (and can be free=
+ to disconnect a peer for sending an unknown message)?
+>=20
+> I think that works fine, so if indeed there will be software that will exp=
+ect things to operate this way then I can withdraw the suggestion I've made i=
+n this thread. However I wanted to clarify that this is what you suggest, b=
+ecause there is another downside to this approach (beyond the sequential nat=
+ure of sequence numbers that you mention) -- if a software implementation mi=
+sses a proposed new protocol upgrade, and thus fails to parse (and ignore) s=
+ome proposed new message, the result can be a network split down the road as=
+ incompatible clients get slowly upgraded over time.=20
+>=20
+> I think this coordination cost is something to be concerned about -- for i=
+nstance, the lack of response to my wtxid-relay proposal made me wonder if o=
+ther software would be implementing something to account for the new message=
+ that proposal introduces (for clients with version >=3D 70016). It's reaso=
+nable for people to be busy and miss things like this, and I think it's wort=
+h considering whether there's a safer way for us to deploy changes.
+>=20
+> That said, I don't think this coordination cost is unbearable, so as long a=
+s we have a process for making p2p protocol improvements I'm not too worried=
+ about what mechanism we use. So if this concern over coordination of chang=
+es doesn't sway you, I think we can continue to just bump protocol version a=
+t the same time as deploying new messages, as we have been doing, and hope t=
+hat we don't run into problems down the road. =20
+>=20
+> If I have misunderstood how you think we should be making future protocol c=
+hanges, please let me know.
+>=20
+> Thanks,
+> Suhas
+>=20
+>=20
+>=20
+>> On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil <eric@voskuil.org> wrote:
+>> A requirement to ignore unknown (invalid) messages is not only a protocol=
+ breaking change but poor protocol design. The purpose of version negotiatio=
+n is to determine the set of valid messages. Changes to version negotiation i=
+tself are very problematic.
+>>=20
+>> The only limitation presented by versioning is that the system is sequent=
+ial. As such, clients that do not wish to implement (or operators who do not=
+ wish to enable) them are faced with a problem when wanting to support later=
+ features. This is resolvable by making such features optional at the new pr=
+otocol level. This allows each client to limit its communication to the nego=
+tiated protocol, and allows ignoring of known but unsupported/disabled featu=
+res.
+>>=20
+>> Sorry I missed your earlier post. Been distracted for a while.
+>>=20
+>> e
+>>=20
+>>=20
+>>>> On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev <bitcoin-dev@l=
+ists.linuxfoundation.org> wrote:
+>>>>=20
+>>> =EF=BB=BF
+>>> Hi,
+>>>=20
+>>> Back in February I posted a proposal for WTXID-based transaction relay[1=
+] (now known as BIP 339), which included a proposal for feature negotiation t=
+o take place prior to the VERACK message being received by each side. In my=
+ email to this list, I had asked for feedback as to whether that proposal wa=
+s problematic, and didn't receive any responses.
+>>>=20
+>>> Since then, the implementation of BIP 339 has been merged into Bitcoin C=
+ore, though it has not yet been released.
+>>>=20
+>>> In thinking about the mechanism used there, I thought it would be helpfu=
+l to codify in a BIP the idea that Bitcoin network clients should ignore unk=
+nown messages received before a VERACK. A draft of my proposal is available=
+ here[2].
+>>>=20
+>>> I presume that software upgrading past protocol version 70016 was alread=
+y planning to either implement BIP 339, or ignore the wtxidrelay message pro=
+posed in BIP 339 (if not, then this would create network split concerns in t=
+he future -- so I hope that someone would speak up if this were a problem). =
+ When we propose future protocol upgrades that would benefit from feature ne=
+gotiation at the time of connection, I think it would be nice to be able to u=
+se the same method as proposed in BIP 339, without even needing to bump the p=
+rotocol version. So having an understanding that this is the standard of ho=
+w other network clients operate would be helpful.
+>>>=20
+>>> If, on the other hand, this is problematic for some reason, I look forwa=
+rd to hearing that as well, so that we can be careful about how we deploy fu=
+ture p2p changes to avoid disruption.
+>>>=20
+>>> Thanks,
+>>> Suhas Daftuar
+>>>=20
+>>> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Februar=
+y/017648.html
+>>>=20
+>>> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-ne=
+gotiation/bip-p2p-feature-negotiation.mediawiki
+>>> _______________________________________________
+>>> bitcoin-dev mailing list
+>>> bitcoin-dev@lists.linuxfoundation.org
+>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
+
+--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7
+Content-Type: text/html;
+ charset=utf-8
+Content-Transfer-Encoding: quoted-printable
+
+<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
+utf-8"></head><body dir=3D"auto"><div dir=3D"ltr">Hi Suhas,</div><div dir=3D=
+"ltr"><br></div><div dir=3D"ltr">It seems to me that your first two paragrap=
+hs contradict each other, so I=E2=80=99m not sure we have understanding. As y=
+ou say in the first paragraph, a peer would never get messages that it does n=
+ot understand, so there is no chance that missing a protocol change would ma=
+tter.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">In case it=E2=80=99s n=
+ot understood, version negotiation provides each peer with the maximum suppo=
+rted protocol version of the other. Once complete both have negotiated to th=
+e highest common version. No message not supported at that version may be se=
+nt by either.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">If the protoc=
+ol was to accept *any* message traffic then it will cease to be a protocol. P=
+eople will drop in their changes without obtaining broad support, and peers w=
+ill evolve to the point of no longer being peers. They won=E2=80=99t speak t=
+he same language and the =E2=80=9Cnetwork=E2=80=9D will be little more than a=
+ broadcast transport, broadcasting all traffic to all peers, whether it=E2=80=
+=99s for them or not.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Peopl=
+e need to either build support or build a distinct network. That=E2=80=99s t=
+he actual coordination issue, which is inherent to protocol development.</di=
+v><div dir=3D"ltr"><br></div><div dir=3D"ltr">Best,</div><div dir=3D"ltr">e<=
+/div><div dir=3D"ltr"><br><blockquote type=3D"cite">On Aug 17, 2020, at 13:4=
+0, Suhas Daftuar &lt;sdaftuar@gmail.com&gt; wrote:<br><br></blockquote></div=
+><blockquote type=3D"cite"><div dir=3D"ltr"><div dir=3D"ltr">Hi Eric,<div><b=
+r></div><div>Thanks for your response.&nbsp; If I understand correctly, you'=
+re suggesting that in the future we do the same as what was done in BIP 339,=
+ of accompanying new messages (which are optional) with a protocol version b=
+ump, so that network clients are never reading unknown messages from a peer (=
+and can be free to disconnect a peer for sending an unknown message)?</div><=
+div><br></div><div>I think that works fine, so if indeed there will be softw=
+are that will expect things to operate this way then I can withdraw the sugg=
+estion I've made in this thread.&nbsp; However I wanted to clarify that this=
+ is what you suggest, because there is another downside to this approach (be=
+yond the sequential nature of sequence numbers that you mention) -- if a sof=
+tware implementation misses a proposed new protocol upgrade, and thus fails t=
+o parse (and ignore) some proposed new message, the result can be a network s=
+plit down the road as incompatible clients get slowly upgraded over time.&nb=
+sp;</div><div><br></div><div>I think this coordination cost is something to b=
+e concerned about -- for instance, the lack of response to my wtxid-relay pr=
+oposal made me wonder if other software would be implementing something to a=
+ccount for the new message that proposal introduces (for clients with versio=
+n &gt;=3D 70016).&nbsp; It's reasonable for people to be busy and miss thing=
+s like this, and I think it's worth considering whether there's a safer way f=
+or us to deploy changes.</div><div><br></div><div>That said, I don't think t=
+his coordination cost is unbearable, so as long as we have a process for mak=
+ing p2p protocol improvements I'm not too worried about what mechanism we us=
+e.&nbsp; So if this concern over coordination of changes doesn't sway you, I=
+ think we can continue to just bump protocol version at the same time as dep=
+loying new messages, as we have been doing, and hope that we don't run into p=
+roblems down the road.&nbsp;&nbsp;</div><div><br></div><div>If I have misund=
+erstood how you think we should be making future protocol changes, please le=
+t me know.</div><div><br></div><div>Thanks,</div><div>Suhas</div><div><br></=
+div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla=
+ss=3D"gmail_attr">On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil &lt;<a href=3D=
+"mailto:eric@voskuil.org" target=3D"_blank">eric@voskuil.org</a>&gt; wrote:<=
+br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
+;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto">=
+<div dir=3D"ltr"><div dir=3D"ltr">A requirement to ignore unknown (invalid) m=
+essages is not only a protocol breaking change but poor protocol design. The=
+ purpose of version negotiation is to determine the set of valid messages. C=
+hanges to version negotiation itself are very problematic.</div><div dir=3D"=
+ltr"><br></div><div dir=3D"ltr">The only limitation presented by versioning i=
+s that the system is sequential. As such, clients that do not wish to implem=
+ent (or operators who do not wish to enable) them are faced with a problem w=
+hen wanting to support later features. This is resolvable by making such fea=
+tures optional at the new protocol level. This allows each client to limit i=
+ts communication to the negotiated protocol, and allows ignoring of known bu=
+t unsupported/disabled features.</div><div dir=3D"ltr"><br></div><div dir=3D=
+"ltr">Sorry I missed your earlier post. Been distracted for a while.</div><d=
+iv dir=3D"ltr"><br></div><div dir=3D"ltr">e</div><div dir=3D"ltr"><br></div>=
+<div dir=3D"ltr"><br></div></div><div dir=3D"ltr"><blockquote type=3D"cite">=
+On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev &lt;<a href=3D"mail=
+to:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@list=
+s.linuxfoundation.org</a>&gt; wrote:<br><br></blockquote></div><blockquote t=
+ype=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"ltr">Hi,<div><br></div><d=
+iv>Back in February I posted a proposal for WTXID-based transaction relay[1]=
+ (now known as BIP 339), which included a proposal for feature negotiation t=
+o take place prior to the VERACK message being received by each side.&nbsp; I=
+n my email to this list, I had asked for feedback as to whether that proposa=
+l was problematic, and didn't receive any responses.</div><div><br></div><di=
+v>Since then, the implementation&nbsp;of BIP 339 has been merged into Bitcoi=
+n Core, though it has not yet been released.</div><div><br></div><div>In thi=
+nking about the mechanism used there, I thought it would&nbsp;be helpful to c=
+odify in a BIP the idea that Bitcoin network clients should ignore unknown m=
+essages received before a VERACK.&nbsp; A draft of my proposal is available h=
+ere[2].</div><div><br></div><div>I presume that software upgrading past prot=
+ocol version 70016 was already planning to either implement BIP 339, or igno=
+re the wtxidrelay message proposed in BIP 339 (if not, then this would creat=
+e network split concerns in the future -- so I hope that someone would speak=
+ up if this were a problem).&nbsp; When we propose future protocol upgrades t=
+hat would benefit from feature negotiation at the time of connection, I thin=
+k it would be nice to be able to use the same method as proposed in BIP 339,=
+ without even needing to bump the protocol version.&nbsp; So having an under=
+standing that this is the standard of how other network clients operate woul=
+d be helpful.</div><div><br></div><div>If, on the other hand, this is proble=
+matic for some reason, I look forward to hearing that as well, so that we ca=
+n be careful about how we deploy future p2p changes to avoid disruption.</di=
+v><div><br></div><div>Thanks,</div><div>Suhas Daftuar</div><div><br></div><d=
+iv>[1]&nbsp;<a href=3D"https://lists.linuxfoundation.org/pipermail/bitcoin-d=
+ev/2020-February/017648.html" target=3D"_blank">https://lists.linuxfoundatio=
+n.org/pipermail/bitcoin-dev/2020-February/017648.html</a><br></div><div><br>=
+</div><div>[2]&nbsp;<a href=3D"https://github.com/sdaftuar/bips/blob/2020-08=
+-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki" targ=
+et=3D"_blank">https://github.com/sdaftuar/bips/blob/2020-08-generalized-feat=
+ure-negotiation/bip-p2p-feature-negotiation.mediawiki</a></div></div>
+<span>_______________________________________________</span><br><span>bitcoi=
+n-dev mailing list</span><br><span><a href=3D"mailto:bitcoin-dev@lists.linux=
+foundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><=
+/span><br><span><a href=3D"https://lists.linuxfoundation.org/mailman/listinf=
+o/bitcoin-dev" target=3D"_blank">https://lists.linuxfoundation.org/mailman/l=
+istinfo/bitcoin-dev</a></span><br></div></blockquote></div></blockquote></di=
+v>
+</div></blockquote></body></html>=
+
+--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7--
+