diff options
author | Eric Voskuil <eric@voskuil.org> | 2020-08-17 14:21:53 -0700 |
---|---|---|
committer | bitcoindev <bitcoindev@gnusha.org> | 2020-08-17 21:51:26 +0000 |
commit | 3b29a6562f68d57c2813404e1cbcd899570a9908 (patch) | |
tree | 1e6f8f7e4592d69f19851895a8f912ce3360dee5 | |
parent | 20995872dc2a0ccf1aacf20fbfdd9cba8c3c1255 (diff) | |
download | pi-bitcoindev-3b29a6562f68d57c2813404e1cbcd899570a9908.tar.gz pi-bitcoindev-3b29a6562f68d57c2813404e1cbcd899570a9908.zip |
Re: [bitcoin-dev] Generalizing feature negotiation when new p2p connections are setup
-rw-r--r-- | c3/ab56d3586b358b17963b74301812ce8bd710da | 343 |
1 files changed, 343 insertions, 0 deletions
diff --git a/c3/ab56d3586b358b17963b74301812ce8bd710da b/c3/ab56d3586b358b17963b74301812ce8bd710da new file mode 100644 index 000000000..8d869f59b --- /dev/null +++ b/c3/ab56d3586b358b17963b74301812ce8bd710da @@ -0,0 +1,343 @@ +Return-Path: <eric@voskuil.org> +Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) + by lists.linuxfoundation.org (Postfix) with ESMTP id 08777C0051 + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 17 Aug 2020 21:51:26 +0000 (UTC) +Received: from localhost (localhost [127.0.0.1]) + by silver.osuosl.org (Postfix) with ESMTP id BD35F2035C + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 17 Aug 2020 21:51:25 +0000 (UTC) +X-Virus-Scanned: amavisd-new at osuosl.org +Received: from silver.osuosl.org ([127.0.0.1]) + by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 1GFKIYmVxNqY + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 17 Aug 2020 21:51:21 +0000 (UTC) +X-Greylist: delayed 00:23:37 by SQLgrey-1.7.6 +Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com + [209.85.167.195]) + by silver.osuosl.org (Postfix) with ESMTPS id 04B5F20450 + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 17 Aug 2020 21:51:20 +0000 (UTC) +Received: by mail-oi1-f195.google.com with SMTP id j7so16140871oij.9 + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 17 Aug 2020 14:51:20 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=voskuil-org.20150623.gappssmtp.com; s=20150623; + h=content-transfer-encoding:from:mime-version:subject:date:message-id + :references:cc:in-reply-to:to; + bh=F5YBEVlobZzZ6l9qMRq1mzwd98RmqT7VPrHutsyhf5U=; + b=iWxPh4DA1jP9EWH2b5Asl2EiDML1tk3LGgBJVwPkPpUqyoze3P5xBTu1gi7ZbculWe + i4StOgolAFgU0I1vrZ47mJHkR4vYnlSLfU58MFxntnSz7mba28aZiVh80HF9vn9afUFA + qWzBYoDk66QOIfhkcAFf05ug51f0VQME493jPVy4TEE0BEh2mWnQzaNJ70zuajE7jooL + J4ho9lsJZvBTLpjiruw10w74Dyw1KAuDrUWk71DAa0bRu3e+ZgT6SUaWqw6dLs/1eZbC + DO1vf1/CRYxpG3oQiyHRArLEkUeujLNk1+uN4rlG+PDFgHqVjYO12S3GhnlIFVEFB68n + 7AeA== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20161025; + h=x-gm-message-state:content-transfer-encoding:from:mime-version + :subject:date:message-id:references:cc:in-reply-to:to; + bh=F5YBEVlobZzZ6l9qMRq1mzwd98RmqT7VPrHutsyhf5U=; + b=Cc6+k8aw9TPO/pAsGLaiJD80cjzHAMYNzN/uQsOfRN4DTNQvl3PlBnCXx+sBiCKEbv + 6saJOmXg3S0YrbTe8G187hW8yU256NX9eb57hfxICt8v1Wr/DcWrox6Gt2I+OeNYMYUk + pFdUc4JLFfKer0zIlLUHsaNCpLnQ3uW8M7K/cQCvM77Tu54iUNvFjBm12z0a/8QR1MSd + gOGq0dMfgMBlt3gSKRkZNcm4wFC0fH0576trxKaP1BOIv531ciRGzQNC4a9+tgtHhE9z + v/qnWqmNwnw6wBKVJKWB5xRj5YvMJcPa0Lia4PAJr4UY2DYLJkcfjnEPTn9ArP0qcY71 + 7GcQ== +X-Gm-Message-State: AOAM532tr4vsIpWGKo8BBgY/IDGQRFI2KQXU6uAA9xlOAQ1KQRnAbUne + R4NfTBWNkTG4ZEXBdzdZIvckqKOeRTdBTw== +X-Google-Smtp-Source: ABdhPJyQuDp36nUf/1q+MpGqFqe3lVdSXwH5wD9QD1fjgdRzBWTVTlZ+e7yR79imwiQ7uRDB8TGxoA== +X-Received: by 2002:a17:90a:5298:: with SMTP id + w24mr13343912pjh.221.1597699316124; + Mon, 17 Aug 2020 14:21:56 -0700 (PDT) +Received: from ?IPv6:2600:380:7066:ce6e:c976:12fb:75f1:c075? + ([2600:380:7066:ce6e:c976:12fb:75f1:c075]) + by smtp.gmail.com with ESMTPSA id x14sm21340292pfj.157.2020.08.17.14.21.54 + (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); + Mon, 17 Aug 2020 14:21:55 -0700 (PDT) +Content-Type: multipart/alternative; + boundary=Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7 +Content-Transfer-Encoding: 7bit +From: Eric Voskuil <eric@voskuil.org> +Mime-Version: 1.0 (1.0) +Date: Mon, 17 Aug 2020 14:21:53 -0700 +Message-Id: <6FCCEC96-02F7-48D7-92EC-52234B94805B@voskuil.org> +References: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com> +In-Reply-To: <CAFp6fsH==PJ38vGvCGUr3AVS8XmQEjvLkNaopAhu0vY7xQjgAw@mail.gmail.com> +To: Suhas Daftuar <sdaftuar@gmail.com> +X-Mailer: iPhone Mail (17G68) +X-Mailman-Approved-At: Mon, 17 Aug 2020 21:53:52 +0000 +Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> +Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p + connections are setup +X-BeenThere: bitcoin-dev@lists.linuxfoundation.org +X-Mailman-Version: 2.1.15 +Precedence: list +List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> +List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> +List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> +List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> +List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> +List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> +X-List-Received-Date: Mon, 17 Aug 2020 21:51:26 -0000 + + +--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7 +Content-Type: text/plain; + charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Hi Suhas, + +It seems to me that your first two paragraphs contradict each other, so I=E2= +=80=99m not sure we have understanding. As you say in the first paragraph, a= + peer would never get messages that it does not understand, so there is no c= +hance that missing a protocol change would matter. + +In case it=E2=80=99s not understood, version negotiation provides each peer w= +ith the maximum supported protocol version of the other. Once complete both h= +ave negotiated to the highest common version. No message not supported at th= +at version may be sent by either. + +If the protocol was to accept *any* message traffic then it will cease to be= + a protocol. People will drop in their changes without obtaining broad suppo= +rt, and peers will evolve to the point of no longer being peers. They won=E2= +=80=99t speak the same language and the =E2=80=9Cnetwork=E2=80=9D will be li= +ttle more than a broadcast transport, broadcasting all traffic to all peers,= + whether it=E2=80=99s for them or not. + +People need to either build support or build a distinct network. That=E2=80=99= +s the actual coordination issue, which is inherent to protocol development. + +Best, +e + +> On Aug 17, 2020, at 13:40, Suhas Daftuar <sdaftuar@gmail.com> wrote: +>=20 +> Hi Eric, +>=20 +> Thanks for your response. If I understand correctly, you're suggesting th= +at in the future we do the same as what was done in BIP 339, of accompanying= + new messages (which are optional) with a protocol version bump, so that net= +work clients are never reading unknown messages from a peer (and can be free= + to disconnect a peer for sending an unknown message)? +>=20 +> I think that works fine, so if indeed there will be software that will exp= +ect things to operate this way then I can withdraw the suggestion I've made i= +n this thread. However I wanted to clarify that this is what you suggest, b= +ecause there is another downside to this approach (beyond the sequential nat= +ure of sequence numbers that you mention) -- if a software implementation mi= +sses a proposed new protocol upgrade, and thus fails to parse (and ignore) s= +ome proposed new message, the result can be a network split down the road as= + incompatible clients get slowly upgraded over time.=20 +>=20 +> I think this coordination cost is something to be concerned about -- for i= +nstance, the lack of response to my wtxid-relay proposal made me wonder if o= +ther software would be implementing something to account for the new message= + that proposal introduces (for clients with version >=3D 70016). It's reaso= +nable for people to be busy and miss things like this, and I think it's wort= +h considering whether there's a safer way for us to deploy changes. +>=20 +> That said, I don't think this coordination cost is unbearable, so as long a= +s we have a process for making p2p protocol improvements I'm not too worried= + about what mechanism we use. So if this concern over coordination of chang= +es doesn't sway you, I think we can continue to just bump protocol version a= +t the same time as deploying new messages, as we have been doing, and hope t= +hat we don't run into problems down the road. =20 +>=20 +> If I have misunderstood how you think we should be making future protocol c= +hanges, please let me know. +>=20 +> Thanks, +> Suhas +>=20 +>=20 +>=20 +>> On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil <eric@voskuil.org> wrote: +>> A requirement to ignore unknown (invalid) messages is not only a protocol= + breaking change but poor protocol design. The purpose of version negotiatio= +n is to determine the set of valid messages. Changes to version negotiation i= +tself are very problematic. +>>=20 +>> The only limitation presented by versioning is that the system is sequent= +ial. As such, clients that do not wish to implement (or operators who do not= + wish to enable) them are faced with a problem when wanting to support later= + features. This is resolvable by making such features optional at the new pr= +otocol level. This allows each client to limit its communication to the nego= +tiated protocol, and allows ignoring of known but unsupported/disabled featu= +res. +>>=20 +>> Sorry I missed your earlier post. Been distracted for a while. +>>=20 +>> e +>>=20 +>>=20 +>>>> On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev <bitcoin-dev@l= +ists.linuxfoundation.org> wrote: +>>>>=20 +>>> =EF=BB=BF +>>> Hi, +>>>=20 +>>> Back in February I posted a proposal for WTXID-based transaction relay[1= +] (now known as BIP 339), which included a proposal for feature negotiation t= +o take place prior to the VERACK message being received by each side. In my= + email to this list, I had asked for feedback as to whether that proposal wa= +s problematic, and didn't receive any responses. +>>>=20 +>>> Since then, the implementation of BIP 339 has been merged into Bitcoin C= +ore, though it has not yet been released. +>>>=20 +>>> In thinking about the mechanism used there, I thought it would be helpfu= +l to codify in a BIP the idea that Bitcoin network clients should ignore unk= +nown messages received before a VERACK. A draft of my proposal is available= + here[2]. +>>>=20 +>>> I presume that software upgrading past protocol version 70016 was alread= +y planning to either implement BIP 339, or ignore the wtxidrelay message pro= +posed in BIP 339 (if not, then this would create network split concerns in t= +he future -- so I hope that someone would speak up if this were a problem). = + When we propose future protocol upgrades that would benefit from feature ne= +gotiation at the time of connection, I think it would be nice to be able to u= +se the same method as proposed in BIP 339, without even needing to bump the p= +rotocol version. So having an understanding that this is the standard of ho= +w other network clients operate would be helpful. +>>>=20 +>>> If, on the other hand, this is problematic for some reason, I look forwa= +rd to hearing that as well, so that we can be careful about how we deploy fu= +ture p2p changes to avoid disruption. +>>>=20 +>>> Thanks, +>>> Suhas Daftuar +>>>=20 +>>> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Februar= +y/017648.html +>>>=20 +>>> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-ne= +gotiation/bip-p2p-feature-negotiation.mediawiki +>>> _______________________________________________ +>>> bitcoin-dev mailing list +>>> bitcoin-dev@lists.linuxfoundation.org +>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev + +--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7 +Content-Type: text/html; + charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D= +utf-8"></head><body dir=3D"auto"><div dir=3D"ltr">Hi Suhas,</div><div dir=3D= +"ltr"><br></div><div dir=3D"ltr">It seems to me that your first two paragrap= +hs contradict each other, so I=E2=80=99m not sure we have understanding. As y= +ou say in the first paragraph, a peer would never get messages that it does n= +ot understand, so there is no chance that missing a protocol change would ma= +tter.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">In case it=E2=80=99s n= +ot understood, version negotiation provides each peer with the maximum suppo= +rted protocol version of the other. Once complete both have negotiated to th= +e highest common version. No message not supported at that version may be se= +nt by either.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">If the protoc= +ol was to accept *any* message traffic then it will cease to be a protocol. P= +eople will drop in their changes without obtaining broad support, and peers w= +ill evolve to the point of no longer being peers. They won=E2=80=99t speak t= +he same language and the =E2=80=9Cnetwork=E2=80=9D will be little more than a= + broadcast transport, broadcasting all traffic to all peers, whether it=E2=80= +=99s for them or not.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Peopl= +e need to either build support or build a distinct network. That=E2=80=99s t= +he actual coordination issue, which is inherent to protocol development.</di= +v><div dir=3D"ltr"><br></div><div dir=3D"ltr">Best,</div><div dir=3D"ltr">e<= +/div><div dir=3D"ltr"><br><blockquote type=3D"cite">On Aug 17, 2020, at 13:4= +0, Suhas Daftuar <sdaftuar@gmail.com> wrote:<br><br></blockquote></div= +><blockquote type=3D"cite"><div dir=3D"ltr"><div dir=3D"ltr">Hi Eric,<div><b= +r></div><div>Thanks for your response. If I understand correctly, you'= +re suggesting that in the future we do the same as what was done in BIP 339,= + of accompanying new messages (which are optional) with a protocol version b= +ump, so that network clients are never reading unknown messages from a peer (= +and can be free to disconnect a peer for sending an unknown message)?</div><= +div><br></div><div>I think that works fine, so if indeed there will be softw= +are that will expect things to operate this way then I can withdraw the sugg= +estion I've made in this thread. However I wanted to clarify that this= + is what you suggest, because there is another downside to this approach (be= +yond the sequential nature of sequence numbers that you mention) -- if a sof= +tware implementation misses a proposed new protocol upgrade, and thus fails t= +o parse (and ignore) some proposed new message, the result can be a network s= +plit down the road as incompatible clients get slowly upgraded over time.&nb= +sp;</div><div><br></div><div>I think this coordination cost is something to b= +e concerned about -- for instance, the lack of response to my wtxid-relay pr= +oposal made me wonder if other software would be implementing something to a= +ccount for the new message that proposal introduces (for clients with versio= +n >=3D 70016). It's reasonable for people to be busy and miss thing= +s like this, and I think it's worth considering whether there's a safer way f= +or us to deploy changes.</div><div><br></div><div>That said, I don't think t= +his coordination cost is unbearable, so as long as we have a process for mak= +ing p2p protocol improvements I'm not too worried about what mechanism we us= +e. So if this concern over coordination of changes doesn't sway you, I= + think we can continue to just bump protocol version at the same time as dep= +loying new messages, as we have been doing, and hope that we don't run into p= +roblems down the road. </div><div><br></div><div>If I have misund= +erstood how you think we should be making future protocol changes, please le= +t me know.</div><div><br></div><div>Thanks,</div><div>Suhas</div><div><br></= +div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla= +ss=3D"gmail_attr">On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil <<a href=3D= +"mailto:eric@voskuil.org" target=3D"_blank">eric@voskuil.org</a>> wrote:<= +br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex= +;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto">= +<div dir=3D"ltr"><div dir=3D"ltr">A requirement to ignore unknown (invalid) m= +essages is not only a protocol breaking change but poor protocol design. The= + purpose of version negotiation is to determine the set of valid messages. C= +hanges to version negotiation itself are very problematic.</div><div dir=3D"= +ltr"><br></div><div dir=3D"ltr">The only limitation presented by versioning i= +s that the system is sequential. As such, clients that do not wish to implem= +ent (or operators who do not wish to enable) them are faced with a problem w= +hen wanting to support later features. This is resolvable by making such fea= +tures optional at the new protocol level. This allows each client to limit i= +ts communication to the negotiated protocol, and allows ignoring of known bu= +t unsupported/disabled features.</div><div dir=3D"ltr"><br></div><div dir=3D= +"ltr">Sorry I missed your earlier post. Been distracted for a while.</div><d= +iv dir=3D"ltr"><br></div><div dir=3D"ltr">e</div><div dir=3D"ltr"><br></div>= +<div dir=3D"ltr"><br></div></div><div dir=3D"ltr"><blockquote type=3D"cite">= +On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev <<a href=3D"mail= +to:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@list= +s.linuxfoundation.org</a>> wrote:<br><br></blockquote></div><blockquote t= +ype=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"ltr">Hi,<div><br></div><d= +iv>Back in February I posted a proposal for WTXID-based transaction relay[1]= + (now known as BIP 339), which included a proposal for feature negotiation t= +o take place prior to the VERACK message being received by each side. I= +n my email to this list, I had asked for feedback as to whether that proposa= +l was problematic, and didn't receive any responses.</div><div><br></div><di= +v>Since then, the implementation of BIP 339 has been merged into Bitcoi= +n Core, though it has not yet been released.</div><div><br></div><div>In thi= +nking about the mechanism used there, I thought it would be helpful to c= +odify in a BIP the idea that Bitcoin network clients should ignore unknown m= +essages received before a VERACK. A draft of my proposal is available h= +ere[2].</div><div><br></div><div>I presume that software upgrading past prot= +ocol version 70016 was already planning to either implement BIP 339, or igno= +re the wtxidrelay message proposed in BIP 339 (if not, then this would creat= +e network split concerns in the future -- so I hope that someone would speak= + up if this were a problem). When we propose future protocol upgrades t= +hat would benefit from feature negotiation at the time of connection, I thin= +k it would be nice to be able to use the same method as proposed in BIP 339,= + without even needing to bump the protocol version. So having an under= +standing that this is the standard of how other network clients operate woul= +d be helpful.</div><div><br></div><div>If, on the other hand, this is proble= +matic for some reason, I look forward to hearing that as well, so that we ca= +n be careful about how we deploy future p2p changes to avoid disruption.</di= +v><div><br></div><div>Thanks,</div><div>Suhas Daftuar</div><div><br></div><d= +iv>[1] <a href=3D"https://lists.linuxfoundation.org/pipermail/bitcoin-d= +ev/2020-February/017648.html" target=3D"_blank">https://lists.linuxfoundatio= +n.org/pipermail/bitcoin-dev/2020-February/017648.html</a><br></div><div><br>= +</div><div>[2] <a href=3D"https://github.com/sdaftuar/bips/blob/2020-08= +-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki" targ= +et=3D"_blank">https://github.com/sdaftuar/bips/blob/2020-08-generalized-feat= +ure-negotiation/bip-p2p-feature-negotiation.mediawiki</a></div></div> +<span>_______________________________________________</span><br><span>bitcoi= +n-dev mailing list</span><br><span><a href=3D"mailto:bitcoin-dev@lists.linux= +foundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><= +/span><br><span><a href=3D"https://lists.linuxfoundation.org/mailman/listinf= +o/bitcoin-dev" target=3D"_blank">https://lists.linuxfoundation.org/mailman/l= +istinfo/bitcoin-dev</a></span><br></div></blockquote></div></blockquote></di= +v> +</div></blockquote></body></html>= + +--Apple-Mail-5EE96465-E05B-480D-B8C1-43DC55EBB4B7-- + |