path: root/5f/582856b8559d823826903f21bc25d62b471b63

Return-Path: <eric@voskuil.org>
Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138])
 by lists.linuxfoundation.org (Postfix) with ESMTP id EA302C0051
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 18 Aug 2020 18:16:59 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by whitealder.osuosl.org (Postfix) with ESMTP id D69178686A
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 18 Aug 2020 18:16:59 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from whitealder.osuosl.org ([127.0.0.1])
 by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id q7kXTUMzkVn3
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 18 Aug 2020 18:16:58 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com
 [209.85.217.43])
 by whitealder.osuosl.org (Postfix) with ESMTPS id 167168680C
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 18 Aug 2020 18:16:58 +0000 (UTC)
Received: by mail-vs1-f43.google.com with SMTP id o184so10613620vsc.0
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 18 Aug 2020 11:16:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=voskuil-org.20150623.gappssmtp.com; s=20150623;
 h=content-transfer-encoding:from:mime-version:subject:date:message-id
 :references:cc:in-reply-to:to;
 bh=cEtGqtkQWq0l6LICXtCzt1ILvK1uA58i+kdZUP+bVpU=;
 b=Yx/4gi1UozZ4k35h+A+6ce3cqF4syloJMFw2aHU9Xqg6bZgvWb2rjYFwEJaYxc4sgn
 r4cfzFkbDfJNLeHTG+aVPX+rSVY/hiawzNJSHVUMGghKbE76IfYB6DpnShSIU7xqW0Sh
 t6rURh7OzUqwyiBouwG/5ewbBzrZMFDvoKpRKSD4J+7d79gfad/9V9fECGiaObUTyUFA
 EPfyvltQ1x1CFCzy9kRPpn/J2ksKHO0zUbLIMUMFkGd62iakdkEJgJw5cyZsVGEGgc0p
 ImZR3NNbEv4ifUuAORXQIPS4QOGfSj4sDcgWfFPW0gyeuEIOn5zwWQ8suBNDWOAnmqxI
 /WMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:content-transfer-encoding:from:mime-version
 :subject:date:message-id:references:cc:in-reply-to:to;
 bh=cEtGqtkQWq0l6LICXtCzt1ILvK1uA58i+kdZUP+bVpU=;
 b=WO3usaC0r4aLei+0RELI23lWzzzaU1rau+BIw7Yt0ZK30VgZxR2M5Uj/Gk7L1yilMY
 lzLOeioE2d+oplGx66Vk719OD3auBtRb/dIN7RalHaULxsyKfFzQLxwnDIzfoFNRquZr
 7jPhcaT4x3d1noHc9TH4NCbZKmCeaJLWYUlWJSZZ+n5ShvmkItebhMj5sqjrROdQIRsc
 ORbKyIa2AOUbQAcq1Dh6RkYXjwJgeasbGV4/KSLTd+nGkhn+Bu5uU2lF4vFSRtHut7D1
 UKSy+XyyDo9LhQzFge+Tcs7BICfZDMf0PfhZNWzcMcqWTcsDyu7NHsLAs4xrS+DHreIX
 WJMA==
X-Gm-Message-State: AOAM533dZuHAwpu63T6gpteKmchCOibCY/GFwdFrORL+lWZOQFzW7zB1
 ZzbZOUQgRstDz4VpfqVO/Nk7ZD262skl+g==
X-Google-Smtp-Source: ABdhPJxqTCAfm0JM/5un4S64tqPioi2LXjUFFMO6Rq60BWs/Bv74iTCe3LPgpm2uc0Zz2eg5ue/CSQ==
X-Received: by 2002:a17:902:b18e:: with SMTP id
 s14mr8867891plr.160.1597774276214; 
 Tue, 18 Aug 2020 11:11:16 -0700 (PDT)
Received: from ?IPv6:2600:380:7066:ce6e:35fb:26cc:838e:c70d?
 ([2600:380:7066:ce6e:35fb:26cc:838e:c70d])
 by smtp.gmail.com with ESMTPSA id q82sm27621823pfc.139.2020.08.18.11.11.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 18 Aug 2020 11:11:15 -0700 (PDT)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: Eric Voskuil <eric@voskuil.org>
Mime-Version: 1.0 (1.0)
Date: Tue, 18 Aug 2020 11:11:12 -0700
Message-Id: <95DD247F-028E-432A-9C26-6337F5819A31@voskuil.org>
References: <6ccf6ff8-6b4f-c8f4-0dbb-36c5d076528f@mattcorallo.com>
In-Reply-To: <6ccf6ff8-6b4f-c8f4-0dbb-36c5d076528f@mattcorallo.com>
To: Matt Corallo <lf-lists@mattcorallo.com>
X-Mailer: iPhone Mail (17G68)
X-Mailman-Approved-At: Tue, 18 Aug 2020 18:21:57 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p
	connections are setup
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Aug 2020 18:17:00 -0000


> On Aug 18, 2020, at 10:26, Matt Corallo <lf-lists@mattcorallo.com> wrote:
>=20
> =EF=BB=BFThere are several cases where a new message has been sent as a pa=
rt of a negotiation without changing the protocol version.

Such as?

> You may chose to ignore that, but that doesn't mean that it isn't an under=
stood and even relied upon feature of the Bitcoin P2P protocol. If you wish t=
o fail connections to new nodes (and risk network splits, as Suhas points ou=
t), then you may do so, but that doesn't make it a part of the Bitcoin P2P p=
rotocol that you must do so. Of course there is no "official document" by wh=
ich we can make a formal appeal, but historical precedent suggests otherwise=
.

You are misrepresenting =E2=80=9Chistorical precedent=E2=80=9D. I=E2=80=99ve=
 seen several attempts to require arbitrary traffic over the years and none h=
ave been realized.

> Still, I think we're talking pedantics here, and not in a useful way.

Not to be pedantic, but I don=E2=80=99t know what that means.

> Ultimately we need some kind of negotiation which is flexible in allowing d=
ifferent software to negotiate different features without a global lock-step=
 version number increase.

I have shown below how that already works.

> Or, to put it another way, if a feature is fully optional, why should ther=
e be a version number increase for it

For the reasons previously given.

> - the negotiation of it is independent and a version number only increases=
 confusion over which change "owns" a given version number.

Presumably this is why we have a standards process. Any new message implies o=
wnership. Deconflicting that is required, which implies it can easily be ver=
sion associated (as it has been).

> I presume you'd support a single message that lists the set of features wh=
ich a node (optionally) wishes to support on the connection. This proposal i=
s fully equivalent to that, instead opting to list them as individual messag=
es instead of one message, which is a bit nicer in that they can be handled m=
ore independently or by different subsystems including even the message hash=
ing.

This presumes an implementation. As part of the handshake, collection of an a=
rbitrary set of messages is a significant and unnecessary complication *of t=
he protocol*. Extension of the verack is not. It is the simplest change poss=
ible to implement the desired behavior. Each peer simply supplies the matrix=
 of sub-protocols it supports and only those supported by both are allowed. T=
here is no reason for the protocol to split that matrix up into multiple mes=
sages, requiring termination. Independent messages exist because of timing o=
r ordering requirements. Implementing dependent messages as if they were ind=
ependent is wasteful and complicating.

I=E2=80=99m well aware of the inefficiency produced by version linearity in t=
he face of optional sub-protocols. The protocol must negotiate to the versio=
n where it can then negotiate support, which has been done. I support creati=
ng a simpler system, eliminating these extra messages. The existing numeric v=
ersion can be reserved exclusively for =E2=80=9Cmust=E2=80=9D implement, and=
 can be used to signal an extension to the verack. The verack can then carry=
 a list of =E2=80=9Cmay=E2=80=9D or =E2=80=9Cshould=E2=80=9D sub-protocols f=
or final negotiation.

The format of the matrix is arbitrary, but the requirement is to list a set o=
f optional sub-protocols. This implies a namespace. This implies =E2=80=9Cow=
nership=E2=80=9C of names. In other words, that coordination requirement is n=
ot eliminated.

e

> Matt
>=20
>> On 8/18/20 12:54 PM, Eric Voskuil wrote:
>> =E2=80=9CBitcoin protocol has always expected clients to ignore unknown m=
essages=E2=80=9D
>> This is not true. Bitcoin has long implemented version negotiation, which=
 is the opposite expectation. Libbitcoin=E2=80=99s p2p protocol implementati=
on immediately drops a peer that sends an invalid message according to the n=
egotiated version. The fact that a given client does not validate the protoc=
ol does not make it an expectation that the protocol not be validated.
>> Features can clearly be optional within an actual protocol. There have be=
en post-handshake negotiations implemented for optional messages which are v=
alid at the negotiated version. The protocol may be flexible while remaining=
 validateable. There is no reason to force a client to accept unknown messag=
e traffic.
>> A generalized versioning change can be implemented in or after the handsh=
ake. The latter is already done on an ad-hoc basis. The former is possible a=
s long as the peer=E2=80=99s version is sufficient to be aware of the behavi=
or. This does not imply any need to send invalid messages. The verack itself=
 can simply be extended with a matrix of feature support. There is no reason=
 to complicate negotiation with an additional message(s).
>> FWIW, bip37 did this poorly, adding a feature field to the version messag=
e, resulting in bip60. Due to this design, older protocol-validating clients=
 were broken. In this case it was message length that was presumed to not be=
 validated.
>> e
>>>> On Aug 18, 2020, at 07:59, Matt Corallo via bitcoin-dev <bitcoin-dev@li=
sts.linuxfoundation.org> wrote:
>>>=20
>>> =EF=BB=BFThis sounds like a great idea!
>>>=20
>>> Bitcoin is no longer a homogeneous network of one client - it is many, w=
ith different features implemented in each. The Bitcoin protocol hasn't (ful=
ly) evolved to capture that reality. Initially the Bitcoin protocol had a si=
mple numerical version field, but that is wholly impractical for any diverse=
 network - some clients may not wish to implement every possible new relay m=
echanic, and why should they have to in order to use other new features?
>>>=20
>>> Bitcoin protocol changes have, many times in recent history, been made v=
ia new dummy "negotiation" messages, which take advantage of the fact that t=
he Bitcoin protocol has always expected clients to ignore unknown messages. G=
iven that pattern, it makes sense to have an explicit negotiation phase - af=
ter version and before verack, just send the list of features that you suppo=
rt to negotiate what the connection will be capable of. The exact way we do t=
hat doesn't matter much, and sending it as a stream of messages which each i=
ndicate support for a given protocol feature perfectly captures the pattern t=
hat has been used in several recent network upgrades, keeping consistency.
>>>=20
>>> Matt
>>>=20
>>> On 8/14/20 3:28 PM, Suhas Daftuar via bitcoin-dev wrote:
>>>> Hi,
>>>> Back in February I posted a proposal for WTXID-based transaction relay[=
1] (now known as BIP 339), which included a proposal for feature negotiation=
 to take place prior to the VERACK message being received by each side.  In m=
y email to this list, I had asked for feedback as to whether that proposal w=
as problematic, and didn't receive any responses.
>>>> Since then, the implementation of BIP 339 has been merged into Bitcoin C=
ore, though it has not yet been released.
>>>> In thinking about the mechanism used there, I thought it would be helpf=
ul to codify in a BIP the idea that Bitcoin network clients should ignore un=
known messages received before a VERACK.  A draft of my proposal is availabl=
e here[2].
>>>> I presume that software upgrading past protocol version 70016 was alrea=
dy planning to either implement BIP 339, or ignore the wtxidrelay message pr=
oposed in BIP 339 (if not, then this would create network split concerns in t=
he future -- so I hope that someone would speak up if this were a problem). =
 When we propose future protocol upgrades that would benefit from feature ne=
gotiation at the time of connection, I think it would be nice to be able to u=
se the same method as proposed in BIP 339, without even needing to bump the p=
rotocol version.  So having an understanding that this is the standard of ho=
w other network clients operate would be helpful.
>>>> If, on the other hand, this is problematic for some reason, I look forw=
ard to hearing that as well, so that we can be careful about how we deploy f=
uture p2p changes to avoid disruption.
>>>> Thanks,
>>>> Suhas Daftuar
>>>> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Februa=
ry/017648.html <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020=
-February/017648.html>
>>>> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-n=
egotiation/bip-p2p-feature-negotiation.mediawiki <https://github.com/sdaftua=
r/bips/blob/2020-08-generalized-feature-negotiation/bip-p2p-feature-negotiat=
ion.mediawiki>
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev