path: root/f6/6ea4470d932b37d5439fb2898dff427abf8dcb

Return-Path: <gloriajzhao@gmail.com>
Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136])
 by lists.linuxfoundation.org (Postfix) with ESMTP id CD57BC002D
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 18 May 2022 18:41:13 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp3.osuosl.org (Postfix) with ESMTP id BAFEA60F1F
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 18 May 2022 18:41:13 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level: 
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp3.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp3.osuosl.org ([127.0.0.1])
 by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id l_GUS20JgRx4
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 18 May 2022 18:41:11 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com
 [IPv6:2607:f8b0:4864:20::b29])
 by smtp3.osuosl.org (Postfix) with ESMTPS id EDABA60E2D
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 18 May 2022 18:41:10 +0000 (UTC)
Received: by mail-yb1-xb29.google.com with SMTP id d137so5142563ybc.13
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 18 May 2022 11:41:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=1NOG41SCcm6xHuJU5eQXn2x4LdvjtNQqQgmZ9wfw1bM=;
 b=CIUSoa+9Gsvc3f3T6TPKWApnByzUAclwQmPbfl/ke88XuyIabXg7UJ/lXjkI9wH7Np
 C5mgRUHUTgFasic3UphjWgvgmD2pCO+7uxQkxN3ApDUa4lG2oxT83YiypL4ODOAcF+pb
 ReAFg/nmZWc/s3V7q3WKkNl2YGpgrKnS6A0tgUpDwBZ9Ih5C8gOMPpwnX4I5lEJvKD5u
 UnsIBe3F4DCJRiakxwULA3s8O3ejHFgIsFL1Sw1mxScM6Ofly9nVdzpfEFQoWifj4ajv
 jR08TVvzw2mXBt8XVuw+w9LL33C0FjCu1LKgOyx4y3Gvu/Gkf+L/PAUeGdFJFOYzxV31
 3j6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=1NOG41SCcm6xHuJU5eQXn2x4LdvjtNQqQgmZ9wfw1bM=;
 b=fwIMUoVMhBjU17SdRIprqdYxsKWMasN67XGIlq1/FY9twWgh2jtDQFt7UiafCMyecq
 tNfxU1qVAPBt1ljWr7A8cAneTbQVokKu9ftPasLO+yuHqWYLmH1qWLAvtqPoT6fpPrk1
 svCteyVo45Nz9RrRoVAoDpZsr6hNA60WTEOumbXH8zUsVJJgm5zLFST1l7q0xwcuOXna
 GoebGc5kWgDFI0fNgO7EGdRswox80KGtYuiCAkMZBW2EHjakcjT9ocZ4OIdm3ZnonXDu
 Y+ILZq0XWRyNMqipCgWLSiFnzOcBeup51TQuavs6Omg2pL09fZxeSyk60JyHfQF8kDPd
 lJHw==
X-Gm-Message-State: AOAM531Ijc9Df34wt7M7v/kg3+5TRaqqod0ZQMoK8/v/XYGiXk/hv+V2
 QCOvUM0AcFPfWtkrGWYio0maJWK9NBQzW4a+5X2Dabuwezc=
X-Google-Smtp-Source: ABdhPJxYRdxwOPAM9e8wDVmkh1mtzUpR/qkNX/JRM4IVf6UgU7H1uS25Bx5HH4WG2xTi++g+sN/utZpD3KpCVcAWEQg=
X-Received: by 2002:a05:6902:13c4:b0:64d:5e80:142 with SMTP id
 y4-20020a05690213c400b0064d5e800142mr923805ybu.55.1652899269693; Wed, 18 May
 2022 11:41:09 -0700 (PDT)
MIME-Version: 1.0
References: <CAFXO6=JROe_9ih2h+_CCH-UbxehsM5RQ6YyNnPesEpveBEtdow@mail.gmail.com>
 <20220518003531.GA4402@erisian.com.au>
In-Reply-To: <20220518003531.GA4402@erisian.com.au>
From: Gloria Zhao <gloriajzhao@gmail.com>
Date: Wed, 18 May 2022 14:40:58 -0400
Message-ID: <CAFXO6=LWM4eHM=zJhejw5981+8h7QHTbwpz0jEbWkrLOX0037Q@mail.gmail.com>
To: Anthony Towns <aj@erisian.com.au>
Content-Type: multipart/alternative; boundary="000000000000d1d5c905df4d9983"
X-Mailman-Approved-At: Wed, 18 May 2022 18:43:15 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Package Relay Proposal
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Wed, 18 May 2022 18:41:13 -0000

--000000000000d1d5c905df4d9983
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

(To everyone):
I should have made it much clearer that version 1 is only supposed to solve
1 of the 2 use cases. I was a lot more focused on the fee-bumping use case,
since it=E2=80=99s more important. Orphan-fetching was added to the motivat=
ion
section last-minute because John Newbery mentioned to me =E2=80=9Chey you c=
ould
deal with orphans really easily with this.=E2=80=9D Of course,
child-with-unconfirmed-parents packages aren=E2=80=99t very useful for
orphan-fetching since non-parent ancestors are quite common.

Maybe a version 2 package for orphan-fetching could look like this:

=E2=80=9Cpckginfo2=E2=80=9D message contains a tx with all of its ancestors

=E2=80=9CMSG_PCKG2=E2=80=9D inv type refers to a =E2=80=9Cpckginfo2=E2=80=
=9D for a tx. You don=E2=80=99t send
inv(MSG_PCKG2), but a node can request getdata(MSG_PCKG2) for a transaction
they want the ancestors for, provided they sent sendpackages(version=3D2)
ahead of time. It seems to me that orphan-fetching only ever needs to be
receiver-initiated.

Protocol flow would look like this:
https://user-images.githubusercontent.com/25183001/168891185-1630f583-de47-=
4937-86b1-2652cf8852f2.png

We don=E2=80=99t have a policy for dealing with anything more than a child =
with its
direct parents, but I also don=E2=80=99t think anybody is relying on fee-bu=
mping
more than 2 generations, so the validation logic here could probably just
submit them all individually. Maybe they can request a pckginfo1 if they
see something that=E2=80=99s too-low-fee, and/or use the
child-with-unconfirmed-parents logic opportunistically.

Thanks aj for the feedback! Responding:

> The "PCKG" abbreviation threw me for a loop; isn't the usual
> abbreviation "PKG" ?

Oh I didn't know that. I could change it if people feel strongly.

> Does it make sense for these to be configurable, rather than implied
> by the version?
> =E2=80=A6 would it be better to either just not do sendpackages
> at all if you're limiting ancestors in the mempool incompatibly

Effectively: if you=E2=80=99re setting your ancestor/descendant limits lowe=
r than
the default, you can=E2=80=99t do package relay. I wonder if this might be
controversial, since it adds pressure to adhere to Bitcoin Core=E2=80=99s c=
urrent
mempool policy? I would be happy to do it this way, though - makes things
easier to implement.

> > 5. If 'fRelay=3D=3Dfalse' in a peer's version message, the node must no=
t
> >    send "sendpackages" to them. If a "sendpackages" message is
> > received by a peer after sending `fRelay=3D=3Dfalse` in their version
> > message, the sender should be disconnected.

> Seems better to just say "if you set fRelay=3Dfalse in your version
> message, you must not send sendpackages"? You already won't do packages
> with the peer if they don't also announce sendpackages.

I guess, theoretically, if you allow bloom filters with this peer, it=E2=80=
=99s
plausible they=E2=80=99re saying =E2=80=9CfRelay=3Dfalse, I=E2=80=99ll send=
 you a bloom filter later,
and I=E2=80=99ll also want to talk about packages.=E2=80=9D
I don=E2=80=99t know if that=E2=80=99s a use case we want to support - my g=
ut reaction is
no.

> Maybe: "You must not send sendpackages unless you also send wtxidrelay" ?

Do you mean if we get a verack, and the peer sent =E2=80=9Csendpackages=E2=
=80=9D but not
=E2=80=9Cwtxidrelay,=E2=80=9D we should disconnect them?

> As I understand it, the two cases for the protocol flow are "I received
> an orphan, and I'd like its ancestors please" which seems simple enough,
> and "here's a child you may be interested in, even though you possibly
> weren't interested in the parents of that child".

(Btw, please see my notes at the top of this email about separating those
two use cases. sorry for the confusion).

> I think the logic for the latter is: [=E2=80=A6]

I have it as: we send a PCKG INV when this transaction=E2=80=99s feerate is=
 above
the fee filter, but one or more of its parents don=E2=80=99t. I don=E2=80=
=99t think using
ancestor feerate is better.
See this counterexample:
https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garde=
n/abc_1parent_2kids.png
A (0fee) has 2 kids, B (3sat/vB) and C (20sat/vB), everything=E2=80=99s the=
 same
vsize. Let=E2=80=99s say the fee filter is 3sat/vB.
If we do it based on ancestor feerate, we won=E2=80=99t send B. But B is ac=
tually
fine; C is paying for A.

> Are "getpckgtxns" / "pcktxns" really limited to packages, or are they
> just a general way to request a batch of transactions?

> Maybe call those messages "getbatchtxns" and "batchtxns" and allow them t=
o
> be used more generally, potentially in ways unrelated to packages/cpfp?

Indeed, it=E2=80=99s a general way to request a batch of transactions. I=E2=
=80=99ll
highlight that it is =E2=80=9Call or nothing,=E2=80=9D i.e. if the sender i=
s missing any of
them, they=E2=80=99ll just send a notfound.
The idea here was to avoid downloading any transactions that can=E2=80=99t =
be
validated right away. With packages, this makes sense, because there are
dependency relationships. But if you=E2=80=99re requesting multiple unrelat=
ed
transactions, for example, it=E2=80=99s unnecessary. You might end up with =
even
more transaction data that=E2=80=99s just sitting around waiting to be vali=
dated.

> The "only be sent if both peers agreed to do package relay" rule could
> simply be dropped, I think.

Wouldn=E2=80=99t we need some way of saying =E2=80=9Chey I support batchtxn=
s?=E2=80=9D Otherwise
you would have to guess by sending a request and waiting to see if it=E2=80=
=99s
ignored?

> Shouldn't the sender only be sending package announcements when they know
> the recipient will be interested in the package, based on their feefilter=
?

I think there are cases where the sender doesn=E2=80=99t necessarily know.
Consider this example:
https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garde=
n/rich_parent_bad_cpfp.png
D (5sat/vB) has 2 parents, A (0sat/vB) and B (20sat/vB). All same size.
Feefilter is 3sat/vB.
If the receiver already has B, they=E2=80=99ll know they can just reject th=
e
package already based on the pckginfo.
But the sender doesn=E2=80=99t really know that. The sender just knows A is=
 below
feerate and D is above. D is above the fee filter, and its ancestor feerate
is above the fee filter.

> CAmount in consensus/amount.h is a int64_t
> The maximum block weight is 4M=E2=80=A6

 Oops yes. I think we just usually use int64_t for vsizes afaik. Agree that
it should be 8 bytes for fee, and 4 bytes is enough for vsize.

> I guess tx relay is low priority enough that it wouldn't be worth tagging
> some peers as "high bandwidth" and having them immediately announce the
> PCKGINFO1 message, and skip the INV/GETDATA step?

I had the same idea as well, but seemed unnecessary. It would reduce the
number of round trips, but I don=E2=80=99t think an extra round trip is tha=
t big of
a deal for transaction relay. Block relay, yes of course, but I don=E2=80=
=99t think
we care that much if it takes an extra second to send a transaction?

Best,
Gloria

On Tue, May 17, 2022 at 8:35 PM Anthony Towns <aj@erisian.com.au> wrote:

> On Tue, May 17, 2022 at 12:01:04PM -0400, Gloria Zhao via bitcoin-dev
> wrote:
> > =3D=3D=3D=3DNew Messages=3D=3D=3D=3D
> > Three new protocol messages are added for use in any version of
> > package relay. Additionally, each version of package relay must define
> > its own inv type and "pckginfo" message version, referred to in this
> > document as "MSG_PCKG" and "pckginfo" respectively. See
> > BIP-v1-packages for a concrete example.
>
> The "PCKG" abbreviation threw me for a loop; isn't the usual
> abbreviation "PKG" ?
>
> > =3D=3D=3D=3D=3Dsendpackages=3D=3D=3D=3D=3D
> > |version || uint32_t || 4 || Denotes a package version supported by the
> > node.
> > |max_count || uint32_t || 4 ||Specifies the maximum number of
> transactions
> > per package this node is
> > willing to accept.
> > |max_weight || uint32_t || 4 ||Specifies the maximum total weight per
> > package this node is willing
> > to accept.
>
> Does it make sense for these to be configurable, rather than implied
> by the version?
>
> I presume the idea is to cope with people specifying different values for
> -limitancestorcount or -limitancestorsize, but if people are regularly
> relaying packages around, it seems like it becomes hard to have those
> values really be configurable while being compatible with that?
>
> I guess I'm asking: would it be better to either just not do sendpackages
> at all if you're limiting ancestors in the mempool incompatibly; or
> alternatively, would it be better to do the package relay, then reject
> the particular package if it turns out too big, and log that you've
> dropped it so that the node operator has some way of realising "whoops,
> I'm not relaying packages properly because of how I configured my node"?
>
> > 5. If 'fRelay=3D=3Dfalse' in a peer's version message, the node must no=
t
> >    send "sendpackages" to them. If a "sendpackages" message is
> > received by a peer after sending `fRelay=3D=3Dfalse` in their version
> > message, the sender should be disconnected.
>
> Seems better to just say "if you set fRelay=3Dfalse in your version
> message, you must not send sendpackages"? You already won't do packages
> with the peer if they don't also announce sendpackages.
>
> > 7. If both peers send "wtxidrelay" and "sendpackages" with the same
> >    version, the peers should announce, request, and send package
> > information to each other.
>
> Maybe: "You must not send sendpackages unless you also send wtxidrelay" ?
>
>
> As I understand it, the two cases for the protocol flow are "I received
> an orphan, and I'd like its ancestors please" which seems simple enough,
> and "here's a child you may be interested in, even though you possibly
> weren't interested in the parents of that child". I think the logic for
> the latter is:
>
>  * if tx C's fee rate is less than the peer's feefilter, skip it
>    (will maybe treat it as a parent in some package later though)
>  * if tx C's ancestor fee rate is less than the peer's feefilter, skip
>    it?
>  * look at the lowest ancestor fee rate for any of C's in-mempool
>    parents
>  * if that is higher than the peer's fee filter, send a normal INV
>  * if it's lower than the peer's fee filter, send a PCKG INV
>
> Are "getpckgtxns" / "pcktxns" really limited to packages, or are they
> just a general way to request a batch of transactions? Particularly in
> the case of requesting the parents of an orphan tx you already have,
> it seems hard for the node receiving getpckgtxns to validate that the
> txs are related in some way; but also it doesn't seem very necessary?
>
> Maybe call those messages "getbatchtxns" and "batchtxns" and allow them t=
o
> be used more generally, potentially in ways unrelated to packages/cpfp?
> The "only be sent if both peers agreed to do package relay" rule could
> simply be dropped, I think.
>
> > 4. The reciever uses the package information to decide how to request
> >    the transactions. For example, if the receiver already has some of
> > the transactions in their mempool, they only request the missing ones.
> > They could also decide not to request the package at all based on the
> > fee information provided.
>
> Shouldn't the sender only be sending package announcements when they know
> the recipient will be interested in the package, based on their feefilter=
?
>
> > =3D=3D=3D=3D=3Dpckginfo1=3D=3D=3D=3D=3D
> > {|
> > |  Field Name  ||  Type  ||  Size  ||   Purpose
> > |-
> > |blockhash || uint256 || 32 || The chain tip at which this package is
> > defined.
> > |-
> > |pckg_fee||CAmount||4|| The sum total fees paid by all transactions in
> the
> > package.
>
> CAmount in consensus/amount.h is a int64_t so shouldn't this be 8
> bytes? If you limit a package to 101kvB, an int32_t is enough to cover
> any package with a fee rate of about 212 BTC/block or lower, though.
>
> > |pckg_weight||int64_t||8|| The sum total weight of all transactions in
> the
> > package.
>
> The maximum block weight is 4M, and the default -limitancestorsize
> presumably implies a max package weight of 404k; seems odd to provide
> a uint64_t rather than an int32_t here, which easily allows either of
> those values?
>
> > 2. ''Only 1 child with unconfirmed parents.'' The package must consist
> >    of one transaction and its unconfirmed parents. There must not be
> > any other transactions in the package. Other dependency relationships
> > may exist within the package (e.g. one parent may spend the output of
> > another parent) provided that topological order is respected.
>
> I think this means that some of the parents could also have unconfirmed
> parents, but they won't be included in the package, and must be requested
> via the recipient-initiated approach?
>
> > 5. ''Total fees and weight.'' The 'total_fee' and 'total_weight'
> >    fields must accurately represent the sum total of all transactions'
> >    fees and weights as defined in BIP141, respectively.
>
> Presumably this excludes any unconfirmed grandparents and earlier
> ancestors since they aren't part of the package, in this approach? Doesn'=
t
> that make this both harder to calculate (assuming we already have
> ancestor summaries) and less useful, in the case where those ancestors
> have a lower fee rate?
>
> > ''Q: Can "getpckgtxns" and "pckgtxns" messages contain only one
> > transaction?''
> > Yes.
>
> This would be normal if you're requesting a single missing parent for
> an orphan you've received, I think?
>
> I'm slightly surprised the process is:
>
>  ->  INV PCKG1 C
>   <- GETDATA PCKG1 C
>  ->  PCKGINFO1 blockhash A B C fee weight
>
> rather than announcing the package fee info in the first message.
> But if the sender is already applying the feefilter to the package before
> announcing it, it probably doesn't matter, and means you're only getting
> a 32B INV from every peer, rather than a 32*(n+2) PCKGINFO1 message from
> every peer.
>
> I guess tx relay is low priority enough that it wouldn't be worth tagging
> some peers as "high bandwidth" and having them immediately announce the
> PCKGINFO1 message, and skip the INV/GETDATA step?
>
> Cheers,
> aj
>
>

--000000000000d1d5c905df4d9983
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div>(To everyone):<br>I should have made it much clea=
rer that version 1 is only supposed to solve 1 of the 2 use cases. I was a =
lot more focused on the fee-bumping use case, since it=E2=80=99s more impor=
tant. Orphan-fetching was added to the motivation section last-minute becau=
se John Newbery mentioned to me =E2=80=9Chey you could deal with orphans re=
ally easily with this.=E2=80=9D Of course, child-with-unconfirmed-parents p=
ackages aren=E2=80=99t very useful for orphan-fetching since non-parent anc=
estors are quite common.<br><br>Maybe a version 2 package for orphan-fetchi=
ng could look like this:<br><br>=E2=80=9Cpckginfo2=E2=80=9D message contain=
s a tx with all of its ancestors<br><br>=E2=80=9CMSG_PCKG2=E2=80=9D inv typ=
e refers to a =E2=80=9Cpckginfo2=E2=80=9D for a tx. You don=E2=80=99t send =
inv(MSG_PCKG2), but a node can request getdata(MSG_PCKG2) for a transaction=
 they want the ancestors for, provided they sent sendpackages(version=3D2) =
ahead of time. It seems to me that orphan-fetching only ever needs to be re=
ceiver-initiated.<br><br>Protocol flow would look like this:<br><a href=3D"=
https://user-images.githubusercontent.com/25183001/168891185-1630f583-de47-=
4937-86b1-2652cf8852f2.png">https://user-images.githubusercontent.com/25183=
001/168891185-1630f583-de47-4937-86b1-2652cf8852f2.png</a> <br><br>We don=
=E2=80=99t have a policy for dealing with anything more than a child with i=
ts direct parents, but I also don=E2=80=99t think anybody is relying on fee=
-bumping more than 2 generations, so the validation logic here could probab=
ly just submit them all individually. Maybe they can request a pckginfo1 if=
 they see something that=E2=80=99s too-low-fee, and/or use the child-with-u=
nconfirmed-parents logic opportunistically.<br><br>Thanks aj for the feedba=
ck! Responding:<br><br>&gt; The &quot;PCKG&quot; abbreviation threw me for =
a loop; isn&#39;t the usual<br>&gt; abbreviation &quot;PKG&quot; ?<br><br>O=
h I didn&#39;t know that. I could change it if people feel strongly.<br><br=
>&gt; Does it make sense for these to be configurable, rather than implied<=
br>&gt; by the version?<br>&gt; =E2=80=A6 would it be better to either just=
 not do sendpackages<br>&gt; at all if you&#39;re limiting ancestors in the=
 mempool incompatibly<br><br>Effectively: if you=E2=80=99re setting your an=
cestor/descendant limits lower than the default, you can=E2=80=99t do packa=
ge relay. I wonder if this might be controversial, since it adds pressure t=
o adhere to Bitcoin Core=E2=80=99s current mempool policy? I would be happy=
 to do it this way, though - makes things easier to implement.<br><br>&gt; =
&gt; 5. If &#39;fRelay=3D=3Dfalse&#39; in a peer&#39;s version message, the=
 node must not<br>&gt; &gt;=C2=A0 =C2=A0 send &quot;sendpackages&quot; to t=
hem. If a &quot;sendpackages&quot; message is<br>&gt; &gt; received by a pe=
er after sending `fRelay=3D=3Dfalse` in their version<br>&gt; &gt; message,=
 the sender should be disconnected.<br><br>&gt; Seems better to just say &q=
uot;if you set fRelay=3Dfalse in your version<br>&gt; message, you must not=
 send sendpackages&quot;? You already won&#39;t do packages<br>&gt; with th=
e peer if they don&#39;t also announce sendpackages.<br><br>I guess, theore=
tically, if you allow bloom filters with this peer, it=E2=80=99s plausible =
they=E2=80=99re saying =E2=80=9CfRelay=3Dfalse, I=E2=80=99ll send you a blo=
om filter later, and I=E2=80=99ll also want to talk about packages.=E2=80=
=9D<br>I don=E2=80=99t know if that=E2=80=99s a use case we want to support=
 - my gut reaction is no.<br><br>&gt; Maybe: &quot;You must not send sendpa=
ckages unless you also send wtxidrelay&quot; ?<br><br>Do you mean if we get=
 a verack, and the peer sent =E2=80=9Csendpackages=E2=80=9D but not =E2=80=
=9Cwtxidrelay,=E2=80=9D we should disconnect them?<br><br>&gt; As I underst=
and it, the two cases for the protocol flow are &quot;I received<br>&gt; an=
 orphan, and I&#39;d like its ancestors please&quot; which seems simple eno=
ugh,<br>&gt; and &quot;here&#39;s a child you may be interested in, even th=
ough you possibly<br>&gt; weren&#39;t interested in the parents of that chi=
ld&quot;.<br><br>(Btw, please see my notes at the top of this email about s=
eparating those two use cases. sorry for the confusion).<br><br>&gt; I thin=
k the logic for the latter is: [=E2=80=A6]<br><br>I have it as: we send a P=
CKG INV when this transaction=E2=80=99s feerate is above the fee filter, bu=
t one or more of its parents don=E2=80=99t. I don=E2=80=99t think using anc=
estor feerate is better.<br>See this counterexample: <a href=3D"https://raw=
.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garden/abc_1pare=
nt_2kids.png">https://raw.githubusercontent.com/glozow/bitcoin-notes/master=
/mempool_garden/abc_1parent_2kids.png</a><br>A (0fee) has 2 kids, B (3sat/v=
B) and C (20sat/vB), everything=E2=80=99s the same vsize. Let=E2=80=99s say=
 the fee filter is 3sat/vB.<br>If we do it based on ancestor feerate, we wo=
n=E2=80=99t send B. But B is actually fine; C is paying for A.<br><br>&gt; =
Are &quot;getpckgtxns&quot; / &quot;pcktxns&quot; really limited to package=
s, or are they<br>&gt; just a general way to request a batch of transaction=
s?<br><br>&gt; Maybe call those messages &quot;getbatchtxns&quot; and &quot=
;batchtxns&quot; and allow them to<br>&gt; be used more generally, potentia=
lly in ways unrelated to packages/cpfp?<br><br>Indeed, it=E2=80=99s a gener=
al way to request a batch of transactions. I=E2=80=99ll highlight that it i=
s =E2=80=9Call or nothing,=E2=80=9D i.e. if the sender is missing any of th=
em, they=E2=80=99ll just send a notfound.<br>The idea here was to avoid dow=
nloading any transactions that can=E2=80=99t be validated right away. With =
packages, this makes sense, because there are dependency relationships. But=
 if you=E2=80=99re requesting multiple unrelated transactions, for example,=
 it=E2=80=99s unnecessary. You might end up with even more transaction data=
 that=E2=80=99s just sitting around waiting to be validated.<br><br>&gt; Th=
e &quot;only be sent if both peers agreed to do package relay&quot; rule co=
uld<br>&gt; simply be dropped, I think.<br><br>Wouldn=E2=80=99t we need som=
e way of saying =E2=80=9Chey I support batchtxns?=E2=80=9D Otherwise you wo=
uld have to guess by sending a request and waiting to see if it=E2=80=99s i=
gnored?<br><br>&gt; Shouldn&#39;t the sender only be sending package announ=
cements when they know<br>&gt; the recipient will be interested in the pack=
age, based on their feefilter?<br><br>I think there are cases where the sen=
der doesn=E2=80=99t necessarily know.<br>Consider this example: <a href=3D"=
https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garde=
n/rich_parent_bad_cpfp.png">https://raw.githubusercontent.com/glozow/bitcoi=
n-notes/master/mempool_garden/rich_parent_bad_cpfp.png</a> <br>D (5sat/vB) =
has 2 parents, A (0sat/vB) and B (20sat/vB). All same size. Feefilter is 3s=
at/vB.<br>If the receiver already has B, they=E2=80=99ll know they can just=
 reject the package already based on the pckginfo.<br>But the sender doesn=
=E2=80=99t really know that. The sender just knows A is below feerate and D=
 is above. D is above the fee filter, and its ancestor feerate is above the=
 fee filter.<br><br>&gt; CAmount in consensus/amount.h is a int64_t<br>&gt;=
 The maximum block weight is 4M=E2=80=A6<br><br>=C2=A0Oops yes. I think we =
just usually use int64_t for vsizes afaik. Agree that it should be 8 bytes =
for fee, and 4 bytes is enough for vsize.<br><br>&gt; I guess tx relay is l=
ow priority enough that it wouldn&#39;t be worth tagging<br>&gt; some peers=
 as &quot;high bandwidth&quot; and having them immediately announce the<br>=
&gt; PCKGINFO1 message, and skip the INV/GETDATA step?<br><br>I had the sam=
e idea as well, but seemed unnecessary. It would reduce the number of round=
 trips, but I don=E2=80=99t think an extra round trip is that big of a deal=
 for transaction relay. Block relay, yes of course, but I don=E2=80=99t thi=
nk we care that much if it takes an extra second to send a transaction?<br>=
<br>Best,<br>Gloria</div></div><br><div class=3D"gmail_quote"><div dir=3D"l=
tr" class=3D"gmail_attr">On Tue, May 17, 2022 at 8:35 PM Anthony Towns &lt;=
<a href=3D"mailto:aj@erisian.com.au">aj@erisian.com.au</a>&gt; wrote:<br></=
div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, May 17, 2022 =
at 12:01:04PM -0400, Gloria Zhao via bitcoin-dev wrote:<br>
&gt; =3D=3D=3D=3DNew Messages=3D=3D=3D=3D<br>
&gt; Three new protocol messages are added for use in any version of<br>
&gt; package relay. Additionally, each version of package relay must define=
<br>
&gt; its own inv type and &quot;pckginfo&quot; message version, referred to=
 in this<br>
&gt; document as &quot;MSG_PCKG&quot; and &quot;pckginfo&quot; respectively=
. See<br>
&gt; BIP-v1-packages for a concrete example.<br>
<br>
The &quot;PCKG&quot; abbreviation threw me for a loop; isn&#39;t the usual<=
br>
abbreviation &quot;PKG&quot; ?<br>
<br>
&gt; =3D=3D=3D=3D=3Dsendpackages=3D=3D=3D=3D=3D<br>
&gt; |version || uint32_t || 4 || Denotes a package version supported by th=
e<br>
&gt; node.<br>
&gt; |max_count || uint32_t || 4 ||Specifies the maximum number of transact=
ions<br>
&gt; per package this node is<br>
&gt; willing to accept.<br>
&gt; |max_weight || uint32_t || 4 ||Specifies the maximum total weight per<=
br>
&gt; package this node is willing<br>
&gt; to accept.<br>
<br>
Does it make sense for these to be configurable, rather than implied<br>
by the version? <br>
<br>
I presume the idea is to cope with people specifying different values for<b=
r>
-limitancestorcount or -limitancestorsize, but if people are regularly<br>
relaying packages around, it seems like it becomes hard to have those<br>
values really be configurable while being compatible with that?<br>
<br>
I guess I&#39;m asking: would it be better to either just not do sendpackag=
es<br>
at all if you&#39;re limiting ancestors in the mempool incompatibly; or<br>
alternatively, would it be better to do the package relay, then reject<br>
the particular package if it turns out too big, and log that you&#39;ve<br>
dropped it so that the node operator has some way of realising &quot;whoops=
,<br>
I&#39;m not relaying packages properly because of how I configured my node&=
quot;?<br>
<br>
&gt; 5. If &#39;fRelay=3D=3Dfalse&#39; in a peer&#39;s version message, the=
 node must not<br>
&gt;=C2=A0 =C2=A0 send &quot;sendpackages&quot; to them. If a &quot;sendpac=
kages&quot; message is<br>
&gt; received by a peer after sending `fRelay=3D=3Dfalse` in their version<=
br>
&gt; message, the sender should be disconnected.<br>
<br>
Seems better to just say &quot;if you set fRelay=3Dfalse in your version<br=
>
message, you must not send sendpackages&quot;? You already won&#39;t do pac=
kages<br>
with the peer if they don&#39;t also announce sendpackages.<br>
<br>
&gt; 7. If both peers send &quot;wtxidrelay&quot; and &quot;sendpackages&qu=
ot; with the same<br>
&gt;=C2=A0 =C2=A0 version, the peers should announce, request, and send pac=
kage<br>
&gt; information to each other.<br>
<br>
Maybe: &quot;You must not send sendpackages unless you also send wtxidrelay=
&quot; ?<br>
<br>
<br>
As I understand it, the two cases for the protocol flow are &quot;I receive=
d<br>
an orphan, and I&#39;d like its ancestors please&quot; which seems simple e=
nough,<br>
and &quot;here&#39;s a child you may be interested in, even though you poss=
ibly<br>
weren&#39;t interested in the parents of that child&quot;. I think the logi=
c for<br>
the latter is:<br>
<br>
=C2=A0* if tx C&#39;s fee rate is less than the peer&#39;s feefilter, skip =
it<br>
=C2=A0 =C2=A0(will maybe treat it as a parent in some package later though)=
<br>
=C2=A0* if tx C&#39;s ancestor fee rate is less than the peer&#39;s feefilt=
er, skip<br>
=C2=A0 =C2=A0it?<br>
=C2=A0* look at the lowest ancestor fee rate for any of C&#39;s in-mempool<=
br>
=C2=A0 =C2=A0parents<br>
=C2=A0* if that is higher than the peer&#39;s fee filter, send a normal INV=
<br>
=C2=A0* if it&#39;s lower than the peer&#39;s fee filter, send a PCKG INV<b=
r>
<br>
Are &quot;getpckgtxns&quot; / &quot;pcktxns&quot; really limited to package=
s, or are they<br>
just a general way to request a batch of transactions? Particularly in<br>
the case of requesting the parents of an orphan tx you already have,<br>
it seems hard for the node receiving getpckgtxns to validate that the<br>
txs are related in some way; but also it doesn&#39;t seem very necessary?<b=
r>
<br>
Maybe call those messages &quot;getbatchtxns&quot; and &quot;batchtxns&quot=
; and allow them to<br>
be used more generally, potentially in ways unrelated to packages/cpfp?<br>
The &quot;only be sent if both peers agreed to do package relay&quot; rule =
could<br>
simply be dropped, I think.<br>
<br>
&gt; 4. The reciever uses the package information to decide how to request<=
br>
&gt;=C2=A0 =C2=A0 the transactions. For example, if the receiver already ha=
s some of<br>
&gt; the transactions in their mempool, they only request the missing ones.=
<br>
&gt; They could also decide not to request the package at all based on the<=
br>
&gt; fee information provided.<br>
<br>
Shouldn&#39;t the sender only be sending package announcements when they kn=
ow<br>
the recipient will be interested in the package, based on their feefilter?<=
br>
<br>
&gt; =3D=3D=3D=3D=3Dpckginfo1=3D=3D=3D=3D=3D<br>
&gt; {|<br>
&gt; |=C2=A0 Field Name=C2=A0 ||=C2=A0 Type=C2=A0 ||=C2=A0 Size=C2=A0 ||=C2=
=A0 =C2=A0Purpose<br>
&gt; |-<br>
&gt; |blockhash || uint256 || 32 || The chain tip at which this package is<=
br>
&gt; defined.<br>
&gt; |-<br>
&gt; |pckg_fee||CAmount||4|| The sum total fees paid by all transactions in=
 the<br>
&gt; package.<br>
<br>
CAmount in consensus/amount.h is a int64_t so shouldn&#39;t this be 8<br>
bytes? If you limit a package to 101kvB, an int32_t is enough to cover<br>
any package with a fee rate of about 212 BTC/block or lower, though.<br>
<br>
&gt; |pckg_weight||int64_t||8|| The sum total weight of all transactions in=
 the<br>
&gt; package.<br>
<br>
The maximum block weight is 4M, and the default -limitancestorsize<br>
presumably implies a max package weight of 404k; seems odd to provide<br>
a uint64_t rather than an int32_t here, which easily allows either of<br>
those values?<br>
<br>
&gt; 2. &#39;&#39;Only 1 child with unconfirmed parents.&#39;&#39; The pack=
age must consist<br>
&gt;=C2=A0 =C2=A0 of one transaction and its unconfirmed parents. There mus=
t not be<br>
&gt; any other transactions in the package. Other dependency relationships<=
br>
&gt; may exist within the package (e.g. one parent may spend the output of<=
br>
&gt; another parent) provided that topological order is respected.<br>
<br>
I think this means that some of the parents could also have unconfirmed<br>
parents, but they won&#39;t be included in the package, and must be request=
ed<br>
via the recipient-initiated approach?<br>
<br>
&gt; 5. &#39;&#39;Total fees and weight.&#39;&#39; The &#39;total_fee&#39; =
and &#39;total_weight&#39;<br>
&gt;=C2=A0 =C2=A0 fields must accurately represent the sum total of all tra=
nsactions&#39;<br>
&gt;=C2=A0 =C2=A0 fees and weights as defined in BIP141, respectively.<br>
<br>
Presumably this excludes any unconfirmed grandparents and earlier<br>
ancestors since they aren&#39;t part of the package, in this approach? Does=
n&#39;t<br>
that make this both harder to calculate (assuming we already have<br>
ancestor summaries) and less useful, in the case where those ancestors<br>
have a lower fee rate?<br>
<br>
&gt; &#39;&#39;Q: Can &quot;getpckgtxns&quot; and &quot;pckgtxns&quot; mess=
ages contain only one<br>
&gt; transaction?&#39;&#39;<br>
&gt; Yes.<br>
<br>
This would be normal if you&#39;re requesting a single missing parent for<b=
r>
an orphan you&#39;ve received, I think?<br>
<br>
I&#39;m slightly surprised the process is:<br>
<br>
=C2=A0-&gt;=C2=A0 INV PCKG1 C<br>
=C2=A0 &lt;- GETDATA PCKG1 C<br>
=C2=A0-&gt;=C2=A0 PCKGINFO1 blockhash A B C fee weight<br>
<br>
rather than announcing the package fee info in the first message.<br>
But if the sender is already applying the feefilter to the package before<b=
r>
announcing it, it probably doesn&#39;t matter, and means you&#39;re only ge=
tting<br>
a 32B INV from every peer, rather than a 32*(n+2) PCKGINFO1 message from<br=
>
every peer.<br>
<br>
I guess tx relay is low priority enough that it wouldn&#39;t be worth taggi=
ng<br>
some peers as &quot;high bandwidth&quot; and having them immediately announ=
ce the<br>
PCKGINFO1 message, and skip the INV/GETDATA step?<br>
<br>
Cheers,<br>
aj<br>
<br>
</blockquote></div>

--000000000000d1d5c905df4d9983--