path: root/02/f086343faaea93f4d07f0f8687a636bdfe3de3

Return-Path: <alicexbt@protonmail.com>
Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 3258EC0033
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Mon, 26 Sep 2022 17:51:04 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp1.osuosl.org (Postfix) with ESMTP id F3D4E82CA5
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Mon, 26 Sep 2022 17:51:03 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org F3D4E82CA5
Authentication-Results: smtp1.osuosl.org;
 dkim=pass (2048-bit key) header.d=protonmail.com header.i=@protonmail.com
 header.a=rsa-sha256 header.s=protonmail3 header.b=O3FutBa0
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Level: 
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001]
 autolearn=ham autolearn_force=no
Received: from smtp1.osuosl.org ([127.0.0.1])
 by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id WuLQCQWf_wha
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Mon, 26 Sep 2022 17:51:01 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3AE7782C21
Received: from mail-4318.protonmail.ch (mail-4318.protonmail.ch [185.70.43.18])
 by smtp1.osuosl.org (Postfix) with ESMTPS id 3AE7782C21
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Mon, 26 Sep 2022 17:51:01 +0000 (UTC)
Date: Mon, 26 Sep 2022 17:50:48 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail3; t=1664214658; x=1664473858;
 bh=9ExWb2kiwyDh+jSWcQ+EIqY2cNoseSdZezaEt3NS680=;
 h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
 Message-ID;
 b=O3FutBa0/TrG9sc7N1/1lBb/eRIxKSmTObfqZYpbL73HZUj/odVWh7cVGGgF0zu6I
 nzJLIFP7qlFswPgkc4d1Ea2fisv6Ef7JAyNqBAftEey3Y7Kg7CPml3VirAzEZRz0nG
 ZakjA/8K9mI/QIPXLZC9b3I086KREy72fh5jq7jlC+GYOoHZEccm+zRFDb1RzrIiNz
 +J+jPj5IBftvw7GJGgm69PUaSX9FcgxcOwvEBILGno2fDLxBH0fjg+xK/1iQhaADGW
 SqSZ+sR9HuG92IcF+Ct2nM1//HEuVj32uSE2802Qelh7UzAJPb3RcJPW+6RaUEY5Ck
 uEFK/Y9uSkBUg==
To: eric@voskuil.org
From: alicexbt <alicexbt@protonmail.com>
Message-ID: <EDCbz3IJLDkExNWyM1WLLSh5qUioK6HdMEh-eofWmete0baz1lo8uF1YFzLaTXxgQhzvtDtqpX2mTMaDpB7_4-SRNnpJX4F8zgL5e6NGS0U=@protonmail.com>
In-Reply-To: <005e01d87b89$3d99df60$b8cd9e20$@voskuil.org>
References: <005e01d87b89$3d99df60$b8cd9e20$@voskuil.org>
Feedback-ID: 40602938:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Mailman-Approved-At: Mon, 26 Sep 2022 17:52:57 +0000
Cc: 'Bitcoin Protocol Discussion' <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Packaged Transaction Relay
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Sep 2022 17:51:04 -0000

Hi Eric,


This email wasn't answered by anyone on mailing list however I did some res=
earch about packages yesterday including this email and below are my observ=
ations, questions etc.

> The sole objective, as expressed in the OP proposal, is to:
>=20
> "Propagate transactions that are incentive-compatible to mine, even if th=
ey don't meet minimum feerate alone."


According to [bitcoinops][1]: Without package relay, it=E2=80=99s not possi=
ble to effectively CPFP fee bump a transaction that=E2=80=99s below the min=
imum feerate nodes accept.

Matt Corallo's thoughts in a bitcoin core [issue][2]:

"Matt Corallo recently wrote about an example on the bitcoin-dev mailing li=
st involving lightning transactions, where pre-signed transactions might be=
 broadcast to the blockchain long after they were generated, and thus not h=
ave been created with a fee that is sufficient to be confirmed quickly (or =
even be accepted to node mempools). In such situations, channel participant=
s may need to use chained transactions (CPFP) in order to increase the conf=
irmation speed of such transactions, and that implies we may need to introd=
uce a mechanism for those parent transactions to be relayed along with thei=
r higher feerate children, even if the parent transaction would be rejected=
 by itself."

1)Is it possible to have multiple pre-signed transactions with different fe=
e rates in a range? Example: PSBT1: 5 sat/vbyte, PSBT2: 10 sat/vbyte, PSBT3=
: 20 sat/vbyte and PSBT4: 100 sat/vbyte
2)How would covenants affect this problem?
3)How often does it happen that a pre-signed tx gets rejected by nodes beca=
use it did not meet the minimum fee rate? Is it predictable and could be ma=
naged in a different way?

After reading several links related to packages and bitcoin core pull reque=
sts, I found it anti-bitcoin to introduce so much complexity because its no=
t possible to CPFP fee bump a tx below minimum fee rate.=20


> Furthermore any tx that is "stuck" can be freed by simply sending another=
 tx. The nodes at which the tx has become stuck will just package it up and=
 relay it to peers. In other words, there is no impact on wallet implementa=
tion apart from raising the aggregate fee using a descendant transaction.

It is easy to send another tx if there is only one user involved however pa=
ckages are trying to fix issues in which multiple users and transaction pre=
-signed between them are involved. So, it will be difficult to coordinate a=
nd create new pre-signed transactions in some cases although it is possible=
 for some use cases.


> This is barely a protocol change - it's primarily implementation. All tha=
t should be required is an additional INV element type, such as MSG_TX_PACK=
AGE.

> * All elements of MSG_TX_PACKAGE in one INV message MUST to be of the sam=
e package.
> * A package MUST must define a set that can be mined into one block (size=
/sigops constraint).
> * A package SHOULD not contain confirmed txs (a race may cause this).
> * A package MUST minimally satisfy peer.feerate.
> * A partial tx order, as in the manner of the block.txs ordering, MUST be=
 imposed.
> * A node SHOULD drop a peer that sends a package (or tx) below node.feera=
te.
> * A node MAY drop a peer that sends a non-minimal package according to no=
de.feerate.

This makes sense particularly if multiple node implementations are used in =
future.=20

My other questions:

a)If a package has tx1, tx2, tx3, tx4 and tx5 and miner just include tx1 an=
d tx2 in the block, how does this affect the projects considered for packag=
es proposal?

b)How does changing the order of txs in a package affect these transactions=
?

c)Do packages introduce more attack vectors in bitcoin for front running or=
 MEV? MEV in bitcoin currently only affects the projects that are considere=
d in packages proposal.

d)What if the package contains a transactions with sanctioned address?

e)Why would miners use packages if the existing scenario in terms of fees p=
er block is beneficial for them?


/dev/fd0

[1]: https://bitcoinops.org/en/topics/package-relay/
[2]: https://github.com/bitcoin/bitcoin/issues/14895

Sent with Proton Mail secure email.

------- Original Message -------
On Thursday, June 9th, 2022 at 4:13 AM, Eric Voskuil via bitcoin-dev <bitco=
in-dev@lists.linuxfoundation.org> wrote:


> Hi Suhas/Gloria,
>=20
> Good questions. I've started a new thread because it became something els=
e...
>=20
> Various ideas about packaging seem to be focused on the idea of an atomic=
 message that is gossiped around the network like a transaction or block. F=
rom my perspective that seems to create a set of problems without good solu=
tions, and it is not a proper analogy to those atomic structures. It may be=
 worth taking the time to step back and take a close look at the underlying=
 objective.
>=20
> The sole objective, as expressed in the OP proposal, is to:
>=20
> "Propagate transactions that are incentive-compatible to mine, even if th=
ey don't meet minimum feerate alone."
>=20
> Effectively producing this outcome with an atomic packaging approach whil=
e at the same time maintaining network invariants seems unlikely, if not im=
possible.
>=20
> Fees:
>=20
> A node knows what fee rate a peer will accept, and announces individual t=
xs that satisfy peer.feerate. Similarly a node knows its own feerate, and S=
HOULD drop any peer that announces txs that do not satisfy node.feerate.
>=20
> Orphans:
>=20
> A node MAY drop a peer that announces txs that the node sees as orphans a=
gainst its DAG. It SHOULD drop the orphan tx and MAY request missing ancest=
ors. Presumably after some amount of time connected to peer, node does not =
expect to see any more orphans from that peer, so these choices could evolv=
e with the channel. However, the design that can only consider each tx in i=
solation will continue to cause orphan announcements on the channel. A belo=
w peer.feerate tx does not get announced to peer, and later a descendant hi=
gh peer.feerate does get announced to the peer - as an orphan.
>=20
> BIP133 (feefilter):
>=20
> "There could be a small number of edge cases where a node's mempool min f=
ee is actually less than the filter value a peer is aware of and transactio=
ns with fee rates between these values will now be newly inhibited."
>=20
> https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki
>=20
> Whether the problem is "small" or not depends on the disparity between no=
de fee rates, which is not a matter of protocol. This is an existing proble=
m that can and should be dealt with in packaging, as part of the above obje=
ctive.
>=20
> Packaged Transaction Relay:
>=20
> One might instead think of packaging as a per-connection function, operat=
ing over its transaction (input->output) DAG and the feerate of its own nod=
e and that of the peer. Logically a "package" is nothing more than a set of=
 transactions (optimized by announcement). Only a node can effectively dete=
rmine the packaging required by each of its peers, since only the node is a=
ware of peer.feerate.
>=20
>=20
> The only way to avoid dead-ending packages (including individual transact=
ions, as is the objective) is for a node to package txs for each peer. The =
origination of any package is then just a wallet peer doing what a node doe=
s - packaging transactions that satisfy peer.feerate (i.e. that of its node=
).
>=20
> Current transaction relay (txB->txA):
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> Node0
> txA.feerate > node.feerate, and not orphaned (accept txA)
>=20
> txA.feerate > peer1.feerate (announce txA to peer1)
>=20
> txA.feerate < peer2.feerate (do not announce txA to peer2)
> -----
> txB.feerate > node.feerate (accept txB)
>=20
> txB.feerate > peer1.feerate (announce txB to peer1)
>=20
> txB.feerate > peer2.feerate (announce txB to peer2)
>=20
>=20
> Node1
> Sees/accepts txA and txB.
>=20
> Node2
> Never sees txA, sees/rejects txB (as an orphan).
>=20
> Packaged transaction relay (txB->txA):
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> Node0
> txA.feerate > node.feerate, and not orphaned (accept txA)
>=20
> txA.feerate > peer1.feerate (announce txA to peer1)
>=20
> txA.feerate < peer2.feerate (do not announce txA to peer2)
> -----
> txB.feerate > node1.feerate (accept txB)
>=20
> txB.feerate > peer1.feerate (announce txB to peer1)
>=20
> txB.feerate > peer2.feerate (do not announce txB to peer2) <=3D=3D avoid =
predictable orphan
>=20
> txA.feerate + txB.feerate > peer2.feerate (announce pkg(A, B) to peer2) <=
=3D create minimal package
>=20
>=20
> Node1
> Sees/accepts txA and txB.
>=20
> Node2
> pkg(A, B) > node2.feerate (accept txA, txB)
>=20
> txA.feerate > peer3.feerate (announce txA to peer3)
>=20
> txB.feerate > peer3.feerate (announce txB to peer3)
>=20
>=20
> Sees/accepts pkg(A, B).
>=20
> Node3
> Sees/accepts txA and txB. <=3D avoided unnecessary packaging
>=20
> Summary:
>=20
> In this design, any node that receives an announcement for a pkg (or tx) =
later determined to be less than node.feerate SHOULD drop the announcing pe=
er. Unlike with existing tx relay, a node can become "current" and subseque=
ntly see few if any tx or pkg orphans, and MAY at some point decide to drop=
 any peer that announces one. Notice that packages are created dynamically,=
 and any package that doesn't need to be grouped gets trimmed down to indiv=
idual transactions. Furthermore any tx that is "stuck" can be freed by simp=
ly sending another tx. The nodes at which the tx has become stuck will just=
 package it up and relay it to peers. In other words, there is no impact on=
 wallet implementation apart from raising the aggregate fee using a descend=
ant transaction.
>=20
> This is barely a protocol change - it's primarily implementation. All tha=
t should be required is an additional INV element type, such as MSG_TX_PACK=
AGE.
>=20
> Additional constraints:
>=20
> * All elements of MSG_TX_PACKAGE in one INV message MUST to be of the sam=
e package.
> * A package MUST must define a set that can be mined into one block (size=
/sigops constraint).
> * A package SHOULD not contain confirmed txs (a race may cause this).
> * A package MUST minimally satisfy peer.feerate.
> * A partial tx order, as in the manner of the block.txs ordering, MUST be=
 imposed.
> * A node SHOULD drop a peer that sends a package (or tx) below node.feera=
te.
> * A node MAY drop a peer that sends a non-minimal package according to no=
de.feerate.
>=20
> The partial ordering of block.txs introduces an ordering constraint that =
precludes full parallelism in validating input attachment. This is an imple=
mentation artifact that made its way into consensus. However in the case of=
 packaging, the set of txs is not presumed to be valid under the proof of w=
ork DoS guard. As such constraints should minimize the work/traffic require=
d to invalidate the message. The partial order constraint ensures that the =
DAG can be built incrementally, dropping the attempt (and peer as desired) =
as soon as the first orphan is discovered. As a result the network traffic =
and work required is not materially different than with tx relay, with two =
exceptions.
>=20
> These are the two central aspects of this approach (Avoiding Predictable =
Orphans and Creating Minimal Packages). These are graph search algorithms, =
some basic computer science. Minimality requires only that the package does=
 not introduce txs that are not necessary to reach the peer.feerate (as the=
se can always be packaged separately). It does not require that nodes all g=
enerate the same packages. It does not require negotiation, package identit=
y, cryptography, or hashing. As a graph search it should be O(n) where n is=
 the unconfirmed ancestry of the package, but should typically be much lowe=
r, if not a single step.
>=20
> Sufficiently-low-fee nodes will see only single txs. Moderate-fee nodes m=
ay cause partial breakup of packages. Sufficiently high fee nodes will caus=
e peers (having received and completed the acceptance of a tx/pkg with pkg.=
feerate < peer.feerate) to navigate from each tx/package external input unt=
il reaching txs above peer.feerate, or confirmed (both of which the peer is=
 presumed to already have). If the pkg.feerate is sufficiently high to conn=
ect all external inputs to the intervening txs, they are added to the packa=
ge and it is announced to the high fee peer. Note that the individual tx.fe=
erate > peer.feerate is insufficient to ensure that the peer should have th=
e tx, as there may be ancestor txs that do not, and for which the tx was in=
sufficient to cause them to be packaged. So a non-caching algorithm must be=
 able to chase each package external input to a confirmed tx (or cache the =
unconfirmed ancestry fee rate at each tx). Note that fee rates are not dire=
ctly additive, both size/
>=20
> weight and fee are required for summation (and aggregate sigops should be=
 considered).
>=20
> This makes no assumptions about current implementations. The design would=
 call for maintenance of a transaction (input->output) DAG with tx.feerate =
on each tx. This could be the unconfirmed tx graph (i.e. "memory pool") tho=
ugh it does not require maintenance of anything more than the parameters ne=
cessary to confirm a set of validated txs within a block. It is very reason=
able to require this of any participating node. A simple version negotiatio=
n can identify a package-accepting/sending nodes.
>=20
>=20
> I have thought about this for some time, but have not implemented either =
the graph search, source code, or BIP. Just wrote this off the top of my he=
ad. So I am sure there are some things I have incorrect or failed to consid=
er. But I think it's worth discussing it at this point.
>=20
> e
>=20
> > -----Original Message-----
> > From: bitcoin-dev bitcoin-dev-bounces@lists.linuxfoundation.org On
> > Behalf Of Suhas Daftuar via bitcoin-dev
> > Sent: Wednesday, June 8, 2022 8:59 AM
> > To: Bitcoin Protocol Discussion bitcoin-dev@lists.linuxfoundation.org
> > Subject: Re: [bitcoin-dev] Package Relay Proposal
> >=20
> > Hi,
> >=20
> > Thanks again for your work on this!
> >=20
> > One question I have is about potential bandwidth waste in the case of n=
odes
> > running with different policy rules. Here's my understanding of a scena=
rio I
> > think could happen:
> >=20
> > 1) Transaction A is both low-fee and non-standard to some nodes on the
> > network.
> > 2) Whenever a transaction T that spends A is relayed, new nodes will se=
nd
> > INV(PKGINFO1, T) to all package-relay peers.
> > 3) Nodes on the network that have implemented package relay, but do not
> > accept A, will send getdata(PKGINFO1, T) and learn all of T's unconfirm=
ed
> > parents (~32 bytes * number of parents(T)).
> > 4) Such nodes will reject T. But because of transaction malleability, a=
nd to
> > avoid being blinded to a transaction unnecessarily, these nodes will li=
kely still
> > send getdata(PKGINFO1, T) to every node that announces T, in case
> > someone has a transaction that includes an alternate set of parent
> > transactions that would pass policy checks.
> >=20
> > Is that understanding correct? I think a good design goal would be to n=
ot
> > waste bandwidth in non-adversarial situations. In this case, there woul=
d be
> > bandwidth waste from downloading duplicate data from all your peers, ju=
st
> > because the announcement doesn't commit to the set of parent wtxids tha=
t
> > we'd get from the peer (and so we are unable to determine that all our =
peers
> > would be telling us the same thing, just based on the announcement).
> >=20
> > Some ways to mitigate this might be to: (a) include a hash (maybe even =
just a
> > 20-byte hash -- is that enough security?) of the package wtxids (in som=
e
> > canonical ordering) along with the wtxid of the child in the initial
> > announcement; (b) limit the use of v1 packages to transactions with ver=
y few
> > parents (I don't know if this is reasonable for the use cases we have i=
n mind).
> >=20
> > Another point I wanted to bring up is about the rules around v1 package
> > validation generally, and the use of a blockhash in transaction relay
> > specifically. My first observation is that it won't always be the case =
that a v1
> > package relay node will be able to validate that a set of package trans=
actions
> > is fully sorted topologically, because there may be (non-parent) ancest=
ors
> > that are missing from the package and the best a peer can validate is
> > topology within the package -- this means that a peer can validly (unde=
r this
> > BIP) relay transaction packages out of the true topological sort (if al=
l
> > ancestors were included).
> >=20
> > This makes me wonder how useful this topological rule is. I suppose the=
re is
> > some value in preventing completely broken implementations from staying
> > connected and so there is no harm in having the rule, but perhaps it wo=
uld
> > be helpful to add that nodes SHOULD order transactions based on topolog=
ical
> > sort in the complete transaction graph, so that if missing-from-package
> > ancestors are already known by a peer (which is the expected case when
> > using v1 package relay on transactions that have more than one generati=
on
> > of unconfirmed ancestor) then the remaining transactions are already
> > properly ordered, and this is helpful even if unenforceable in general.
> >=20
> > The other observation I wanted to make was that having transaction rela=
y
> > gated on whether two nodes agree on chain tip seems like an overly
> > restrictive criteria. I think an important design principle is that we =
want to
> > minimize disruption from network splits -- if there are competing block=
s
> > found in a small window of time, it's likely that the utxo set is not m=
aterially
> > different on the two chains (assuming miners are selecting from roughly=
 the
> > same sets of transactions when this happens, which is typical). Having
> > transaction relay bifurcate on the two network halves would seem to
> > exacerbate the difference between the two sides of the split -- users o=
ught
> > to be agnostic about how benign splits are resolved and would likely wa=
nt
> > their transactions to relay across the whole network.
> >=20
> > Additionally, use of a chain tip might impose a larger burden than is n=
ecessary
> > on software that would seek to participate in transaction relay without
> > implementing headers sync/validation. I don't know what software exists=
 on
> > the network, but I imagine there are a lot of scripts out there for tra=
nsaction
> > submission to the public p2p network, and in thinking about modifying s=
uch a
> > script to utilize package relay it seems like an unnecessary added burd=
en to
> > first learn a node's tip before trying to relay a transaction.
> >=20
> > Could you explain again what the benefit of including the blockhash is?=
 It
> > seems like it is just so that a node could prioritize transaction relay=
 from
> > peers with the same chain tip to maximize the likelihood of transaction
> > acceptance, but in the common case this seems like a pretty negligible
> > concern, and in the case of a chain fork that persists for many minutes=
 it
> > seems better to me that we not partition the network into package-relay
> > regimes and just risk a little extra bandwidth in one direction or the =
other. If
> > we solve the problem I brought up at the beginning (of de-duplicating
> > package data across peers with a package-wtxid-commitment in the
> > announcement), I think this is just some wasted pkginfo bandwidth on a
> > single-link, and not across links (as we could cache validation failure=
 for a
> > package-hash to avoid re-requesting duplicate pkginfo1 messages).
> >=20
> > Best,
> > Suhas
> >=20
> > On Tue, Jun 7, 2022 at 1:57 PM Gloria Zhao via bitcoin-dev <bitcoin-
> > dev@lists.linuxfoundation.org <mailto:bitcoin-
> > dev@lists.linuxfoundation.org> > wrote:
> >=20
> > Hi Eric, aj, all,
> >=20
> > Sorry for the delayed response. @aj I'm including some paraphrased
> > points from our offline discussion (thanks).
> >=20
> > > Other idea: what if you encode the parent txs as a short hash of the
> > > wtxid (something like bip152 short ids? perhaps seeded per peer so co=
llisions
> > > will be different per peer?) and include that in the inv announcement=
?
> > > Would that work to avoid a round trip almost all of the time, while s=
till giving
> > > you enough info to save bw by deduping parents?
> >=20
> > > As I suggested earlier, a package is fundamentally a compact block
> > > (or
> > > block) announcement without the header. Compact block (BIP152)
> > > announcement
> > > is already well-defined and widely implemented...
> >=20
> > > Let us not reinvent the wheel and/or introduce accidental
> > > complexity. I see
> > > no reason why packaging is not simply BIP152 without the 'header'
> > > field, an
> > > updated protocol version, and the following sort of changes to
> > > names
> >=20
> > Interestingly, "why not use BIP 152 shortids to save bandwidth?" is
> > by far the most common suggestion I hear (including offline feedback).
> > Here's a full explanation:
> >=20
> > BIP 152 shortens transaction hashes (32 bytes) to shortids (6 bytes)
> > to save a significant amount of network bandwidth, which is extremely
> > important in block relay. However, this comes at the expense of
> > computational complexity. There is no way to directly calculate a trans=
action
> > hash from a shortid; upon receipt of a compact block, a node is expecte=
d to
> > calculate the shortids of every unconfirmed transaction it knows about =
to
> > find the matches (BIP 152: 1, Bitcoin Core: [2]). This is expensive but
> > appropriate for block relay, since the block must have a valid Proof of=
 Work
> > and new blocks only come every ~10 minutes. On the other hand, if we
> > require nodes to calculate shortids for every transaction in their memp=
ools
> > every time they receive a package, we are creating a DoS vector.
> > Unconfirmed transactions don't need PoW and, to have a live transaction
> > relay network, we should expect nodes to handle transactions at a high-=
ish
> > rate (i.e. at least 1000s of times more transactions than blocks). We c=
an't pre-
> > calculate or cache shortids for mempool transactions, since the SipHash=
 key
> > depends on the block hash and a per-connection salt.
> >=20
> > Additionally, shortid calculation is not designed to prevent intentiona=
l
> > individual collisions. If we were to use these shortids to deduplicate
> > transactions we've supposedly already seen, we may have a censorship
> > vector. Again, these tradeoffs make sense for compact block relay (see
> > shortid section in BIP 152 [3]), but not package relay.
> >=20
> > TLDR: DoSy if we calculate shortids on every package and censorship
> > vector if we use shortids for deduplication.
> >=20
> > > Given this message there is no reason
> > > to send a (potentially bogus) fee rate with every package. It can
> > > only be
> > > validated by obtaining the full set of txs, and the only recourse is
> > > dropping (etc.) the peer, as is the case with single txs.
> >=20
> > Yeah, I agree with this. Combined with the previous discussion with
> > aj (i.e. we can't accurately communicate the incentive compatibility of=
 a
> > package without sending the full graph, and this whole dance is to avoi=
d
> > downloading a few low-fee transactions in uncommon edge cases), I've
> > realized I should remove the fee + weight information from pkginfo. Yay=
 for
> > less complexity!
> >=20
> > Also, this might be pedantic, but I said something incorrect earlier
> > and would like to correct myself:
> >=20
> > > > In theory, yes, but maybe it was announced earlier (while our
> > > > node was down?) or had dropped from our mempool or similar, either =
way
> > > > we don't have those txs yet.
> >=20
> > I said "It's fine if they have Erlay, since a sender would know in
> > advance that B is missing and announce it as a package." But this isn't=
 true
> > since we're only using reconciliation in place of flooding to announce
> > transactions as they arrive, not for rebroadcast, and we're not doing f=
ull
> > mempool set reconciliation. In any case, making sure a node receives th=
e
> > transactions announced when it was offline is not something we guarante=
e,
> > not an intended use case for package relay, and not worsened by this.
> >=20
> > Thanks for your feedback!
> >=20
> > Best,
> >=20
> > Gloria
> >=20
> > 0152.mediawiki#cmpctblock
> > [2]:
> > https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.cpp#L=
49
> > [3]: https://github.com/bitcoin/bips/blob/master/bip-
> > 0152.mediawiki#short-transaction-id-calculation
> >=20
> > On Thu, May 26, 2022 at 3:59 AM <eric@voskuil.org
> > mailto:eric@voskuil.org > wrote:
> >=20
> > Given that packages have no header, the package requires
> > identity in a
> > BIP152 scheme. For example 'header' and 'blockhash' fields
> > can be replaced
> > with a Merkle root (e.g. "identity" field) for the package,
> > uniquely
> > identifying the partially-ordered set of txs. And use of
> > 'getdata' (to
> > obtain a package by hash) can be eliminated (not a use case).
> >=20
> > e
> >=20
> > > -----Original Message-----
> > > From: eric@voskuil.org mailto:eric@voskuil.org
> > <eric@voskuil.org mailto:eric@voskuil.org >
> > > Sent: Wednesday, May 25, 2022 1:52 PM
> > > To: 'Anthony Towns' <aj@erisian.com.au
> > mailto:aj@erisian.com.au >; 'Bitcoin Protocol Discussion'
> > > <bitcoin-dev@lists.linuxfoundation.org <mailto:bitcoin-
> > dev@lists.linuxfoundation.org> >; 'Gloria Zhao'
> > > <gloriajzhao@gmail.com mailto:gloriajzhao@gmail.com >
> > > Subject: RE: [bitcoin-dev] Package Relay Proposal
> > >
> > > > From: bitcoin-dev <bitcoin-dev-
> > bounces@lists.linuxfoundation.org <mailto:bitcoin-dev-
> > bounces@lists.linuxfoundation.org> > On
> > > Behalf
> > > > Of Anthony Towns via bitcoin-dev
> > > > Sent: Wednesday, May 25, 2022 11:56 AM
> > >
> > > > So the other thing is what happens if the peer
> > announcing packages to us
> > > is
> > > > dishonest?
> > > >
> > > > They announce pkg X, say X has parents A B C and the fee
> > rate is
> > garbage.
> > > But
> > > > actually X has parent D and the fee rate is excellent. Do
> > we request the
> > > > package from another peer, or every peer, to double
> > check? Otherwise
> > > we're
> > > > allowing the first peer we ask about a package to censor
> > that tx from
> > us?
> > > >
> > > > I think the fix for that is just to provide the fee and weight
> > when
> > > announcing
> > > > the package rather than only being asked for its info?
> > Then if one peer
> > > makes
> > > > it sound like a good deal you ask for the parent txids from
> > them,
> > dedupe,
> > > > request, and verify they were honest about the parents.
> > >
> > > Single tx broadcasts do not carry an advertised fee rate,
> > however the'
> > > feefilter' message (BIP133) provides this distinction. This
> > should be
> > > interpreted as applicable to packages. Given this message
> > there is no
> > reason
> > > to send a (potentially bogus) fee rate with every package. It
> > can only be
> > > validated by obtaining the full set of txs, and the only
> > recourse is
> > > dropping (etc.) the peer, as is the case with single txs.
> > Relying on the
> > > existing message is simpler, more consistent, and more
> > efficient.
> > >
> > > > >> Is it plausible to add the graph in?
> > > >
> > > > Likewise, I think you'd have to have the graph info from
> > many nodes if
> > > you're
> > > > going to make decisions based on it and don't want
> > hostile peers to be
> > > able to
> > > > trick you into ignoring txs.
> > > >
> > > > Other idea: what if you encode the parent txs as a short
> > hash of the
> > wtxid
> > > > (something like bip152 short ids? perhaps seeded per
> > peer so collisions
> > > will
> > > > be different per peer?) and include that in the inv
> > announcement? Would
> > > > that work to avoid a round trip almost all of the time,
> > while still
> > giving
> > > you
> > > > enough info to save bw by deduping parents?
> > >
> > > As I suggested earlier, a package is fundamentally a
> > compact block (or
> > > block) announcement without the header. Compact block
> > (BIP152)
> > > announcement
> > > is already well-defined and widely implemented. A node
> > should never be
> > > required to retain an orphan, and BIP152 ensures this is not
> > required.
> > >
> > > Once a validated set of txs within the package has been
> > obtained with
> > > sufficient fee, a fee-optimal node would accept the largest
> > subgraph of
> > the
> > > package that conforms to fee constraints and drop any
> > peer that provides a
> > > package for which the full graph does not.
> > >
> > > Let us not reinvent the wheel and/or introduce accidental
> > complexity. I
> > see
> > > no reason why packaging is not simply BIP152 without the
> > 'header' field,
> > an
> > > updated protocol version, and the following sort of changes
> > to names:
> > >
> > > sendpkg
> > > MSG_CMPCT_PKG
> > > cmpctpkg
> > > getpkgtxn
> > > pkgtxn
> > >
> > > > > For a maximum 25 transactions,
> > > > >2324/2 =3D 276, seems like 36 bytes for a child-with-
> > parents package.
> > > >
> > > > If you're doing short ids that's maybe 254B=3D100B
> > already, then the
> > above
> > > is
> > > > up to 36% overhead, I guess. Might be worth thinking
> > more about, but
> > > maybe
> > > > more interesting with ancestors than just parents.
> > > >
> > > > >Also side note, since there are no size/count params,
> > >
> > > Size is restricted in the same manner as block and
> > transaction broadcasts,
> > > by consensus. If the fee rate is sufficient there would be no
> > reason to
> > > preclude any valid size up to what can be mined in one
> > block (packaging
> > > across blocks is not economically rational under the
> > assumption that one
> > > miner cannot expect to mine multiple blocks in a row).
> > Count is
> > incorporated
> > > into BIP152 as 'shortids_length'.
> > >
> > > > > wondering if we
> > > > >should just have "version" in "sendpackages" be a bit
> > field instead of
> > > > >sending a message for each version. 32 versions should
> > be enough right?
> > >
> > > Adding versioning to individual protocols is just a reflection
> > of the
> > > insufficiency of the initial protocol versioning design, and
> > that of the
> > > various ad-hoc changes to it (including yet another
> > approach in this
> > > proposal) that have been introduced to compensate for it,
> > though I'll
> > > address this in an independent post at some point.
> > >
> > > Best,
> > > e
> > >
> > > > Maybe but a couple of messages per connection doesn't
> > really seem worth
> > > > arguing about?
> > > >
> > > > Cheers,
> > > > aj
> > > >
> > > >
> > > > --
> > > > Sent from my phone.
> > > >
> > _______________________________________________
> > > > bitcoin-dev mailing list
> > > > bitcoin-dev@lists.linuxfoundation.org <mailto:bitcoin-
> > dev@lists.linuxfoundation.org>
> > > >
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >=20
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists.linuxfoundation.org <mailto:bitcoin-
> > dev@lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>=20
>=20
>=20
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev