path: root/1e/4856ef65a64b6db6be542d3e0aca616a61057a

Return-Path: <james.obeirne@gmail.com>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
 by lists.linuxfoundation.org (Postfix) with ESMTP id CA54AC000B
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Feb 2022 19:40:37 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp2.osuosl.org (Postfix) with ESMTP id 9DB8D40156
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Feb 2022 19:40:37 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level: 
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp2.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp2.osuosl.org ([127.0.0.1])
 by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id oEOc2Zx5AltH
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Feb 2022 19:40:35 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com
 [IPv6:2607:f8b0:4864:20::b36])
 by smtp2.osuosl.org (Postfix) with ESMTPS id 417F64038E
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Feb 2022 19:40:35 +0000 (UTC)
Received: by mail-yb1-xb36.google.com with SMTP id p5so18405954ybd.13
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Feb 2022 11:40:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:from:date:message-id:subject:to;
 bh=e6+lnZPDz42qX7SLF422G/edsa6da8QGahgTF0fSjkE=;
 b=JMRJjDdcfATDD6rpQ01kMmwNlFe//PsNZN8XYfqWbOCyPUKgjVVpLo0l2CWrzu+lRt
 uIJbmVz2sTE4UwscpjS7H/PY5D6vtJAmAYTJvG4FyyVle9x5sECbAPQ02ck3bBsorHQ5
 HK8Qy/IwxEB72X/3pielyDxVwtcg/rj7RN9AQ3frEKtYOZrajdX9yeBmTJfGgotxi9WE
 WSdk05tDo+QHdIwRy83/o9ZcEksz6FQ9p+bNMReP2LO7AxkbnIh+oMLA0x4KemfBz3R3
 1GFns61MpeHA9hSNCjcFO7dlzm8dCZvecbh1tr+kXfeGcaiIVsT1NAN8n4khCp6GLJNw
 N4ag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=e6+lnZPDz42qX7SLF422G/edsa6da8QGahgTF0fSjkE=;
 b=Sjl505txm04tkFjs8B4U50/a6acQvUlPpTl7HdEdKvLhYUZpbbtV7xetMdrtna1gIC
 O8VkSi9rx8a0DshMfybEDnsBxnW0IR5zt+XC+JHpe6LlZLxIdrrLGukNSsv7I+OBswDe
 aTfUtZJOCjoH8UBfZ/hYsmxCOgFSfPesUGDYFAvgpLnk26ouGBSUX++eV+K54ndJDJic
 ZbDjwx1AyZBbZqDj0YmoBlFc1eTNlW9Da0qb5Wo0fFWKSotFPP+LrPWTVOyHC7KZYbLH
 HdEXWlSEOQgBWb2noWrGTkd2GANXdEgKeODuy4Wz3vWUemFN299Agl4va8bKVNX0n1gK
 WcEQ==
X-Gm-Message-State: AOAM531E6/OwgEQb+R+sOoxLRkMMj5vN5ULbUb9iS9NB0EDlC7N3UUqI
 tQGVBhkYe/+GDhT4jNknACPFbgvVTfbViSVqDPDOe0LjBUA=
X-Google-Smtp-Source: ABdhPJz4WvvMXxGqqrsKAo08yjfntEnHOgurSxyxIiECDlc2YTBuhbG5AZKbqwOzi4t/o+9u42fCwKbxsBbQ0YWD7fU=
X-Received: by 2002:a81:4528:: with SMTP id s40mr9192272ywa.188.1644522033643; 
 Thu, 10 Feb 2022 11:40:33 -0800 (PST)
MIME-Version: 1.0
From: "James O'Beirne" <james.obeirne@gmail.com>
Date: Thu, 10 Feb 2022 14:40:22 -0500
Message-ID: <CAPfvXfKrnju1fzxOKs3Fx00NOPWHjedF7e4xMSGs8buwc0O2kw@mail.gmail.com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000a402c005d7af1fb0"
X-Mailman-Approved-At: Thu, 10 Feb 2022 19:48:22 +0000
Subject: [bitcoin-dev] Thoughts on fee bumping
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 10 Feb 2022 19:40:37 -0000

--000000000000a402c005d7af1fb0
Content-Type: text/plain; charset="UTF-8"

There's been much talk about fee-bumping lately, and for good reason -
dynamic fee management is going to be a central part of bitcoin use as
the mempool fills up (lord willing) and right now fee-bumping is
fraught with difficulty and pinning peril.

Gloria's recent post on the topic[0] was very lucid and highlights a
lot of the current issues, as well as some proposals to improve the
situation.

As others have noted, the post was great. But throughout the course
of reading it and the ensuing discussion, I became troubled by the
increasing complexity of both the status quo and some of the
proposed remedies.

Layering on special cases, more carve-outs, and X and Y percentage
thresholds is going to make reasoning about the mempool harder than it
already is. Special consideration for "what should be in the next
block" and/or the caching of block templates seems like an imposing
dependency, dragging in a bunch of state and infrastructure to a
question that should be solely limited to mempool feerate aggregates
and the feerate of the particular txn package a wallet is concerned
with.

This is bad enough for protocol designers and Core developers, but
making the situation any more intractable for "end-users" and wallet
developers feels wrong.

I thought it might be useful to step back and reframe. Here are a few
aims that are motivated chiefly by the quality of end-user experience,
constrained to obey incentive compatibility (i.e. miner reward, DoS
avoidance). Forgive the abstract dalliance for a moment; I'll talk
through concretes afterwards.


# Purely additive feerate bumps should never be impossible

Any user should always be able to add to the incentive to mine any
transaction in a purely additive way. The countervailing force here
ends up being spam prevention (a la min-relay-fee) to prevent someone
from consuming bandwidth and mempool space with a long series of
infinitesimal fee-bumps.

A fee bump, naturally, should be given the same per-byte consideration
as a normal Bitcoin transaction in terms of relay and block space,
although it would be nice to come up with a more succinct
representation. This leads to another design principle:


# The bandwidth and chain space consumed by a fee-bump should be minimal

Instead of prompting a rebroadcast of the original transaction for
replacement, which contains a lot of data not new to the network, it
makes more sense to broadcast the "diff" which is the additive
contribution towards some txn's feerate.

This dovetails with the idea that...


# Special transaction structure should not be required to bump fees

In an ideal design, special structural foresight would not be needed
in order for a txn's feerate to be improved after broadcast.

Anchor outputs specified solely for CPFP, which amount to many bytes of
wasted chainspace, are a hack. It's probably uncontroversial at this
point to say that even RBF itself is kind of a hack - a special
sequence number should not be necessary for post-broadcast contribution
toward feerate. Not to mention RBF's seemingly wasteful consumption of
bandwidth due to the rebroadcast of data the network has already seen.

In a sane design, no structural foresight - and certainly no wasted
bytes in the form of unused anchor outputs - should be needed in order
to add to a miner's reward for confirming a given transaction.

Planning for fee-bumps explicitly in transaction structure also often
winds up locking in which keys are required to bump fees, at odds
with the idea that...


# Feerate bumps should be able to come from anywhere

One of the practical downsides of CPFP that I haven't seen discussed in
this conversation is that it requires the transaction to pre-specify the
keys needed to sign for fee bumps. This is problematic if you're, for
example, using a vault structure that makes use of pre-signed
transactions.

What if the key you specified n the anchor outputs for a bunch of
pre-signed txns is compromised? What if you'd like to be able to
dynamically select the wallet that bumps fees? CPFP does you no favors
here.

There is of course a tension between allowing fee bumps to come from
anywhere and the threat of pinning-like attacks. So we should venture
to remove pinning as a possibility, in line with the first design
principle I discuss.


---

Coming down to earth, the "tabula rasa" thought experiment above has led
me to favor an approach like the transaction sponsors design that Jeremy
proposed in a prior discussion back in 2020[1].

Transaction sponsors allow feerates to be bumped after a transaction's
broadcast, regardless of the structure of the original transaction.
No rebroadcast (wasted bandwidth) is required for the original txn data.
No wasted chainspace on only-maybe-used prophylactic anchor outputs.

The interface for end-users is very straightforward: if you want to bump
fees, specify a transaction that contributes incrementally to package
feerate for some txid. Simple.

In the original discussion, there were a few main objections that I noted:

1. In Jeremy's original proposal, only one sponsor txn per txid is
   allowed by policy. A malicious actor could execute a pinning-like
   attack by specifying an only-slightly-helpful feerate sponsor that
   then precludes other larger bumps.

I think there are some ways around this shortcoming. For example: what
if, by policy, sponsor txns had additional constraints that

  - each input must be signed {SIGHASH_SINGLE,SIGHASH_NONE}|ANYONECANPAY,
  - the txn must be specified RBFable,
  - a replacement for the sponsor txn must raise the sponsor feerate,
    including ancestors (maybe this is inherent in "is RBFable," but
    I don't want to conflate absolute feerates into this).

That way, there is still at most a single sponsor txn per txid in the
mempool, but anyone can "mix in" inputs which bump the effective
feerate of the sponsor.

This may not be the exact solution we want, but I think it demonstrates
that the sponsors design has some flexibility and merits some thinking.

The second objection about sponsors was

2. (from Suhas) sponsors break the classic invariant: "once a valid
   transaction is created, it should not become invalid later on unless
   the inputs are double-spent."

This doesn't seem like a huge concern to me if you consider the txid
being sponsored as a sort of spiritual input to the sponsor. While the
theoretical objection against broadening where one has to look in a txn
to determine its dependencies is understandable, I don't see what the
practical cost here is.

Reorg complexity seems comparable if not identical, especially if we
broaden sponsor rules to allow blocks to contain sponsor txns that are
both for txids in the same block _or_ already included in the chain.

This theoretical concession seems preferable to heaping more rules onto
an already labyrinthine mempool policy that is difficult for both
implementers and users to reason about practically and conceptually.

A third objection that wasn't posed, IIRC, but almost certainly would
be:

3. Transaction sponsors requires a soft-fork.

Soft-forks are no fun, but I'll tell you what also isn't fun: being on
the hook to model (and sometimes implement) a dizzying potpourri of
mempool policies and special-cases. Expecting wallet implementers to
abide by a maze of rules faithfully in order to ensure txn broadcast and
fee management invites bugs for perpetuity and network behavior that is
difficult to reason about a priori. Use of CPFP in the long-term also
risks needless chain waste.

If a soft-fork is the cost of cleaning up this essential process,
consideration should be given to paying it as a one-time cost. This
topic merits a separate post, but consider that in the 5 years leading
up to the 2017 SegWit drama, we averaged about a soft-fork a year.
Uncontroversial, "safe" changes to the consensus protocol shouldn't be
out of the question when significant practical benefit is plain to see.

---

I hope this message has added some framing to the discussion on fees,
as well prompting other participants to go back and give the
transaction sponsor proposal a serious look. The sponsors interface is
about the simplest I can imagine for wallets, and it seems easy to
reason about for implementers on Core and elsewhere.

I'm not out to propose soft-forks lightly, but the current complexity
in fee management feels untenable, and as evidenced by all the
discussion lately, fees are an increasingly crucial part of the system.



[0]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
[1]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168.html

--000000000000a402c005d7af1fb0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>There&#39;s been much talk about fee-bumping lately, =
and for good reason -<br>dynamic fee management is going to be a central pa=
rt of bitcoin use as<br>the mempool fills up (lord willing) and right now f=
ee-bumping is<br>fraught with difficulty and pinning peril.<br><br>Gloria&#=
39;s recent post on the topic[0] was very lucid and highlights a<br>lot of =
the current issues, as well as some proposals to improve the<br>situation.<=
br><br>As others have noted, the post was great. But throughout the course<=
br>of reading it and the ensuing discussion, I became troubled by the<br>in=
creasing complexity of both the status quo and some of the <br></div><div>p=
roposed remedies. <br></div><div><br>Layering on special cases, more carve-=
outs, and X and Y percentage<br>thresholds is going to make reasoning about=
 the mempool harder than it<br>already is. Special consideration for &quot;=
what should be in the next<br>block&quot; and/or the caching of block templ=
ates seems like an imposing<br>dependency, dragging in a bunch of state and=
 infrastructure to a<br>question that should be solely limited to mempool f=
eerate aggregates<br>and the feerate of the particular txn package a wallet=
 is concerned<br>with. <br><br>This is bad enough for protocol designers an=
d Core developers, but<br>making the situation any more intractable for &qu=
ot;end-users&quot; and wallet<br>developers feels wrong.<br><br>I thought i=
t might be useful to step back and reframe. Here are a few<br>aims that are=
 motivated chiefly by the quality of end-user experience,<br>constrained to=
 obey incentive compatibility (i.e. miner reward, DoS<br>avoidance). Forgiv=
e the abstract dalliance for a moment; I&#39;ll talk<br>through concretes a=
fterwards.<br><br><br># Purely additive feerate bumps should never be impos=
sible<br><br>Any user should always be able to add to the incentive to mine=
 any<br>transaction in a purely additive way. The countervailing force here=
<br>ends up being spam prevention (a la min-relay-fee) to prevent someone<b=
r>from consuming bandwidth and mempool space with a long series of<br>infin=
itesimal fee-bumps. <br><br>A fee bump, naturally, should be given the same=
 per-byte consideration<br>as a normal Bitcoin transaction in terms of rela=
y and block space,<br>although it would be nice to come up with a more succ=
inct<br>representation. This leads to another design principle:<br><br><br>=
# The bandwidth and chain space consumed by a fee-bump should be minimal<br=
><br>Instead of prompting a rebroadcast of the original transaction for<br>=
replacement, which contains a lot of data not new to the network, it<br>mak=
es more sense to broadcast the &quot;diff&quot; which is the additive<br>co=
ntribution towards some txn&#39;s feerate. <br><br>This dovetails with the =
idea that...<br><br><br># Special transaction structure should not be requi=
red to bump fees<br><br>In an ideal design, special structural foresight wo=
uld not be needed <br>in order for a txn&#39;s feerate to be improved after=
 broadcast.<br><br>Anchor outputs specified solely for CPFP, which amount t=
o many bytes of<br>wasted chainspace, are a hack. It&#39;s probably uncontr=
oversial at this<br>point to say that even RBF itself is kind of a hack - a=
 special<br>sequence number should not be necessary for post-broadcast cont=
ribution<br>toward feerate. Not to mention RBF&#39;s seemingly wasteful con=
sumption of<br>bandwidth due to the rebroadcast of data the network has alr=
eady seen.<br><br>In a sane design, no structural foresight - and certainly=
 no wasted<br>bytes in the form of unused anchor outputs - should be needed=
 in order<br>to add to a miner&#39;s reward for confirming a given transact=
ion.<br><br>Planning for fee-bumps explicitly in transaction structure also=
 often<br>winds up locking in which keys are required to bump fees, at odds=
<br>with the idea that...<br><br><br># Feerate bumps should be able to come=
 from anywhere<br><br>One of the practical downsides of CPFP that I haven&#=
39;t seen discussed in<br>this conversation is that it requires the transac=
tion to pre-specify the<br>keys needed to sign for fee bumps. This is probl=
ematic if you&#39;re, for<br>example, using a vault structure that makes us=
e of pre-signed<br>transactions. <br><br>What if the key you specified n th=
e anchor outputs for a bunch of<br>pre-signed txns is compromised? What if =
you&#39;d like to be able to<br>dynamically select the wallet that bumps fe=
es? CPFP does you no favors<br>here.<br><br>There is of course a tension be=
tween allowing fee bumps to come from<br>anywhere and the threat of pinning=
-like attacks. So we should venture<br>to remove pinning as a possibility, =
in line with the first design<br>principle I discuss.<br><br><br>---<br><br=
>Coming down to earth, the &quot;tabula rasa&quot; thought experiment above=
 has led<br>me to favor an approach like the transaction sponsors design th=
at Jeremy<br>proposed in a prior discussion back in 2020[1].<br><br>Transac=
tion sponsors allow feerates to be bumped after a transaction&#39;s<br>broa=
dcast, regardless of the structure of the original transaction.<br>No rebro=
adcast (wasted bandwidth) is required for the original txn data.<br>No wast=
ed chainspace on only-maybe-used prophylactic anchor outputs. <br><br>The i=
nterface for end-users is very straightforward: if you want to bump<br>fees=
, specify a transaction that contributes incrementally to package<br>feerat=
e for some txid. Simple.<br><br>In the original discussion, there were a fe=
w main objections that I noted:<br><br>1. In Jeremy&#39;s original proposal=
, only one sponsor txn per txid is<br>=C2=A0 =C2=A0allowed by policy. A mal=
icious actor could execute a pinning-like <br>=C2=A0 =C2=A0attack by specif=
ying an only-slightly-helpful feerate sponsor that <br>=C2=A0 =C2=A0then pr=
ecludes other larger bumps.<br><br>I think there are some ways around this =
shortcoming. For example: what<br>if, by policy, sponsor txns had additiona=
l constraints that <br><br>=C2=A0 - each input must be signed {SIGHASH_SING=
LE,SIGHASH_NONE}|ANYONECANPAY,<br>=C2=A0 - the txn must be specified RBFabl=
e,<br>=C2=A0 - a replacement for the sponsor txn must raise the sponsor fee=
rate, <br>=C2=A0 =C2=A0 including ancestors (maybe this is inherent in &quo=
t;is RBFable,&quot; but <br>=C2=A0 =C2=A0 I don&#39;t want to conflate abso=
lute feerates into this).<br><br>That way, there is still at most a single =
sponsor txn per txid in the<br>mempool, but anyone can &quot;mix in&quot; i=
nputs which bump the effective<br>feerate of the sponsor.<br><br>This may n=
ot be the exact solution we want, but I think it demonstrates<br>that the s=
ponsors design has some flexibility and merits some thinking.<br><br>The se=
cond objection about sponsors was<br><br>2. (from Suhas) sponsors break the=
 classic invariant: &quot;once a valid<br>=C2=A0 =C2=A0transaction is creat=
ed, it should not become invalid later on unless <br>=C2=A0 =C2=A0the input=
s are double-spent.&quot;<br><br>This doesn&#39;t seem like a huge concern =
to me if you consider the txid<br>being sponsored as a sort of spiritual in=
put to the sponsor. While the<br>theoretical objection against broadening w=
here one has to look in a txn<br>to determine its dependencies is understan=
dable, I don&#39;t see what the<br>practical cost here is. <br><br>Reorg co=
mplexity seems comparable if not identical, especially if we<br>broaden spo=
nsor rules to allow blocks to contain sponsor txns that are<br>both for txi=
ds in the same block _or_ already included in the chain.<br><br>This theore=
tical concession seems preferable to heaping more rules onto<br>an already =
labyrinthine mempool policy that is difficult for both<br>implementers and =
users to reason about practically and conceptually.<br><br>A third objectio=
n that wasn&#39;t posed, IIRC, but almost certainly would<br>be:<br><br>3. =
Transaction sponsors requires a soft-fork.<br><br>Soft-forks are no fun, bu=
t I&#39;ll tell you what also isn&#39;t fun: being on<br>the hook to model =
(and sometimes implement) a dizzying potpourri of<br>mempool policies and s=
pecial-cases. Expecting wallet implementers to<br>abide by a maze of rules =
faithfully in order to ensure txn broadcast and<br>fee management invites b=
ugs for perpetuity and network behavior that is<br>difficult to reason abou=
t a priori. Use of CPFP in the long-term also<br>risks needless chain waste=
.<br><br>If a soft-fork is the cost of cleaning up this essential process,<=
br>consideration should be given to paying it as a one-time cost. This<br>t=
opic merits a separate post, but consider that in the 5 years leading<br>up=
 to the 2017 SegWit drama, we averaged about a soft-fork a year.<br>Uncontr=
oversial, &quot;safe&quot; changes to the consensus protocol shouldn&#39;t =
be<br>out of the question when significant practical benefit is plain to se=
e.<br><br>---<br><br>I hope this message has added some framing to the disc=
ussion on fees,<br>as well prompting other participants to go back and give=
 the<br>transaction sponsor proposal a serious look. The sponsors interface=
 is<br>about the simplest I can imagine for wallets, and it seems easy to<b=
r>reason about for implementers on Core and elsewhere. =C2=A0 =C2=A0 =C2=A0=
 =C2=A0<br><br>I&#39;m not out to propose soft-forks lightly, but the curre=
nt complexity<br>in fee management feels untenable, and as evidenced by all=
 the<br>discussion lately, fees are an increasingly crucial part of the sys=
tem. <br><br><br><br>[0]: <a href=3D"https://lists.linuxfoundation.org/pipe=
rmail/bitcoin-dev/2022-January/019817.html">https://lists.linuxfoundation.o=
rg/pipermail/bitcoin-dev/2022-January/019817.html</a><br>[1]: <a href=3D"ht=
tps://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168=
.html">https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Septemb=
er/018168.html</a><br></div></div>

--000000000000a402c005d7af1fb0--