Return-Path: Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 12C9DC0011 for ; Wed, 9 Feb 2022 17:59:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A11AB827A9 for ; Wed, 9 Feb 2022 17:58:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rsnHqEn1LBTe for ; Wed, 9 Feb 2022 17:58:19 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [IPv6:2a00:1450:4864:20::129]) by smtp1.osuosl.org (Postfix) with ESMTPS id C731E82542 for ; Wed, 9 Feb 2022 17:58:18 +0000 (UTC) Received: by mail-lf1-x129.google.com with SMTP id i17so5681851lfg.11 for ; Wed, 09 Feb 2022 09:58:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=eSJmreFU3jUon8Fdsjw1Ftcp++TFdghneOFhloS6Hvw=; b=OXM2mU85t08PvDn+QFafEBA6JEUzmwIKy44DXvsdrWhQXP5IiwhcWEvrAdOBy77QIR bz0ah8RNEG9KPs9tjApFqFPW+yVWSdLEP7jSNvKXdf6ZyNWtQDUTWsjX74qxk541WXxr QjvG2SvH592B9p/esk2EvBavSdmFyO7FeHy+5hbL1c/L7H8RLJKqpfBtvJerX/F5XFqc osXNBJnTXOyDnm+aYLqQLE2QiDYZS4AgR2XTQbosX0kAAth86MBgnqbrWSrlOpoJe9mU 5v5/ep+nRcW7hn5ZrgySwyBNf5vzfO93KgJRJAucDKTWJ+te9ohMlEi8HZLf5HIzX0yk NdZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=eSJmreFU3jUon8Fdsjw1Ftcp++TFdghneOFhloS6Hvw=; b=fnIdhJz3nT2EFNoK95M5JdBN2+T6ZC7ifkEPqiXBWtc97JByIAS3BRa68rt56aESqS 5oVCi4wcfkIF5CbLlIb++vqb1CwWxX4ENTQeesXesKiVQP803j6s3k/qV8sbIdweNZEa 20WvhRe+0l5lP9U4TfrpxMSWOdj1vBtbTzXoX0o1AmSpXJkVTPJW57V6Lmp9dN8vnpUV zPpSwkk5CjKdxyF5wNLsNQjEer6t4Kh2lpnUWisQYeLy4cQxZBKz4/jIvEpWBpLNjAkG tcxqOrFVJ6Q/U2JnnADbRUHq5RvFSvIrpqOhZQdHOTkZ06HIuznV8E5X1k+ow2BMZy4e C8qw== X-Gm-Message-State: AOAM5309DfPcVS6ipBPIZXlqNejUkxWrAlHrWhfTTj48FdJO5/xSCyg4 4qST9fP74TBqp14RKJUQDB/zDl7I5jrNdbinqkWs6pbeuso= X-Google-Smtp-Source: ABdhPJxqd1mrPAQ70YOkvzw3l/Mhd24eFH6BoM0EvnmjQp5T4/6SRt9iOaN7ifCYdZ9LeLqIKwM3erXE86K0DtohSR8= X-Received: by 2002:a05:6512:12d3:: with SMTP id p19mr1934796lfg.72.1644429496139; Wed, 09 Feb 2022 09:58:16 -0800 (PST) MIME-Version: 1.0 From: lisa neigut Date: Wed, 9 Feb 2022 11:57:59 -0600 Message-ID: To: bitcoin-dev@lists.linuxfoundation.org Content-Type: multipart/alternative; boundary="000000000000f9c1c105d799930c" X-Mailman-Approved-At: Wed, 09 Feb 2022 18:14:56 +0000 Subject: Re: [bitcoin-dev] Improving RBF Policy X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Feb 2022 17:59:59 -0000 --000000000000f9c1c105d799930c Content-Type: text/plain; charset="UTF-8" Changing the way that RBF works is an excellent idea. Bitcoin is overdue for revisiting these rules. Thanks to @glowzo et al for kicking off the discussion. I've been thinking about RBF for a very long time[1], and it's been fun to see other people's thoughts on the topics. Here's my current thinking about this, and proposal for how we should update the rules. ### Changing the Rules? How to Change Them Regarding how to change them, Bram and aj are right -- we should move to a model where transaction relay candidates are evaluated on their net increase in fees per byte paid, and remove the requirement that the gross fee of the preceding transaction is met or exceeded. Our current ruleset is over complicated because it attempts to solve two problems at once: the miner's best interests (highest possible fee take) and relay policy. I believe this is a mistake and the mempool should change its goals. Instead, the mempool relay design for RBFs should be built around 1) increasing the the per-byte fees paid of a transaction and 2) providing a simple policy for applications building on top of bitcoin, such that knowledge of the mempool is not required for successfully issuing relay-able RBF transactions. (A simple "must increase the per-byte feerate" heuristic for RBF relay candidates has the nice benefit of being easy to program to on the application side, and only requires knowledge of the previous candidate transaction, not the entire mempool or any previous tx's relative position within it.) Finally, when blockspace is competitive , this simple policy ensures that the per-byte value of every subsequent relayed transaction increases the per-byte value of pending bytes for the next block. This provides a measure of DoS protection and ensures that every relayed byte is more valuable (to the miner/network) than the last. *The only time that RBF is critical for relay is during full block periods -- if there's not enough transactions to fill a block, using RBF to ensure that a transaction is mined in a timely manner is moot. As such, RBF rules should not be concerned with low-block environments. ### Mempools and Relay The mempool currently serves two masters: the profit motive of the miner and the relay motive of a utxo holder. It is in the interest of a user to send the miner a high per-byte tx, such that it might end up in the next block. It is in the miner's best interest to include the highest per-byte set of transactions in their block template. There is some conflation here in the current RBF policies between what is in the mempool and what is considered a candidate for the block template. If a miner has already included a more profitable package of txs into their block template than a more valuable per-byte tx that the network has relayed to them, it should be the responsibility of the block template constructor to reject the new proposed tx, not the nodes relaying the transaction to said miner. This is a policy that the miner can (and should) implement at the level of the template construction, however. Is it the responsibility of the mempool to provide the best "historical" block opportunity for a miner (e.g. the highest paying block given all txs it's ever seen)? I would say no, that the ability of a utxo owner to re-state the spend condition of a pending transaction is more important, from a use-case perspective, and that the mempool should concern itself solely with relaying increasingly more profitable bytes to miners. Let the miners concern themselves with deciding what the best policy for their own block construction is, and the mempool with relaying the highest value bytes for the network. Net-net, this will benefit everyone as it becomes easier for users to re-submit txs with increasingly greater feerates, creating more active competition for available blockspace as more applications are able to include it as a feature (and it works, reliable, as a relay mechanism). ### Packages and RBF Packages make the increasing per-byte rule less of a guarantee that increasing the per-byte value of a single transaction will net a given miner more fees than including the entire package. Let's decompose this a bit. It's helpful to think of tx packages as 'composable txs'. Basically when you consider a 'package' it is actually a large tx with sub-components, the individual txs. As a 'composed tx', you can calculate the per-byte feerate of the entire set. This is the number that you, as someone issuing an RBF, would need to beat in order to move your tx up in the pending block queue. RBF, however, is a transaction level policy: it allows you to replace any *one* component of a package, or tree, with the side effect of possibly invalidating other candidate txs. If the 'composed tx' (aka package) had a net per-byte value that was higher than the new replacement transaction because of a leaf tx that had an outsized per-byte feerate, then it would be more profitable for the miner to have mined the entire package rather than the replacement. This edge case complicates the picture for the miner. Considered from the viewpoint of the user issuing the RBF, however, it is far simpler. In the ideal case, a person is issuing an RBF because the previous tx tree, even with its high fee sub-component, was not a candidate for the next block. And, in some cases, increasing the sub-component's per-byte feerate will not achieve the goal of moving the tx any closer to being mined. It's only by increasing the feerate above the present feerate of the candidate plus desendents (tx package) that the transaction will advance in the queue. While not uncomplicated, this is a simple enough metric for a wallet to track, and does not require any knowledge of the current mempool to effectively surpass. It's the wallet's responsibility to track this though; failure to take descendants into account when deciding on the next per-byte feerate for an RBF *will* mean that your RBF will be ineffective at achieving the goal of getting your UTXO spent. Any wallet is incentivized to always provide a higher per-byte feerate than the 'composed tx' (tx and its descendants), so as to ensure an actual improvement in the unconfirmed transaction's position in the block queue, so to speak. Note that the v2 protocol for channel opens in lightning uses an RBF negotiation that adheres basically to these rules (ea transaction must have a strictly greater per-byte feerate). We enforce a rate of 65/64th as the required increase in feerate for each subsequent channel open transaction. https://github.com/lightning/bolts/pull/851/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR1154 ### RBF and DoS Admittedly, changing these rules will increase the number of times that any UTXO is eligible to be retransmitted (relayed) via the bitcoin node network. Strictly increasing the per-byte feerate however ensures that this re-relay is increasingly more expensive to the UTXO owner, however. ### in exitus These are the things I've been thinking about with regards to RBF. I hope they can help to highlight the challenges in the RBF design space a bit more clearly, as well as spelling out the case for using a simple heuristic such as "solely increasing per-byte feerate" as a good candidate for the revised RBF policy. ~niftynei [1] https://basicbitch.software/posts/2018-12-27-Explaining-Replace-By-Fee.html --000000000000f9c1c105d799930c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Changing the way that RBF works is an excellent idea. Bitcoin is o= verdue for revisiting these rules. Thanks to @glowzo et al for kicking off = the discussion.

I've been thinking about RBF = for a very long time[1], and it's been fun to see other people's th= oughts on the topics. Here's my current thinking about this, and propos= al for how we should update the rules.

### Changing the Rules? How to Change Them<= br style=3D"color:rgb(32,33,36);font-family:Roboto,Arial,sans-serif;font-va= riant-ligatures:none;letter-spacing:0.1px;white-space:pre-wrap">Regarding how to ch= ange them, Bram and aj are right -- we should move to a model where transac= tion relay candidates are evaluated on their net increase in fees per byte = paid, and remove the requirement that the gross fee of the preceding transa= ction is met or exceeded.

Our current ruleset is over complicated because it attempts to solve tw= o problems at once: the miner's best interests (highest possible fee ta= ke) and relay policy.

= I believe this is a mistake and the mempool should change its goals. Instea= d, the mempool relay design for RBFs should be built around 1) increasing t= he the per-byte fees paid of a transaction and 2) providing a simple policy= for applications building on top of bitcoin, such that knowledge of the me= mpool is not required for successfully issuing relay-able RBF transactions.=
(A simple "mus= t increase the per-byte feerate" heuristic for RBF relay candidates ha= s the nice benefit of being easy to program to on the application side, and= only requires knowledge of the previous candidate transaction, not the ent= ire mempool or any previous tx's relative position within it.)
Finally, when blockspace is co= mpetitive , this simple policy ensures that the per-byte value of every sub= sequent relayed transaction increases the per-byte value of pending bytes f= or the next block. This provides a measure of DoS protection and ensures th= at every relayed byte is more valuable (to the miner/network) than the last= .
*The only time that= RBF is critical for relay is during full block periods -- if there's n= ot enough transactions to fill a block, using RBF to ensure that a transact= ion is mined in a timely manner is moot. As such, RBF rules should not be c= oncerned with low-block environments.

### Mempools and Relay
The mempool currently serves two masters: the pr= ofit motive of the miner and the relay motive of a utxo holder. It is in th= e interest of a user to send the miner a high per-byte tx, such that it mig= ht end up in the next block. It is in the miner's best interest to incl= ude the highest per-byte set of transactions in their block template.

There is some conflation he= re in the current RBF policies between what is in the mempool and what is c= onsidered a candidate for the block template. If a miner has already includ= ed a more profitable package of txs into their block template than a more v= aluable per-byte tx that the network has relayed to them, it should be the = responsibility of the block template constructor to reject the new proposed= tx, not the nodes relaying the transaction to said miner.

This is a policy that the miner can (= and should) implement at the level of the template construction, however.

Is it the responsibilit= y of the mempool to provide the best "historical" block opportuni= ty for a miner (e.g. the highest paying block given all txs it's ever s= een)? I would say no, that the ability of a utxo owner to re-state the spen= d condition of a pending transaction is more important, from a use-case per= spective, and that the mempool should concern itself solely with relaying = increasingly more profitable bytes to miners. Let the miners concern themse= lves with deciding what the best policy for their own block construction is= , and the mempool with relaying the highest value bytes for the network. Ne= t-net, this will benefit everyone as it becomes easier for users to re-subm= it txs with increasingly greater feerates, creating more active competition= for available blockspace as more applications are able to include it as a = feature (and it works, reliable, as a relay mechanism).

### Packages and RBF
Packages make the increasing per= -byte rule less of a guarantee that increasing the per-byte value of a sing= le transaction will net a given miner more fees than including the entire p= ackage.


<= /span>
RBF, however, is a transaction level policy: it allows you to repl= ace any *one* component of a package, or tree, with the side effect of poss= ibly invalidating other candidate txs. If the 'composed tx' (aka pa= ckage) had a net per-byte value that was higher than the new replacement tr= ansaction because of a leaf tx that had an outsized per-byte feerate, then = it would be more profitable for the miner to have mined the entire package = rather than the replacement.

This edge case complicates the picture for the miner. Considered fro= m the viewpoint of the user issuing the RBF, however, it is far simpler. In= the ideal case, a person is issuing an RBF because the previous tx tree, e= ven with its high fee sub-component, was not a candidate for the next block= . And, in some cases, increasing the sub-component's per-byte feerate w= ill not achieve the goal of moving the tx any closer to being mined. It'= ;s only by increasing the feerate above the present feerate of the candidat= e plus desendents (tx package) that the transaction will advance in the que= ue.
=
While not uncompli= cated, this is a simple enough metric for a wallet to track, and does not = require any knowledge of the current mempool to effectively surpass. It'= ;s the wallet's responsibility to track this though; failure to take de= scendants into account when deciding on the next per-byte feerate for an RB= F *will* mean that your RBF will be ineffective at achieving the goal of ge= tting your UTXO spent. Any wallet is incentivized to always provide a highe= r per-byte feerate than the 'composed tx' (tx and its descendants),= so as to ensure an actual improvement in the unconfirmed transaction's= position in the block queue, so to speak.

Note that the v2 protocol for cha= nnel opens in lightning uses an RBF negotiation that adheres basically to t= hese rules (ea transaction must have a strictly greater per-byte feerate).<= /span>

We enforce a rate of 6= 5/64th as the required increase in feerate for each subsequent channel open= transaction.

= https://gith= ub.com/lightning/bolts/pull/851/files#diff-ed04ca2c673fd6aabde69389511fa9ee= 60cb44d6b2ef6c88b549ffaa753d6afeR1154

### = RBF and DoS

Admittedly, changing these rules will = increase the number of times that any UTXO is eligible to be retransmitted = (relayed) via the bitcoin node network. Strictly increasing the per-byte fe= erate however ensures that this re-relay is increasingly more expensive to = the UTXO owner, however.


### in exi= tus
These are the things I've been thinking about with regard= s to RBF. I hope they can help=C2=A0to highlight the challenges in the RBF = design space a bit more clearly, as well as spelling out the case for using= a simple heuristic such as "solely increasing per-byte feerate" = as a good candidate for the revised RBF policy.

~n= iftynei

--000000000000f9c1c105d799930c--