Return-Path: Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1B08EC001E for ; Thu, 13 Jan 2022 21:06:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id EE12C403B8 for ; Thu, 13 Jan 2022 21:06:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -4.197 X-Spam-Level: X-Spam-Status: No, score=-4.197 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JrDT-5240pxn for ; Thu, 13 Jan 2022 21:06:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by smtp4.osuosl.org (Postfix) with ESMTPS id 67A5C4032D for ; Thu, 13 Jan 2022 21:06:52 +0000 (UTC) Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (authenticated bits=0) (User authenticated as jlrubin@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 20DL6nVb008683 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 13 Jan 2022 16:06:50 -0500 Received: by mail-lf1-f53.google.com with SMTP id x22so23851418lfd.10 for ; Thu, 13 Jan 2022 13:06:50 -0800 (PST) X-Gm-Message-State: AOAM53092W2QitEPJh5BXgqhwDUB4EizRpMGH39Rqc4UOkYfCiuAPcEJ ysq97pL2q6h/GIT2K9iq+CrEJ3/VChlUI27IJLQ= X-Google-Smtp-Source: ABdhPJxt4vQ1zFQFK1kE/QW2S/2e28Li/J/PdlerbiBfoXFFC/aS+5Z+sltAbGpQwnrAwFKH+hoC9YIJ5huQ9X9EqgE= X-Received: by 2002:a05:6512:33ce:: with SMTP id d14mr3538062lfg.516.1642108008527; Thu, 13 Jan 2022 13:06:48 -0800 (PST) MIME-Version: 1.0 From: Jeremy Date: Thu, 13 Jan 2022 13:06:37 -0800 X-Gmail-Original-Message-ID: Message-ID: To: Bitcoin development mailing list Content-Type: multipart/alternative; boundary="00000000000087fdeb05d57d1057" Subject: [bitcoin-dev] Documenting the lifetime of a transaction during mempool congestion from the perspective of a rational user X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jan 2022 21:06:55 -0000 --00000000000087fdeb05d57d1057 Content-Type: text/plain; charset="UTF-8" Devs, This email is primarily about existing wallet behaviors and user preferences, and not about CTV. However, towards the end I will describe the relevance of CTV, but the email is worth reading even if you have no interest in CTV as the problems described exist today. One point of confusion I've seen while discussing CTV based congestion control is that it requires a bunch of new wallet software. Most of the software requirements that would make CTV work well are things that either already exist in Bitcoin Core, or are 'bugs' (where bug is defined as deviation from rational utility maximizing behavior) that should be fixed *whether or not CTV exists.* In this post, I am going to walk through what I expect rational behavior to be for a not unlikely circumstance. First, let's define what rational behavior for a wallet is. A rational wallet should have a few goals: 1) Maximize 'fully trusted' balance (fully confirmed + unconfirmed change outputs from our own txns) 2) Process payments requested by the owner within the "urgency budget" requested by the user. 3) Maximize "privacy" (this is a vague goal, so we'll largely ignore it here.). Rational wallet behavior may not be possible without metadata. For example, a rational wallet might prompt the user for things like "how much do you trust the sender of this payment to not RBF this transaction?", or "how much do you trust this sender to not double spend you?". For example, a self-transfer from cold wallet to hot wallet could have a trust score of 1, whereas a payment from an anonymous source would have a trust score of 0. Exchanges where you have a legal agreement to not RBF might sit somewhere in between. Other pieces of exogenous information that could inform wallet behavior include "has hashrate decreased recently, making longer reorgs likely". In the model above, a user does not request transactions, they request payments. The rational wallet serves as an agent to assist the user in completing these payments. For example, if I have a wallet with a single unconfirmed output, and I spend from it to pay Alice, if the unconfirmed gets replaced, my wallet should track that it was replaced and prompt me to re-sign a new transaction. Rational wallets that maximize balance should be careful to ensure that replaced payments are exclusive, guaranteed either through sufficient confirmations or 'impossibility proofs' by reusing an input (preventing double-send behavior). ----------------------------- Now that we've sketched out a basic framework for what a rational wallet should be doing, we can describe what the process of receiving a payment is. Suppose I have a wallet with a bevy of fully confirmed coins such that for my future payments I am sufficiently funded. Then, I receive a payment from a highly trusted source (e.g., self transfer) that is unconfirmed. I then seek to make an outgoing payment. I should have no preference towards or against spending the unconfirmed transfer, I should simply account for it's cost in coin selection of CPFP-ing the parent transaction. If fees are presently historically low, I may have a preference to spend it so as to not have a higher fee later (consolidation). Later, I receive payment from an untrusted source (e.g., an anonymous donation to me). I have no reason to trust this won't be double spent. Perhaps I can even observe that this output has been RBF'd many times previously. I do not count on this money arriving. The feerate on the transaction suggests it won't be confirmed immediately. In order to maximize balance, I should prioritize spending from this output (even if I don't have a payment I desire to make) in order to CPFP it to the top of the mempool and guarantee I am paid. This is inherently "free" since my cost to CPFP would be checked to be lower than the funds I am receiving, and I have no expected value to receive the payment if it is not confirmed. If I do have a transaction I desire to do, I should prioritize spending this output at that time. If not, I would do a CPFP just in favor of balance maximizing. Perhaps I can piggyback something useful, like speculatively opening a lightning channel. If I just self-spend to CPFP, it is very simple since the only party set up for disappointment is myself (note: I patched the behavior in this case to accurately *not* count this as a trusted balance in https://github.com/bitcoin/bitcoin/pull/16766, since a parent could disrupt this). However, if I try to make a payment, my wallet agent must somehow prompt me to re-sign or automatically sign an alternative payment once it is proven (e.g. 6 blocks) I won't receive the output, or to re-sign on a mutually exclusive output (e.g., fee bumping RBF) such that issuing two payments will not causes a double-send error. This adds complexity to both the user story and logic, but is still rational. Now, suppose that I receive a new payment from a **trusted** source that is a part of a "long chain". A long chain is a series of transactions that are all unconfirmed in the mempool. This long-chain is in the bottom of the mempool, and is not expected to confirm any time soon. My wallet should be configured such that it saves not only all ancestors of the transaction paying me, but also all descendants of the root unconfirmed transaction paying me. If I do not save all ancestor transactions, then it would be impossible for me to claim this payment at a future date, and would violate balance maximization. But why save descendants, if they do not concern me? Descendant transactions are critical for balance maximization. Someone else's spend of an output is a "free" CPFP subsidy for driving my transaction to completion (perhaps "descendants that increase the feerate of any parent" is the correct thing to save). Therefore if I want to maximize balance, I would rather keep these transactions around should I ever need to rebroadcast the transactions as it should be cheaper than having to CPFP by myself. Now, suppose that I receive a similar payment in a longchain from a series of untrusted sources. The same arguments apply, but now I may have even higher incentive to prioritize spending this coin since, if sender's trust scores are independent, my total trust in that payment is decomposed worst-case geometrically. It may not be a good assumption that trust scores are independent, since a long chain might be generated as e.g. a series of batch payments from one service provider. Briefly mentioned above is rebroadcasting. This is sort of an orthogonal behavior to the above, but it is "simple" to explain. Wallet agents should retransmit txns to the network if they believe other nodes are not aware of them and they are likely to go into a block. This encapsulates personal transactions as well as arbitrary transactions from third parties. There are many privacy implications of rebroadcasting that are out of scope for this post. ----------------- All of the behaviors documented above are things that should happen today if you would like to have a rational wallet that maximizes your balance and makes payments. The reasons we don't do some of these things are, as far as I can tell: 1) Engineering Complexity 2) UX Complexity (simpler to make unconfirmed outputs "unspendable" to minimize transaction reissuing) 3) Mempool backlog is low, things are confirmed quickly if a sender pays a relatively low fee Certain wallets already have parts of this functionality baked in to an extent. For example, in Lightning Channels, you will drive payments to completion to prevent HTLC timeouts during contested closes (HTLCs == low trust score payments). Should Bitcoin see development of a more robust fee market, it is highly likely the rational behaviors described above would be emergent among all bitcoin wallets (who would want to use a Bitcoin wallet that gets you less money over time?). This email is not just a "Bitcoin Core" thing, hence not being an issue on Bitcoin Core where there are currently deviations from the above behaviors. ----------------- What's CTV got to do with it? A common critique of congestion control using CTV is that it complicates wallet behavior because congestion control is designed to be useful in the circumstances above. CTV and congestion control do not cause these conditions. These conditions already exist whether or not we have congestion control. Where congestion control helps is that, in a world with a full mempool, you have fewer payments that are *actually* unconfirmed because exchanges that batch can fully confirm a constant sized root transaction and the sub-trees of transactions locked in by CTV can be treated as fully trusted. This helps reduce the need for the (rational) behavior of CPFP bumping your own payments on receipt from lower trust senders. Further, the expansion of the transaction tree can be done by other users receiving, so you have an incentive to wait for funds to mature as someone else might pay for expansion. These two factors mean that CTV congestion control can exert a dramatic back pressure on transaction urgency by unbundling the blockspace demand for spending and receiving coins. There are other synergies -- such as non-interactive channel opens -- that further improve the amount of reduction of time-preference in full on-chain resolution. I hope this email helps clarify why CTV Congestion Control isn't particularly creating a wallet architecture problem, it's helping to solve an existing one. Best, Jeremy -- @JeremyRubin --00000000000087fdeb05d57d1057 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Devs,

This email is= primarily about existing wallet behaviors and user preferences, and not ab= out CTV. However, towards the end I will describe the relevance of CTV, but= the email is worth reading even if you have no interest in CTV as the prob= lems described exist today.

One point of confusion I've seen whil= e discussing CTV based congestion control is that it requires a bunch of ne= w wallet software.

Most of the software requirements that would make = CTV work well are things that either already exist in Bitcoin Core, or are = 'bugs' (where bug is defined as deviation from rational utility max= imizing behavior) that should be fixed whether or not CTV exists.

In this post, I am going to walk through what I expect rational = behavior to be for a not unlikely circumstance.

First, let's defi= ne what rational behavior for a wallet is. A rational wallet should have a = few goals:

1) Maximize 'fully trusted' balance (fully confirm= ed=C2=A0+ unconfirmed change outputs from our own txns)
2) Process payments requested by the owner within the &q= uot;urgency budget" requested by the user.
3) Maximize "privacy" (this is a vague goal, so we'= ;ll largely ignore it here.).

Rational wallet behavior may not be pos= sible without metadata. For example, a rational wallet might prompt the use= r for things like "how much do you trust the sender of this payment to= not RBF this transaction?", or "how much do you trust this sende= r to not double=C2=A0spend you?". For example, a self-transfer from co= ld wallet to hot wallet could have a trust score of 1, whereas a payment fr= om an anonymous source would have a trust score of 0. Exchanges where you h= ave a legal agreement to not RBF might sit somewhere in between. Other piec= es of exogenous information that could inform wallet behavior include "= ;has hashrate decreased recently, making longer reorgs likely".
<= div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif= ;font-size:small;color:#000000">
= In the model above, a user does not request transactions, they request paym= ents. The rational wallet serves as an agent to assist the user in completi= ng these payments. For example, if I have a wallet with a single unconfirme= d output, and I spend from it to pay Alice, if the unconfirmed gets replace= d, my wallet should track that it was replaced and prompt me to re-sign a n= ew transaction. Rational wallets that maximize balance should be careful to= ensure that replaced payments are exclusive, guaranteed either through suf= ficient confirmations or 'impossibility proofs' by reusing an input= (preventing double-send behavior).
<= br>
-----------------------------

Now that we've sketched out a basic framework for what a rational wa= llet should be doing, we can describe what the process of receiving a payme= nt is.

Suppose I have a wallet with a bevy of fully confirmed coins s= uch that for my future payments I am sufficiently funded.

Then, I r= eceive a payment from a highly trusted source (e.g., self transfer) that is= unconfirmed.

I then seek to make an outgoing payment. I should have = no preference towards or against spending the unconfirmed transfer, I shoul= d simply account for it's cost in coin selection of CPFP-ing the parent= transaction. If fees are presently historically low, I may have a preferen= ce to spend it so as to not have a higher fee later (consolidation).
<= div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif= ;font-size:small;color:#000000">
= Later, I receive=C2=A0payment from an untrusted source (e.g., an anonymous = donation to me). I have no reason to trust this won't be double spent. = Perhaps I can even observe that this output has been RBF'd many times p= reviously. I do not count on this money arriving. The feerate on the transa= ction suggests it won't be confirmed immediately.

In order to max= imize balance, I should prioritize spending from this output (even if I don= 't have a payment I desire to make) in order to CPFP it to the top of t= he mempool and guarantee I am paid. This is inherently "free" sin= ce my cost to CPFP would be checked to be lower than the funds I am receivi= ng, and I have no expected value to receive the payment if it is not confir= med. If I do have a transaction I desire to do, I should prioritize spendin= g this output at that time. If not, I would do a CPFP just in favor of bala= nce maximizing. Perhaps I can piggyback something useful, like speculativel= y opening a lightning channel.

If I just self-spend to CPFP, it is ve= ry simple since the only party set up for disappointment is myself (note: I= patched the behavior in this case to accurately *not* count this as a trus= ted balance in=C2=A0https://github.com/bitcoin/bitcoin/pull/16766, since a parent could = disrupt this). However, if I try to make a payment, my wallet agent must so= mehow prompt me to re-sign or automatically sign an alternative payment onc= e it is proven (e.g. 6 blocks) I won't receive the output, or to re-sig= n on a mutually exclusive output (e.g., fee bumping RBF) such that issuing = two payments will not causes a double-send error. This adds complexity to b= oth the user story and logic, but is still rational.=C2=A0

Now, supp= ose that I receive a new payment from =C2=A0a *trusted* source that = is a part of a "long chain". A long chain is a series of transact= ions that are all unconfirmed in the mempool. This long-chain is in the bot= tom of the mempool, and is not expected to confirm any time soon.

M= y wallet should be configured such that it saves not only all ancestors of = the transaction paying me, but also all descendants of the root unconfirmed= transaction paying me. If I do not save all ancestor transactions, then it= would be impossible for me to claim this payment at a future date, and wou= ld violate balance maximization. But why save descendants, if they do not c= oncern me? Descendant transactions are critical for balance maximization. S= omeone else's spend of an output is a "free" CPFP subsidy for= driving my transaction to completion (perhaps "descendants that incre= ase the feerate of any parent" is the correct thing to save). Therefor= e if I want to maximize balance, I would rather keep these transactions aro= und should I ever need to rebroadcast the transactions as it should be chea= per than having to CPFP by myself.
<= br>
Now, suppose that I receive a sim= ilar payment in a longchain from a series of untrusted sources. The same ar= guments apply, but now I may have even higher incentive to prioritize spend= ing this coin since, if sender's trust scores are independent, my total= trust in that payment is decomposed worst-case geometrically. It may not b= e a good assumption that trust scores are independent, since a long chain m= ight be generated as e.g. a series of batch payments from one service provi= der.

Briefly mentioned above is rebroadcasting. This is sort of an or= thogonal behavior to the above, but it is "simple" to explain. Wa= llet agents should retransmit txns to the network if they believe other nod= es are not aware of them and they are likely to go into a block. This encap= sulates personal transactions as well as arbitrary transactions from third = parties. There are many privacy implications of rebroadcasting that are out= of scope for this post.

-----------------

All of the behaviors= documented above are things that should happen today if you would like to = have a rational wallet that maximizes your balance and makes payments.

The reasons we don't do some of these things are, as far as I can tel= l:

1) Engineering Complexity
2= ) UX Complexity (simpler to make unconfirmed outputs "unspendable"= ; to minimize transaction reissuing)
= 3) Mempool backlog is low, things are confirmed quickly if a sender pays a = relatively low fee

Certain wallets already have parts of this functio= nality baked in to an extent. For example, in Lightning Channels, you will = drive payments to completion to prevent HTLC timeouts during contested clos= es (HTLCs =3D=3D low trust score payments).

Should Bitcoin see develo= pment of a more robust fee market, it is highly likely the rational behavio= rs described above would be emergent among all bitcoin wallets (who would w= ant to use a Bitcoin wallet that gets you less money over time?). This emai= l is not just a "Bitcoin Core" thing, hence not being an issue on= Bitcoin Core where there are currently deviations from the above behaviors= .

-----------------

<= div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif= ;font-size:small;color:#000000">What's CTV got to do with it?

A= common critique of congestion control using CTV is that it complicates wal= let behavior because congestion control is designed to be useful in the cir= cumstances above. CTV and congestion control do not cause these conditions.= These conditions already exist whether or not we have congestion control.<= /div>

Where congestion control helps is that, in a world with a full mempoo= l, you have fewer payments that are *actually* unconfirmed because exchange= s that batch can fully confirm a constant sized root transaction and the su= b-trees of transactions locked in by CTV can be treated as fully trusted. T= his helps reduce the need for the (rational) behavior of CPFP bumping your = own payments on receipt from lower trust senders. Further, the expansion of= the transaction tree can be done by other users receiving, so you have an = incentive to wait for funds to mature as someone else might pay for expansi= on. These two factors mean that CTV congestion control can exert a dramatic= back pressure on transaction urgency by unbundling the blockspace demand f= or spending and receiving coins. There are other synergies -- such as non-i= nteractive channel opens -- that further improve the amount of reduction of= time-preference in full on-chain resolution.

I hope this email helps= clarify why CTV Congestion Control isn't particularly creating a walle= t architecture problem, it's helping to solve an existing one.

B= est,

Jeremy



--00000000000087fdeb05d57d1057--