MIME-Version: 1.0
References: <a09f5291-e7c0-0aca-6971-03ace0c38dff@mattcorallo.com>
 <CAO3Pvs-SbA+6b2c3Pg-ohvovVtTenx8ve1BZWGgCiLAcSZNNVw@mail.gmail.com>
 <10406f38-fc9d-ac0b-cfd2-0b507d8ec573@mattcorallo.com>
In-Reply-To: <10406f38-fc9d-ac0b-cfd2-0b507d8ec573@mattcorallo.com>
From: Olaoluwa Osuntokun <laolu32@gmail.com>
Date: Wed, 22 Apr 2020 16:13:01 -0700
Message-ID: <CAO3Pvs8Dx8ATXfQRA3p7DzzHcYZz1nF4g=+ZHvSukL0jMbxAOw@mail.gmail.com>
To: Matt Corallo <lf-lists@mattcorallo.com>
Content-Type: multipart/alternative; boundary="00000000000035720105a3e947ea"
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>,
 lightning-dev <lightning-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] RBF Pinning with Counterparties and Competing
	Interest
Precedence: list

--00000000000035720105a3e947ea
Content-Type: text/plain; charset="UTF-8"

> This seems like a somewhat unnecessary drive-by insult of a project you
> don't contribute to, but feel free to start with a concrete suggestion
> here :).

This wasn't intended as an insult at all. I'm simply saying if there's
concern about worst case eviction/replacement, optimizations likely exist.
Other developers that are interested in more complex multi-transaction
contracts have realized this as well, and there're various open PRs that
attempt to propose such optimizations [1].

> Hmm, maybe the proposal wasn't clear. The idea isn't to add signatures to
> braodcasted transactions, but instead to CPFP a maybe-broadcasted
> transaction by sending a transaction which spends it and seeing if it is
> accepted

Sorry I still don't follow. By "we clearly need to go the other direction -
all HTLC output spends need to be pre-signed.", you don't mean that the HTLC
spends of the non-broadcaster also need to be an off-chain 2-of-2 multi-sig
covenant? If the other party isn't restricted w.r.t _how_ they can spend the
output (non-rbf'd, ect), then I don't see how that addresses anything.

Also see my mail elsewhere in the thread that the other party is actually
forced to spend their HTLC output using an RBF-replaceable transaction. With
that, I think we're all good here? In the end both sides have the ability to
raise the fee rate of their spending transactions with the highest winning.
As long as one of them confirms within the CLTV-delta, then everyone is
made whole.


[1]: https://github.com/bitcoin/bitcoin/pull/18191


On Wed, Apr 22, 2020 at 9:50 AM Matt Corallo <lf-lists@mattcorallo.com>
wrote:

> A few replies inline.
>
> On 4/22/20 12:13 AM, Olaoluwa Osuntokun wrote:
> > Hi Matt,
> >
> >
> >> While this is somewhat unintuitive, there are any number of good
> anti-DoS
> >> reasons for this, eg:
> >
> > None of these really strikes me as "good" reasons for this limitation,
> which
> > is at the root of this issue, and will also plague any more complex
> Bitcoin
> > contracts which rely on nested trees of transaction to confirm (CTV,
> Duplex,
> > channel factories, etc). Regarding the various (seemingly arbitrary)
> package
> > limits it's likely the case that any issues w.r.t computational
> complexity
> > that may arise when trying to calculate evictions can be ameliorated with
> > better choice of internal data structures.
> >
> > In the end, the simplest heuristic (accept the higher fee rate package)
> side
> > steps all these issues and is also the most economically rationale from a
> > miner's perspective. Why would one prefer a higher absolute fee package
> > (which could be very large) over another package with a higher total _fee
> > rate_?
>
> This seems like a somewhat unnecessary drive-by insult of a project you
> don't contribute to, but feel free to start with
> a concrete suggestion here :).
>
> >> You'll note that B would be just fine if they had a way to safely
> monitor the
> >> global mempool, and while this seems like a prudent mitigation for
> >> lightning implementations to deploy today, it is itself a quagmire of
> >> complexity
> >
> > Is it really all that complex? Assuming we're talking about just watching
> > for a certain script template (the HTLC scipt) in the mempool to be able
> to
> > pull a pre-image as soon as possible. Early versions of lnd used the
> mempool
> > for commitment broadcast detection (which turned out to be a bad idea so
> we
> > removed it), but at a glance I don't see why watching the mempool is so
> > complex.
>
> Because watching your own mempool is not guaranteed to work, and during
> upgrade cycles that include changes to the
> policy rules an attacker could exploit your upgraded/non-upgraded status
> to perform the same attack.
>
> >> Further, this is a really obnoxious assumption to hoist onto lightning
> >> nodes - having an active full node with an in-sync mempool is a lot more
> >> CPU, bandwidth, and complexity than most lightning users were expecting
> to
> >> face.
> >
> > This would only be a requirement for Lightning nodes that seek to be a
> part
> > of the public routing network with a desire to _forward_ HTLCs. This
> isn't
> > doesn't affect laptops or mobile phones which likely mostly have private
> > channels and don't participate in HTLC forwarding. I think it's pretty
> > reasonable to expect a "proper" routing node on the network to be backed
> by
> > a full-node. The bandwidth concern is valid, but we'd need concrete
> numbers
> > that compare the bandwidth over head of mempool awareness (assuming the
> > latest and greatest mempool syncing) compared with the overhead of the
> > channel update gossip and gossip queries over head which LN nodes face
> today
> > as is to see how much worse off they really would be.
>
> If mempool-watching were practical, maybe, though there are a number of
> folks who are talking about designing
> partially-offline local lightning hubs which would be rendered impractical.
>
> > As detailed a bit below, if nodes watch the mempool, then this class of
> > attack assuming the anchor output format as described in the open
> > lightning-rfc PR is mitigated. At a glance, watching the mempool seems
> like
> > a far less involved process compared to modifying the state machine as
> its
> > defined today. By watching the mempool and implementing the changes in
> > #lightning-rfc/688, then this issue can be mitigated _today_. lnd 0.10
> > doesn't yet watch the mempool (but does include anchors [1]), but unless
> I'm
> > missing something it should be pretty straight forward to add which mor
> or less
> > resolves this issue all together.
> >
> >> not fixing this issue seems to render the whole exercise somewhat
> useless
> >
> > Depends on if one considers watching the mempool a fix. But even with
> that a
> > base version of anchors still resolves a number of issues including:
> > eliminating the commitment fee guessing game, allowing users to pay less
> on
> > force close, being able to coalesce 2nd level HTLC transactions with the
> > same CLTV expiry, and actually being able to reliably enforce multi-hop
> HTLC
> > resolution.
> >
> >> Instead of making the HTLC output spending more free-form with
> >> SIGHASH_ANYONECAN_PAY|SIGHASH_SINGLE, we clearly need to go the other
> >> direction - all HTLC output spends need to be pre-signed.
> >
> > I'm not sure this is actually immediately workable (need to think about
> it
> > more). To see why, remember that the commit_sig message includes HTLC
> > signatures for the _remote_ party's commitment transaction, so they can
> > spend the HTLCs if they broadcast their version of the commitment (force
> > close). If we don't somehow also _gain_ signatures (our new HTLC
> signatures)
> > allowing us to spend HTLCs on _their_ version of the commitment, then if
> > they broadcast that commitment (without revoking), then we're unable to
> > redeem any of those HTLCs at all, possibly losing money.
>
> Hmm, maybe the proposal wasn't clear. The idea isn't to add signatures to
> braodcasted transactions, but instead to CPFP
> a maybe-broadcasted transaction by sending a transaction which spends it
> and seeing if it is accepted. You only need to
> know the transaction's exact format (ie txid, which we do, since we sent a
> signature for it long ago) to do this, you
> don't have to actually *have* the fully-signed transaction (and you don't).
>
> > In an attempt to counteract this, we might say ok, the revoke message
> also
> > now includes HTLC signatures for their new commitment allowing us to
> spend
> > our HTLCs. This resolves things in a weaker security model, but doesn't
> > address the issue generally, as after they receive the commit_sig, they
> can
> > broadcast immediately, again leaving us without a way to redeem our
> HTLCs.
> >
> > I'd need to think about it more, but it seems that following this path
> would
> > require an overhaul in the channel state machine to make presenting a new
> > commitment actually take at least _two phases_ (at least a full round
> trip).
> > The first phase would tender the commitment, but render them unable to
> > broadcast it. The second phase would then <insert something something
> > scriptless scripts here> enter a new sub-protocol which upon conclusion,
> > gives the commitment proposer valid HTLC signatures, and gives the
> responder
> > what they need to be able to broadcast their commitment and claim their
> > HTCLs in an atomic manner.
> >
> > -- Laolu
> >
> > [1]: https://github.com/lightningnetwork/lnd/pull/3821
>

--00000000000035720105a3e947ea
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">&gt; This seems like a somewhat unnecessary drive-by insul=
t of a project you<br>&gt; don&#39;t contribute to, but feel free to start =
with a concrete suggestion<br>&gt; here :).<br><br>This wasn&#39;t intended=
 as an insult at all. I&#39;m simply saying if there&#39;s<br>concern about=
 worst case eviction/replacement, optimizations likely exist.<br>Other deve=
lopers that are interested in more complex multi-transaction<br>contracts h=
ave realized this as well, and there&#39;re various open PRs that<br>attemp=
t to propose such optimizations [1].<br><br>&gt; Hmm, maybe the proposal wa=
sn&#39;t clear. The idea isn&#39;t to add signatures to<br>&gt; braodcasted=
 transactions, but instead to CPFP a maybe-broadcasted<br>&gt; transaction =
by sending a transaction which spends it and seeing if it is<br>&gt; accept=
ed<br><br>Sorry I still don&#39;t follow. By &quot;we clearly need to go th=
e other direction -<br>all HTLC output spends need to be pre-signed.&quot;,=
 you don&#39;t mean that the HTLC<br>spends of the non-broadcaster also nee=
d to be an off-chain 2-of-2 multi-sig<br>covenant? If the other party isn&#=
39;t restricted w.r.t _how_ they can spend the<br>output (non-rbf&#39;d, ec=
t), then I don&#39;t see how that addresses anything.<br><br>Also see my ma=
il elsewhere in the thread that the other party is actually<br>forced to sp=
end their HTLC output using an RBF-replaceable transaction. With<br>that, I=
 think we&#39;re all good here? In the end both sides have the ability to<b=
r>raise the fee rate of their spending transactions with the highest winnin=
g.<br>As long as one of them confirms within the CLTV-delta, then everyone =
is<br>made whole.<br><br><br>[1]: <a href=3D"https://github.com/bitcoin/bit=
coin/pull/18191">https://github.com/bitcoin/bitcoin/pull/18191</a><br><br><=
/div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">O=
n Wed, Apr 22, 2020 at 9:50 AM Matt Corallo &lt;<a href=3D"mailto:lf-lists@=
mattcorallo.com">lf-lists@mattcorallo.com</a>&gt; wrote:<br></div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">A few replies inline.<br>
<br>
On 4/22/20 12:13 AM, Olaoluwa Osuntokun wrote:<br>
&gt; Hi Matt,<br>
&gt; <br>
&gt; <br>
&gt;&gt; While this is somewhat unintuitive, there are any number of good a=
nti-DoS<br>
&gt;&gt; reasons for this, eg:<br>
&gt; <br>
&gt; None of these really strikes me as &quot;good&quot; reasons for this l=
imitation, which<br>
&gt; is at the root of this issue, and will also plague any more complex Bi=
tcoin<br>
&gt; contracts which rely on nested trees of transaction to confirm (CTV, D=
uplex,<br>
&gt; channel factories, etc). Regarding the various (seemingly arbitrary) p=
ackage<br>
&gt; limits it&#39;s likely the case that any issues w.r.t computational co=
mplexity<br>
&gt; that may arise when trying to calculate evictions can be ameliorated w=
ith<br>
&gt; better choice of internal data structures.<br>
&gt; <br>
&gt; In the end, the simplest heuristic (accept the higher fee rate package=
) side<br>
&gt; steps all these issues and is also the most economically rationale fro=
m a<br>
&gt; miner&#39;s perspective. Why would one prefer a higher absolute fee pa=
ckage<br>
&gt; (which could be very large) over another package with a higher total _=
fee<br>
&gt; rate_?<br>
<br>
This seems like a somewhat unnecessary drive-by insult of a project you don=
&#39;t contribute to, but feel free to start with<br>
a concrete suggestion here :).<br>
<br>
&gt;&gt; You&#39;ll note that B would be just fine if they had a way to saf=
ely monitor the<br>
&gt;&gt; global mempool, and while this seems like a prudent mitigation for=
<br>
&gt;&gt; lightning implementations to deploy today, it is itself a quagmire=
 of<br>
&gt;&gt; complexity<br>
&gt; <br>
&gt; Is it really all that complex? Assuming we&#39;re talking about just w=
atching<br>
&gt; for a certain script template (the HTLC scipt) in the mempool to be ab=
le to<br>
&gt; pull a pre-image as soon as possible. Early versions of lnd used the m=
empool<br>
&gt; for commitment broadcast detection (which turned out to be a bad idea =
so we<br>
&gt; removed it), but at a glance I don&#39;t see why watching the mempool =
is so<br>
&gt; complex.<br>
<br>
Because watching your own mempool is not guaranteed to work, and during upg=
rade cycles that include changes to the<br>
policy rules an attacker could exploit your upgraded/non-upgraded status to=
 perform the same attack.<br>
<br>
&gt;&gt; Further, this is a really obnoxious assumption to hoist onto light=
ning<br>
&gt;&gt; nodes - having an active full node with an in-sync mempool is a lo=
t more<br>
&gt;&gt; CPU, bandwidth, and complexity than most lightning users were expe=
cting to<br>
&gt;&gt; face.<br>
&gt; <br>
&gt; This would only be a requirement for Lightning nodes that seek to be a=
 part<br>
&gt; of the public routing network with a desire to _forward_ HTLCs. This i=
sn&#39;t<br>
&gt; doesn&#39;t affect laptops or mobile phones which likely mostly have p=
rivate<br>
&gt; channels and don&#39;t participate in HTLC forwarding. I think it&#39;=
s pretty<br>
&gt; reasonable to expect a &quot;proper&quot; routing node on the network =
to be backed by<br>
&gt; a full-node. The bandwidth concern is valid, but we&#39;d need concret=
e numbers<br>
&gt; that compare the bandwidth over head of mempool awareness (assuming th=
e<br>
&gt; latest and greatest mempool syncing) compared with the overhead of the=
<br>
&gt; channel update gossip and gossip queries over head which LN nodes face=
 today<br>
&gt; as is to see how much worse off they really would be.<br>
<br>
If mempool-watching were practical, maybe, though there are a number of fol=
ks who are talking about designing<br>
partially-offline local lightning hubs which would be rendered impractical.=
<br>
<br>
&gt; As detailed a bit below, if nodes watch the mempool, then this class o=
f<br>
&gt; attack assuming the anchor output format as described in the open<br>
&gt; lightning-rfc PR is mitigated. At a glance, watching the mempool seems=
 like<br>
&gt; a far less involved process compared to modifying the state machine as=
 its<br>
&gt; defined today. By watching the mempool and implementing the changes in=
<br>
&gt; #lightning-rfc/688, then this issue can be mitigated _today_. lnd 0.10=
<br>
&gt; doesn&#39;t yet watch the mempool (but does include anchors [1]), but =
unless I&#39;m<br>
&gt; missing something it should be pretty straight forward to add which mo=
r or less<br>
&gt; resolves this issue all together.<br>
&gt; <br>
&gt;&gt; not fixing this issue seems to render the whole exercise somewhat =
useless<br>
&gt; <br>
&gt; Depends on if one considers watching the mempool a fix. But even with =
that a<br>
&gt; base version of anchors still resolves a number of issues including:<b=
r>
&gt; eliminating the commitment fee guessing game, allowing users to pay le=
ss on<br>
&gt; force close, being able to coalesce 2nd level HTLC transactions with t=
he<br>
&gt; same CLTV expiry, and actually being able to reliably enforce multi-ho=
p HTLC<br>
&gt; resolution.<br>
&gt; <br>
&gt;&gt; Instead of making the HTLC output spending more free-form with<br>
&gt;&gt; SIGHASH_ANYONECAN_PAY|SIGHASH_SINGLE, we clearly need to go the ot=
her<br>
&gt;&gt; direction - all HTLC output spends need to be pre-signed.<br>
&gt; <br>
&gt; I&#39;m not sure this is actually immediately workable (need to think =
about it<br>
&gt; more). To see why, remember that the commit_sig message includes HTLC<=
br>
&gt; signatures for the _remote_ party&#39;s commitment transaction, so the=
y can<br>
&gt; spend the HTLCs if they broadcast their version of the commitment (for=
ce<br>
&gt; close). If we don&#39;t somehow also _gain_ signatures (our new HTLC s=
ignatures)<br>
&gt; allowing us to spend HTLCs on _their_ version of the commitment, then =
if<br>
&gt; they broadcast that commitment (without revoking), then we&#39;re unab=
le to<br>
&gt; redeem any of those HTLCs at all, possibly losing money.<br>
<br>
Hmm, maybe the proposal wasn&#39;t clear. The idea isn&#39;t to add signatu=
res to braodcasted transactions, but instead to CPFP<br>
a maybe-broadcasted transaction by sending a transaction which spends it an=
d seeing if it is accepted. You only need to<br>
know the transaction&#39;s exact format (ie txid, which we do, since we sen=
t a signature for it long ago) to do this, you<br>
don&#39;t have to actually *have* the fully-signed transaction (and you don=
&#39;t).<br>
<br>
&gt; In an attempt to counteract this, we might say ok, the revoke message =
also<br>
&gt; now includes HTLC signatures for their new commitment allowing us to s=
pend<br>
&gt; our HTLCs. This resolves things in a weaker security model, but doesn&=
#39;t<br>
&gt; address the issue generally, as after they receive the commit_sig, the=
y can<br>
&gt; broadcast immediately, again leaving us without a way to redeem our HT=
LCs.<br>
&gt; <br>
&gt; I&#39;d need to think about it more, but it seems that following this =
path would<br>
&gt; require an overhaul in the channel state machine to make presenting a =
new<br>
&gt; commitment actually take at least _two phases_ (at least a full round =
trip).<br>
&gt; The first phase would tender the commitment, but render them unable to=
<br>
&gt; broadcast it. The second phase would then &lt;insert something somethi=
ng<br>
&gt; scriptless scripts here&gt; enter a new sub-protocol which upon conclu=
sion,<br>
&gt; gives the commitment proposer valid HTLC signatures, and gives the res=
ponder<br>
&gt; what they need to be able to broadcast their commitment and claim thei=
r<br>
&gt; HTCLs in an atomic manner.<br>
&gt; <br>
&gt; -- Laolu<br>
&gt; <br>
&gt; [1]: <a href=3D"https://github.com/lightningnetwork/lnd/pull/3821" rel=
=3D"noreferrer" target=3D"_blank">https://github.com/lightningnetwork/lnd/p=
ull/3821</a><br>
</blockquote></div>

--00000000000035720105a3e947ea--