Return-Path: Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 16E25C0032; Sat, 21 Oct 2023 20:05:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id D61304016C; Sat, 21 Oct 2023 20:05:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D61304016C Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=jytSe7Bp X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZjtDjhETQTJf; Sat, 21 Oct 2023 20:05:48 +0000 (UTC) Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by smtp2.osuosl.org (Postfix) with ESMTPS id 4F3D0400D7; Sat, 21 Oct 2023 20:05:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4F3D0400D7 Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-7a6a8db9252so52920639f.3; Sat, 21 Oct 2023 13:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697918747; x=1698523547; darn=lists.linuxfoundation.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TeSGgsAIqKpC/VzcuaCt4rAVE4NCkvkU9eJh8CsoVpI=; b=jytSe7Bpz/9nVzKG/0HorSv3xk5aNLO/yKpqXyIDsTjO+MQD3P4wc8baiGAUU7Ju1i BroXv9AyGu08PE8gqvaUNL8eLH9XKxtjHH6tO5tTqmZ4CXfAZkjzjUTDb59YgZu0H/CO +GmpUE4FbJXf6jRdWZBWk8mTYByFLsP2hMG2RC028F9RlfiBi4zhKXNNwLFmoHURS7SO jDZaUhZx/GvDMWS2o0uu7zF2J2O6ImPz2rFkWIL8PnD8GfxGRAfaXLD8i6jNXa1NUS1k fsDRtDeck85DLI6fgQPTDhWC7lYXMrUCJLV9jBz3E6eYJCUSgdfZDOx0zVMwj8gbVYJK nkeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697918747; x=1698523547; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TeSGgsAIqKpC/VzcuaCt4rAVE4NCkvkU9eJh8CsoVpI=; b=pxqqBKQN+W2A4a0Tyz2uulycdlEisZ/k1NaKI9rwwMo1b67W/+VHie1srcb807F0/f xctN9S5tpB8yPDbGAnyOcGKj090mGBqjjwzGtRgtA6RS9vffW0vGYaCo7Ox+GJBlg/AB +KNMTk4pXX2yDm43GbclYR9N1vuGxXxJriiZ0N3MNmPRjEXcuidIif7UU4xKlj5apSHW +0aFvVYqwrawp5Y5PiW0/G9BNxdPmQyF0iAcGaw9W45D10u0GjEDXX83qOj4bkp4CW0a WYDAJXMiaF10h2fPT2SqFI9L98RJRWHKkcwtxGEm+fYJSxE5WAL2mlThTkXZnC/1jMh8 Q/SQ== X-Gm-Message-State: AOJu0YwyKrMeWkJ4On88ua/C/s/vm6dfgL5VYEobD1fxLKJq3vk7v34A 3Cm66pkGeQ9cAaSoiHBf+rkPysYqvxs9oNYin6sYg+FVp8zgUtY4 X-Google-Smtp-Source: AGHT+IHUL4m59qz88vBQoDY7jWCrP2JQGTrIaKkNe5wlVupEBw9gYhLLOPmZet/Y5zNEqyRvavBiHgDui8X8xTuWgpM= X-Received: by 2002:a05:6602:2e12:b0:7a6:9947:ec76 with SMTP id o18-20020a0566022e1200b007a69947ec76mr6682996iow.9.1697918746854; Sat, 21 Oct 2023 13:05:46 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Antoine Riard Date: Sat, 21 Oct 2023 21:05:35 +0100 Message-ID: To: Bitcoin Protocol Discussion , "lightning-dev\\\\@lists.linuxfoundation.org" Content-Type: multipart/alternative; boundary="000000000000c388e406083f8305" X-Mailman-Approved-At: Sun, 22 Oct 2023 01:29:30 +0000 Cc: security@ariard.me Subject: Re: [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us" X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Oct 2023 20:05:54 -0000 --000000000000c388e406083f8305 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, As I've been shown offline Twitter posts misrepresenting my previous mail, I think it's good to correct them. The security flaws are not "intentional backdoor" or whatever misrepresentation that would question the competence and know-how of the Bitcoin and Lightning development community. The replacement cycling issue discovered has been known by a small circle of Bitcoin developers since December 2022. As it appears to some experts and it has been commented publicly, changes at the bitcoin base-layer might be the most substantial fixes. Those changes take time and here this is akin to how the linux kernel, bsds and OS vendors are working [0]. All I can say is that we had recently had internal discussion on how to improve coordinated security fixes and patching processes for the coming decades. This is an area of concern where I've always been at the forefront as early as 2020 / 2021 [1]. In the meanwhile, lightning experts have already deployed mitigations which are hardening the lightning ecosystem significantly in face of simple or medium attacks. More advanced attacks can only be mounted if you have sufficient p2p and mempool knowledge as was pointed out by other bitcoin experts like Matt or Peter (which take years to acquire for average bitcoin developers) and the months of preparation to attempt them. If you're a journalist reporting on the information in mainstream crypto publications, I'll suggest waiting to do so before expert reporters of bitcoin circles who have more in-field knowledge can do so and qualify the technical situation with more distance. As I've already been interviewed by top financial publication years ago for my work on bitcoin, as a journalist you're engaging your own reputation on the information you're reporting. Thanks for being responsible here. This is the nature of the electronic communication and contemporaneous media that information is extremely fluid and there is no native anti-DoS mechanism to slow down the propagation of sensitive information where mitigations are still in deployment. A reason I'm not on social media of any kind [2]. In the meanwhile, it's good to go to read senecca and marcus aurelius take the situation with stoicism and with a zelt of meditation [3]= . All my previous statements are mostly technically correct (even if some could have been written with more clarity once again I'm not an english native [4]). While I wish to wait the week of the 30th Oct o discuss further what is best fix and what are the trade-offs as a community as a wide (give time some laggard lightning implementations ship fixes), though I'll comment further on the mailing list if the flow of information on "social media" is DoSing the ability of the bitcoin community to work on the long-term appropriate fixes in a responsible and constructive fashion. [0] See meltdown class of vulnerability and how operating systems are handling hardware-sourced vulnerabilities https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability). Most of the time they do their best on the software side and they go to see with hardware vendors how to do the necessary updates. [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-April/003002= .html [2] And for the wider analysis on contemporaneous culture of information propagation and network effect, I can only recommend to read venkatesh rao's ribbonfarm essays http://ribbonfarm.com [3] There are very good reasons why some executives at top modern technology companies are doing meditation daily, some even hours. "mind illuminated" is a good read. [4] While my former employer, Chaincode Labs, paid for my english lessons back in 2020. Generally it was a good insight from them to train people on how to communicate in a crisis. Le ven. 20 oct. 2023 =C3=A0 07:56, Antoine Riard = a =C3=A9crit : > Hi, > > After writing the mail reply on the economics of sequential malicious > replacement of honest HTLC-timeout, I did write one more test to verify t= he > behavior on core mempool, and it works as expected. > > > https://github.com/ariard/bitcoin/commit/30f5d5b270e4ff195e8dcb9ef6b7ddcc= 5f6a1bf2 > > Responsible disclosure process has followed the lines of hardware issues > affecting operating system, as documented for the Linux kernel, while > adapted to the bitcoin ecosystem: > > https://docs.kernel.org/6.1/process/embargoed-hardware-issues.html > > Effective now, I'm halting my involvement with the development of the > lightning network and its implementations, including coordinating the > handling of security issues at the protocol level (I informed some senior > lightning devs in that sense before). > > Closed the very old issue which was affected to me at this purpose on the > bolt repository: > > https://github.com/lightning/bolts/pull/772 > > I think this new class of replacement cycling attacks puts lightning in a > very perilous position, where only a sustainable fix can happen at the > base-layer, e.g adding a memory-intensive history of all-seen transaction= s > or some consensus upgrade. Deployed mitigations are worth something in fa= ce > of simple attacks, though I don't think they're stopping advanced attacke= rs > as said in the first full disclosure mail. > > Those types of changes are the ones necessitating the utmost transparency > and buy-in of the community as a whole, as we're altering the full-nodes > processing requirements or the security architecture of the decentralized > bitcoin ecosystem in its integrality. > > On the other hand fully explaining why such changes would be warranted fo= r > the sake of lightning and for designing them well, we might need to lay o= ut > in complete state practical and critical attacks on a ~5 355 public BTC > ecosystem. > > Hard dilemma. > > There might be a lesson in terms of bitcoin protocol deployment, we might > have to get them right at first try. Little second chance to fix them in > flight. > > I'll be silent on those issues on public mailing lists until the week of > the 30 oct. Enough material has been published and other experts are > available. Then I'll be back focusing more on bitcoin core. > > Best, > Antoine > > Le lun. 16 oct. 2023 =C3=A0 17:57, Antoine Riard a > =C3=A9crit : > >> (cross-posting mempool issues identified are exposing lightning chan to >> loss of funds risks, other multi-party bitcoin apps might be affected) >> >> Hi, >> >> End of last year (December 2022), amid technical discussions on eltoo >> payment channels and incentives compatibility of the mempool anti-DoS >> rules, a new transaction-relay jamming attack affecting lightning channe= ls >> was discovered. >> >> After careful analysis, it turns out this attack is practical and >> immediately exposed lightning routing hops carrying HTLC traffic to loss= of >> funds security risks, both legacy and anchor output channels. A potentia= l >> exploitation plausibly happening even without network mempools congestio= n. >> >> Mitigations have been designed, implemented and deployed by all major >> lightning implementations during the last months. >> >> Please find attached the release numbers, where the mitigations should b= e >> present: >> - LDK: v0.0.118 - CVE-2023 -40231 >> - Eclair: v0.9.0 - CVE-2023-40232 >> - LND: v.0.17.0-beta - CVE-2023-40233 >> - Core-Lightning: v.23.08.01 - CVE-2023-40234 >> >> While neither replacement cycling attacks have been observed or reported >> in the wild since the last ~10 months or experimented in real-world >> conditions on bitcoin mainet, functional test is available exercising th= e >> affected lightning channel against bitcoin core mempool (26.0 release >> cycle). >> >> It is understood that a simple replacement cycling attack does not deman= d >> privileged capabilities from an attacker (e.g no low-hashrate power) and >> only access to basic bitcoin and lightning software. Yet I still think >> executing such an attack successfully requests a fair amount of bitcoin >> technical know-how and decent preparation. >> >> From my understanding of those issues, it is yet to be determined if the >> mitigations deployed are robust enough in face of advanced replacement >> cycling attackers, especially ones able to combine different classes of >> transaction-relay jamming such as pinnings or vetted with more privilege= d >> capabilities. >> >> Please find a list of potential affected bitcoin applications in this >> full disclosure report using bitcoin script timelocks or multi-party >> transactions, albeit no immediate security risk exposure as severe as th= e >> ones affecting lightning has been identified. Only cursory review of >> non-lightning applications has been conducted so far. >> >> There is a paper published summarizing replacement cycling attacks on th= e >> lightning network: >> >> https://github.com/ariard/mempool-research/blob/2023-10-replacement-pape= r/replacement-cycling.pdf >> >> ## Problem >> >> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted >> HTLC on incoming link and offered HTLC on outgoing link) should settle t= he >> outgoing state with either a success or timeout before the incoming stat= e >> timelock becomes final and an asymmetric defavorable settlement might >> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" >> section 2.3 for a classical exposition of this lightning security proper= ty). >> >> Failure to satisfy this settlement requirement exposes a forwarding hop >> to a loss of fund risk where the offered HTLC is spent by the outgoing l= ink >> counterparty's HTLC-preimage and the accepted HTLC is spent by the incom= ing >> link counterparty's HTLC-timeout. >> >> The specification mandates the incoming HTLC expiration timelock to be >> spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC >> expiration timelock, this exact interval value being an implementation a= nd >> node policy setting. As a minimal value, the specification recommends 34 >> blocks of interval. If the timelock expiration I of the inbound HTLC is >> equal to 100 from chain tip, the timelock expiration O of the outbound H= TLC >> must be equal to 66 blocks from chain tip, giving a reasonable buffer of >> reaction to the lightning forwarding node. >> >> In the lack of cooperative off-chain settlement of the HTLC on the >> outgoing link negotiated with the counterparty (either >> `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the >> lightning node should broadcast its commitment transaction. Once the >> commitment is confirmed (if anchor and the 1 CSV encumbrance is present)= , >> the lightning node broadcasts and confirms its HTLC-timeout before I hei= ght >> is reached. >> >> Here enter a replacement cycling attack. A malicious channel counterpart= y >> can broadcast its HTLC-preimage transaction with a higher absolute fee a= nd >> higher feerate than the honest HTLC-timeout of the victim lightning node >> and triggers a replacement. Both for legacy and anchor output channels, = a >> HTLC-preimage on a counterparty commitment transaction is malleable, i.e >> additional inputs or outputs can be added. The HTLC-preimage spends an >> unconfirmed and unrelated to the channel parent transaction M and confli= cts >> its child. >> >> As the HTLC-preimage spends an unconfirmed input that was already >> included in the unconfirmed and unrelated child transaction (rule 2), pa= ys >> an absolute higher fee of at least the sum paid by the HTLC-timeout and >> child transaction (rule 3) and the HTLC-preimage feerate is greater than >> all directly conflicting transactions (rule 6), the replacement is >> accepted. The honest HTLC-timeout is evicted out of the mempool. >> >> In an ulterior move, the malicious counterparty can replace the parent >> transaction itself with another candidate N satisfying the replacement >> rules, triggering the eviction of the malicious HTLC-preimage from the >> mempool as it was a child of the parent T. >> >> There is no spending candidate of the offered HTLC output for the curren= t >> block laying in network mempools. >> >> This replacement cycling tricks can be repeated for each rebroadcast >> attempt of the HTLC-timeout by the honest lightning node until expiratio= n >> of the inbound HTLC timelock I. Once this height is reached a HTLC-timeo= ut >> is broadcast by the counterparty's on the incoming link in collusion wit= h >> the one on the outgoing link broadcasting its own HTLC-preimage. >> >> The honest Lightning node has been "double-spent" in its HTLC forwarding= . >> >> As a notable factor impacting the success of the attack, a lightning >> node's honest HTLC-timeout might be included in the block template of th= e >> miner winning the block race and therefore realizes a spent of the offer= ed >> output. In practice, a replacement cycling attack might over-connect to >> miners' mempools and public reachable nodes to succeed in a fast evictio= n >> of the HTLC-timeout by its HTLC-preimage. As this latter transaction can >> come with a better ancestor-score, it should be picked up on the flight = by >> economically competitive miners. >> >> A functional test exercising a simple replacement cycling of a HTLC >> transaction on bitcoin core mempool is available: >> https://github.com/ariard/bitcoin/commits/2023-test-mempool >> >> ## Deployed LN mitigations >> >> Aggressive rebroadcasting: As the replacement cycling attacker benefits >> from the HTLC-timeout being usually broadcast by lightning nodes only on= ce >> every block, or less the replacement cycling malicious transactions paid >> only equal the sum of the absolute fees paid by the HTLC, adjusted with = the >> replacement penalty. Rebroadcasting randomly and multiple times before t= he >> next block increases the absolute fee cost for the attacker. >> >> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> >> Local-mempool preimage monitoring: As the replacement cycling attacker i= n >> a simple setup broadcast the HTLC-preimage to all the network mempools, = the >> honest lightning node is able to catch on the flight the unconfirmed >> HTLC-preimage, before its subsequent mempool replacement. The preimage c= an >> be extracted from the second-stage HTLC-preimage and used to fetch the >> off-chain inbound HTLC with a cooperative message or go on-chain with it= to >> claim the accepted HTLC output. >> >> Implemented and deployed by Eclair and LND. >> >> CLTV Expiry Delta: With every jammed block comes an absolute fee cost >> paid by the attacker, a risk of the HTLC-preimage being detected or >> discovered by the honest lightning node, or the HTLC-timeout to slip in = a >> winning block template. Bumping the default CLTV delta hardens the odds = of >> success of a simple replacement cycling attack. >> >> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >> >> ## Affected Bitcoin Protocols and Applications >> >> From my understanding the following list of Bitcoin protocols and >> applications could be affected by new denial-of-service vectors under so= me >> level of network mempools congestion. Neither tests or advanced review o= f >> specifications (when available) has been conducted for each of them: >> - on-chain DLCs >> - coinjoins >> - payjoins >> - wallets with time-sensitive paths >> - peerswap and submarine swaps >> - batch payouts >> - transaction "accelerators" >> >> Inviting their developers, maintainers and operators to investigate how >> replacement cycling attacks might disrupt their in-mempool chain of >> transactions, or fee-bumping flows at the shortest delay. Simple flows a= nd >> non-multi-party transactions should not be affected to the best of my >> understanding. >> >> ## Open Problems: Package Malleability >> >> Pinning attacks have been known for years as a practical vector to >> compromise lightning channels funds safety, under different scenarios (c= f. >> current bip331's motivation section). Mitigations at the mempool level h= ave >> been designed, discussed and are under implementation by the community >> (ancestor package relay + nverrsion=3D3 policy). Ideally, they should >> constraint a pinning attacker to always attach a high feerate package >> (commitment + CPFP) to replace the honest package, or allow a honest >> lightning node to overbid a malicious pinning package and get its >> time-sensitive transaction optimistically included in the chain. >> >> Replacement cycling attack seem to offer a new way to neutralize the >> design goals of package relay and its companion nversion=3D3 policy, whe= re an >> attacker package RBF a honest package out of the mempool to subsequently >> double-spend its own high-fee child with a transaction unrelated to the >> channel. As the remaining commitment transaction is pre-signed with a >> minimal relay fee, it can be evicted out of the mempool. >> >> A functional test exercising a simple replacement cycling of a lightning >> channel commitment transaction on top of the nversion=3D3 code branch is >> available: >> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >> >> ## Discovery >> >> In 2018, the issue of static fees for pre-signed lightning transactions >> is made more widely known, the carve-out exemption in mempool rules to >> mitigate in-mempool package limits pinning and the anchor output pattern >> are proposed. >> >> In 2019, bitcoin core 0.19 is released with carve-out support. Continued >> discussion of the anchor output pattern as a dynamic fee-bumping method. >> >> In 2020, draft of anchor output submitted to the bolts. Initial finding >> of economic pinning against lightning commitment and second-stage HTLC >> transactions. Subsequent discussions of a preimage-overlay network or >> package-relay as mitigations. Public call made to inquiry more on potent= ial >> other transaction-relay jamming attacks affecting lightning. >> >> In 2021, initial work in bitcoin core 22.0 of package acceptance. >> Continued discussion of the pinning attacks and shortcomings of current >> mempool rules during community-wide online workshops. Later the year, in >> light of all issues for bitcoin second-layers, a proposal is made about >> killing the mempool. >> >> In 2022, bip proposed for package relay and new proposed v3 policy desig= n >> proposed for a review and implementation. Mempoolfullrbf is supported in >> bitcoin core 24.0 and conceptual questions about alignment of mempool ru= les >> w.r.t miners incentives are investigated. >> >> Along this year 2022, eltoo lightning channels design are discussed, >> implemented and reviewed. In this context and after discussions on mempo= ol >> anti-DoS rules, I discovered this new replacement cycling attack was >> affecting deployed lightning channels and immediately reported the findi= ng >> to some bitcoin core developers and lightning maintainers. >> >> ## Timeline >> >> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Gre= g >> Sanders and Gloria Zhao >> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien >> Teinturier, Matt Corallo and Olaoluwa Osuntunkun >> - 2022-12-23: Sharing to Eugene Siegel (LND) >> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot >> (non-lightning potential affected projects) >> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and >> cross-layers issuers) and initial proposal of an early public disclosure >> - 2022-01-19: Collection of analysis if other second-layers and >> multi-party applications affected. LN mitigations development starts. >> - 2023-05-04: Sharing to Wilmer Paulino (LDK) >> - 2023-06-20: LN mitigations implemented and progressively released. Wee= k >> of the 16 october proposed for full disclosure. >> - 2023-08-10: CVEs assigned by MITRE >> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling >> attack existence to security@bitcoincore.org. >> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / >> CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks >> >> ## Conclusion >> >> Despite the line of mitigations adopted and deployed by current major >> lightning implementations, I believe replacement cycling attacks are sti= ll >> practical for advanced attackers. Beyond this new attack might come as a >> way to partially or completely defeat some of the pinning mitigations wh= ich >> have been working for years as a community. >> >> As of today, it is uncertain to me if lightning is not affected by a mor= e >> severe long-term package malleability critical security issue under curr= ent >> consensus rules, and if any other time-sensitive multi-party protocol, >> designed or deployed isn't de facto affected too (loss of funds or denia= l >> of service). >> >> Assuming analysis on package malleability is correct, it is unclear to m= e >> if it can be corrected by changes in replacement / eviction rules or >> mempool chain of transactions processing strategy. Inviting my technical >> peers and the bitcoin community to look more on this issue, including to >> dissent. I'll be the first one pleased if I'm fundamentally wrong on tho= se >> issues, or if any element has not been weighted with the adequate techni= cal >> accuracy it deserves. >> >> Do not trust, verify. All mistakes and opinions are my own. >> >> Antoine >> >> "meet with Triumph and Disaster. And treat those two impostors just the >> same" - K. >> > --000000000000c388e406083f8305 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

As I've been shown offline Twit= ter posts misrepresenting my previous mail, I think it's good to correc= t them. The security flaws are not "intentional backdoor" or what= ever misrepresentation that would question the competence and know-how of t= he Bitcoin and Lightning development community.

Th= e replacement cycling issue discovered has been known by a small circle of = Bitcoin developers since December 2022. As it appears to some experts and i= t has been commented publicly, changes at the bitcoin base-layer might be t= he most substantial fixes. Those changes take time and here this is akin to= how the linux kernel, bsds and OS vendors are working [0].

<= /div>
All I can say is that we had recently had internal discussion on = how to improve coordinated security fixes and patching processes for the co= ming decades. This is an area of concern where I've always been at the = forefront as early as 2020 / 2021 [1].

In the mean= while, lightning experts have already=C2=A0deployed mitigations which are h= ardening the lightning ecosystem significantly in face of simple or medium = attacks. More advanced attacks can only be mounted if you have sufficient p= 2p and mempool knowledge as was pointed out by other bitcoin experts like M= att or Peter (which take years to acquire for average bitcoin developers) a= nd the months of preparation to attempt them.

If y= ou're a journalist reporting on the information in mainstream crypto pu= blications, I'll suggest waiting to do so before expert reporters of bi= tcoin circles who have more in-field knowledge can do so and qualify the te= chnical situation with more distance. As I've already been interviewed = by top financial publication years ago for my work on bitcoin, as a journal= ist you're engaging your own reputation on the information you're r= eporting. Thanks for being responsible here.

This = is the nature of the electronic communication and contemporaneous media tha= t information is extremely fluid and there is no native anti-DoS mechanism = to slow down the propagation of sensitive information where mitigations are= still in deployment. A reason I'm not on social media of any kind [2].= In the meanwhile, it's good to go to read senecca and marcus aurelius = take the situation with stoicism and with a zelt of meditation [3].

All my previous statements are mostly technically correct= (even if some could have been written with more clarity once again I'm= not an english native [4]). While I wish to wait the week of the 30th Oct = o discuss further what is best fix and what are the trade-offs as a communi= ty as a wide (give time some laggard lightning implementations ship fixes),= though I'll comment further on the mailing list if the flow of informa= tion on "social media" is DoSing the ability of the bitcoin commu= nity to work on the long-term appropriate fixes in a responsible and constr= uctive fashion.


Le=C2=A0ven. 20 o= ct. 2023 =C3=A0=C2=A007:56, Antoine Riard <antoine.riard@gmail.com> a =C3=A9crit=C2=A0:
=
Hi,

After writing the = mail reply on the economics of sequential malicious replacement of honest H= TLC-timeout, I did write one more test to verify the behavior on core mempo= ol, and it works as expected.


Responsible disclo= sure process has followed the lines of hardware issues affecting operating = system, as documented for the Linux kernel, while adapted to the bitcoin ec= osystem:


Effective now, I'm halting my involvement with the development of= the lightning=C2=A0network and its implementations, including coordinating= the handling of security=C2=A0issues at the protocol=C2=A0level (I informe= d some senior lightning devs in that sense before).

Closed the very old issue which was affected to me at this purpose on the= bolt repository:


I think this new class of repla= cement cycling attacks puts lightning in a very perilous position, where on= ly a sustainable fix can happen at the base-layer, e.g adding a memory-inte= nsive history of all-seen transactions or some consensus upgrade. Deployed = mitigations are worth something in face of simple attacks, though I don'= ;t think they're stopping advanced attackers as said in the first full = disclosure mail.

Those types of changes are the on= es necessitating the utmost transparency and buy-in of the community as a w= hole, as we're altering the full-nodes processing requirements or the s= ecurity architecture of the decentralized bitcoin ecosystem in its integral= ity.

On the other hand fully explaining why such c= hanges would be warranted for the sake of lightning and for designing them = well, we might need to lay out in complete state practical and critical att= acks on a ~5 355 public BTC ecosystem.

Hard dilemm= a.

There might be a lesson in terms of bitcoin pro= tocol deployment, we might have to get them right at first try. Little seco= nd chance to fix them in flight.

I'll be silen= t on those issues on public mailing lists until the week of the 30 oct. Eno= ugh material has been published and other experts are available. Then I'= ;ll be back focusing more on bitcoin core.

Best,
Antoine

Le=C2=A0lun. 16 oct. 2023 =C3=A0=C2=A017:57, Antoine = Riard <anto= ine.riard@gmail.com> a =C3=A9crit=C2=A0:
=
(cross-posting mempool issues identified are exposing= lightning chan to loss of funds risks, other multi-party bitcoin apps migh= t be affected)

Hi,

End of last year = (December 2022), amid technical discussions on eltoo payment channels and i= ncentives compatibility of the mempool anti-DoS rules, a new transaction-re= lay jamming attack affecting lightning channels was discovered.
<= br>
After careful analysis, it turns out this attack is practical= and immediately=C2=A0exposed lightning routing hops carrying HTLC traffic = to loss of funds security risks, both legacy and anchor=C2=A0output channel= s. A potential exploitation plausibly happening even without network mempoo= ls congestion.

Mitigations have been designed, imp= lemented and deployed by all major lightning implementations during the las= t months.

Please find attached the release numbers= , where the mitigations should be present:
- LDK: v0.0.118 - CVE-= 2023 -40231
- Eclair: v0.9.0 - CVE-2023-40232
- LND: v.= 0.17.0-beta - CVE-2023-40233
- Core-Lightning: v.23.08.01 - CVE-2= 023-40234

While neither replacement cycling attack= s have been observed or reported in the wild since the last ~10 months or e= xperimented in real-world conditions on bitcoin mainet, functional test is = available exercising the affected lightning channel against bitcoin core me= mpool (26.0 release cycle).

It is understood that = a simple replacement cycling attack does not demand privileged capabilities= from an attacker (e.g no low-hashrate power) and only access to basic bitc= oin and lightning software. Yet I still think executing such an attack succ= essfully requests a fair amount of bitcoin technical know-how and decent pr= eparation.

From my understanding of those issues, = it is yet to be determined if the mitigations deployed are robust enough in= face of advanced replacement cycling attackers, especially ones able to co= mbine different classes of transaction-relay jamming such as pinnings or ve= tted with more privileged capabilities.

Please fin= d a list of potential affected bitcoin applications in this full disclosure= report using bitcoin script timelocks or multi-party transactions, albeit = no immediate security risk exposure as severe as the ones affecting lightni= ng has been identified. Only cursory review of non-lightning applications h= as been conducted so far.

There is a paper publish= ed summarizing replacement cycling attacks on the lightning network:
<= div>https://github.com= /ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling= .pdf

=C2=A0## Problem

A lightning node allows HTLCs forwarding (in bolt3's parlance accepted= HTLC on incoming link and offered HTLC on outgoing link) should settle the= outgoing state with either a success or timeout before the incoming state = timelock becomes final and an asymmetric defavorable settlement might happe= n (cf "Flood & Loot: A Systematic Attack on The Lightning Network&= quot; section 2.3 for a classical exposition of this lightning security pro= perty).

Failure to satisfy this settlement require= ment exposes a forwarding hop to a loss of fund risk where the offered HTLC= is spent by the outgoing link counterparty's HTLC-preimage and the acc= epted HTLC is spent by the incoming link counterparty's HTLC-timeout.

The specification mandates the incoming HTLC expira= tion timelock to be spaced out by an interval of `cltv_expiry_delta` from t= he outgoing HTLC expiration timelock, this exact interval value being an im= plementation and node policy setting. As a minimal value, the specification= recommends 34 blocks of interval. If the timelock expiration I of the inbo= und HTLC is equal to 100 from chain tip, the timelock expiration O of the o= utbound HTLC must be equal to 66 blocks from chain tip, giving a reasonable= buffer of reaction to the lightning forwarding node.

<= div>In the lack of cooperative off-chain settlement of the HTLC on the outg= oing link negotiated with the counterparty (either `update_fulfill_htlc` or= `update_fail_htlc`) when O is reached, the lightning node should broadcast= its commitment transaction. Once the commitment is confirmed (if anchor an= d the 1 CSV encumbrance is present), the lightning node broadcasts and conf= irms its HTLC-timeout before I height is reached.

= Here enter a replacement cycling attack. A malicious channel counterparty c= an broadcast its HTLC-preimage transaction with a higher absolute fee and h= igher feerate than the honest HTLC-timeout of the victim lightning node and= triggers a replacement. Both for legacy and anchor output channels, a HTLC= -preimage on a counterparty commitment transaction is malleable, i.e additi= onal inputs or outputs can be added. The HTLC-preimage spends an unconfirme= d and unrelated to the channel parent transaction M and conflicts its child= .

As the HTLC-preimage spends an unconfirmed input= that was already included in the unconfirmed and unrelated child transacti= on (rule 2), pays an absolute higher fee of at least the sum paid by the HT= LC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is = greater than all directly conflicting transactions (rule 6), the replacemen= t is accepted. The honest HTLC-timeout is evicted out of the mempool.
=

In an ulterior move, the malicious counterparty can rep= lace the parent transaction itself with another candidate N satisfying the = replacement rules, triggering the eviction of the malicious HTLC-preimage f= rom the mempool as it was a child of the parent T.

There is no spending candidate of the offered HTLC output for the current = block laying in network mempools.

This replacement= cycling tricks can be repeated for each rebroadcast attempt of the HTLC-ti= meout by the honest lightning node until expiration of the inbound HTLC tim= elock I. Once this height is reached a HTLC-timeout is broadcast by the cou= nterparty's on the incoming link in collusion with the one on the outgo= ing link broadcasting its own HTLC-preimage.

The h= onest Lightning node has been "double-spent" in its HTLC forwardi= ng.

As a notable factor impacting the success of t= he attack, a lightning node's honest HTLC-timeout might be included in = the block template of the miner winning the block race and therefore realiz= es a spent of the offered output. In practice, a replacement cycling attack= might over-connect to miners' mempools and public reachable nodes to s= ucceed in a fast eviction of the HTLC-timeout by its HTLC-preimage. As this= latter transaction can come with a better ancestor-score, it should be pic= ked up on the flight by economically competitive miners.

A functional test exercising a simple replacement cycling of a HTLC = transaction on bitcoin core mempool is available:

## Deployed LN mitigations

Aggre= ssive rebroadcasting: As the replacement cycling attacker benefits from the= HTLC-timeout being usually broadcast by lightning nodes only once every bl= ock, or less the replacement cycling malicious transactions paid only equal= the sum of the absolute fees paid by the HTLC, adjusted with the replaceme= nt penalty. Rebroadcasting randomly and multiple times before the next bloc= k increases the absolute fee cost for the attacker.

Implemented and deployed by Eclair, Core-Lightning, LND and LDK .

Local-mempool preimage monitoring: As the replacement cyc= ling attacker in a simple setup broadcast the HTLC-preimage to all the netw= ork mempools, the honest lightning node is able to catch on the flight the = unconfirmed HTLC-preimage, before its subsequent mempool replacement. The p= reimage can be extracted from the second-stage HTLC-preimage and used to fe= tch the off-chain inbound HTLC with a cooperative message or go on-chain wi= th it to claim the accepted HTLC output.

Implement= ed and deployed by Eclair and LND.

CLTV Expiry= Delta: With every jammed block comes an absolute fee cost paid by the atta= cker, a risk of the HTLC-preimage being detected or discovered by the hones= t lightning node, or the HTLC-timeout to slip in a winning block template. = Bumping the default CLTV delta hardens the odds of success of a simple repl= acement cycling attack.

Default setting: Eclair 14= 4, Core-Lightning 34, LND 80 and LDK 72.

## Affect= ed Bitcoin Protocols and Applications

From my unde= rstanding the following list of Bitcoin protocols and applications could be= affected by new denial-of-service vectors under some level of network memp= ools congestion. Neither tests or advanced review of specifications (when a= vailable) has been conducted for each of them:
- on-chain DLCs
- coinjoins
- payjoins
- wallets with time-sens= itive paths
- peerswap and submarine swaps
- batch payo= uts
- transaction "accelerators"

Inviting their developers, maintainers and operators to investigate how = replacement cycling attacks might disrupt their in-mempool chain of transac= tions, or fee-bumping flows at the shortest delay. Simple flows and non-mul= ti-party transactions should not be affected to the best of my understandin= g.

## Open Problems: Package Malleability

Pinning attacks have been known for years as a practical v= ector to compromise lightning channels funds safety, under different scenar= ios (cf. current bip331's motivation section). Mitigations at the mempo= ol level have been designed, discussed and are under implementation by the = community (ancestor package relay=C2=A0+ nverrsion=3D3 policy). Ideally, th= ey should constraint a pinning attacker to always attach a high feerate pac= kage (commitment=C2=A0+ CPFP) to replace the honest package, or allow a hon= est lightning node to overbid a malicious pinning package and get its time-= sensitive transaction optimistically included in the chain.

<= /div>
Replacement cycling attack seem to offer a new way to neutralize = the design goals of package relay and its companion nversion=3D3 policy, wh= ere an attacker package RBF a honest package out of the mempool to subseque= ntly double-spend its own high-fee child with a transaction unrelated to th= e channel. As the remaining commitment transaction is pre-signed with a min= imal relay fee, it can be evicted out of the mempool.

<= div>A functional test exercising a simple replacement cycling of a lightnin= g channel commitment transaction on top of the nversion=3D3 code branch is = available:

## Discover= y

In 2018, the issue of static fees for pre-signed= lightning transactions is made more widely known, the carve-out exemption = in mempool rules to mitigate in-mempool package limits pinning and the anch= or output pattern are proposed.

In 2019, bitcoin c= ore 0.19 is released with carve-out support. Continued discussion of the an= chor output pattern as a dynamic fee-bumping method.

In 2020, draft of anchor output submitted to the bolts. Initial finding = of economic pinning against lightning commitment and second-stage HTLC tran= sactions. Subsequent discussions of a preimage-overlay network or package-r= elay as mitigations. Public call made to inquiry more on potential other tr= ansaction-relay jamming attacks affecting lightning.

In 2021, initial work in bitcoin core 22.0 of package acceptance. Contin= ued discussion of the pinning attacks and shortcomings of current mempool r= ules during community-wide online workshops. Later the year, in light of al= l issues for bitcoin second-layers, a proposal is made about killing the me= mpool.

In 2022, bip proposed for package relay and= new proposed v3 policy design proposed for a review and implementation. Me= mpoolfullrbf is supported in bitcoin core 24.0 and conceptual questions abo= ut alignment of mempool rules w.r.t miners incentives are investigated.

Along this year 2022, eltoo lightning channels design= are discussed, implemented and reviewed. In this context and after discuss= ions on mempool anti-DoS rules, I discovered this new replacement cycling a= ttack was affecting deployed lightning channels and immediately reported th= e finding to some bitcoin core developers and lightning maintainers.
<= div>
## Timeline

- 2022-12-16: Repor= t of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Z= hao
- 2022-12-16: Report to LN maintainers: Rusty Russell, Bastie= n Teinturier, Matt Corallo and Olaoluwa Osuntunkun
- 2022-12-23: = Sharing to Eugene Siegel (LND)
- 2022-12-24: Sharing to James O&#= 39;Beirne and Antoine Poinsot (non-lightning potential affected projects)
- 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cros= s-layers issuers) and initial proposal of an early public disclosure=C2=A0<= /div>
- 2022-01-19: Collection of analysis if other second-layers and m= ulti-party applications affected. LN mitigations development starts.
<= div>- 2023-05-04: Sharing to Wilmer Paulino (LDK)
- 2023-06-20: L= N mitigations implemented and progressively released. Week of the 16 octobe= r proposed for full disclosure.
- 2023-08-10: CVEs assigned by MI= TRE
- 2023-10-05: Pre-disclosure of LN CVEs numbers and replaceme= nt cycling attack existence to security@bitcoincore.org.
- 2023-10-16: Ful= l disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023= -40234 and replacement cycling attacks

## Conclusi= on=C2=A0

Despite the line of mitigations adopted a= nd deployed by current major lightning implementations, I believe replaceme= nt cycling attacks are still practical for advanced attackers. Beyond this = new attack might come as a way to partially or completely defeat some of th= e pinning mitigations which have been working for years as a community.

As of today, it is uncertain to me if lightning is no= t affected by a more severe long-term package malleability critical securit= y issue under current consensus rules, and if any other time-sensitive mult= i-party protocol, designed or deployed isn't de facto affected too (los= s of funds or denial of service).

Assuming analysi= s on package malleability is correct, it is unclear to me if it can be corr= ected by changes in replacement / eviction rules or mempool chain of transa= ctions processing strategy. Inviting my technical peers and the bitcoin com= munity to look more on this issue, including to dissent. I'll be the fi= rst one pleased if I'm fundamentally wrong on those issues, or if any e= lement has not been weighted with the adequate technical accuracy it deserv= es.

Do not trust, verify. All mistakes and opinion= s are my own.

Antoine

&qu= ot;meet with Triumph and Disaster. And treat those two impostors just the s= ame" - K.
--000000000000c388e406083f8305--