Return-Path: Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id AB4C9C000E for ; Tue, 26 Oct 2021 23:45:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 879F9403E6 for ; Tue, 26 Oct 2021 23:45:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id guC9uAnaco6X for ; Tue, 26 Oct 2021 23:44:59 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by smtp4.osuosl.org (Postfix) with ESMTPS id 22F6740235 for ; Tue, 26 Oct 2021 23:44:59 +0000 (UTC) Received: by mail-wr1-x429.google.com with SMTP id d3so967169wrh.8 for ; Tue, 26 Oct 2021 16:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=vGm8PaQupt1LhBLMa+SVxKlWGB1tU0IBKUjd2w/Xc3U=; b=oTuH2Kb0ldx8jYGQlE5kHQIbz7oTkkvDAFAqqPfSe9gcsZf0ijQp2M1Fyj64yrex+D k5L6dTAy/gDUCc7Nd/iHyUMhy8AXWdg/QsXRyWfDrWXujFZTFVrqBIuAAs0R9tZDFxB4 El1d22kNCNIvlagT40qHzBD5DMnFvORcKBJh7DF0fOjtMm8kLt33R8forYdja10gia+9 ipIpm44kPvwM3R3VPPcYCqWP89hSPcAByJB6IFQLC1E8GjJG9bjbKrO8iYUNX/vvsBrg SJfuX4pXYiSCrstB4OY8qSYysPwVtbG5BrbXOJ47ara0Ii7k6HchbSa16f8aIogey4/A nq9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=vGm8PaQupt1LhBLMa+SVxKlWGB1tU0IBKUjd2w/Xc3U=; b=EJnlBil7abrcW/1FEuRkcmxHQSnKYNeUCaNAfiXGtvJtz1IftkWmf0DLQLF1bfyyb0 N/m6//dgu80jEgbKu/8ZcyaGKBrQzL3TMQwvhadV8MAznpJ9ugjjHfkqv9orLwi/LRAK ZbQcmsAD5aWalQXQ7q7+t7aRbGSI+psFwn1ms+vlPgv3Upk1l7x7sTjyxGBb4QMNr7C9 zQ2/hKUQdibOnwPTWJNdMmx2RaiB1aedPFHVkRef5bUJyBhhfrHhJPSd7wyf3gvDtxJD 0zVhvjYaoEwtYXTOkYViiQlSdYbwFTzuBJagVEMW4l/m2uYKALCBhZB3tVSsWnwvAVSF Q1lQ== X-Gm-Message-State: AOAM533GS9PX6weMvdtonYegC9dalnjX5CWm3IMZsepVU+AImZiDcu2X CHGn6C6AzaX9H8QX1T7D8Gs/DY/8kyJk9MKScZNMkJMdkynjqQ== X-Google-Smtp-Source: ABdhPJxcLWUtA162Ap7REdouitYq1p2yvTmwjFaP/eC+Sqr/w3TYaotoyzeFgQMYi7trXl6fitTyzxy+eOIXgcyBfik= X-Received: by 2002:a5d:5274:: with SMTP id l20mr9042439wrc.328.1635291897361; Tue, 26 Oct 2021 16:44:57 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Antoine Riard Date: Tue, 26 Oct 2021 19:44:45 -0400 Message-ID: To: lisa neigut , Bitcoin Protocol Discussion Content-Type: multipart/alternative; boundary="000000000000a577a705cf4a1065" X-Mailman-Approved-At: Wed, 27 Oct 2021 07:37:33 +0000 Subject: Re: [bitcoin-dev] death to the mempool, long live the mempool X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Oct 2021 23:45:00 -0000 --000000000000a577a705cf4a1065 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Lisa, Network mempools constitute a blockspace marketplace where block demand meets the offer in real-time. Block producers are acting to discover the best feerate bids compensating for their operational costs and transaction proposers are acting to offer the best feerate in function of their confirmation preferences. Of course in a distributed system like bitcoin, we can't guarantee perfect information from the market participants. But moving away from this model by decreasing the ability of the non-mining nodes to observe the current demand is softening the requirements for potential attackers. As transaction proposers are competing with each other to publish, they have an interest to "front-run" each other by querying the pending transactions to the block producers instead of observing only the published blocks. Therefore good connections to the block producers are now critical and censorship-resistance of the mining endpoints must be guaranteed. Such a list of endpoints couldn't be static otherwise it's an artificial barrier to enter in the mining competition, and as such a centralization vector. Dynamic, trust-minimized discovery of the mining endpoints assumes an address-relay network, of which the robustness must be high enough against sophisticated sybil attacks. One current defense mechanism in core to achieve that is selecting outbound peers based in different /16 subnets as it's harder for an attacker to obtain IP addresses. Replicating this mechanism for the mining endpoints binds the mining topology to the Internet one, which is downgrading the mining competition. Relying on tor to guarantee the confidentiality of the transaction announcement is raising its own issues. Flowing by default all the bitcoin traffic over tor will change the incentive structure of tor attackers, potentially attracting a new class of attackers able to do deanonymization attacks, not that expensive in practice [0]. Tor bridges are another censorship vector as the fingerprint of the bitcoin traffic (a block every 10 min, etc) make it possible to drop or delay the tor channel, in the lack of high-bandwidth consuming "synthetic" traffic. Further, identified mining endpoints make it easier to launch partition attacks, where mining mempools are sent low-feerate clusters of transactions, to prevent the replacement by a better feerate offer. This is especially concerning for L2 nodes with time-sensitive requirements [1] Lastly, removing the mempool won't solve the current issues inherent with pre-signed transactions under the mempool min fee as ultimately miner's mempools are also finite in memory and a dynamic lower bound must exist to prevent spam. These lower bounds potentially increase after the signature exchange of the time-sensitive transactions. Antoine [0] https://www.usenix.org/system/files/sec19-jansen.pdf [1] See "The Ugly" https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002758.= html Le mar. 26 oct. 2021 =C3=A0 03:37, lisa neigut via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> a =C3=A9crit : > Hi all, > > In a recent conversation with @glozow, I had the realization that the > mempool is obsolete and should be eliminated. > > Instead, users should submit their transactions directly to mining pools, > preferably over an anonymous communication network such as tor. This can > easily be achieved by mining pools running a tor onion node for this > express purpose (or via a lightning network extension etc) > > Mempools make sense in a world where mining is done by a large number of > participating nodes, eg where the block template is constructed by a > majority of the participants on the network. In this case, it is necessar= y > to socialize pending transaction data to all participants, as you don=E2= =80=99t > know which participant will be constructing the winning block template. > > In reality however, mempool relay is unnecessary where the majority of > hashpower and thus block template creation is concentrated in a > semi-restricted set. > > Removing the mempool would greatly reduce the bandwidth requirement for > running a node, keep intentionality of transactions private until > confirmed/irrevocable, and naturally resolve all current issues inherent = in > package relay and rbf rules. It also resolves the recent minimum relay > questions, as relay is no longer a concern for unmined transactions. > > Provided the number of block template producing actors remains beneath, > say 1000, it=E2=80=99d be quite feasible to publish a list of tor endpoin= ts that > nodes can independently + directly submit their transactions to. In fact= , > merely allowing users to select their own list of endpoints to use > alternatively to the mempool would be a low effort starting point for the > eventual replacement. > > On the other hand, removing the mempool would greatly complicate solo > mining and would also make BetterHash proposals, which move the block > template construction away from a centralized mining pool back to the > individual miner, much more difficult. It also makes explicit the target > for DoS attacks. > > A direct communication channel between block template construction venues > and transaction proposers also provides a venue for direct feedback wrt > acceptable feerates at the time, which both makes transaction confirmatio= n > timelines less variable as well as provides block producers a mechanism f= or > (independently) enforcing their own minimum security budget. In other > words, expressing a minimum acceptable feerate for continued operation. > > Initial feerate estimation would need to be based on published blocks, no= t > pending transactions (as this information would no longer be available), = or > from direct interactions with block producers. > > > ~niftynei > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --000000000000a577a705cf4a1065 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Lisa,

Network mempools constitute a blocksp= ace marketplace where block demand meets the offer in real-time. Block prod= ucers are acting to discover the best feerate bids compensating for their o= perational costs and transaction proposers are acting to offer the best fee= rate in function of their confirmation preferences.

Of course in a d= istributed system like bitcoin, we can't guarantee perfect information = from the market participants. But moving away from this model by decreasing= the ability of the non-mining nodes to observe the current demand is softe= ning the requirements for potential attackers.

As transaction propos= ers are competing with each other to publish, they have an interest to &quo= t;front-run" each other by querying the pending transactions to the bl= ock producers instead of observing only the published blocks. Therefore goo= d connections to
the block producers are now critical and censorship-res= istance of the mining endpoints must be guaranteed.

Such a list of e= ndpoints couldn't be static otherwise it's an artificial barrier to= enter in the mining competition, and as such a centralization vector. Dyna= mic, trust-minimized discovery of the mining endpoints assumes an address-r= elay network, of which the robustness must be high enough against sophistic= ated sybil attacks. One current defense mechanism in core to achieve that i= s selecting outbound peers based in different /16 subnets as it's harde= r for an attacker to obtain IP addresses. Replicating this mechanism for th= e mining endpoints binds the mining topology to the Internet one, which is = downgrading the mining competition.

Relying on tor to guarantee the = confidentiality of the transaction announcement is raising its own issues. = Flowing by default all the bitcoin traffic over tor will change the incenti= ve structure of tor attackers, potentially attracting a new class of attack= ers able to do deanonymization attacks, not that expensive in practice [0].= Tor bridges are another censorship vector as the fingerprint of the bitcoi= n traffic (a block every 10 min, etc) make it possible to drop or delay the= tor channel, in the lack of high-bandwidth consuming "synthetic"= traffic.

Further, identified mining endpoints make it easier to lau= nch partition attacks, where mining mempools are sent low-feerate clusters = of transactions, to prevent the replacement by a better feerate offer. This= is especially concerning for L2 nodes with time-sensitive requirements [1]=

Lastly, removing the mempool won't solve the current issues inh= erent with pre-signed transactions under the mempool min fee as ultimately = miner's mempools are also finite in memory and a dynamic lower bound mu= st exist to prevent spam. These lower bounds potentially increase after the= signature exchange of the time-sensitive transactions.

Antoin= e

Le=C2=A0mar. 26 oc= t. 2021 =C3=A0=C2=A003:37, lisa neigut via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.= org> a =C3=A9crit=C2=A0:
Hi all,

<= /div>
In a recent conversation with @glozow, I had the rea= lization that the mempool is obsolete and should be eliminated.

Instead, users should submit their = transactions directly to mining pools, preferably over an anonymous communi= cation network such as tor. This can easily be achieved by mining pools run= ning a tor onion node for this express purpose (or via a lightning network = extension etc)

Mempools = make sense in a world where mining is done by a large number of participati= ng nodes, eg where the block template is constructed by a majority of the p= articipants on the network. In this case, it is necessary to socialize pend= ing transaction data to all participants, as you don=E2=80=99t know which p= articipant will be constructing the winning block template.

In reality however, mempool relay is = unnecessary where the majority of hashpower and thus block template creatio= n is concentrated in a semi-restricted set.=C2=A0
Removing the mempool would greatly reduce the ban= dwidth requirement for running a node, keep intentionality of transactions = private until confirmed/irrevocable, and naturally resolve all current issu= es inherent in package relay and rbf rules. It also resolves the recent min= imum relay questions, as relay is no longer a concern for unmined transacti= ons.

Provided the number= of block template producing actors remains beneath, say 1000, it=E2=80=99d= be quite feasible to publish a list of tor endpoints that nodes can indepe= ndently =C2=A0+ directly submit their transactions to. In fact, merely allo= wing users to select their own list of endpoints to use alternatively to th= e mempool would be a low effort starting point for the eventual replacement= .

On the other hand, rem= oving the mempool would greatly complicate solo mining and would also make = BetterHash proposals, which move the block template construction away from = a centralized mining pool back to the individual miner, much more difficult= . It also makes explicit the target for DoS attacks.

A direct communication channel between block t= emplate construction venues and transaction proposers also provides a venue= for direct feedback wrt acceptable feerates at the time, which both makes = transaction confirmation timelines less variable as well as provides block = producers a mechanism for (independently) enforcing their own minimum secur= ity budget. In other words, expressing a minimum acceptable feerate for con= tinued operation.

Initia= l feerate estimation would need to be based on published blocks, not pendin= g transactions (as this information would no longer be available), or from = direct interactions with block producers.

=

~niftynei
_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev
--000000000000a577a705cf4a1065--