path: root/be/12a18276aad203bce633d5222b3b604b042339

Return-Path: <antoine.riard@gmail.com>
Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137])
 by lists.linuxfoundation.org (Postfix) with ESMTP id AB4C9C000E
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 26 Oct 2021 23:45:00 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp4.osuosl.org (Postfix) with ESMTP id 879F9403E6
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 26 Oct 2021 23:45:00 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level: 
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp4.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp4.osuosl.org ([127.0.0.1])
 by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id guC9uAnaco6X
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 26 Oct 2021 23:44:59 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [IPv6:2a00:1450:4864:20::429])
 by smtp4.osuosl.org (Postfix) with ESMTPS id 22F6740235
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 26 Oct 2021 23:44:59 +0000 (UTC)
Received: by mail-wr1-x429.google.com with SMTP id d3so967169wrh.8
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 26 Oct 2021 16:44:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
 bh=vGm8PaQupt1LhBLMa+SVxKlWGB1tU0IBKUjd2w/Xc3U=;
 b=oTuH2Kb0ldx8jYGQlE5kHQIbz7oTkkvDAFAqqPfSe9gcsZf0ijQp2M1Fyj64yrex+D
 k5L6dTAy/gDUCc7Nd/iHyUMhy8AXWdg/QsXRyWfDrWXujFZTFVrqBIuAAs0R9tZDFxB4
 El1d22kNCNIvlagT40qHzBD5DMnFvORcKBJh7DF0fOjtMm8kLt33R8forYdja10gia+9
 ipIpm44kPvwM3R3VPPcYCqWP89hSPcAByJB6IFQLC1E8GjJG9bjbKrO8iYUNX/vvsBrg
 SJfuX4pXYiSCrstB4OY8qSYysPwVtbG5BrbXOJ47ara0Ii7k6HchbSa16f8aIogey4/A
 nq9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to;
 bh=vGm8PaQupt1LhBLMa+SVxKlWGB1tU0IBKUjd2w/Xc3U=;
 b=EJnlBil7abrcW/1FEuRkcmxHQSnKYNeUCaNAfiXGtvJtz1IftkWmf0DLQLF1bfyyb0
 N/m6//dgu80jEgbKu/8ZcyaGKBrQzL3TMQwvhadV8MAznpJ9ugjjHfkqv9orLwi/LRAK
 ZbQcmsAD5aWalQXQ7q7+t7aRbGSI+psFwn1ms+vlPgv3Upk1l7x7sTjyxGBb4QMNr7C9
 zQ2/hKUQdibOnwPTWJNdMmx2RaiB1aedPFHVkRef5bUJyBhhfrHhJPSd7wyf3gvDtxJD
 0zVhvjYaoEwtYXTOkYViiQlSdYbwFTzuBJagVEMW4l/m2uYKALCBhZB3tVSsWnwvAVSF
 Q1lQ==
X-Gm-Message-State: AOAM533GS9PX6weMvdtonYegC9dalnjX5CWm3IMZsepVU+AImZiDcu2X
 CHGn6C6AzaX9H8QX1T7D8Gs/DY/8kyJk9MKScZNMkJMdkynjqQ==
X-Google-Smtp-Source: ABdhPJxcLWUtA162Ap7REdouitYq1p2yvTmwjFaP/eC+Sqr/w3TYaotoyzeFgQMYi7trXl6fitTyzxy+eOIXgcyBfik=
X-Received: by 2002:a5d:5274:: with SMTP id l20mr9042439wrc.328.1635291897361; 
 Tue, 26 Oct 2021 16:44:57 -0700 (PDT)
MIME-Version: 1.0
References: <CAM1a7P04apCwwmqNp4VLRam5_uk59tVRWv74UVD_g0-DUGNghw@mail.gmail.com>
In-Reply-To: <CAM1a7P04apCwwmqNp4VLRam5_uk59tVRWv74UVD_g0-DUGNghw@mail.gmail.com>
From: Antoine Riard <antoine.riard@gmail.com>
Date: Tue, 26 Oct 2021 19:44:45 -0400
Message-ID: <CALZpt+E_47wmDg3Z2N5=z5x3cca5vSxP6fgYzMK_pybwp1161g@mail.gmail.com>
To: lisa neigut <niftynei@gmail.com>, 
 Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000a577a705cf4a1065"
X-Mailman-Approved-At: Wed, 27 Oct 2021 07:37:33 +0000
Subject: Re: [bitcoin-dev] death to the mempool, long live the mempool
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Oct 2021 23:45:00 -0000

--000000000000a577a705cf4a1065
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Lisa,

Network mempools constitute a blockspace marketplace where block demand
meets the offer in real-time. Block producers are acting to discover the
best feerate bids compensating for their operational costs and transaction
proposers are acting to offer the best feerate in function of their
confirmation preferences.

Of course in a distributed system like bitcoin, we can't guarantee perfect
information from the market participants. But moving away from this model
by decreasing the ability of the non-mining nodes to observe the current
demand is softening the requirements for potential attackers.

As transaction proposers are competing with each other to publish, they
have an interest to "front-run" each other by querying the pending
transactions to the block producers instead of observing only the published
blocks. Therefore good connections to
the block producers are now critical and censorship-resistance of the
mining endpoints must be guaranteed.

Such a list of endpoints couldn't be static otherwise it's an artificial
barrier to enter in the mining competition, and as such a centralization
vector. Dynamic, trust-minimized discovery of the mining endpoints assumes
an address-relay network, of which the robustness must be high enough
against sophisticated sybil attacks. One current defense mechanism in core
to achieve that is selecting outbound peers based in different /16 subnets
as it's harder for an attacker to obtain IP addresses. Replicating this
mechanism for the mining endpoints binds the mining topology to the
Internet one, which is downgrading the mining competition.

Relying on tor to guarantee the confidentiality of the transaction
announcement is raising its own issues. Flowing by default all the bitcoin
traffic over tor will change the incentive structure of tor attackers,
potentially attracting a new class of attackers able to do deanonymization
attacks, not that expensive in practice [0]. Tor bridges are another
censorship vector as the fingerprint of the bitcoin traffic (a block every
10 min, etc) make it possible to drop or delay the tor channel, in the lack
of high-bandwidth consuming "synthetic" traffic.

Further, identified mining endpoints make it easier to launch partition
attacks, where mining mempools are sent low-feerate clusters of
transactions, to prevent the replacement by a better feerate offer. This is
especially concerning for L2 nodes with time-sensitive requirements [1]

Lastly, removing the mempool won't solve the current issues inherent with
pre-signed transactions under the mempool min fee as ultimately miner's
mempools are also finite in memory and a dynamic lower bound must exist to
prevent spam. These lower bounds potentially increase after the signature
exchange of the time-sensitive transactions.

Antoine

[0] https://www.usenix.org/system/files/sec19-jansen.pdf
[1] See "The Ugly"
https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002758.=
html

Le mar. 26 oct. 2021 =C3=A0 03:37, lisa neigut via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> a =C3=A9crit :

> Hi all,
>
> In a recent conversation with @glozow, I had the realization that the
> mempool is obsolete and should be eliminated.
>
> Instead, users should submit their transactions directly to mining pools,
> preferably over an anonymous communication network such as tor. This can
> easily be achieved by mining pools running a tor onion node for this
> express purpose (or via a lightning network extension etc)
>
> Mempools make sense in a world where mining is done by a large number of
> participating nodes, eg where the block template is constructed by a
> majority of the participants on the network. In this case, it is necessar=
y
> to socialize pending transaction data to all participants, as you don=E2=
=80=99t
> know which participant will be constructing the winning block template.
>
> In reality however, mempool relay is unnecessary where the majority of
> hashpower and thus block template creation is concentrated in a
> semi-restricted set.
>
> Removing the mempool would greatly reduce the bandwidth requirement for
> running a node, keep intentionality of transactions private until
> confirmed/irrevocable, and naturally resolve all current issues inherent =
in
> package relay and rbf rules. It also resolves the recent minimum relay
> questions, as relay is no longer a concern for unmined transactions.
>
> Provided the number of block template producing actors remains beneath,
> say 1000, it=E2=80=99d be quite feasible to publish a list of tor endpoin=
ts that
> nodes can independently  + directly submit their transactions to. In fact=
,
> merely allowing users to select their own list of endpoints to use
> alternatively to the mempool would be a low effort starting point for the
> eventual replacement.
>
> On the other hand, removing the mempool would greatly complicate solo
> mining and would also make BetterHash proposals, which move the block
> template construction away from a centralized mining pool back to the
> individual miner, much more difficult. It also makes explicit the target
> for DoS attacks.
>
> A direct communication channel between block template construction venues
> and transaction proposers also provides a venue for direct feedback wrt
> acceptable feerates at the time, which both makes transaction confirmatio=
n
> timelines less variable as well as provides block producers a mechanism f=
or
> (independently) enforcing their own minimum security budget. In other
> words, expressing a minimum acceptable feerate for continued operation.
>
> Initial feerate estimation would need to be based on published blocks, no=
t
> pending transactions (as this information would no longer be available), =
or
> from direct interactions with block producers.
>
>
> ~niftynei
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--000000000000a577a705cf4a1065
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi Lisa,<br><br>Network mempools constitute a blocksp=
ace marketplace where block demand meets the offer in real-time. Block prod=
ucers are acting to discover the best feerate bids compensating for their o=
perational costs and transaction proposers are acting to offer the best fee=
rate in function of their confirmation preferences.<br><br>Of course in a d=
istributed system like bitcoin, we can&#39;t guarantee perfect information =
from the market participants. But moving away from this model by decreasing=
 the ability of the non-mining nodes to observe the current demand is softe=
ning the requirements for potential attackers.<br><br>As transaction propos=
ers are competing with each other to publish, they have an interest to &quo=
t;front-run&quot; each other by querying the pending transactions to the bl=
ock producers instead of observing only the published blocks. Therefore goo=
d connections to<br>the block producers are now critical and censorship-res=
istance of the mining endpoints must be guaranteed.<br><br>Such a list of e=
ndpoints couldn&#39;t be static otherwise it&#39;s an artificial barrier to=
 enter in the mining competition, and as such a centralization vector. Dyna=
mic, trust-minimized discovery of the mining endpoints assumes an address-r=
elay network, of which the robustness must be high enough against sophistic=
ated sybil attacks. One current defense mechanism in core to achieve that i=
s selecting outbound peers based in different /16 subnets as it&#39;s harde=
r for an attacker to obtain IP addresses. Replicating this mechanism for th=
e mining endpoints binds the mining topology to the Internet one, which is =
downgrading the mining competition.<br><br>Relying on tor to guarantee the =
confidentiality of the transaction announcement is raising its own issues. =
Flowing by default all the bitcoin traffic over tor will change the incenti=
ve structure of tor attackers, potentially attracting a new class of attack=
ers able to do deanonymization attacks, not that expensive in practice [0].=
 Tor bridges are another censorship vector as the fingerprint of the bitcoi=
n traffic (a block every 10 min, etc) make it possible to drop or delay the=
 tor channel, in the lack of high-bandwidth consuming &quot;synthetic&quot;=
 traffic.<br><br>Further, identified mining endpoints make it easier to lau=
nch partition attacks, where mining mempools are sent low-feerate clusters =
of transactions, to prevent the replacement by a better feerate offer. This=
 is especially concerning for L2 nodes with time-sensitive requirements [1]=
<br><br>Lastly, removing the mempool won&#39;t solve the current issues inh=
erent with pre-signed transactions under the mempool min fee as ultimately =
miner&#39;s mempools are also finite in memory and a dynamic lower bound mu=
st exist to prevent spam. These lower bounds potentially increase after the=
 signature exchange of the time-sensitive transactions.<br><br></div>Antoin=
e<br><div><br>[0] <a href=3D"https://www.usenix.org/system/files/sec19-jans=
en.pdf">https://www.usenix.org/system/files/sec19-jansen.pdf</a><br>[1] See=
 &quot;The Ugly&quot; <a href=3D"https://lists.linuxfoundation.org/pipermai=
l/lightning-dev/2020-June/002758.html">https://lists.linuxfoundation.org/pi=
permail/lightning-dev/2020-June/002758.html</a><br></div></div><br><div cla=
ss=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Le=C2=A0mar. 26 oc=
t. 2021 =C3=A0=C2=A003:37, lisa neigut via bitcoin-dev &lt;<a href=3D"mailt=
o:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfoundation.=
org</a>&gt; a =C3=A9crit=C2=A0:<br></div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa=
dding-left:1ex"><div><div dir=3D"auto">Hi all,</div><div dir=3D"auto"><br><=
/div><div dir=3D"auto">In a recent conversation with @glozow, I had the rea=
lization that the mempool is obsolete and should be eliminated.</div><div d=
ir=3D"auto"><br></div><div dir=3D"auto">Instead, users should submit their =
transactions directly to mining pools, preferably over an anonymous communi=
cation network such as tor. This can easily be achieved by mining pools run=
ning a tor onion node for this express purpose (or via a lightning network =
extension etc)</div><div dir=3D"auto"><br></div><div dir=3D"auto">Mempools =
make sense in a world where mining is done by a large number of participati=
ng nodes, eg where the block template is constructed by a majority of the p=
articipants on the network. In this case, it is necessary to socialize pend=
ing transaction data to all participants, as you don=E2=80=99t know which p=
articipant will be constructing the winning block template.</div><div dir=
=3D"auto"><br></div><div dir=3D"auto">In reality however, mempool relay is =
unnecessary where the majority of hashpower and thus block template creatio=
n is concentrated in a semi-restricted set.=C2=A0</div><div dir=3D"auto"><b=
r></div><div dir=3D"auto">Removing the mempool would greatly reduce the ban=
dwidth requirement for running a node, keep intentionality of transactions =
private until confirmed/irrevocable, and naturally resolve all current issu=
es inherent in package relay and rbf rules. It also resolves the recent min=
imum relay questions, as relay is no longer a concern for unmined transacti=
ons.</div><div dir=3D"auto"><br></div><div dir=3D"auto">Provided the number=
 of block template producing actors remains beneath, say 1000, it=E2=80=99d=
 be quite feasible to publish a list of tor endpoints that nodes can indepe=
ndently =C2=A0+ directly submit their transactions to. In fact, merely allo=
wing users to select their own list of endpoints to use alternatively to th=
e mempool would be a low effort starting point for the eventual replacement=
.</div><div dir=3D"auto"><br></div><div dir=3D"auto">On the other hand, rem=
oving the mempool would greatly complicate solo mining and would also make =
BetterHash proposals, which move the block template construction away from =
a centralized mining pool back to the individual miner, much more difficult=
. It also makes explicit the target for DoS attacks.</div><div dir=3D"auto"=
><br></div><div dir=3D"auto">A direct communication channel between block t=
emplate construction venues and transaction proposers also provides a venue=
 for direct feedback wrt acceptable feerates at the time, which both makes =
transaction confirmation timelines less variable as well as provides block =
producers a mechanism for (independently) enforcing their own minimum secur=
ity budget. In other words, expressing a minimum acceptable feerate for con=
tinued operation.</div><div dir=3D"auto"><br></div><div dir=3D"auto">Initia=
l feerate estimation would need to be based on published blocks, not pendin=
g transactions (as this information would no longer be available), or from =
direct interactions with block producers.</div><div dir=3D"auto"><br></div>=
<div dir=3D"auto"><br></div><div dir=3D"auto">~niftynei</div>
</div>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div>

--000000000000a577a705cf4a1065--