MIME-Version: 1.0
References: <CALZpt+GBPbf+Pgctm5NViNons50aQs1RPQkEo3FW5RM4fL9ztA@mail.gmail.com>
In-Reply-To: <CALZpt+GBPbf+Pgctm5NViNons50aQs1RPQkEo3FW5RM4fL9ztA@mail.gmail.com>
From: Igor Cota <igor@codexapertus.com>
Date: Thu, 7 May 2020 18:40:49 +0200
Message-ID: <CAJx8jdwHF3wF+q+JafuDhMtca45Wb2vS_gvB0yw71jNGz=ZHzA@mail.gmail.com>
To: Antoine Riard <antoine.riard@gmail.com>
Content-Type: multipart/alternative; boundary="000000000000be348b05a5118bb8"
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>,
 "lightning-dev\\\\@lists.linuxfoundation.org"
 <lightning-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of
 onboarding millions of LN mobile clients
Precedence: list

--000000000000be348b05a5118bb8
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Antoine et al,

Maybe I'm completely wrong, missing some numbers, and it's maybe fine to
> just rely on few thousands of full-node operators being nice and servicin=
g
> friendly millions of LN mobiles clients. But just in case it may be good =
to
> consider a reasonable alternative.
>


> So you may want to separate control/data plane, get filters from CDN and
> headers as check-and-control directly from the backbone network. "Hybrid"
> models should clearly be explored.


For some months now I've been exploring the feasibility of running full
nodes on everyday phones [1]. One of my first thoughts was how to avoid the
phones mooching off the network. Obviously due to battery, storage and
bandwidth constraints it is not reasonable to expect pocket full nodes to
serve blocks during day time.

Huge exception to this is the time we are asleep and our phones are
connected to wifi and charging. IMO this is a huge untapped resource that
would allow mobile nodes to earn their keep. If we limit full node
operation to sleepy night time the only constraining resource is storage:
512 gb of internal storage in phones is quite rare, probably about $100 for
an SD card with full archival node capacity but phones with memory card
slots rarer still - no one is going to bother.

So depending on their storage capacity phone nodes could decide to store
and serve just a randomly selected range of blocks during their nighttime
operation. With trivial changes to P2P they could advertise the blocks they
are able to serve.
If there comes a time that normal full nodes feel DoS'ed they can challenge
such nodes to produce the blocks they advertise and ban them as moochers if
they fail to do so. Others may elect to be more charitable and serve
everyone.

These types of nodes would truly be part-timing since they only carry a
subset of the blockchain and work while their operator is asleep. Probably
should be called part-time or Sleeper Nodes=E2=84=A2.

They could be user friendly as well, with Assume UTXO they could be
bootstrapped quickly and while they do the IBD in the background instead of
traditional pruning they can keep the randomly assigned bit of blockchain
to later serve the network.

Save for the elderly, all the people I know could run such a node, and I
don't live in a first world country.

There is also the feel-good kumbaya aspect of American phone nodes serving
the African continent while the Americans are asleep, Africans and
Europeans serving the Asians in kind. By plugging in our phones and going
to sleep we could blanket the whole world in (somewhat) full nodes!

Cheers,
Igor

[1] https://icota.github.io/

On Tue, 5 May 2020 at 12:18, Antoine Riard <antoine.riard@gmail.com> wrote:

> Hi,
>
> (cross-posting as it's really both layers concerned)
>
> Ongoing advancement of BIP 157 implementation in Core maybe the
> opportunity to reflect on the future of light client protocols and use th=
is
> knowledge to make better-informed decisions about what kind of
> infrastructure is needed to support mobile clients at large scale.
>
> Trust-minimization of Bitcoin security model has always relied first and
> above on running a full-node. This current paradigm may be shifted by LN
> where fast, affordable, confidential, censorship-resistant payment servic=
es
> may attract a lot of adoption without users running a full-node. Assuming=
 a
> user adoption path where a full-node is required to benefit for LN may
> deprive a lot of users, especially those who are already denied a real
> financial infrastructure access. It doesn't mean we shouldn't foster node
> adoption when people are able to do so, and having a LN wallet maybe even=
 a
> first-step to it.
>
> Designing a mobile-first LN experience opens its own gap of challenges
> especially in terms of security and privacy. The problem can be scoped as
> how to build a scalable, secure, private chain access backend for million=
s
> of LN clients ?
>
> Light client protocols for LN exist (either BIP157 or Electrum are used),
> although their privacy and security guarantees with regards to
> implementation on the client-side may still be an object of concern
> (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted f=
ee
> estimation). That said, one of the bottlenecks is likely the number of
> full-nodes being willingly to dedicate resources to serve those clients.
> It's not about _which_ protocol is deployed but more about _incentives_ f=
or
> node operators to dedicate long-term resources to client they have lower
> reasons to care about otherwise.
>
> Even with cheaper, more efficient protocols like BIP 157, you may have a
> huge discrepancy between what is asked and what is offered. Assuming 10M
> light clients [0] each of them consuming ~100MB/month for filters/headers=
,
> that means you're asking 1PB/month of traffic to the backbone network. If
> you assume 10K public nodes, like today, assuming _all_ of them opt-in to
> signal BIP 157, that's an increase of 100GB/month for each. Which is
> consequent with regards to the estimated cost of 350GB/month for running =
an
> actual public node. Widening full-node adoption, specially in term of
> geographic distribution means as much as we can to bound its operational
> cost.
>
> Obviously,  deployment of more efficient tx-relay protocol like Erlay wil=
l
> free up some resources but it maybe wiser to dedicate them to increase
> health and security of the backbone network like deploying more outbound
> connections.
>
> Unless your light client protocol is so ridiculous cheap to rely on
> niceness of a subset of node operators offering free resources, it won't
> scale. And it's likely you will always have a ratio disequilibrium betwee=
n
> numbers of clients and numbers of full-node, even worst their growth rate
> won't be the same, first ones are so much easier to setup.
>
> It doesn't mean servicing filters for free won't work for now, numbers of
> BIP157 clients is still pretty low, but what is worrying is  wallet vendo=
rs
> building such chain access backend, hitting a bandwidth scalability wall
> few years from now instead of pursuing better solutions. And if this
> happen, maybe suddenly, isn't the quick fix going to be to rely on
> centralized services, so much easier to deploy ?
>
> Of course, it may be brought that actually current full-node operators
> don't get anything back from servicing blocks, transactions, addresses...
> It may be replied that you have an indirect incentive to participate in
> network relay and therefore guarantee censorship-resistance, instead of
> directly connecting to miners. You do have today ways to select your
> resources exposure like pruning, block-only or being private but the wide=
r
> point is the current (non?)-incentives model seems to work for the base
> layer. For light clients data, are node operators going to be satisfied t=
o
> serve this new *class* of traffic en masse ?
>
> This doesn't mean you won't find BIP157 servers, ready to serve you with
> unlimited credit, but it's more likely their intentions maybe not aligned=
,
> like spying on your transaction broadcast or block fetched. And you do wa=
nt
> peer diversity to avoid every BIP157 servers being on few ASNs for
> fault-tolerance. Do people expect a scenario a la Cloudflare, where
> everyone connections is to far or less the same set of entities ?
>
> Moreover, the LN security model diverges hugely from basic on-chain
> transactions. Worst-case attack on-chain a malicious light client server
> showing a longest, invalid, PoW-signed chain to double-spend the user. On
> LN, the *liveliness* requirement means the entity owning your view of the
> chain can lie to you on whether your channel has been spent by a revoked
> commitment, the real tip of the blockchain or even dry-up block
> announcement to trigger unexpected behavior in the client logic. A
> malicious light client server may just drop any filters/utxos spends, wha=
t
> your LN client should do in this case ? [1]
>
> Therefore, you may want to introduce monetary compensation in exchange of
> servicing filters. Light client not dedicating resources to maintain the
> network but free-riding on it, you may use their micro-payment capabiliti=
es
> to price chain access resources [3]. This proposition may suit within the
> watchtower paradigm, where another entity is delegated some part of
> protocol execution, alleviating client onliness requirement. It needs
> further analysis but how your funds may be compromised by a watchtower ar=
e
> likely to be the same scenario that how a chain validation provider can
> compromise you. That said, how do you avoid such "chain access" market
> turning as an oligopoly is an open question. You may "bind" them to
> internet topology or ask for fidelity bonds and create some kind of
> scarcity but still...
>
> Maybe I'm completely wrong, missing some numbers, and it's maybe fine to
> just rely on few thousands of full-node operators being nice and servicin=
g
> friendly millions of LN mobiles clients. But just in case it may be good =
to
> consider a reasonable alternative.
>
> Thanks Gleb for many points exposed here but all mistakes are my own.
>
> Cheers,
>
> Antoine
>
> [0] UTXO set size may be a bottleneck, but still if you have 2 channels b=
y
> clients that's 20M utxos, just roughly ~x3 than today.
>
> [1] And committing filters as part of headers may not solve everything as
> an attacker can just delay or slow announcements to you, so you still nee=
d
> network access to at least one honest node.
>
> [2]  It maybe argue that distinction client-vs-peer doesn't hold because
> you may start as a client and start synchronizing the chain, relaying
> blocks, etc. AFAIK, there is no such hybrid implementation and that's not
> what you want to run in a mobile.
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>


--=20
*Igor Cota*
Codex Apertus Ltd

--000000000000be348b05a5118bb8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi Antoine et al,</div><div><br></div><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex">Maybe I&#39;m completely wrong, missing =
some numbers, and it&#39;s maybe fine to just rely on few thousands of full=
-node operators being nice and servicing friendly millions of LN mobiles cl=
ients. But just in case it may be good to consider a reasonable alternative=
.<br></blockquote><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd=
ing-left:1ex">So you may want to separate control/data plane, get filters f=
rom CDN and headers as check-and-control directly from the backbone network=
. &quot;Hybrid&quot; models should clearly be explored.</blockquote><div></=
div><div><u><br></u></div><div><u></u></div><div>For some months now I&#39;=
ve been exploring the feasibility of running full nodes on everyday phones =
[1]. One of my first thoughts was how to avoid the phones mooching off the =
network. Obviously due to battery, storage and bandwidth constraints it is =
not reasonable to expect pocket full nodes to serve blocks during day time.=
</div><div><br></div><div></div><div>Huge exception to this is the time we =
are asleep and our phones are connected to wifi and charging. IMO this is a=
 huge untapped resource that would allow mobile nodes to earn their keep. I=
f we limit full node operation to sleepy night time the only constraining r=
esource is storage: 512 gb of internal storage in phones is quite rare, pro=
bably about $100 for an SD card with full archival node capacity but phones=
 with memory card slots rarer still - no one is going to bother.</div><div>=
<br></div><div></div><div>So depending on their storage capacity phone node=
s could decide to store and serve just a randomly selected range of blocks =
during their nighttime operation. With trivial changes to P2P they could ad=
vertise the blocks they are able to serve. </div><div></div><div>If there c=
omes a time that normal full nodes feel DoS&#39;ed they can challenge such =
nodes to produce the blocks they advertise and ban them as moochers if they=
 fail to do so. Others may elect to be more charitable and serve everyone.<=
/div><div><br></div><div></div><div>These types of nodes would truly be par=
t-timing since they only carry a subset of the blockchain and work while th=
eir operator is asleep. Probably should be called part-time or Sleeper Node=
s<span style=3D"color:rgb(32,33,34)">=E2=84=A2.</span></div><div><span styl=
e=3D"color:rgb(32,33,34)"><br></span></div><div><span style=3D"color:rgb(32=
,33,34)"></span></div><div><span style=3D"color:rgb(32,33,34)">They could b=
e user friendly as well, with Assume UTXO they could be bootstrapped quickl=
y and while they do the IBD in the background instead of traditional prunin=
g they can keep the randomly assigned bit of blockchain to later serve the =
network.</span></div><div><span style=3D"color:rgb(32,33,34)"><br></span></=
div><div><span style=3D"color:rgb(32,33,34)"></span></div><div><span style=
=3D"color:rgb(32,33,34)">Save for the elderly, all the people I know could =
run such a node, and I don&#39;t live in a first world country.</span></div=
><div><span style=3D"color:rgb(32,33,34)"><br></span></div><div><span style=
=3D"color:rgb(32,33,34)"></span></div><div><span style=3D"color:rgb(32,33,3=
4)">There is also the feel-good kumbaya aspect of American phone nodes serv=
ing the African continent while the Americans are asleep, Africans and Euro=
peans serving the Asians in kind. By plugging in our phones and going to sl=
eep we could blanket the whole world in (somewhat) full nodes!</span></div>=
<div><span style=3D"color:rgb(32,33,34)"><br></span></div><div><span style=
=3D"color:rgb(32,33,34)"></span></div><div><span style=3D"color:rgb(32,33,3=
4)">Cheers,</span></div><div><span style=3D"color:rgb(32,33,34)">Igor</span=
></div><div><span style=3D"color:rgb(32,33,34)"><br></span></div><div><span=
 style=3D"color:rgb(32,33,34)"></span></div><div><span style=3D"color:rgb(3=
2,33,34)">[1] </span><a href=3D"https://icota.github.io/" rev=3D"en_rl_none=
" target=3D"_blank">https://icota.github.io/</a></div></div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, 5 May 2020 a=
t 12:18, Antoine Riard &lt;<a href=3D"mailto:antoine.riard@gmail.com" targe=
t=3D"_blank">antoine.riard@gmail.com</a>&gt; wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Hi,</div><div><br=
></div><div>(cross-posting as it&#39;s really both layers concerned)<br></d=
iv><div><div><br>Ongoing advancement of BIP 157 implementation in Core mayb=
e the opportunity to reflect on the future of light client protocols and us=
e this knowledge to make better-informed decisions about what kind of infra=
structure is needed to support mobile clients at large scale.<br><br></div>=
<div>Trust-minimization of Bitcoin security model has always relied first a=
nd above on running a full-node. This current paradigm may be shifted by LN=
 where fast, affordable, confidential, censorship-resistant payment service=
s may attract a lot of adoption without users running a full-node. Assuming=
 a user adoption path where a full-node is required to benefit for LN may d=
eprive a lot of users, especially those who are already denied a real finan=
cial infrastructure access. It doesn&#39;t mean we shouldn&#39;t foster nod=
e adoption when people are able to do so, and having a LN wallet maybe even=
 a first-step to it.<br><br></div><div>Designing a mobile-first LN experien=
ce opens its own gap of challenges especially in terms of security and priv=
acy. The problem can be scoped as how to build a scalable, secure, private =
chain access backend for millions of LN clients ?<br><br></div><div>Light c=
lient protocols for LN exist (either BIP157 or Electrum are used), although=
 their privacy and security guarantees with regards to implementation on th=
e client-side may still be an object of concern (aggressive tx-rebroadcast,=
 sybillable outbound peer selection, trusted fee estimation). That said, on=
e of the bottlenecks is likely the number of full-nodes being willingly to =
dedicate resources to serve those clients. It&#39;s not about _which_ proto=
col is deployed but more about _incentives_ for node operators to dedicate =
long-term resources to client they have lower reasons to care about otherwi=
se.<br><br></div><div>Even with cheaper, more efficient protocols like BIP =
157, you may have a huge discrepancy between what is asked and what is offe=
red. Assuming 10M light clients [0] each of them consuming ~100MB/month for=
 filters/headers, that means you&#39;re asking 1PB/month of traffic to the =
backbone network. If you assume 10K public nodes, like today, assuming _all=
_ of them opt-in to signal BIP 157, that&#39;s an increase of 100GB/month f=
or each. Which is consequent with regards to the estimated cost of 350GB/mo=
nth for running an actual public node. Widening full-node adoption, special=
ly in term of geographic distribution means as much as we can to bound its =
operational cost.<br><br></div><div>Obviously,=C2=A0 deployment of more eff=
icient tx-relay protocol like Erlay will free up some resources but it mayb=
e wiser to dedicate them to increase health and security of the backbone ne=
twork like deploying more outbound connections.<br><br></div><div>Unless yo=
ur light client protocol is so ridiculous cheap to rely on niceness of a su=
bset of node operators offering free resources, it won&#39;t scale. And it&=
#39;s likely you will always have a ratio disequilibrium between numbers of=
 clients and numbers of full-node, even worst their growth rate won&#39;t b=
e the same, first ones are so much easier to setup.<br><br></div><div>It do=
esn&#39;t mean servicing filters for free won&#39;t work for now, numbers o=
f BIP157 clients is still pretty low, but what is worrying is=C2=A0 wallet =
vendors building such chain access backend, hitting a bandwidth scalability=
 wall few years from now instead of pursuing better solutions. And if this =
happen, maybe suddenly, isn&#39;t the quick fix going to be to rely on cent=
ralized services, so much easier to deploy ?<br><br></div><div>Of course, i=
t may be brought that actually current full-node operators don&#39;t get an=
ything back from servicing blocks, transactions, addresses... It may be rep=
lied that you have an indirect incentive to participate in network relay an=
d therefore guarantee censorship-resistance, instead of directly connecting=
 to miners. You do have today ways to select your resources exposure like p=
runing, block-only or being private but the wider point is the current (non=
?)-incentives model seems to work for the base layer. For light clients dat=
a, are node operators going to be satisfied to serve this new *class* of tr=
affic en masse ?<br><br>This doesn&#39;t mean you won&#39;t find BIP157 ser=
vers, ready to serve you with unlimited credit, but it&#39;s more likely th=
eir intentions maybe not aligned, like spying on your transaction broadcast=
 or block fetched. And you do want peer diversity to avoid every BIP157 ser=
vers being on few ASNs for fault-tolerance. Do people expect a scenario a l=
a Cloudflare, where everyone connections is to far or less the same set of =
entities ?<br><br>Moreover, the LN security model diverges hugely from basi=
c on-chain transactions. Worst-case attack on-chain a malicious light clien=
t server showing a longest, invalid, PoW-signed chain to double-spend the u=
ser. On LN, the *liveliness* requirement means the entity owning your view =
of the chain can lie to you on whether your channel has been spent by a rev=
oked commitment, the real tip of the blockchain or even dry-up block announ=
cement to trigger unexpected behavior in the client logic. A malicious ligh=
t client server may just drop any filters/utxos spends, what your LN client=
 should do in this case ? [1]<br><br>Therefore, you may want to introduce m=
onetary compensation in exchange of servicing filters. Light client not ded=
icating resources to maintain the network but free-riding on it, you may us=
e their micro-payment capabilities to price chain access resources [3]. Thi=
s proposition may suit within the watchtower paradigm, where another entity=
 is delegated some part of protocol execution, alleviating client onliness =
requirement. It needs further analysis but how your funds may be compromise=
d by a watchtower are likely to be the same scenario that how a chain valid=
ation provider can compromise you. That said, how do you avoid such &quot;c=
hain access&quot; market turning as an oligopoly is an open question. You m=
ay &quot;bind&quot; them to internet topology or ask for fidelity bonds and=
 create some kind of scarcity but still...<br><br>Maybe I&#39;m completely =
wrong, missing some numbers, and it&#39;s maybe fine to just rely on few th=
ousands of full-node operators being nice and servicing friendly millions o=
f LN mobiles clients. But just in case it may be good to consider a reasona=
ble alternative.<br><br>Thanks Gleb for many points exposed here but all mi=
stakes are my own.<br><br></div><div>Cheers,<br><br></div><div>Antoine<br><=
br>[0] UTXO set size may be a bottleneck, but still if you have 2 channels =
by clients that&#39;s 20M utxos, just roughly ~x3 than today.<br><br>[1] An=
d committing filters as part of headers may not solve everything as an atta=
cker can just delay or slow announcements to you, so you still need network=
 access to at least one honest node.<br><br>[2]=C2=A0 It maybe argue that d=
istinction client-vs-peer doesn&#39;t hold because you may start as a clien=
t and start synchronizing the chain, relaying blocks, etc. AFAIK, there is =
no such hybrid implementation and that&#39;s not what you want to run in a =
mobile.<br><br></div></div></div>
_______________________________________________<br>
Lightning-dev mailing list<br>
<a href=3D"mailto:Lightning-dev@lists.linuxfoundation.org" target=3D"_blank=
">Lightning-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev=
" rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/ma=
ilman/listinfo/lightning-dev</a><br>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
><div dir=3D"ltr"><div><div dir=3D"ltr"><b>Igor Cota</b><div>Codex Apertus =
Ltd<br></div></div></div></div></div>

--000000000000be348b05a5118bb8--