Return-Path: Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3EA44C07FF; Sat, 9 May 2020 07:23:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3A66688CC9; Sat, 9 May 2020 07:23:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0h2Mx3SXvxEo; Sat, 9 May 2020 07:23:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by hemlock.osuosl.org (Postfix) with ESMTPS id 4F65988CB4; Sat, 9 May 2020 07:23:04 +0000 (UTC) Received: by mail-pf1-f172.google.com with SMTP id z1so2181472pfn.3; Sat, 09 May 2020 00:23:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3BrB+llrWT6UlzVRlF4ANF66KEiGtm78vy7jlMU5eJM=; b=i5t066z2XS6KKWUKKWopgU+R+R4koMsu1nbYXx8ldhHopOAHgubuaYKk0kUjleNzfd uZVVz9l/kY/AHEBTNLQSJ3OyRTh2Qw6Tmfc9sfrn6yIdszL/Nr7jxUVTJ8AN/jyeLmGt /2n+rmyddF2dh3KBc26D7P2T2h6hMUbtpvcpW3wyGBBiHDAt3bhnNQt1qFU2S1QCVrpA yB08+m+eSxV2EyOGAQ1lHbYTCCYu+awOjbzW0gbTNgIJqHR6fNYp9O7fVL9B5K3hUHRV le0ZKGoYQ6jfAo1GvESEI3sDIYa+iP/Sf3rnw9t0VkKAoVuLBMePsS2mHkwZD7/np8VK zdBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3BrB+llrWT6UlzVRlF4ANF66KEiGtm78vy7jlMU5eJM=; b=FNhSfIRWkkKhysc23mQ7LrKYzBI2PfWvEOc6EXxqP/Cv1iKDnQQIuXhcRffeBDaNPW 0gzCdhhmegGcxqIL3KzojzMpDBR7qcM1WdHSabvd9mg6zNrYhnXzumqCQiOylj/0Xxus AWti0C5vFwqLvDuiA31VN5kL27oL6YzB6cpoZE3QJ2FpkBNJ+XU9jdBhknIch8OIGlyi VlIsdYFhL6S/6sM+pQpOtJLIN0Y9Jajhop4E9QKsPXylEAJIL9oNQD+uaGM613LnJE4N lInbX8XfphdNEoISqMdMJch2IENUTsTU74szpjC2xQB8RW29vdGUJcCaIDu5S9gpmOrr reNQ== X-Gm-Message-State: AGi0PuYm9TfSjSuH5C8R1gTe/ZF3Uz7ub4qWYbdnUYbBTgoo8MszFZGU TBEMWo766u/ErOrM+QC7/h7iO6N8JcAtGmfFF0F2kBSn X-Google-Smtp-Source: APiQypKyNkFMvO5ASkLyasaeN4GmcB8niQXA8wBX5pz7Yqc62lvBHbdteB7gZqz4swqO7aKNJ4AzPsnYs+wth9vGay8= X-Received: by 2002:a63:5a5d:: with SMTP id k29mr5366652pgm.176.1589008983781; Sat, 09 May 2020 00:23:03 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Antoine Riard Date: Sat, 9 May 2020 03:22:52 -0400 Message-ID: To: Igor Cota Content-Type: multipart/alternative; boundary="00000000000005fb0b05a531fc74" X-Mailman-Approved-At: Sat, 09 May 2020 07:24:34 +0000 Cc: Bitcoin Protocol Discussion , "lightning-dev\\\\@lists.linuxfoundation.org" Subject: Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2020 07:23:06 -0000 --00000000000005fb0b05a531fc74 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Igor, Thanks for sharing about what it's technically possible to do for a full-node on phone, specially with regards to lower grade devices. I do see 2 limitations for sleeping nodes: - a lightning specific one, i.e you need to process block data real-time in case of incoming HTLC you need to claim on chain or a HTLC timeout. There is a bunch of timelocks implications in LN, with regards to CSV, CLTV_DELTA, incoming policy, outgoing policy, ... and you can't really afford to be late without loosing a payment. I don't see timelocks being increase, that would hinder liquidity. - a p2p bandwidth concern, even if this new class of nodes turn as public ones, they would still have a heavy sync period due to be fallen-behind during the day, so you would have huge bandwidth spikes every a timezone falls asleep and a risk of choking upload links of stable full-nodes. I think assume-utxo may be interesting in the future in case of long-fork detection, you may be able to download a utxo-set on the fly, and fall-back to a full-node. But that would be only an emergency measure, not a regular cost on the backbone network. Antoine Le jeu. 7 mai 2020 =C3=A0 12:41, Igor Cota a =C3=A9= crit : > Hi Antoine et al, > > Maybe I'm completely wrong, missing some numbers, and it's maybe fine to >> just rely on few thousands of full-node operators being nice and servici= ng >> friendly millions of LN mobiles clients. But just in case it may be good= to >> consider a reasonable alternative. >> > > >> So you may want to separate control/data plane, get filters from CDN and >> headers as check-and-control directly from the backbone network. "Hybrid= " >> models should clearly be explored. > > > For some months now I've been exploring the feasibility of running full > nodes on everyday phones [1]. One of my first thoughts was how to avoid t= he > phones mooching off the network. Obviously due to battery, storage and > bandwidth constraints it is not reasonable to expect pocket full nodes to > serve blocks during day time. > > Huge exception to this is the time we are asleep and our phones are > connected to wifi and charging. IMO this is a huge untapped resource that > would allow mobile nodes to earn their keep. If we limit full node > operation to sleepy night time the only constraining resource is storage: > 512 gb of internal storage in phones is quite rare, probably about $100 f= or > an SD card with full archival node capacity but phones with memory card > slots rarer still - no one is going to bother. > > So depending on their storage capacity phone nodes could decide to store > and serve just a randomly selected range of blocks during their nighttime > operation. With trivial changes to P2P they could advertise the blocks th= ey > are able to serve. > If there comes a time that normal full nodes feel DoS'ed they can > challenge such nodes to produce the blocks they advertise and ban them as > moochers if they fail to do so. Others may elect to be more charitable an= d > serve everyone. > > These types of nodes would truly be part-timing since they only carry a > subset of the blockchain and work while their operator is asleep. Probabl= y > should be called part-time or Sleeper Nodes=E2=84=A2. > > They could be user friendly as well, with Assume UTXO they could be > bootstrapped quickly and while they do the IBD in the background instead = of > traditional pruning they can keep the randomly assigned bit of blockchain > to later serve the network. > > Save for the elderly, all the people I know could run such a node, and I > don't live in a first world country. > > There is also the feel-good kumbaya aspect of American phone nodes servin= g > the African continent while the Americans are asleep, Africans and > Europeans serving the Asians in kind. By plugging in our phones and going > to sleep we could blanket the whole world in (somewhat) full nodes! > > Cheers, > Igor > > [1] https://icota.github.io/ > > On Tue, 5 May 2020 at 12:18, Antoine Riard > wrote: > >> Hi, >> >> (cross-posting as it's really both layers concerned) >> >> Ongoing advancement of BIP 157 implementation in Core maybe the >> opportunity to reflect on the future of light client protocols and use t= his >> knowledge to make better-informed decisions about what kind of >> infrastructure is needed to support mobile clients at large scale. >> >> Trust-minimization of Bitcoin security model has always relied first and >> above on running a full-node. This current paradigm may be shifted by LN >> where fast, affordable, confidential, censorship-resistant payment servi= ces >> may attract a lot of adoption without users running a full-node. Assumin= g a >> user adoption path where a full-node is required to benefit for LN may >> deprive a lot of users, especially those who are already denied a real >> financial infrastructure access. It doesn't mean we shouldn't foster nod= e >> adoption when people are able to do so, and having a LN wallet maybe eve= n a >> first-step to it. >> >> Designing a mobile-first LN experience opens its own gap of challenges >> especially in terms of security and privacy. The problem can be scoped a= s >> how to build a scalable, secure, private chain access backend for millio= ns >> of LN clients ? >> >> Light client protocols for LN exist (either BIP157 or Electrum are used)= , >> although their privacy and security guarantees with regards to >> implementation on the client-side may still be an object of concern >> (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted = fee >> estimation). That said, one of the bottlenecks is likely the number of >> full-nodes being willingly to dedicate resources to serve those clients. >> It's not about _which_ protocol is deployed but more about _incentives_ = for >> node operators to dedicate long-term resources to client they have lower >> reasons to care about otherwise. >> >> Even with cheaper, more efficient protocols like BIP 157, you may have a >> huge discrepancy between what is asked and what is offered. Assuming 10M >> light clients [0] each of them consuming ~100MB/month for filters/header= s, >> that means you're asking 1PB/month of traffic to the backbone network. I= f >> you assume 10K public nodes, like today, assuming _all_ of them opt-in t= o >> signal BIP 157, that's an increase of 100GB/month for each. Which is >> consequent with regards to the estimated cost of 350GB/month for running= an >> actual public node. Widening full-node adoption, specially in term of >> geographic distribution means as much as we can to bound its operational >> cost. >> >> Obviously, deployment of more efficient tx-relay protocol like Erlay >> will free up some resources but it maybe wiser to dedicate them to incre= ase >> health and security of the backbone network like deploying more outbound >> connections. >> >> Unless your light client protocol is so ridiculous cheap to rely on >> niceness of a subset of node operators offering free resources, it won't >> scale. And it's likely you will always have a ratio disequilibrium betwe= en >> numbers of clients and numbers of full-node, even worst their growth rat= e >> won't be the same, first ones are so much easier to setup. >> >> It doesn't mean servicing filters for free won't work for now, numbers o= f >> BIP157 clients is still pretty low, but what is worrying is wallet vend= ors >> building such chain access backend, hitting a bandwidth scalability wall >> few years from now instead of pursuing better solutions. And if this >> happen, maybe suddenly, isn't the quick fix going to be to rely on >> centralized services, so much easier to deploy ? >> >> Of course, it may be brought that actually current full-node operators >> don't get anything back from servicing blocks, transactions, addresses..= . >> It may be replied that you have an indirect incentive to participate in >> network relay and therefore guarantee censorship-resistance, instead of >> directly connecting to miners. You do have today ways to select your >> resources exposure like pruning, block-only or being private but the wid= er >> point is the current (non?)-incentives model seems to work for the base >> layer. For light clients data, are node operators going to be satisfied = to >> serve this new *class* of traffic en masse ? >> >> This doesn't mean you won't find BIP157 servers, ready to serve you with >> unlimited credit, but it's more likely their intentions maybe not aligne= d, >> like spying on your transaction broadcast or block fetched. And you do w= ant >> peer diversity to avoid every BIP157 servers being on few ASNs for >> fault-tolerance. Do people expect a scenario a la Cloudflare, where >> everyone connections is to far or less the same set of entities ? >> >> Moreover, the LN security model diverges hugely from basic on-chain >> transactions. Worst-case attack on-chain a malicious light client server >> showing a longest, invalid, PoW-signed chain to double-spend the user. O= n >> LN, the *liveliness* requirement means the entity owning your view of th= e >> chain can lie to you on whether your channel has been spent by a revoked >> commitment, the real tip of the blockchain or even dry-up block >> announcement to trigger unexpected behavior in the client logic. A >> malicious light client server may just drop any filters/utxos spends, wh= at >> your LN client should do in this case ? [1] >> >> Therefore, you may want to introduce monetary compensation in exchange o= f >> servicing filters. Light client not dedicating resources to maintain the >> network but free-riding on it, you may use their micro-payment capabilit= ies >> to price chain access resources [3]. This proposition may suit within th= e >> watchtower paradigm, where another entity is delegated some part of >> protocol execution, alleviating client onliness requirement. It needs >> further analysis but how your funds may be compromised by a watchtower a= re >> likely to be the same scenario that how a chain validation provider can >> compromise you. That said, how do you avoid such "chain access" market >> turning as an oligopoly is an open question. You may "bind" them to >> internet topology or ask for fidelity bonds and create some kind of >> scarcity but still... >> >> Maybe I'm completely wrong, missing some numbers, and it's maybe fine to >> just rely on few thousands of full-node operators being nice and servici= ng >> friendly millions of LN mobiles clients. But just in case it may be good= to >> consider a reasonable alternative. >> >> Thanks Gleb for many points exposed here but all mistakes are my own. >> >> Cheers, >> >> Antoine >> >> [0] UTXO set size may be a bottleneck, but still if you have 2 channels >> by clients that's 20M utxos, just roughly ~x3 than today. >> >> [1] And committing filters as part of headers may not solve everything a= s >> an attacker can just delay or slow announcements to you, so you still ne= ed >> network access to at least one honest node. >> >> [2] It maybe argue that distinction client-vs-peer doesn't hold because >> you may start as a client and start synchronizing the chain, relaying >> blocks, etc. AFAIK, there is no such hybrid implementation and that's no= t >> what you want to run in a mobile. >> >> _______________________________________________ >> Lightning-dev mailing list >> Lightning-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >> > > > -- > *Igor Cota* > Codex Apertus Ltd > --00000000000005fb0b05a531fc74 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Igor,

Thanks= for sharing about what it's technically possible to do for a full-node= on phone, specially with regards to lower grade devices.

I do= see 2 limitations for sleeping nodes:
- a lightning specific one,= i.e you need to process block data real-time in case of incoming HTLC you = need to claim on chain or a HTLC timeout. There is a bunch of timelocks imp= lications in LN,=C2=A0 with regards to CSV, CLTV_DELTA, incoming policy, ou= tgoing policy, ... and you can't really afford to be late without loosi= ng a payment. I don't see timelocks being increase, that would hinder l= iquidity.
- a p2p bandwidth concern, even if this new class of nod= es turn as public ones, they would still have a heavy sync period due to be= fallen-behind during the day, so you would have huge bandwidth spikes ever= y a timezone falls asleep and a risk of choking upload links of stable full= -nodes.

I think assume-utxo may be interesting in the future i= n case of long-fork detection, you may be able to download a utxo-set on th= e fly, and fall-back to a full-node. But that would be only an emergency me= asure, not a regular cost on the backbone network.

Antoine
=


Le=C2=A0jeu. 7 m= ai 2020 =C3=A0=C2=A012:41, Igor Cota <igor@codexapertus.com> a =C3=A9crit=C2=A0:
Hi Antoine e= t al,

Maybe I'm completely wrong, missing some numbers, and it's maybe f= ine to just rely on few thousands of full-node operators being nice and ser= vicing friendly millions of LN mobiles clients. But just in case it may be = good to consider a reasonable alternative.
=C2=A0
<= /div>
So you may want to s= eparate control/data plane, get filters from CDN and headers as check-and-c= ontrol directly from the backbone network. "Hybrid" models should= clearly be explored.

=
For some months now I've been exploring the feasibility = of running full nodes on everyday phones [1]. One of my first thoughts was = how to avoid the phones mooching off the network. Obviously due to battery,= storage and bandwidth constraints it is not reasonable to expect pocket fu= ll nodes to serve blocks during day time.

Huge exception to this is the time we are asleep and our phones are conn= ected to wifi and charging. IMO this is a huge untapped resource that would= allow mobile nodes to earn their keep. If we limit full node operation to = sleepy night time the only constraining resource is storage: 512 gb of inte= rnal storage in phones is quite rare, probably about $100 for an SD card wi= th full archival node capacity but phones with memory card slots rarer stil= l - no one is going to bother.

So depen= ding on their storage capacity phone nodes could decide to store and serve = just a randomly selected range of blocks during their nighttime operation. = With trivial changes to P2P they could advertise the blocks they are able t= o serve.
If there comes a time that normal full nodes= feel DoS'ed they can challenge such nodes to produce the blocks they a= dvertise and ban them as moochers if they fail to do so. Others may elect t= o be more charitable and serve everyone.

These types of nodes would truly be part-timing since they only carry a s= ubset of the blockchain and work while their operator is asleep. Probably s= hould be called part-time or Sleeper Nodes=E2=84=A2.

They could be user friendly as well, with Assu= me UTXO they could be bootstrapped quickly and while they do the IBD in the= background instead of traditional pruning they can keep the randomly assig= ned bit of blockchain to later serve the network.

Save for t= he elderly, all the people I know could run such a node, and I don't li= ve in a first world country.

There is also the feel-good kum= baya aspect of American phone nodes serving the African continent while the= Americans are asleep, Africans and Europeans serving the Asians in kind. B= y plugging in our phones and going to sleep we could blanket the whole worl= d in (somewhat) full nodes!


On Tue, 5 May 2020 at 12:18, Antoine Riard <antoine.riard@gmail.com<= /a>> wrote:
<= div dir=3D"ltr">
Hi,

(cross-posting as it'= s really both layers concerned)

Ongoing advancement = of BIP 157 implementation in Core maybe the opportunity to reflect on the f= uture of light client protocols and use this knowledge to make better-infor= med decisions about what kind of infrastructure is needed to support mobile= clients at large scale.

Trust-minimization of Bitcoin se= curity model has always relied first and above on running a full-node. This= current paradigm may be shifted by LN where fast, affordable, confidential= , censorship-resistant payment services may attract a lot of adoption witho= ut users running a full-node. Assuming a user adoption path where a full-no= de is required to benefit for LN may deprive a lot of users, especially tho= se who are already denied a real financial infrastructure access. It doesn&= #39;t mean we shouldn't foster node adoption when people are able to do= so, and having a LN wallet maybe even a first-step to it.

Designing a mobile-first LN experience opens its own gap of challenges es= pecially in terms of security and privacy. The problem can be scoped as how= to build a scalable, secure, private chain access backend for millions of = LN clients ?

Light client protocols for LN exist (either = BIP157 or Electrum are used), although their privacy and security guarantee= s with regards to implementation on the client-side may still be an object = of concern (aggressive tx-rebroadcast, sybillable outbound peer selection, = trusted fee estimation). That said, one of the bottlenecks is likely the nu= mber of full-nodes being willingly to dedicate resources to serve those cli= ents. It's not about _which_ protocol is deployed but more about _incen= tives_ for node operators to dedicate long-term resources to client they ha= ve lower reasons to care about otherwise.

Even with cheap= er, more efficient protocols like BIP 157, you may have a huge discrepancy = between what is asked and what is offered. Assuming 10M light clients [0] e= ach of them consuming ~100MB/month for filters/headers, that means you'= re asking 1PB/month of traffic to the backbone network. If you assume 10K p= ublic nodes, like today, assuming _all_ of them opt-in to signal BIP 157, t= hat's an increase of 100GB/month for each. Which is consequent with reg= ards to the estimated cost of 350GB/month for running an actual public node= . Widening full-node adoption, specially in term of geographic distribution= means as much as we can to bound its operational cost.

O= bviously,=C2=A0 deployment of more efficient tx-relay protocol like Erlay w= ill free up some resources but it maybe wiser to dedicate them to increase = health and security of the backbone network like deploying more outbound co= nnections.

Unless your light client protocol is so ridicu= lous cheap to rely on niceness of a subset of node operators offering free = resources, it won't scale. And it's likely you will always have a r= atio disequilibrium between numbers of clients and numbers of full-node, ev= en worst their growth rate won't be the same, first ones are so much ea= sier to setup.

It doesn't mean servicing filters for = free won't work for now, numbers of BIP157 clients is still pretty low,= but what is worrying is=C2=A0 wallet vendors building such chain access ba= ckend, hitting a bandwidth scalability wall few years from now instead of p= ursuing better solutions. And if this happen, maybe suddenly, isn't the= quick fix going to be to rely on centralized services, so much easier to d= eploy ?

Of course, it may be brought that actually curren= t full-node operators don't get anything back from servicing blocks, tr= ansactions, addresses... It may be replied that you have an indirect incent= ive to participate in network relay and therefore guarantee censorship-resi= stance, instead of directly connecting to miners. You do have today ways to= select your resources exposure like pruning, block-only or being private b= ut the wider point is the current (non?)-incentives model seems to work for= the base layer. For light clients data, are node operators going to be sat= isfied to serve this new *class* of traffic en masse ?

This doesn= 9;t mean you won't find BIP157 servers, ready to serve you with unlimit= ed credit, but it's more likely their intentions maybe not aligned, lik= e spying on your transaction broadcast or block fetched. And you do want pe= er diversity to avoid every BIP157 servers being on few ASNs for fault-tole= rance. Do people expect a scenario a la Cloudflare, where everyone connecti= ons is to far or less the same set of entities ?

Moreover, the LN se= curity model diverges hugely from basic on-chain transactions. Worst-case a= ttack on-chain a malicious light client server showing a longest, invalid, = PoW-signed chain to double-spend the user. On LN, the *liveliness* requirem= ent means the entity owning your view of the chain can lie to you on whethe= r your channel has been spent by a revoked commitment, the real tip of the = blockchain or even dry-up block announcement to trigger unexpected behavior= in the client logic. A malicious light client server may just drop any fil= ters/utxos spends, what your LN client should do in this case ? [1]

= Therefore, you may want to introduce monetary compensation in exchange of s= ervicing filters. Light client not dedicating resources to maintain the net= work but free-riding on it, you may use their micro-payment capabilities to= price chain access resources [3]. This proposition may suit within the wat= chtower paradigm, where another entity is delegated some part of protocol e= xecution, alleviating client onliness requirement. It needs further analysi= s but how your funds may be compromised by a watchtower are likely to be th= e same scenario that how a chain validation provider can compromise you. Th= at said, how do you avoid such "chain access" market turning as a= n oligopoly is an open question. You may "bind" them to internet = topology or ask for fidelity bonds and create some kind of scarcity but sti= ll...

Maybe I'm completely wrong, missing some numbers, and it&#= 39;s maybe fine to just rely on few thousands of full-node operators being = nice and servicing friendly millions of LN mobiles clients. But just in cas= e it may be good to consider a reasonable alternative.

Thanks Gleb f= or many points exposed here but all mistakes are my own.

= Cheers,

Antoine

[0] UTXO set size may be a bottlen= eck, but still if you have 2 channels by clients that's 20M utxos, just= roughly ~x3 than today.

[1] And committing filters as part of heade= rs may not solve everything as an attacker can just delay or slow announcem= ents to you, so you still need network access to at least one honest node.<= br>
[2]=C2=A0 It maybe argue that distinction client-vs-peer doesn't= hold because you may start as a client and start synchronizing the chain, = relaying blocks, etc. AFAIK, there is no such hybrid implementation and tha= t's not what you want to run in a mobile.

_______________________________________________
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/ma= ilman/listinfo/lightning-dev


--
Igor Cota
Codex Apertus = Ltd
--00000000000005fb0b05a531fc74--