Return-Path: Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 511BCC07FF for ; Fri, 8 May 2020 20:04:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3B79D89808 for ; Fri, 8 May 2020 20:04:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DTOAb13JHbu3 for ; Fri, 8 May 2020 20:04:55 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) by hemlock.osuosl.org (Postfix) with ESMTPS id 7ADDD897D1 for ; Fri, 8 May 2020 20:04:55 +0000 (UTC) Received: by mail-oi1-f173.google.com with SMTP id t199so9522063oif.7 for ; Fri, 08 May 2020 13:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purse.io; s=google; h=to:cc:references:from:autocrypt:subject:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=wbAvl2+uB37SYwqrIXtzDUdVr/QgEYsWbBsh+/BB5UE=; b=Cx4WaIrJUYX0RUHTIW+RUj3M6ekjaknlcvc39c8MHG/TfDPOB++0qrOK/erh4dkfeU PzQXXz2W7tf8ffjtm6WBZziCLwNfnFLLZVAATFiqrPELpzTWt4C65QJxqdEsAPRy8N/q ifgypY48H0c0iQ3GLAQh+WeQtP9XkGFUg5aPc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:cc:references:from:autocrypt:subject :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding:content-language; bh=wbAvl2+uB37SYwqrIXtzDUdVr/QgEYsWbBsh+/BB5UE=; b=rHvjutxJPNTD7c5uTZqIIkTfs5n92/UNHby5uHeYJz+qNh7Itv93BVfdVnKsir69hP kF34aX5ls7r5NYn/8qJsDm9T6+9uqtBRw1JvkYF/JOutz9G+PPI6nuTJDBNmgp6Lbx7V ySIFUnl8LRODt8Uj7qs44OPcRO7I9VSk0Z+VD/uw8Np9QIL9jSS7bFo4OynEtly+ulBe VHY2dAvFl90z9IX7v2OKqZUg1rAG8GJSoqbELgy0R41SuTgsCxFcdHkj0EDka42q+PLT uvMeSrQB9cNEoGmKQNfY47OEhsoBDk2hpQKV295MjmQY0LuyLwY6Dd/ZnPKnlU1pJoJt NVWQ== X-Gm-Message-State: AGi0PuaXhwTUxYCESE+93H6fT+wAknvPM4oFLbq0UkT4noo4v50vpINb d5MH77vak4a4FDvfltpibF4ho+XVQpNj0g== X-Google-Smtp-Source: APiQypK+N3NHO/arneBn1uBA9OgLF8WKyhcGvr/N0fWZMT5RNPUROHaEretnDT0fJ7FEug00xZqPTg== X-Received: by 2002:a17:90a:f197:: with SMTP id bv23mr7564526pjb.3.1588966437707; Fri, 08 May 2020 12:33:57 -0700 (PDT) Received: from [10.0.0.199] ([66.172.99.113]) by smtp.gmail.com with ESMTPSA id mj2sm3007617pjb.49.2020.05.08.12.33.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 May 2020 12:33:56 -0700 (PDT) To: Olaoluwa Osuntokun , Bitcoin Protocol Discussion , Antoine Riard References: From: Braydon Fuller Autocrypt: addr=braydon@purse.io; keydata= LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tCgptUUlOQkZzRWthc0JFQUM0 b1BKUXNGV0FNME9qTkpsWVV4SmJUVjhibzFUZ1h3WmROWldld0c2ZnZRUStpR0dJCm15L2E1 aDEwLzlWKy9DdGlvOGF5QWZwazZWM3ovdnhxOElkUFZjc1djY0hMbmd1YUFFRWhZbktkR1pm c29oY1EKTU80TFI2M1I2dlByVVlVSmVoVnp0MllaNUY0OTVpUkRLd2JRTFhibVhGOXZMdGFL dWYvNmhiVm1sRzdiTTU5ZQpWS2N2cEU4RUttOTVMcHcrQ0JFM3VLYzNSeFRjS0hnbzZRcmtD bG0xOUdORERrbUh4TS9rK2hHVDNNK3NYR2hnCnBMMjk0MEFHS3NYdVRQZ2hnTFErNXNlQld0 enVXVjMxVWYwbHRIeDZLczF3MG1sZ1paK3U0d2xsak1ISUJLTHIKNGNZRE9GOEFZeHlWY1B6 dVFMZFJqSUlLRWdwTURRVzdmdGJnR00vcWhnR2cybU4wMzZLVFVmVDFIa21UODJaMwpEM3o0 QUNKL2JZL0orbUp3MGJhcWNKcGV6bmRkMU02ZG5ETkY5U2xzdDJNcitMb2VKZE4xdFk1a2s2 L09Xd0FJCjVMMTVobWRpUldVQnJ5SnF1bjdzeTIzY21WUFNPcnRhSWZsNy9TRUZKK011WGJL NTMrbUZHVDJQQ3BkUnpWNE0KcktyOXFiQmFYa2todkVmb0wyejZWd0lGQnBycVpqdGhQSGNI UjhyNlFTVHBaZHNEKzcrcW9xMm80TURVcVA1NAo1YTNrR0kzbDZWVzc2N1ZFL05VSW9jeU83 WjlEL0VaT0NrMjJFbFRMVEN6V20wSnZ4eEdKb2FsZWJMUWNHZXkvCkE5V1dHT3YveWR3dlRy eTNQV0F6cnJYTnQ0M01jSEh3WlhVNHZTTG5BdEF4UXkwNk50OXdzV28rMlFBUkFRQUIKdENG Q2NtRjVaRzl1SUVaMWJHeGxjaUE4WW5KaGVXUnZia0J3ZFhKelpTNXBiejZKQWxRRUV3RUlB RDRDR3dNRgpDd2tJQndJR0ZRZ0pDZ3NDQkJZQ0F3RUNIZ0VDRjRBRkNRVlp2cGdXSVFSYmZj V05rUDdCNlpDakVMcnlUeU10CkVJczYxQVVDWG90VmJBQUtDUkR5VHlNdEVJczYxTVVJRC80 blpZN0lDeUhock1iNDNaOEcrVE45TzE1WlFON2sKWlNzZ2hWeWxpcW15cE1lNWFGOWNkbFUz a1VkR1BEbUVuWjgweTc1bmZheXFpdlpCOG1HYXFJSTkycXVLNFJ2TQphc0kzdlQ5Rkl0Z2g4 Nkh6NWRHa1g4eGVEZms0SkNmWml0SUQydDJVdUR1Z3JGc2VFMVE4OTJsZ21NcGZQQ3VoCk5I NWxDWnUzVjNnNFJDdlAxd1Jjd09XY0dXU0wwR0pvOHJrRS9tYzBKS0NGNDdwbDFFUWVRUVp5 czIxTHBOTnYKWmFMVHJoQUpPdDFIRXBheUlxdXFKTzA4TzBMZGErelF1Z2szV0hHWUNyeXlJ ZFdiSWljdUFsbmowa0J1ajZKagpFdVphNFNRdE0wenF5QzNCbmIzdXBBSkNGOVdmL1lUcyts cUJTNW96ekxkTXJRSlo5VTE4VmxUZCtZeU9icjFBCkY0UW5xT1g2aDJlNE02KzZuOWlIdU9o eWlEdTI4cms2dU9sY1lOQTVXcm9mV0dFK1AyYS9zOExGOGxEZ1BtQlIKMjNmT0hrSmFtK2FB U1R6VVVJSC9GM0NqbkhkTzU1cHJDWDRPOUJWc2FOR2J6WHRhMzFYVnZQbE1qQXpPc3ZWegpT SitzUjRvMjkyVjE3THdkZU11OUFNK3c1YSt3Yk5kVnA2OEtmRnh2OTliaSs0QWgvNlVrQ2Js dHdQWkJqVS9nClp1WGhXM2s0Z3J6a3pwdmd1U3JEU1NHNzdDNC9QTjB4aFFRMTVlaEdlQkNl YWd2Ui9EZWxRQnkwMUYvTC9NV3EKSkJZTTVwN3ptR1htSlJXTWRNMDlDMThBc1NVZnMySWxl dnN4a3RJWFQzSUFsVU9wdG14RlJTRWlLSERwNlJ2NgpweWM2TWJSWWZGN1BGYlFrUW5KaGVX UnZiaUJHZFd4c1pYSWdQR052ZFhKcFpYSkFZbkpoZVdSdmJpNWpiMjArCmlRSllCQk1CQWdC Q0Foc0RCZ3NKQ0FjREFnWVZDQUlKQ2dzRUZnSURBUUllQVFJWGdBVUpCVm0rbUJZaEJGdDkK eFkyUS9zSHBrS01RdXZKUEl5MFFpenJVQlFKZWkxVnlBaGtCQUFvSkVQSlBJeTBRaXpyVTVJ Y1AvMXVmOUlrQgpsd2h3TG5CUjQ2QkM5NTgyOVN3MnBzTTZYTDZ6OXIrVUtmUTFSdm9pTDQ5 V2FzU1Z6QUJGVjlnNVhRbllUajVVCkgrTnFTK29lTXlScTl0d1Avb3JFN2crVDRTNzRpaGFx YzFhZTZmY3pNaDRIc3NkbmtyZ2FLbTVoRXEzQjh1V08KNzRSaHRJUnczUVpzM2lqSHoxU2w3 K1NzNklEOC9IbHFGRzNQaGhJQ2hFU2xna1gwQXlRSHV0ZkwzTmhGTFU5VAo1MU9RSFoyUTBG NUo5cFgxY0JPSlR5WHNwUHppLzhUdWdWcHNqcC9LdVpWbjg2WjhIVksraFhIWCs5bVV0emZy CnQweG91RGJRUHFlN1hRbDBJdnJpVVpaTWhFOXdOMEpMRHljTFhlLzl5RFppRGFsRUV0RENR ZXBpcGx2STdMRlQKRGw3cHArU2xKQWFNMm9RbFRQUGg3a2M4ZThwWnNmQUo2clZKTTlheFUr am0rNFRacFdSUm8rK3NIY00xY2VnZQo5MVI3VnRkREN6cGJRYys3Ymtpa3VHRk9sYkw5OFY5 U2daWVVxWkQrbkI0MC9xeldxQkM5RGU5YWdZWUMxaFhWCmo2YnUvTHhOV0NYZis2YnpKVWJN aWZJc2JBNWhKZzRTbTlVTGFCV0M3eE1WUzFjSE9TSU9iaFl1aVU1Ny9HekkKRW5SMVo5YjU1 UHYwV3IwelJjRW9aM3dSUG5hMzZ4bHU5YXRiSXBuRFQ1VTNSclB1TTdXWWdnWEZaMDRkMm5F TQpndnpYZk5BR2IvMG51K1kxZmwrbjJDYkJIenFleGFTckpYQUw3QjlBbmhock1ETGJxMk51 aVk0dnk5YWFDTFZFCnZvcWpKRzBFQXJacG1BMkt4UG03ZmQrNWJNY1hsb2RsZEFiL3VRSU5C RnNFa2FzQkVBRGY1SlYwRWxST0dNL0gKOG5rYmdEVjEzYThwVkZHbXRiNm5JblJjTnMydEZV NFNmR3ZlaVRQWVhnZTJEdmFoUXZDdzF5NEthQVhFT1hFZwpVeUszTmtCWnU0dCtyMFZIdXlk cGJjOHlXU1BqaDU3T3JkNlpoZFcvY0xlVnpveTUwQTZxUkFvM05xVEpvRC9wCncxV1ZDZFdh dWx5MmtVZTFRMktoNks4ejBoNEkvbmF4eE1memlwNUZtaCthZDUxemNVREVvd24wNW1MZ3lL S0MKeDM4T0hPRlNwc0I0K3NJeVZCQ3d6OGRkSTJ6YXFnd3JjR2pxVndmTlh1cklqM0RYMjJZ V2k1ZlRwUXVjQUF6VgowbGx3SlF2ZlVVdTFvaXNadHhjRDczMXJ4MlU1UnZyeXVQMzBmZFhm TU9NbXh3TUdBbmFiT0s2NUwrZG9iODByCjVGV0ZXSGNxcTFXSUxyYWNhQURXMUxWb3NwbnNv TjdMTzdnV0lUWnNJK0hwdmdpVWhDS1BPckJQSThtMzdReTUKL1lWakdLdjBiQ0F5ZFdvbjk5 aHBQcnZWRUp4U3VTcmFTdFhOMGVyOGRmc0R3Zko5KzF3RXdTM2pKa20zOXRudQp2TEpZbncr VkFrL2NjYXJQNkdrZWJDdkNZbUNQbEZ1YVMxVXpEZEVBU0trbXNRY1BWRHZBaUU4MWluTG9V akFXClVQSENsUkE1UmdZUmxqREQwdEZtSnFrM1FWNE53V25vSlF4R0FrZmJobERNV1R3bmsz ZzhETm1zaGw5QnRydlYKc01EUGYzVFk4TzNycXFJaUliYWE1TnQzeDRoNExQSDFIRmFFT1pC eVhOZWZsYkY4OVRxWFFUMDJBczBXZzBLbQprdHBQUFpMOGJIQm5PdEoyNHd0OVdXRW84T08v WVFBUkFRQUJpUUlsQkJnQkFnQVBCUUpiQkpHckFoc01CUWtCCjRUT0FBQW9KRVBKUEl5MFFp enJVd3VvUC8yOHhjdEdGN25EcWdFcTR2UzJQNEtFdUdSQVF6cnBEczdpNnRhYSsKWHpKLzlP RTJMVDh2SjBZQmhyQUpSYTUzYW1GbzZEaThmUWhTOVNHVXNORnhoell4MS9BbUx4cVB3YnRn TDg1UQpyREFVTjlqSmozTWlUVUxuSmVmb3VLQ1NNNHZRL0pnalF3emFDVHJpMzg4cjVHZkx5 UFBwQ1BCbmdBeXloT21qCnIzd3RKaExzdHk4NFJPbjdmTFBRdGkyUTgvZm9XcWdiYUc1Mk9I VFhrYVFOSWdkWnlTaTNoTWxjcnc0NGM2NHIKQlRQWisxNGF3VjR6aHVaT0lDc0pGZXdDUkZh alJWc0RxRStLY3JSbXYvRXMxSmdqSDNnaGQ3bXIwaEpsenB4Ngo1YktLeFd4Um5GU0Fra2JJ MUs5ZGZESGpPaEpMc0ZmSk9qdXpLT2twM1QyZUVJdmRBWis0ZTZBbi9YOE1KbzIwCnBTN1lM NnRSN1ZUSHIrcTE0SlZTd0F6NXNOOE1GRCs5TFcyZktmMDVVRk01bXBlOEtFdmgvNHdBM1pi Q3ovNHUKWEhBT29ZeGZwZ0w5a0ZZem5wY1lDa3ZYT1AzZWZLREZzM3l6ekplcW94QTJDRk9Q Q1AwaXJKbzFCd283OEN5VgpLWVdWYnBGZlBWaEVOUGlaSXAwQ2lCdDBPcWFDNFp6VG1SUm81 S1NwSnZSc3ZmaUpEOTJiZy9KWEdOSVpJTkd2CnhOQTZOQWJQU3RJU0hwRDR0bjNOR1cyU1lR eVIyVEpaQm5pMWl3RnRDL1pOVDYwSjhvd2hRSm52RVFlNitNQXAKa2lhejBTdXpmTlZMNDl4 R3YyTHFhOHkxczB5NTgvMHpCTEhzK3F3ZTYvODlaMmdqZTdCcWZGak9UVVpNc0FzeQp6TGFK aVFJOEJCZ0JBZ0FtQWhzTUZpRUVXMzNGalpEK3dlbVFveEM2OGs4akxSQ0xPdFFGQWw1OUhN a0ZDUVZaCnZwNEFDZ2tROGs4akxSQ0xPdFFPeGcvNkFqWWoxUlFHL25yTktBNFQ5cnYrV2tr MFo1dXRqTERYa2NmQjV2TTgKa0ZwMCtTTnBWMmVRMmlRTTZXWTRCUVBweTBZTXVuSGVOVmJ5 SHVPZzI2UEluUGc0WWdSZXpvUXhIbmdBdHJqLwowV1BKSXhodHRoWXNSODRhbXZ0TEJ3MWFs T3VRU1daQVNYRUdFcmgrTkQrNFB0N0dobWxEODhROWxmWXpGZHhJCk43V1YxdFBBVHBQeCta M1ljTllaWGQydkswVmluTG9odk4wdW1iOGd2ZUZDVkhYaWliYjZzcFI4Q0VQTVRvSEkKc1JU VWo1S3JEWmhmbUduY0Qrc0NySFNXVDlSbHh2TkZpRjI5RFdzKzdudUJCYU5QQ3hYMHozeTZJ aVg0aFVsaApkcUFOeVR1cHVSaVlGUFpIWjBMSFFzeDQ2WjJjVXE1enQwMUJwV1NCWHlKeEJj TVY3SGtWT2J3ZzJaTVJGbGNJCkRGYy9aWTBWbmlmSDlWRHFpaWZKelhtNUkxaWE1SXU2R3M2 ZXRlZnZGdHQ2QlFKcXd0MlBNcENDOG53dDE3eDEKVlFuZVk4OVZmTTROczlPeC9IU01zODFZ U2wyKzZDSXQxVjVNeGtMYzMxeEJCNXZKTUhsRTRFM0g3VnVidjRicQoyWUxnY29nSlN6WkZv ZDRQUHY1ZkIzbTYybjV3U1R6M2todjBiVVhGOUN2dUxLNkk0dXdpUnBCNHhLRlRSZjZTCm5m RTNzaUcwVWlidFl1cE1wLzJ5RmYrWUpiVVlEZk1XUjhwZWNTRzFJemVjQ3dFNHptU2F6TFN3 czFMc1pYQnAKUVd5U0JvaVBCeEU1akYzOEYvRkRyMll4VEdqNU1KaXRzTUloNkFoZnlCNG1o OW8yVzBlYnh4K2YyNXh4aGZydwpuekk9Cj16T0twCi0tLS0tRU5EIFBHUCBQVUJMSUMgS0VZ IEJMT0NLLS0tLS0K Message-ID: <6bd0f6ff-9c99-d5c8-f6ae-7f604ad3e273@purse.io> Date: Fri, 8 May 2020 12:33:55 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Mailman-Approved-At: Fri, 08 May 2020 21:00:21 +0000 Cc: "lightning-dev\\\\@lists.linuxfoundation.org" Subject: Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2020 20:04:56 -0000 On 5/5/20 5:31 PM, Olaoluwa Osuntokun via bitcoin-dev wrote: > Hi Antoine, > >> Even with cheaper, more efficient protocols like BIP 157, you may have= a >> huge discrepancy between what is asked and what is offered. Assuming 1= 0M >> light clients [0] each of them consuming ~100MB/month for filters/head= ers, >> that means you're asking 1PB/month of traffic to the backbone network.= If >> you assume 10K public nodes, like today, assuming _all_ of them opt-in= to >> signal BIP 157, that's an increase of 100GB/month for each. Which is >> consequent with regards to the estimated cost of 350GB/month for runni= ng >> an actual public node > One really dope thing about BIP 157+158, is that the protocol makes ser= ving > light clients now _stateless_, since the full node doesn't need to perf= orm > any unique work for a given client. As a result, the entire protocol co= uld > be served over something like HTTP, taking advantage of all the establi= shed > CDNs and anycast serving infrastructure, which can reduce syncing time > (less latency to > fetch data) and also more widely distributed the load of light clients = using > the existing web infrastructure. Going further, with HTTP/2's server-pu= sh > capabilities, those serving this data can still push out notifications = for > new headers, etc. The statelessness of compact block filters does look useful. Bloom filters for blocks can be inefficient, during a scan with a BIP37 wallet, it's necessary to discard already received merkle blocks as the filter has been updated and the previous results may have missed transactions. Both bitcoinj [1] and breadwallet-core [2] handle it using a similar method. The alternative of= synchronizing and alternating between requesting blocks and filter updates leads to slow scan times. With compact block filters, a separate wallet process (from the full node) can make adjustments necessary to what it needs to filter without having to communicate with the full node. [1]: https://github.com/bitcoinj/bitcoinj/blob/806afa04419ebdc3c15d5adf065979a= a7303e7f6/core/src/main/java/org/bitcoinj/core/Peer.java#L1076-L1079 [2]: https://github.com/breadwallet/breadwallet-core/blob/8eb05454df3e2d5cca24= 8b4e24eeffa420c97e3a/bitcoin/BRPeer.c#L83-L85