Return-Path: Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id BD8B3C002A; Mon, 24 Apr 2023 16:07:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9808E81F91; Mon, 24 Apr 2023 16:07:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9808E81F91 Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=pnr90WAa X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J-hdu8TsFQXK; Mon, 24 Apr 2023 16:07:12 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1E81D81FAB X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-oa1-x2e.google.com (mail-oa1-x2e.google.com [IPv6:2001:4860:4864:20::2e]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1E81D81FAB; Mon, 24 Apr 2023 16:07:11 +0000 (UTC) Received: by mail-oa1-x2e.google.com with SMTP id 586e51a60fabf-187af4a5453so1607156fac.1; Mon, 24 Apr 2023 09:07:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682352431; x=1684944431; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=b8drZOrvSDdor7tZpS49m3lns3YJ8AGxjTlECcZ1K/0=; b=pnr90WAa6Sny+CD2SfQ680VAn0ARsyZlwJ1rfqjDPimTtAensHbl70b5sW8lQCpsFQ BmHU4whMg7hO007AV/c9pohAfn0qjIeveXwc+Bv3lDPVuj/q2uHLzGAYJ5nPTYdrIuwx 0cpKN6D+EUfDeldrKzO1xGqL/4zJXXtMBzrjNypiDFIq0QR75vCym1LVRS1ae/KVAmU9 aQo0Dtbs+KxFXA++oGr38Upskw/Jnvin7IHd0ZT2fJSJN5B5jFUQpIwN7YKjGtTlf5qW W0U/m3kf/bSg06vvxPP1LG50YbqI/YUKm8YKu6LFQiZMy5g2ZNVNP2ckbt7ZSH7+cLOO eU5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682352431; x=1684944431; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b8drZOrvSDdor7tZpS49m3lns3YJ8AGxjTlECcZ1K/0=; b=W6Q31klfLEOnSLi/FlOsR+XD7CRUiVIaNYh6S0yDtZD/NhZuNvtAMyNSPBKOD0pCup bnvmmqjttr2+Ogwj1V7i4aC4WfDJNAuZnByAswrk60Z+lomEKhEa66BkSHlJGWfr9a23 eNiPPf7p3Nb5r8rP5zj46Ihe8hW6YjAU3FaFfLpkOmtxqkQ4BOFhgIN2g9m1GmL1NKwe JUG9i/toqX1H0rrUDN3OBBwE8rRnZ34okrEcODJiUSHtLQpi1+IBajsPlgO60ztzWmiS QGgSk8WKaEHnWLPq5V3vslBfatQhlLjnj8TIj8UBN/wveBsanvObztZgzN5qLKYSXL62 +ukg== X-Gm-Message-State: AAQBX9e1zxZCFni0BWpQJO1UVJq5TxrhSW2EzliuM5+SN246aGMkqVjT jIyQXIfNujycmJ2YHcMDZtqbCHR4JvCWSE5chW4= X-Google-Smtp-Source: AKy350YwZUBki0sV15RVbNQ3qufxhS5zjlYM90xziO3ip9BP3eCVZcf/zEUrZS1rOCqvDpnp7u4JYbfFGNQIsaanhek= X-Received: by 2002:a05:6870:418f:b0:17a:bf1b:cab8 with SMTP id y15-20020a056870418f00b0017abf1bcab8mr10741052oac.4.1682352430606; Mon, 24 Apr 2023 09:07:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: niftynei Date: Mon, 24 Apr 2023 12:06:59 -0400 Message-ID: To: Michael Folkson Content-Type: multipart/alternative; boundary="000000000000036f4105fa17338e" X-Mailman-Approved-At: Mon, 24 Apr 2023 16:31:38 +0000 Cc: bitcoin-dev@lists.linuxfoundation.org, Lightning Dev Subject: Re: [bitcoin-dev] [Lightning-dev] A new Bitcoin implementation integrated with Core Lightning X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Apr 2023 16:07:13 -0000 --000000000000036f4105fa17338e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Michael, CLN as implemented is currently nicely decoupled from the block source; as a project we assume that the node runner will choose a block backend that fits their self-sovereignty goals. This provides us with a nice separation of concerns. The block source is responsible for ensuring that only consensus valid data is delivered to the node, which in turn allows us to focus on processing and reacting to that data, as necessary. Bitcoin core as a project implements a broad swath of functionality (wallet, consensus, peering, rpc server, coin selection, key management, etc); breaking out the validation and peering functions into more composable parts would def open up more opportunities for building block sources for a wide variety of projects, not just CLN. There=E2=80=99s probably a real opportunity to =E2=80=9Ccomingle=E2=80=9D t= he peering of LN gossip + block data networks, this has been suggested a few times but never seriously pursued from the LN side. Having the peering functions of bitcoin-core broken out into a more composable/reusable piece may be a good first step here, and as a project would largely be on the bitcoin core side. Maybe this work is already in progress? I havent been keeping up with developments there. Thanks for the email, it=E2=80=99s definitely a good question. Lisa On Mon, Apr 24, 2023 at 02:09 Michael Folkson via Lightning-dev < lightning-dev@lists.linuxfoundation.org> wrote: > Any thoughts on this from the Core Lightning contributors? The way I see > it with upcoming proposed changes to default policy (primarily though not > exclusively for Lightning) and a soft fork activation attempt of APO/APOA= S > (primarily though not exclusively for Lightning) that a tighter coupling > between the full node and the Lightning node could eventually make sense. > In a world where transaction fees were much higher you'd think almost eve= ry > full node would also want to be a Lightning node and so the separation of > concerns would make less sense. Having two separate P2P networks and two > separate P2P protocols also wouldn't make much sense in this scenario. Yo= u > could obviously still opt out of Lightning P2P messages if you weren't > interested in Lightning. > > The alternative would be just to focus on Knots style consensus compatibl= e > forks of Core with limited additional functionality. But I think we've > reached the point of no return on Core dominance and not having widely us= ed > "distros". As the ecosystem scales systems and processes should be > constantly evolving and improving and to me if anything Core's seem to be > going backwards. > > Thanks > Michael > > -- > Michael Folkson > Email: michaelfolkson at protonmail.com > Keybase: michaelfolkson > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 > > ------- Original Message ------- > > On Saturday, January 14th, 2023 at 20:26, Michael Folkson < > michaelfolkson@protonmail.com> wrote: > > I tweeted this [0] back in November 2022. > > "With the btcd bugs and the analysis paralysis on a RBF policy option in > Core increasingly thinking @BitcoinKnots and consensus compatible forks o= f > Core are the future. Gonna chalk that one up to another thing @LukeDashjr > was right about all along." > > A new bare bones Knots style Bitcoin implementation (in C++/C) integrated > with Core Lightning was a long term idea I had (and presumably many other= s > have had) but the dysfunction on the Bitcoin Core project this week (if > anything it has been getting worse over time, not better) has made me sta= rt > to take the idea more seriously. It is clear to me that the current way t= he > Bitcoin Core project is being managed is not how I would like an open > source project to be managed. Very little discussion is public anymore an= d > decisions seem to be increasingly made behind closed doors or in private > IRC channels (to the extent that decisions are made at all). Core Lightni= ng > seems to have the opposite problem. It is managed effectively in the open > (admittedly with fewer contributors) but doesn't have the eyeballs or the > usage that Bitcoin Core does. Regardless, selfishly I at some point would > like a bare bones Bitcoin and Lightning implementation integrated in one > codebase. The Bitcoin Core codebase has collected a lot of cruft over tim= e > and the ultra conservatism that is needed when treating (potential) > consensus code seems to permeate into parts of the codebase that no one i= s > using, definitely isn't consensus code and should probably just be remove= d. > > The libbitcoinkernel project was (is?) an attempt to extract the consensu= s > engine out of Core but it seems like it won't achieve that as consensus i= s > just too slippery a concept and Knots style consensus compatible codebase > forks of Bitcoin Core seem to still the model. To what extent you can > safely chop off this cruft and effectively maintain this less crufty fork > of Bitcoin Core also isn't clear to me yet. > > Then there is the question of whether it makes sense to mix C and C++ cod= e > that people have different views on. C++ is obviously a superset of C but > assuming this merging of Bitcoin Core and Core Lightning is/was the optim= al > final destination it surely would have been better if Core Lightning was > written in the same language (i.e. with classes) as Bitcoin Core. > > I'm just floating the idea to (hopefully) hear from people who are much > more familiar with the entirety of the Bitcoin Core and Core Lightning > codebases. It would be an ambitious long term project but it would be nic= e > to focus on some ambitious project(s) (even if just conceptually) for a > while given (thankfully) there seems to be a lull in soft fork activation > chaos. > > Thanks > Michael > > [0]: > https://twitter.com/michaelfolkson/status/1589220155006910464?s=3D20&t=3D= GbPm7w5BqS7rS3kiVFTNcw > > -- > Michael Folkson > Email: michaelfolkson at protonmail.com > Keybase: michaelfolkson > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 > > > _______________________________________________ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > --000000000000036f4105fa17338e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Michael,

CLN as implemented is currently nicely decoupled from the block sourc= e; as a project we assume that the node runner will choose a block backend = that fits their self-sovereignty goals.

This provides us with a nice separation of concerns. The bl= ock source is responsible for ensuring that only consensus valid data is de= livered to the node, which in turn allows us to focus on processing and rea= cting to that data, as necessary.

Bitcoin core as a project implements a broad swath of functionali= ty (wallet, consensus, peering, rpc server, coin selection, key management,= etc); breaking out the validation and peering functions into more composab= le parts would def open up more opportunities for building block sources fo= r a wide variety of projects, not just CLN.

There=E2=80=99s probably a real opportunity to =E2=80= =9Ccomingle=E2=80=9D the peering of LN gossip + block data networks, this h= as been suggested a few times but never seriously pursued from the LN side.= Having the peering functions of bitcoin-core broken out into a more compos= able/reusable piece may be a good first step here, and as a project would l= argely be on the bitcoin core side. Maybe this work is already in progress?= I havent been keeping up with developments there.
<= br>
Thanks for the email, it=E2=80=99s definitely a = good question.

Lisa


= On Mon, Apr 24, 2023 at 02:09 Michael Folkson via Lightning-dev <lightning-dev@lists.lin= uxfoundation.org> wrote:
Any thoughts on this from the Core Lightning contributors? The= way I see it with upcoming proposed changes to default policy (primarily t= hough not exclusively for Lightning) and a soft fork activation attempt of = APO/APOAS (primarily though not exclusively for Lightning) that a tighter c= oupling between the full node and the Lightning node could eventually make = sense. In a world where transaction fees were much higher you'd think a= lmost every full node would also want to be a Lightning node and so the sep= aration of concerns would make less sense. Having two separate P2P networks= and two separate P2P protocols also wouldn't make much sense in this s= cenario. You could obviously still opt out of Lightning P2P messages if you= weren't interested in Lightning.

The alternative would be just to focus on Knots style = consensus compatible forks of Core with limited additional functionality. B= ut I think we've reached the point of no return on Core dominance and n= ot having widely used "distros". As the ecosystem scales systems = and processes should be constantly evolving and improving and to me if anyt= hing Core's seem to be going backwards.

Thanks
Michael

--
Michael FolksonEmail: michaelfolkson at protonmail.com <= /span>
Keybase:= michaelfolkson
PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
=20
=20

------- Original Message -------

On Saturday, January 14th, 2023 at 20:26, Michael Folkson <michaelfolkso= n@protonmail.com> wrote:

I tweeted this = [0] back in November 2022.

"With t= he btcd bugs and the analysis paralysis on a RBF policy option in Core incr= easingly thinking @BitcoinKnots and consensus compatible forks of Core are = the future. Gonna chalk that one up to another thing @LukeDashjr was right = about all along."

A new bare bones= Knots style Bitcoin implementation (in C++/C) integrated with Core Lightni= ng was a long term idea I had (and presumably many others have had) but the= dysfunction on the Bitcoin Core project this week (if anything it has been= getting worse over time, not better) has made me start to take the idea mo= re seriously. It is clear to me that the current way the Bitcoin Core proje= ct is being managed is not how I would like an open source project to be ma= naged. Very little discussion is public anymore and decisions seem to be in= creasingly made behind closed doors or in private IRC channels (to the exte= nt that decisions are made at all). Core Lightning seems to have the opposi= te problem. It is managed effectively in the open (admittedly with fewer co= ntributors) but doesn't have the eyeballs or the usage that Bitcoin Cor= e does. Regardless, selfishly I at some point would like a bare bones Bitco= in and Lightning implementation integrated in one codebase. The Bitcoin Cor= e codebase has collected a lot of cruft over time and the ultra conservatis= m that is needed when treating (potential) consensus code seems to permeate= into parts of the codebase that no one is using, definitely isn't cons= ensus code and should probably just be removed.

The libbitcoinkernel project was (is?) an attempt to extract the c= onsensus engine out of Core but it seems like it won't achieve that as = consensus is just too slippery a concept and Knots style consensus compatib= le codebase forks of Bitcoin Core seem to still the model. To what extent y= ou can safely chop off this cruft and effectively maintain this less crufty= fork of Bitcoin Core also isn't clear to me yet.

Then there is the question of whether it makes sense to mix = C and C++ code that people have different views on. C++ is obviously a supe= rset of C but assuming this merging of Bitcoin Core and Core Lightning is/w= as the optimal final destination it surely would have been better if Core L= ightning was written in the same language (i.e. with classes) as Bitcoin Co= re.

I'm just floating the idea to (= hopefully) hear from people who are much more familiar with the entirety of= the Bitcoin Core and Core Lightning codebases. It would be an ambitious lo= ng term project but it would be nice to focus on some ambitious project(s) = (even if just conceptually) for a while given (thankfully) there seems to b= e a lull in soft fork activation chaos.

Thanks
Michael
<= div style=3D"font-family:Arial;font-size:14px">
=

--
Michael FolksonEmail: michaelfolkson at protonmail.com= =
Keybase: michaelfolkson
PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 21= 4C FEE3

_______________________________________________
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/ma= ilman/listinfo/lightning-dev
--000000000000036f4105fa17338e--