diff options
| author | Eric Voskuil <eric@voskuil.org> | 2016-01-12 11:17:20 -0800 |
|---|---|---|
| committer | bitcoindev <bitcoindev@gnusha.org> | 2016-01-12 19:17:16 +0000 |
| commit | 9a9ac6e15d31ea0b4cb168441463299835d7af4a (patch) | |
| tree | dd088ae052c2d029a667a23d21b9f090aae988cf | |
| parent | 1d096413afc1d22035ff9703c3040f8a05632718 (diff) | |
| download | pi-bitcoindev-9a9ac6e15d31ea0b4cb168441463299835d7af4a.tar.gz pi-bitcoindev-9a9ac6e15d31ea0b4cb168441463299835d7af4a.zip | |
Re: [bitcoin-dev] Libconsensus phase 2
| -rw-r--r-- | 14/397ffe0c57cd90b5603b5ac1840deb9c5b19ac | 247 |
1 files changed, 247 insertions, 0 deletions
diff --git a/14/397ffe0c57cd90b5603b5ac1840deb9c5b19ac b/14/397ffe0c57cd90b5603b5ac1840deb9c5b19ac new file mode 100644 index 000000000..2cf2aaf19 --- /dev/null +++ b/14/397ffe0c57cd90b5603b5ac1840deb9c5b19ac @@ -0,0 +1,247 @@ +Return-Path: <eric@voskuil.org> +Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org + [172.17.192.35]) + by mail.linuxfoundation.org (Postfix) with ESMTPS id 532C187A + for <bitcoin-dev@lists.linuxfoundation.org>; + Tue, 12 Jan 2016 19:17:16 +0000 (UTC) +X-Greylist: whitelisted by SQLgrey-1.7.6 +Received: from mail-pf0-f179.google.com (mail-pf0-f179.google.com + [209.85.192.179]) + by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B3C2A160 + for <bitcoin-dev@lists.linuxfoundation.org>; + Tue, 12 Jan 2016 19:17:14 +0000 (UTC) +Received: by mail-pf0-f179.google.com with SMTP id q63so69332455pfb.1 + for <bitcoin-dev@lists.linuxfoundation.org>; + Tue, 12 Jan 2016 11:17:14 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=voskuil-org.20150623.gappssmtp.com; s=20150623; + h=message-id:date:from:user-agent:mime-version:to:subject:references + :in-reply-to:content-type; + bh=5pHiCIPRhI5CXI5bMkLC1viUCCRDOMK5mTdbdPWlSGY=; + b=Oh1TZrjWOivC/qpT1CuGXxpxWvTydY1Ne4+pIvmk5LgXjQMjoSMXhvjfh7/wXfC2tS + dt+1wYXq3pt4kX7iQC5VzG4tVxXy7OBhf48LoJIM+Kq5kQE3tunjv+pVI0vVKuvQWSZ0 + QgwDCnr3R2Sm3JRnOSLeljPQRdI0z6XmqWCodmkYioK6qUzwsF2GkvXhmzkvfqUIdwb0 + wECM7ciPsuCqa4d3lVCNmnwDs29zwCbarrm7z+MHaWA2p077UdxMVY2LZ1g+U+YkVx99 + Fc0IrBqxroUm65Di0NLCGxSeiZcmz0Q5v9JRlGAEqqxOkTe6njRQTRTrfjz5bfQftl8q + jBow== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to + :subject:references:in-reply-to:content-type; + bh=5pHiCIPRhI5CXI5bMkLC1viUCCRDOMK5mTdbdPWlSGY=; + b=JqaBqbvdz5JQOSDud4AMUVnXp67ferzB11gLAUMjybrKalFo30xdRsWFg3JU+KrIQC + BVGT/IPpDhxU5OegrqeM1EW2uBLq55bizes5vwVfgf9FeqwyQ6jDFzNBzkzXn8JwHquk + T5Qiis7KL5XQAnVOiT5oisSI6+G4APjjBuTaSt3LvyM6W0bYHEhhPnrWgpT+qWQLDGUG + Bmp3gdDK9JcHZ8stUM6xMlrD6VOtIlvosh7jsRmnAuBRuoyF64yZPMfJF+Kmao42Zc8k + tk7etjkeFqdnrCPQPehy3nAwbC4a5CaWvxNTjQk6c7dGtDKNo2sahwh4oM2/ztcUC9F6 + 5O+g== +X-Gm-Message-State: ALoCoQkzM6J3v1qeZB6WWThflgGuzduqONriqy4xrrmFvvuo7Ww4tQaFcGfN4YFMsqe9S0a3RniY8Fx6mJ93lGEbPkVMnnjSfA== +X-Received: by 10.98.68.73 with SMTP id r70mr36533195pfa.12.1452626234394; + Tue, 12 Jan 2016 11:17:14 -0800 (PST) +Received: from ?IPv6:2601:600:9001:8060:149e:5071:8056:988e? + ([2601:600:9001:8060:149e:5071:8056:988e]) + by smtp.googlemail.com with ESMTPSA id + v2sm32134807pfi.93.2016.01.12.11.17.12 + (version=TLSv1/SSLv3 cipher=OTHER); + Tue, 12 Jan 2016 11:17:13 -0800 (PST) +Message-ID: <56955140.7050301@voskuil.org> +Date: Tue, 12 Jan 2016 11:17:20 -0800 +From: Eric Voskuil <eric@voskuil.org> +User-Agent: Mozilla/5.0 (X11; Linux x86_64; + rv:31.0) Gecko/20100101 Thunderbird/31.2.0 +MIME-Version: 1.0 +To: =?UTF-8?B?Sm9yZ2UgVGltw7Nu?= <jtimon@jtimon.cc>, + Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>, + Libbitcoin <libbitcoin@lists.dyne.org> +References: <CABm2gDqPrsuHH4L+dQE-6e162Zj6jbyyB1-6htjB_+2EY2qTXg@mail.gmail.com> +In-Reply-To: <CABm2gDqPrsuHH4L+dQE-6e162Zj6jbyyB1-6htjB_+2EY2qTXg@mail.gmail.com> +Content-Type: multipart/signed; micalg=pgp-sha256; + protocol="application/pgp-signature"; + boundary="d3mnxrHvr0lDe7RX6t03p5jSgAWDXOuWO" +X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on + smtp1.linux-foundation.org +X-Mailman-Approved-At: Tue, 12 Jan 2016 19:21:23 +0000 +Subject: Re: [bitcoin-dev] Libconsensus phase 2 +X-BeenThere: bitcoin-dev@lists.linuxfoundation.org +X-Mailman-Version: 2.1.12 +Precedence: list +List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org> +List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> +List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> +List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> +List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> +List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> +X-List-Received-Date: Tue, 12 Jan 2016 19:17:16 -0000 + +This is an OpenPGP/MIME signed message (RFC 4880 and 3156) +--d3mnxrHvr0lDe7RX6t03p5jSgAWDXOuWO +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Jorge, first, thanks again for your work on this. + +Without creating and using a public blockchain interface in phase 2, how +will you isolate the database dependency from consensus critical code? +Is it that the interface will exist but you will recommend against its us= +e? + +This work presumes that the users of the library reject the argument +that the database implementation is consensus critical code. Faithful +reproduction of stored data is a prerequisite for a validity. But a +common store implementation is only slightly more reasonable for this +library than a common RAM implementation. + +e + +On 01/12/2016 09:53 AM, Jorge Tim=C3=B3n wrote: +> After talking to some people about libconsensus in the last Hong Kong +> conference I realized that my initial plan of exposing one more thing +> at a time would actually probably slow things down. +>=20 +> There's still a promised pdf with pictures that will be released, and +> actually drafting the UML pictures helped realize that the whole +> explanation could be much simpler if #7091 was merged first as the +> last step in phase 1 (that phase has so many contributors that I will +> probably never get finished documenting it). Matt Corallo's idea of +> exposing VerifyScript() through a C API certainly helped a lot in +> cementing the more-minimal-than-earlier dependencies (thanks to Cory +> Fields among many other people before him) that are not part of the +> incomplete but existing libbitcoinconsensus library. +>=20 +> Given this success in protecting encapsulation by exposing things in a +> new library, my instinct was to expose more things: VerifyHeader(), +> VerifyTx() and VerifyBlock() [in that order]. +> But all those three new functions depend on storage in one way or +> another. That was part of my reasoning to expose VerifyHeader() first, +> because I believe there will be less discussion on a common interface +> for the stored longest chain than for the utxo view (which may depend +> on other transactions spent within the same block). +> In any case, I realized we should finish putting all the consensus +> critical code in the libconsensus lib and then worry about its "final" +> API. +>=20 +> Therefore I changed the goal of the phase 2 in my libconsensus +> encapsulation planning from "expose VerifyHeader() in the existing +> libconsensus library" to "build all the consensus critical code within +> the existing libconsensus library, even if we don't expose anything +> else". I believe this is much feasible for a single Bitcoin Core +> release cycle and also more of a priority. Other implementations +> experimenting with libconsensus like +> https://github.com/libbitcoin/libbitcoin-consensus will have the +> chance to compare their reimplementations with the future complete +> libbitcoinconsensus without having to worry about the C API, which +> ideally they will help to define. +>=20 +> I repeat, the goal of phase 2 in my upcoming libconsensus +> encapsulation plan is to fully decouple libconsensus from Bitcoin +> Core. +> In phase 3, we can refine the storage interfaces and focus on a +> quasi-final C API. +> In phase 4, we can refine and take out code that doesn't belong in +> libconsensus like CTxOut::GetDustThreshold() in +> primitives/transaction.h and move all those consensus files to the +> consensus folder before creating a separate sub-repository like for +> libsecp256k1. Note that most of the file-moving work can be in +> parallel to phases 2 and 3 and, in fact, by any new developer that is +> willing to exchange rebase-patience for meaningless github stats (I'll +> do it if nobody else wants, but I'm more than happy to delegate there: +> I have more than enough github meaningless stats already). +>=20 +> As said, the document with pictures and the update to #6714 are still +> promised, but until they're ready, merging/reviewing #7091, #7287, +> #7310 and #7311 could do a great deal to make later steps in +> libconsensus phase 2 more readable. +> Most reviewers probably don't need to see any "big picture" to tell +> whether certain functions on Bitcoin Core are consensus-critical or +> not, or whether consensus critical code needs to depend on util.o or +> not. +> But I wouldn't be writing to the mailing list without a plan with +> further words nor pictures if I didn't had what I believe is a +> complete implementation of what I just defined as "libconsensus phase +> 2". +>=20 +> Phase 3 should finish long pending discussions like "should +> libconsensus be C++14 or plain C" which should NOT delay phase 2. +> Phase 4 should be mostly trivial: rename files to the target dir and +> move the remaining unused code out of libconsensus. +> Phase 5 should make Bitcoin Core eat its own dog food and use +> libbitcoinconsensus oonly by its generic C API (I'm sorry if this +> looks too far away for me to even think about detailing it). +>=20 +> The work in progress branch (but hopefully being finished, nit and +> merged within the 0.12.99 cycle) can be found in: +> https://github.com/jtimon/bitcoin/commits/libconsensus-f2 +>=20 +> Before sipa asks, signing code may make it into a new library but +> SHOULDN'T BE PART OF LIBBITCOINCONSENSUS. Ideally, all exposed +> functions will return true or false and an error string. It is based +> on last-0.12.99 3cd836c1 but by popular demand I can open it as a +> "DEPENDENT-tagged" PR linking to smaller steps and keeping track of +> steps done. Analogous to the about to be replaced (for a simpler and +> more maintainable example of testchain) #6382. If people like +> Wladimir, Cory and Pieter cannot see that I've been able to reduce my +> overall cry-for-review noise thanks to github adoption of emacs' +> org-mode's [ ] vs [X] I can alwways leave those "big picture" branches +> as "private" branches out of the pull request count. +>=20 +> I expect to publish a phase 3 branch very shortly. But as said I +> expect a lot of discussion on the API part, so I don't expect big +> movements in phase 3 until phase 2 is done (as said phase 4 is +> orthogonal to anything, this time git will say "verified MOVEONLY" for +> us). +>=20 +> To finish this long mail, if you are new to free software and would +> like to get familiarized with Bitcoin Core development in particular, +> moving one file is a simple task that you can always besure you can do +> right. +> The way I plan to hand this to you, you won't need to convince anyone +> to publicly confirm that your "MOVEONLY" commit being legit, because +> all your remaining work will be to build on one platform (ideally you +> should do a gitian build, but embarrassingly enough for someone +> touching consensus code I just trust travis ) and trust travis (as +> said, that's what I do from my laptop, but I plan to buy my own +> building machine [and maybe outsource it for free in some protocol +> that hasn't been invented, sorry again for the distraction]) and fix +> the includes that have stopped working. +>=20 +> I intend to create an issue to move all the files in this list one by o= +ne: +>=20 +> https://github.com/bitcoin/bitcoin/pull/7091/files#diff-480477e89f9b6dd= +afb30c4383dcdd705R250 +>=20 +> But don't hesitate to contact me if are eager for moving some files, +> because I believe we can save a few lines of total diff if we chose +> the order of the movements properly. +>=20 +> Sorry, I forgot many people read this list again. +> Happy to answer any question. +>=20 +> Specially about https://github.com/jtimon/bitcoin/commits/libconsensus-= +f2 +>=20 + + +--d3mnxrHvr0lDe7RX6t03p5jSgAWDXOuWO +Content-Type: application/pgp-signature; name="signature.asc" +Content-Description: OpenPGP digital signature +Content-Disposition: attachment; filename="signature.asc" + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1 + +iQEcBAEBCAAGBQJWlVFAAAoJEDzYwH8LXOFONPkH/2XSRbgiomqg7PqP26WWi96h +alrbOCbQ/bZ8hcCpxNlvLRY6PwmfYipbiZEUp5dWuWXzDoLwiDM4O0gqJvEuP3Jv +KvDsuIjBk48QazBDlP8CsVA/35C5KAZQMffUMhbXieMKUejE3RrwCS+eHsN6hffL +2aa1/TacWu4PzrWog35a4PCt7Vi1RL5Ev25gnJ5QnFvXkk9dKpwnwPLz0WPrxGhC +1OvxcSxui+VtzguAogsTVbnJ6A2D6/pktTvA+Wvae/+tN58ofbOzVJLsAR0wclKC +kOF1Vp0omzFlO8f4qUTDAQqAsUkIP3uzNLu0L+tA2oQGuQumbC62IeGPGtaVEeU= +=GyKA +-----END PGP SIGNATURE----- + +--d3mnxrHvr0lDe7RX6t03p5jSgAWDXOuWO-- + |