diff options
| author | Eric Voskuil <eric@voskuil.org> | 2015-07-27 23:40:42 -0700 |
|---|---|---|
| committer | bitcoindev <bitcoindev@gnusha.org> | 2015-07-28 06:40:26 +0000 |
| commit | e4164b960f98da5b4b4f978121b1aaa92e353f99 (patch) | |
| tree | 83864f691bfa6fc3c1ecc156ca78d3897bee0da7 | |
| parent | 521be46edd59e592f753d102b79c608d7fcbf236 (diff) | |
| download | pi-bitcoindev-e4164b960f98da5b4b4f978121b1aaa92e353f99.tar.gz pi-bitcoindev-e4164b960f98da5b4b4f978121b1aaa92e353f99.zip | |
Re: [bitcoin-dev] Libconsensus separated repository (was Bitcoin Core and hard forks)
| -rw-r--r-- | 66/7dfce3f57396321538a30139066f9e657e8877 | 336 |
1 files changed, 336 insertions, 0 deletions
diff --git a/66/7dfce3f57396321538a30139066f9e657e8877 b/66/7dfce3f57396321538a30139066f9e657e8877 new file mode 100644 index 000000000..bc0d899bf --- /dev/null +++ b/66/7dfce3f57396321538a30139066f9e657e8877 @@ -0,0 +1,336 @@ +Return-Path: <eric@voskuil.org> +Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org + [172.17.192.35]) + by mail.linuxfoundation.org (Postfix) with ESMTPS id 8655E408 + for <bitcoin-dev@lists.linuxfoundation.org>; + Tue, 28 Jul 2015 06:40:26 +0000 (UTC) +X-Greylist: whitelisted by SQLgrey-1.7.6 +Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com + [209.85.220.46]) + by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 894678D + for <bitcoin-dev@lists.linuxfoundation.org>; + Tue, 28 Jul 2015 06:40:25 +0000 (UTC) +Received: by padck2 with SMTP id ck2so64740731pad.0 + for <bitcoin-dev@lists.linuxfoundation.org>; + Mon, 27 Jul 2015 23:40:25 -0700 (PDT) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to + :cc:subject:references:in-reply-to:content-type; + bh=aabFVj6kATt7aOeAP1E40FvqxWuZtpvgB2J3qaRHcUY=; + b=mLH7IGRtDbpxHPfxl2MsIton55AJnfZLc+g7X9mO2WbQ/+i15oFowMPLUB7Oa7tk6V + RdzE7ImdzQ6jZluB6wG0cG2MHO+xYHonPK5jz2b1l620jkeFYQjtfP/vTxu8sYfn8I5u + e1I6EyPJOv49I3YLI8FS6kFiE707lFi1P2ngDB8IBR2FSSRaN4lXNPnZFbylCLhVI14E + P0RIC36AohE3BVHVdCJ0+F9b7iNVa2TrNs3hm9E29JCNmD5D1UyHjXDyd6lCR9g282Uh + 0E1fTq0YFlkO4I8JigdbeBRTLsTbKNCy7p7HJoSMzmIRzUbTXv+bO8HwTJ/LvKuumD6w + 3BRg== +X-Gm-Message-State: ALoCoQk9sEQpdEjFWc/AeC7yJrtKBl8RO/BibWVBM2Rkj8GgkA63vbwwt8cqF6MJFWEEE3cr75EM +X-Received: by 10.66.132.98 with SMTP id ot2mr54972089pab.31.1438065625277; + Mon, 27 Jul 2015 23:40:25 -0700 (PDT) +Received: from [10.0.1.14] (c-67-161-88-20.hsd1.wa.comcast.net. [67.161.88.20]) + by smtp.googlemail.com with ESMTPSA id + ym6sm33226179pac.32.2015.07.27.23.40.24 + (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); + Mon, 27 Jul 2015 23:40:24 -0700 (PDT) +Message-ID: <55B723EA.7010700@voskuil.org> +Date: Mon, 27 Jul 2015 23:40:42 -0700 +From: Eric Voskuil <eric@voskuil.org> +User-Agent: Mozilla/5.0 (X11; Linux x86_64; + rv:31.0) Gecko/20100101 Thunderbird/31.2.0 +MIME-Version: 1.0 +To: =?UTF-8?B?Sm9yZ2UgVGltw7Nu?= <jtimon@jtimon.cc> +References: <CABm2gDrApVuxF8DFf32V=pQhDKvvVfcDK=LeCXJ9h9o8CY+wNQ@mail.gmail.com> +In-Reply-To: <CABm2gDrApVuxF8DFf32V=pQhDKvvVfcDK=LeCXJ9h9o8CY+wNQ@mail.gmail.com> +Content-Type: multipart/signed; micalg=pgp-sha1; + protocol="application/pgp-signature"; + boundary="uNOa5RRXaWrqG2OJBmmI6WR7q0NXqmaKV" +X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW + autolearn=ham version=3.3.1 +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on + smtp1.linux-foundation.org +Cc: bitcoin-dev@lists.linuxfoundation.org +Subject: Re: [bitcoin-dev] Libconsensus separated repository (was Bitcoin + Core and hard forks) +X-BeenThere: bitcoin-dev@lists.linuxfoundation.org +X-Mailman-Version: 2.1.12 +Precedence: list +List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org> +List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> +List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> +List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> +List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> +List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> +X-List-Received-Date: Tue, 28 Jul 2015 06:40:26 -0000 + +This is an OpenPGP/MIME signed message (RFC 4880 and 3156) +--uNOa5RRXaWrqG2OJBmmI6WR7q0NXqmaKV +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +On 07/23/2015 07:30 AM, Jorge Tim=C3=B3n wrote: +> On Thu, Jul 23, 2015 at 2:49 AM, Eric Voskuil via bitcoin-dev wrote: +>> On 07/22/2015 05:13 PM, Eric Lombrozo via bitcoin-dev wrote: +>>> Only being partly serious - I strongly am in favor of a sufficiently +>>> modularized codebase that swapping out consensus rules is fairly +>>> straightforward and easy to test... +>> +>> We (libbitcoin) have taken the time to publish and maintain bitcoind's= + +>> "libbitcoinconsensus" source files as an independent C++ library... + +> I think there were some misunderstandings in our previous conversation +> about this topic. +> I completely agree with having a separated repository for libconsensus +> (that's the whole point, alternative implementations can be +> consensus-safe by using it, and in the event of a schism fork[1], they +> can fork just that smaller project without having to relay on Bitcoin +> Core [satoshi] at all). +> But I thought you also wanted Bitcoin Core to use libconsensus instead +> of just having a subtree/subrepository like it currently does with +> libsecp256k1. + +libsecp256k1 has it's own repository, libbitcoinconsensus doesn't. A +separate repository was what I considered as a requirement for us to use = +it. + +> I'm not sure if that would ever be accepted, but in any case we're +> certainly far away from that goal. + +If it's not certain whether this would even be accepted, the commitment +to a community consensus library is too weak to take a strong dependency +on. But for us it's moot, as we have made the already accomplished that +goal. + +> Here are some things that need to +> happen first: +>=20 +> 1) Finish encapsulating consensus code so that it can be built without +> any (we've done it only with script-related code so far). Here are +> some related PRs (other people have done other things that help with +> this as well): +=2E.. +> 2) Finish libconsensus's API: expose more things than VerifyScript, at +> the very least, also expose VerifyTx, VerifyHeader and VerifyBlock. +> Feedback from alternative implementations like libbitcoin is extremely +> valuable here. Some related closed-for-now PRs: + +In our earlier discussion I believe you said that the library would not +be undergoing significant change or feature creep. If this is the very +least that's projected it would seem that constraint will not hold. + +> 3) Move libconsensus to a separate repository as a +> subtree/subrepository of Bitcoin Core. +>=20 +> Only after all that we can discuss whether Bitcoin Core itself should +> include libconsensus' code or just use its API directly. + +I don't think it's a question of whether it *should* use its own library +as it is published for others - this is a practically self-evident +conclusion. + +> I hope that after all this, libbitcoin also reconsiders whether to +> reimplement its own libconsensus or use the "official" one directly +> instead. + +We use a fork of libsecp256k1 and would probably do the same with the +consensus library if it was cleanly isolated. + +>> In any case I agree with your stated need for this isolation (if not t= +he +>> means) for the reasons you state. The community needs to move beyond a= + +>> largely singular and monolithic codebase that is holding that position= + +>> in part due to fear about consensus bug forks. +>=20 +> I completely agree. That's the goal of libconsensus (and an +> alternative implementation like libbitcoin being able to use it +> without sacrificing any of its current or future design differences +> from Bitcoin Core would be a sign of success in this reward). + +It's a performance sacrifice, and then there's the OpenSSL dependency, +but these are both optional within our stack - so the application +developer has the option. So the only downside is that we are +maintaining the conditional compilation. + +> Unfortunately any changes that touch consensus code are risky and +> therefore slow. And when consensus encapsulation changes conflict with +> other changes (not because the other changes need to change consensus +> but because consensus code is still coupled with policy and other +> bitcoind-specific code), refactors are never prioritized. Ironically, +> you need to encapsulate the consensus code to avoid such conflicts, +> which would make all non-consensus changes far less risky (reducing +> the consensus-critical review development bottleneck). +>=20 +> Unfortunately and ironically again, safer, small and incremental +> changes are less interesting for reviewers. +> For example, I've been trying to move consensus code to the consensus +> folder for a long time. The correctness of a MOVEONLY change is +> trivial to review for anyone who knows how to copy/paste in its +> favorite editor and how to use git diff, but will I ever get answers +> to my questions in [1]? + +I think it's worthwhile work, especially if you are passionate about the +longer term objectives. I haven't been involved in these reviews as I +spend very little time with the satoshi client + +> I know there's many people who really care about this, Cory Fields, +> Wladimir and Pieter Wuille to name a few have reviewed many of this +> changes (I've just got used to publicly whine about lack of review on +> this front and policy encapsulation [very related fronts] as an +> attempt to get some attention: not always, but begging for review +> actually works some times). + +Well a cynic might observe that fear of consensus bugs is what keeps +people on the satoshi client, and therefore accelerating the development +of a clean and independent consensus library would be a very low priority= +=2E + +> Another unfortunate fact is that although a script-only libconsensus +> allows you to avoid a big part of all possible consensus fork bugs, +> there cannot be users of a finished libconsensus to ask things to until= + +> a finished libconsensus actually exists. + +Software is never finished, but this exists and we are using it. + +> At the same time, the future +> users (alternative implementations, since bitcoin core is already +> "using libconsensus")=20 + +It is using the same source files, but AFAICT not the library. + +> are the most relevant people to listen when it +> comes to the C API. That's why I beg you to comment on [2], even if +> #5995 is currently closed. Your input on [1] would be very appreciated +> as well (maybe you think it's better to expose verifyTx before +> exposing verifyHeader, even if exposing verifyHeader is something that +> could be done faster). + +I haven't looked at any of these commits, but I'll make some time to at +least give a cursory review. + +> > To make choice regarding consensus an actual choice (and thereby act= +ual +>> consensus) the modularity you suggest is essential. One must be able t= +o +>> take new developments without having to take consensus changes. The +>> option to fork the codebase is not reasonable for most people. At this= + +>> point there is no defensible reason for coupling consensus checks with= + +>> other features. +>=20 +> Would you agree that asking people to fork an independent libconsensus +> project instead of having to fork the full Bitcoin-qt is much more +> reasonable? + +Yes, of course. We've already done it. For each release of the satoshi +client since we made libbitcoin-consensus I've copied the sources. It's +pretty much automated and easy to visually verify that the sources +match. That would be quite a bit more difficult if there wasn't an +independent build. + +> I mean, I agree with your points. If "the specification of the +> consensus rules is an implementation", then that implementation +> shouldn't be coupled with a bunch of policy and non-consensus +> technical choices (storage, dependencies, p2p protocol...). But I +> still think that "the specification of the consensus rules should be a +> concrete implementation" rather than based purely on a natural +> language like English. + +Useful specifications often have two reference implementations. It's the +idea that there can be only one legitimate implementation that's +problematic. + +> I believe that's the only point where we fundamentally disagree, but +> it shouldn't be a barrier in our common goal of taking "power" away +> from Bitcoin Core development. If we're successful Bitcoin Core won't +> have any privileged position with regards to, say, libbitcoin when it +> comes to deciding consensus rules changes. + +I don't think we disagree on anything fundamental. My issues with the +library were (1) the lack of isolation, (2) the fact that the satoshi +client wouldn't actually use the library, and (3) backtracking to use +OpenSSL, which we had recently removed from libbitcoin. + +=2E. + +> 1) libconsensus us finished and used by libbitcoin +> 2) Bitcoin Core was unanimously in favor of Gavin's 32 GB initial +> proposal and the changes are applied to bitcoin/bitcoin and +> bitcoin/libconsensus (or Bitcoin Core has a dictator like Mike +> wants[4] and he accepts it, it doesn't really matter for this +> example). +>=20 +> But let's also assume that X% of the users and 10% of the miners are +> against that Schism hardfork, and they don't want to be forced to +> change the rules by any influential group, mining, economic or user +> majority. +> Libbitcoin cannot be forced to accept the next, controversial version +> of bitcoin/libconsensus, so you guys fork libbitcoin/libconsensus out +> of the last ok version. + +This is already done. + +> Centralized-bitcoin and old-bitcoin would become 2 separated +> currencies and some people would likely lose money in the transition +> from one currency to 2 of them, but the users of old-bitcoin have the +> right of keeping the rules they signed up for and the only responsible +> people for this likely-catastrophic schism would be the Bitcoin core +> developers for trying to impose consensus changes into others against +> their will. Trying to impose consensus changes against the will of +> some users is wrong, and it is irrelevant if that happens in Bitcoin +> Core or Bitcoin Tx (if it is uncontroversial, it's also irrelevant +> where it gets implemented first). + +I don't disagree, but you previously argued that *everyone* had to agree +on consensus. Above you are making the argument that people can choose +to disagree. Yes, this is important. Yet it's unrealistic for any +alternative consensus to overcome inertia unless there are widely +deployed independent implementations. + +> I really believe bitcoin needs competitive alternative implementations +> and I believe libconsensus is a tool to help that happen and reduce +> the "gatekeeping" friction that there's (unfortunately) around Bitcoin +> Core. I look forward to any potential collaboration on this front. +> Even if you still want to maintain a reimplementation of libconsensus +> (which I humbly think it's a mistake, but I don't think there's any +> point on keep discussing that, since we know we disagree) we can +> collaborate on the future common API of a complete libconsensus (with +> verifyBlock and all). I really hope we can do that. + +I'm maintain our fork of the consensus sources until they are (1) +properly isolated from the satoshi client and (2) the satoshi client +uses the actual library. + +I don't see it as important or even productive to try and wedge the +consensus library into the repository/build for bitcoind and Bitcoin-QT. +It's straightforward to maintain the consensus library independently. +Always willing to work with you on it, although we're all busy, and this +isn't my top priority presently. + +e + + +--uNOa5RRXaWrqG2OJBmmI6WR7q0NXqmaKV +Content-Type: application/pgp-signature; name="signature.asc" +Content-Description: OpenPGP digital signature +Content-Disposition: attachment; filename="signature.asc" + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1 + +iQEcBAEBAgAGBQJVtyPqAAoJEDzYwH8LXOFOQPsH/3HZrA7Mzi/zoYYs3+8r4ISe ++BtnXaK5CtYXmOAK4lgZflFlAJpzCBYCC2tUhBftPqm3Z9Ta3OUf+L7St74cZb/y +NXzUgaajGoxw1nWJyECtKKgXHmvRt2ALWfoiVlrT6NgH7ZQDrUuoOw4OW/D7oZzn +0YE5VsOMFrwO+H4QbOLIn/Hkp5OeNWkszJdrodWHqpRfbVX2Qcwx91Iojk53z4UI +XoK8d7G+7EfGAKYYcwxl5mMZvTijYNdJSga8sB2v4RWYxjp6fd14VtWGJNqBZnko +bfpomau/mdYiMSyHQyr5ORRu2mzRRTOjTLDam00Nq06a6j1O3Pp/S+d8OOagSiw= +=fIeR +-----END PGP SIGNATURE----- + +--uNOa5RRXaWrqG2OJBmmI6WR7q0NXqmaKV-- + |