MIME-Version: 1.0
In-Reply-To: <C486E9D9-D799-48B9-B80F-1A165DFD6536@bitsofproof.com>
References: <CABm2gDrApVuxF8DFf32V=pQhDKvvVfcDK=LeCXJ9h9o8CY+wNQ@mail.gmail.com>
	<55B723EA.7010700@voskuil.org>
	<CABm2gDok2gH2R8=x3a8PmPiM56WFg3TKzCum_WS=uV9-T1Ss3g@mail.gmail.com>
	<55B939CF.1080903@voskuil.org>
	<CABm2gDq1wHP01Tncw3hu=SCWQHaAOMjWOVYQWdQsOZ+E7zp9Yw@mail.gmail.com>
	<CABm2gDr_ePfPeWB8pEO8QNHDjUZpybVuCAdDmMxJxMaC8+2bGA@mail.gmail.com>
	<C4EA4A39-1920-4F33-822C-FBD12DF81004@bitsofproof.com>
	<CABm2gDqkF20ZoexQSV8iORb3ukxxZr5RasTLxJqQfSTsTqHvog@mail.gmail.com>
	<3390F712-879A-46E9-ABCD-D35B51190304@bitsofproof.com>
	<CABm2gDpcEmiPNQWeUk5aTjuTSRAJSPYfgAKc7B_qrqw0w04xoA@mail.gmail.com>
	<C486E9D9-D799-48B9-B80F-1A165DFD6536@bitsofproof.com>
Date: Sun, 30 Aug 2015 00:08:24 +0200
Message-ID: <CABm2gDoPdysXRrcrrgH6mizDMpwN7JPr5EOzfrgoXWcgb9fVYA@mail.gmail.com>
From: =?UTF-8?B?Sm9yZ2UgVGltw7Nu?= <jtimon@jtimon.cc>
To: Tamas Blummer <tamas@bitsofproof.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>,
	Libbitcoin <libbitcoin@lists.dyne.org>
Subject: Re: [bitcoin-dev] Libconsensus separated repository (was Bitcoin
 Core and hard forks)
Precedence: list

On Sat, Aug 22, 2015 at 1:04 PM, Tamas Blummer <tamas@bitsofproof.com> wrot=
e:
> On Aug 21, 2015, at 21:46, Jorge Tim=C3=B3n <jtimon@jtimon.cc> wrote:
>
> On Thu, Aug 20, 2015 at 10:35 AM, Tamas Blummer <tamas@bitsofproof.com>
> wrote:
>
> Every re-implementation, re-factoring even copy-paste introduces a risk o=
f
> disagreement,
> but also open the chance of doing the work better, in the sense of softwa=
re
> engineering.
>
>
> But you don't want something better, you want something functionally
> identical.
> You may want to watch sipa's explanation on why "the implementation is
> the specification" and the reasons to separate libconsensus:
> https://youtu.be/l3O4nh79CUU?t=3D764
>
>
> I do want something better, but not for the focus you have.
>
> Not because what you produce was not high quality, but because quality is
> achieved at a very
> high cost and is hard to uphold over generations of developer. You focus =
on
> a single use case
> while there are many out there for distributed ledgers.
>
> I think in an infrastructure for enterprise applications, building consen=
sus
> on the ledger is a
> cornerstone there, but is only a piece of the solution. I built several
> commercially successful
> deployments where I delegated the consensus building to a border router, =
a
> Bitcoin Core,
> then interfaced that trusted peer with my  implementation that accepted
> Core=E2=80=99s decisions
> in an SPV manner. One might think of this setup as wasteful and unsuitabl=
e
> for =E2=80=9Csmall devices=E2=80=9D
> therefore an example of centralization people here try to avoid.
>
> Enterprises have sufficient resources. Solving the business problem is
> valuable to them even at
> magnitudes higher cost than a hobbyist would bear.
>
> For mainstream adoption you need to get enterprises on board too, and  th=
at
> is what I care of.
> Enterprises want code that is not only high quality, but is easy to maint=
ain
> with a development
> team with high attrition. One has to take whatever help is offered for th=
at,
> and one is modern
> languages and runtimes.
>
> Bits of Proof=E2=80=99s own implementation of the scripts was not practic=
ally
> relevant in my commercially
> successful deployments, because of the use of a border router, but it hel=
ped
> development,
> enabling easier debug and precise error feedback esp. end even after Core
> had a reject message.

In fact I have been accused in the past (by at least Peter Todd) of
having "too many cases in mind" or "doing refactors that are good for
altchains".
That's why I'm very cautious about proposing changes that are not
strict improvments in maintainability to bitcoin itself.
But I actually have freicoin, sidechains and private chains (defined
in freimarkets, used in elements alpha as "block signing") in mind.
Some of the consensus changes I have in mind are support for multiple
assets or interest-bearing assets, for example.
But if you need to change the consensus rules you need to change the
code, there's no way around that.
It will be much simpler to only adapt libconsensus to other chains
than it is to adapt the whole Bitcoin Core code base.
Libconsensus can free you from the need of running "border routers"
(which you need to adapt if you depend on them and are supporting
chains with different rules).
When libconsensus has it's own independent repository, will I fork the
project to have a multi-consensus library supporting multiple
different chains (apart from bitcoin and its testchains)? Maybe, I'm
not sure it makes sense, maybe it's just simpler to maintain a
different project for each different chain (ie libfreicoinconsensus,
libbetaconsensus, etc).

> I integrated libconsensus only for the hope that is significantly fastens
> application side tx verification,
>  which it has turned out it does not, until secp265k1 is integrated.

That is very sad to hear. The main reason to integrate libconsensus is
to avoid consensus fork bugs (or to not depend on the "border routers"
to avoid those bugs).

> I would likely use an other extended libconsensus too, but do not think
> there was a dependency on
> that for enterprise development.
>
> It would help there more to have a slim protocol server, no wallet, no rp=
c,
> no qt but a high
> performance remoting API.

That's out of scope for libconsensus which will be stateless and whose
only API would be in C.
But the refactors in Bitcoin Core will hopefully make it easier to
support such a minimal node in it (you know you can "./configure
--disable-wallet --without-gui" already, right?, about RPC, that's the
remaining API!).

> Since you already depend on libconsensus for VerifyScript, wouldn't it
> be nice that it also offered VerifyTx, VerifyHeader and VerifyBlock?
> You would still have complete control over storage, concurrency,
> networking, policy...
> My plan is for the C API to interface with the external storage by
> passing a function pointer to it.
>
>
> Storage and validation is non-trivially interconnected, but I now the
> separation can be done,
> since I did it.
>
> Excuse me, but function pointers is a pattern I used in the 80=E2=80=99s.=
 I know
> that they are behind
> the curtain of modern abstractions with similar use, I still prefer not t=
o
> see them again.

Yes, and the wheel it's an invention used in pre-historic times: that
doesn't make it less useful.
Do you have any other suggestion for interfacing with external storage
using a C API?