From: "Eric Lombrozo" <elombrozo@gmail.com>
To: =?utf-8?q?Jorge=20Tim=c3=b3n?= <jtimon@jtimon.cc>,
	"Wladimir J. van der Laan" <laanwj@gmail.com>
Date: Wed, 23 Sep 2015 00:07:22 +0000
Message-Id: <em9a15c95a-cdb0-451f-8b69-e73572a722d2@platinum>
In-Reply-To: <CABm2gDrx4uDVPFL_tWUsvh_cN6qV903WCmgpA_Kj9Deq+Q3yfA@mail.gmail.com>
Reply-To: "Eric Lombrozo" <elombrozo@gmail.com>
User-Agent: eM_Client/6.0.23181.0
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Cc: Bitcoin development mailing list <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Long-term vision for bitcoind (was libconsensus
 and bitcoin development process)
Precedence: list

Here's what I propose as a long-term plan:

1) libconsensus
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
We should probably start by defining an API for libconsensus. It should=20
support an abstract DB model, track chain state, provide query=20
mechanisms for blocks and transactions with optional pruning and=20
indexing, expose a subscription mechanism for events such as NEW_TIP,=20
REORG, etc, and contain a script interpreter.

We can develop the library in parallel with Bitcoin Core without too=20
much refactoring of Bitcoin Core itself...just moving pieces of Bitcoin=20
Core's consensus code into the new library, tracking code movements to=20
make merging easier. Yes, this is a bit ugly as it requires code=20
duplication...but it will temporarily avoid much of the downstream=20
pushback we're getting. The idea is that we can prove out the library=20
with some simple projects, then start removing the consensus stuff from=20
Bitcoin Core once we have greater acceptance of the library and better=20
documentation.


2) peer services
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
We develop a peer services library that performs the tasks of peer=20
discovery and relay, with the ability to connect to appropriate peers=20
and queue messages. It uses libconsensus for all validation=20
functionality and as a datastore for the consensus state but maintains=20
its own database for peer history and statistics. I believe Cory has=20
been working on this already using libevent. I've already developed an=20
async library for this as well.


3) API/RPC
=3D=3D=3D=3D=3D=3D=3D
We provide high level calls and pub/sub mechanisms. ZMQ has been=20
implemented and added already, but we could support other transports as=20
well.


4) Wallet
=3D=3D=3D=3D=3D=3D
The wallet is split out into a separate process that connects to the=20
stack via the API/RPC layer.


- Eric

------ Original Message ------
From: "Jorge Tim=C3=B3n" <bitcoin-dev@lists.linuxfoundation.org>
To: "Wladimir J. van der Laan" <laanwj@gmail.com>
Cc: "Bitcoin development mailing list"=20
<bitcoin-dev@lists.linuxfoundation.org>
Sent: 9/22/2015 11:36:14 AM
Subject: [bitcoin-dev] Long-term vision for bitcoind (was libconsensus=20
and bitcoin development process)

>On Fri, Sep 18, 2015 at 2:07 AM, Wladimir J. van der Laan via
>bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>  My long-term vision of bitcoind is a P2P node with validation and=20
>>blockchain store, with a couple of data sources that can be subscribed=
=20
>>to or pulled from.
>
>I agree with this long term vision.
>Here's how I think it could happen:
>
>1) Libconsensus is completed and moved to a subtree (which has libsecp
>as an internal subtree)
>
>2) Bitcoind becomes a subtree of bitcoin-wallet (which has
>bitcoin-wallet and bitcoin-qt)
>
>Without aggressively changing it for this purpose, libconsensus should
>tend to become C, like libsecp, which is better for proving
>correctness.
>Hopefully at some point it won't take much to move to C.
>
>Upper layers should move to C++11
>
>Don't focus on the git subtrees, the basic architecture is bitcoin-qt
>on top of bitcoin-wallet, bitcoin-wallet on top of bitcoind (and
>friends like bitcoin-cli and bitcoin-tx), bitcoind on top of
>libconsensus on top of libsecp256k1.
>
>I believe this would maximize the number of people who can safely
>contribute to the project.
>I also believe this is the architecture most contributors have in mind
>for the long term, but I may be wrong about it.
>
>Criticisms to this plan?
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev