Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.215.175 as permitted sender)
	client-ip=209.85.215.175; envelope-from=gavinandresen@gmail.com;
	helo=mail-ey0-f175.google.com; 
MIME-Version: 1.0
In-Reply-To: <CANEZrP2q9a_0rFh+oo6iUFF1goWs0OJO1xPvxC9zqNA-6VnFAQ@mail.gmail.com>
References: <CANEZrP2xnsOHyH+a1g6qSNSx_g+TW-yvL0Due7PVr421U6kRLw@mail.gmail.com>
	<CAAS2fgTNqUeYy+oEFyQWrfs4Xyb=3NXutvCmLusknF-18JmFQg@mail.gmail.com>
	<CANEZrP2q9a_0rFh+oo6iUFF1goWs0OJO1xPvxC9zqNA-6VnFAQ@mail.gmail.com>
Date: Tue, 19 Jun 2012 11:05:21 -0400
Message-ID: <CABsx9T3pQFqL0xsvRfnixYEATO61qMCCDdLmtqZkbVLW0Vxytg@mail.gmail.com>
From: Gavin Andresen <gavinandresen@gmail.com>
To: Mike Hearn <mike@plan99.net>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] LevelDB benchmarking
Precedence: list

> OK, to make progress on this work I need a few decisions (Gavin?)
>
> 1) Shall we do it?

What problem does it solve?

If the problem it will solve is "it will only take 4 hours to download
the entire blockchain next year instead of taking 16 hours" then no, I
don't think we should do it, both 4 and 16 hours to get fully up and
running is too long.

If the problem it will solve is the "too easy to get a DB_RUNRECOVERY
error" because bdb is fragile when it comes to its environment... then
LevelDB looks very interesting.

If the problem is bdb is creaky and old and has obscure semantics and
a hard-to-work-with API, then yes, lets switch (I'm easily seduced by
a pretty API and blazing fast performance).

> 2) LevelDB is obscure, new and has a very minimalist build system. It
> supports "make" but not "make install", for example, and is unlikely
> to be packaged. It's also not very large. I suggest we just check the
> source into the main Bitcoin tree and link it statically rather than
> complicate the build.

As long as it compiles and runs on mac/windows/linux that doesn't
really worry me. I just tried it, and it compiled quickly with no
complaints on my mac.

Lack of infrastructure because it is new does worry me; for example,
could I rework bitcointools to read the LevelDB blockchain?  (are
there python bindings for LevelDB?)

> 3) As the DB format would change and a slow migration period
> necessary, any other tweaks to db format we could make at the same
> time? Right now the key/values are the same as before, though using
> satoshi serialization for everything is a bit odd.

Satoshi rolled his own network serialization because he didn't trust
existing serialization solutions to be 100% secure against remote
exploits. Then it made sense to use the same solution for disk
serialization; I don't see a compelling reason to switch to some other
serialization scheme.

Modifying the database schema during migration to better support
applications like InstaWallet (tens of thousands of separate wallets)
or something like Pieter's ultra-pruning makes sense.

-- 
--
Gavin Andresen