Message-ID: <52388CA6.2050709@monetize.io>
Date: Tue, 17 Sep 2013 10:08:54 -0700
From: Mark Friedenbach <mark@monetize.io>
Organization: Monetize.io Inc.
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8;
	rv:17.0) Gecko/20130801 Thunderbird/17.0.8
MIME-Version: 1.0
To: bitcoin-development@lists.sourceforge.net
References: <CANEZrP1ksdGfB-282giysF-2Lo==NiD-nBdon5FJ9wLBLmX=TA@mail.gmail.com>
In-Reply-To: <CANEZrP1ksdGfB-282giysF-2Lo==NiD-nBdon5FJ9wLBLmX=TA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Subject: Re: [Bitcoin-development] Faster databases than LevelDB
Precedence: list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Also somewhat related, I have been looking for some time now to
abstract out the UTXO and block databases so that a variety of
key/value stores could be used as a backend, configured by a command
line parameter. In particular, it would be interesting for some server
applications to support HyperDex, which is basically a distributed,
fault-tolerant version of LevelDB:

http://hyperdex.org/

By the same mechanism you could just as easily support a Sophia backend.

Mark


On 9/17/13 4:00 AM, Mike Hearn wrote:
> LevelDB is fast - very fast if you give it enough CPU time and
> disk seeks. But it's not the last word in performance.
> 
> HyperLevelDB is a forked LevelDB with some changes, mostly, finer 
> grained locking and changes to how compaction works:
> 
> http://hyperdex.org/performance/leveldb/
> 
> However, it comes with a caveat - one of the changes they made is
> to take away write throttling if compaction falls behind, the app
> itself is expected to do that.
> 
> Sophia is a competitor to LevelDB. The website claims that in
> benchmarks it completely smokes LevelDB. I have not explored how it
> does this or tried to replicate their benchmarks myself:
> 
> http://sphia.org/index.html http://sphia.org/benchmarks.html
> 
> It's written in C and BSD licensed.
> 
> As an example of the kind of speedup they claim to be capable of,
> they say LevelDB could do 167,476 random reads per second on their
> SSD based machine. Sophia could do 438,084 reads/sec. Random reads
> are of course the most interesting for us because that's what UTXO
> lookups involve.
> 
> They also compare against HyperLevelDB, where the differences are
> much less pronounced and actually HyperLevelDB appears to be able
> to do random writes faster than Sophia.
> 
> 
> 
> 
> ------------------------------------------------------------------------------
>
> 
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
> SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New
> Multi-Library Power Pack includes Mobile, Cloud, Java, and UX
> Design. Lowest price ever! Ends 9/20/13. 
> http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
>
> 
> 
> 
> _______________________________________________ Bitcoin-development
> mailing list Bitcoin-development@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSOIymAAoJEAdzVfsmodw4H48QALC+ae4wRLEg3lrg9sgayfOn
ukLM079PXgEbARFPt6WxkLnNGYzEbb7IzT0uvaKH4VIW/rrORy9VqNPmliF+834h
XygUwfAzU04K/oLyCsdWZcOugj2P8aufNeA6whLS5IijDLtHb3Ueu4ORNcfLBGqp
KKfqPj0QHseusiLJ9f3IW+LrdM1vAoT1jryTngpQy2i+qFFDM6CN3THCq4adJvjr
AnYlfLoJSZ0/obz/krwLv6vP1BbwxXzv5CfD0Q2bdoEV/EgWDP3Bd5tUzUCjj53/
qMmhaACoVlarohh64s3JNSDSkHDFSbHFt65ZgNQbNY1wmSeyilQcd8FGWOF/WRzW
Z/pl2IdhoCm3t86xSggRGivj/EVeBJlD36i7ohpDbVWFPsf6B4e5M6xSdso/2WBp
fr55TwehCaGE+UHa0gITkE/si1txvY4gti0bLNvwFDEcZ3qsXRsz4CyLlZLMBbPX
4aRNGyqv2yJ2AivkEyNOUugo1Q8RKEKZWfWWDecI53DHdebzKX1zu9GLJwlGJqGw
Qzm7Tdb7S8J/D6IIHf4Xq2LDhQ2fnPylmGSmtuVFEMxeDhmdbNqKSr3kqlWQf3T8
Oa8bm6kUQFJ+11jLEkVEGZJC4e42+faQBxR+CsqvVsTEezDCP1dE7D3QV8ry9YBc
DwXt3299Q03B5LoxpWTq
=KseH
-----END PGP SIGNATURE-----