1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mh.in.england@gmail.com>) id 1Sggtd-0004yh-IR
for bitcoin-development@lists.sourceforge.net;
Mon, 18 Jun 2012 18:41:53 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 74.125.82.53 as permitted sender)
client-ip=74.125.82.53; envelope-from=mh.in.england@gmail.com;
helo=mail-wg0-f53.google.com;
Received: from mail-wg0-f53.google.com ([74.125.82.53])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1Sggtc-0005NP-I6
for bitcoin-development@lists.sourceforge.net;
Mon, 18 Jun 2012 18:41:53 +0000
Received: by wgbfm10 with SMTP id fm10so4975502wgb.10
for <bitcoin-development@lists.sourceforge.net>;
Mon, 18 Jun 2012 11:41:46 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.216.226.147 with SMTP id b19mr9094149weq.210.1340044903591;
Mon, 18 Jun 2012 11:41:43 -0700 (PDT)
Sender: mh.in.england@gmail.com
Received: by 10.216.254.232 with HTTP; Mon, 18 Jun 2012 11:41:43 -0700 (PDT)
Date: Mon, 18 Jun 2012 20:41:43 +0200
X-Google-Sender-Auth: TGzWPza3bBgzzYpSLwSzQcYwvE0
Message-ID: <CANEZrP2xnsOHyH+a1g6qSNSx_g+TW-yvL0Due7PVr421U6kRLw@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.5 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(mh.in.england[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Sggtc-0005NP-I6
Subject: [Bitcoin-development] LevelDB benchmarking
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 18:41:53 -0000
I switched the transaction database to use the Google LevelDB library,
which is a refactored out part of BigTable.
Here are my results. All tests are done on this hard disk:
http://wdc.custhelp.com/app/answers/detail/a_id/1409/~/specifications-for=
-the-500-gb-caviar-blue-and-caviar-se16-serial-ata-drive
which has an average 8.9msec seek time. It is a 6 core Ubuntu machine.
I used -loadblock on a chain with with 185127 blocks in it, so it has
lots of SatoshiDice traffic.
8.9 ms (average) seek time
>> Regular BDB as we have today:
real 96m6.836s
user 49m55.220s
sys 2m29.850s
Throughput usually 4-5MB/sec according to iotop, pauses of 8-10
seconds for =E2=80=9CFlushing wallet ...=E2=80=9D. 611mb of blkindex.dat
>> BDB without sig checking
Throughput, 12-17mb/sec
real 42m51.508s
user 11m52.700s
sys 2m36.590s
Disabling EC verification halves running time.
>> LevelDB no customized options
(I ran the wrong time command here, hence the different format)
3184.73user 181.02system 51:20.81elapsed 109%CPU (0avgtext+0avgdata
1220096maxresident)k
1104inputs+125851776outputs (293569major+37436202minor)pagefaults 0swaps
So, 50 minutes. Throughput often in range of 20-30mb/sec. 397MB of data fil=
es.
>> LevelDB w/ 10 bit per key bloom filter
real 50m52.740s
user 53m38.870s
sys 3m4.990s
424mb of data files
No change.
>> LevelDB w/ 10 bit per key bloom filter + 30mb cache (up from 8mb by defa=
ult)
real 50m53.054s
user 53m26.910s
sys 3m10.720s
No change. The reason is, signature checking is the bottleneck not IO.
>> LevelDB w/10 bit per key bloom filter, 30mb cache, no sigs
real 12m58.998s
user 11m42.330s
sys 2m5.670s
12 minutes vs 42 minutes for BDB on the same benchmark.
Conclusion: LevelDB is a clear win, taking a sync in the absence of
network delays from 95 minutes to 50, at which point signature
checking becomes the bottleneck. It is nearly 4x as fast when
signature checks are not done (ie, when receiving a block containing
only mempool transactions you already verified).
|
