1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mh.in.england@gmail.com>) id 1TtfKs-0005K3-Bp
for bitcoin-development@lists.sourceforge.net;
Fri, 11 Jan 2013 14:11:54 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.214.178 as permitted sender)
client-ip=209.85.214.178; envelope-from=mh.in.england@gmail.com;
helo=mail-ob0-f178.google.com;
Received: from mail-ob0-f178.google.com ([209.85.214.178])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1TtfKo-0002Wn-Jp
for bitcoin-development@lists.sourceforge.net;
Fri, 11 Jan 2013 14:11:54 +0000
Received: by mail-ob0-f178.google.com with SMTP id eh20so1768684obb.9
for <bitcoin-development@lists.sourceforge.net>;
Fri, 11 Jan 2013 06:11:45 -0800 (PST)
MIME-Version: 1.0
Received: by 10.60.29.70 with SMTP id i6mr46385697oeh.38.1357913505299; Fri,
11 Jan 2013 06:11:45 -0800 (PST)
Sender: mh.in.england@gmail.com
Received: by 10.76.128.139 with HTTP; Fri, 11 Jan 2013 06:11:44 -0800 (PST)
In-Reply-To: <CA+8xBpcB6kXWyRbeUknK6gkcrFMV6YtrDk0c938q1_32U6GtRw@mail.gmail.com>
References: <CANEZrP0XALwBFJyZTzYd5xBp4MRrjv0s_y2tOXbO7UgjWF2HzA@mail.gmail.com>
<20121121151534.GA5540@vps7135.xlshosting.net>
<1353523117.1085.14.camel@localhost.localdomain>
<20121127211019.GA22701@vps7135.xlshosting.net>
<CANEZrP0w052ebao-04H4Wduerm86o6RKBY=ObnJXBX22k--zMA@mail.gmail.com>
<1357876751.1740.4.camel@localhost.localdomain>
<CA+8xBpcB6kXWyRbeUknK6gkcrFMV6YtrDk0c938q1_32U6GtRw@mail.gmail.com>
Date: Fri, 11 Jan 2013 15:11:44 +0100
X-Google-Sender-Auth: llF7-Of9-7B__9_YzOxfYH0H2_I
Message-ID: <CANEZrP2k30UsWFYSZ7Bh5Hm4LJ9vEAMEUgYSrYkcXcDTY2Z79Q@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Jeff Garzik <jgarzik@exmulti.com>
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -1.5 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(mh.in.england[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1TtfKo-0002Wn-Jp
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Fri, 11 Jan 2013 14:11:54 -0000
I did some very rough initial performance tests.
Syncing from a local peer gives me about 50 blocks per second in the
later parts of the chain (post SD), which is about a 10-20x speedup
over what I could do before. This is on a MacBook Pro. But at those
points it's clearly bottlenecked by bitcoind which has saturated its
CPU core. This makes sense - the filtering is much more server than
client intensive because every transaction in every block has to be
loaded and checked.
I think filtering can be fairly well parallelized on the server side.
So the current 10-20x speedup could potentially be larger if the
server becomes more efficient at scanning and filtering blocks. It's
still a very nice win for now, especially bandwidth wise. And if Matt
makes the mempool command filtered it solves a common usability
problem as well.
Once we get this code in, merged and rolled out I think what we need
for bloom v2 is clear:
- Multi-thread the filtering process in bitcoind so transactions can
be checked in parallel. A 4-core server would then get 4x faster at
filtering blocks and assuming it's not too busy doing other stuff we
could maybe sync at more like 200 blocks per second, which is cool ...
more than a days worth of history for each second of syncing.
- Make the client smarter so the FP rate is adapted during the sync
process. An FP rate that makes sense post-SD results in no false
positives pre-SD, more or less.
- Make the client shard its wallet keys over multiple peers, for
better privacy.
- Make the client suck down filtered blocks in parallel from multiple
peers, for better speed.
As it seems the bottleneck for chain sync is now CPU time, the latter
point may be the most important from a practical perspective.
On Fri, Jan 11, 2013 at 6:02 AM, Jeff Garzik <jgarzik@exmulti.com> wrote:
> On Thu, Jan 10, 2013 at 10:59 PM, Matt Corallo <bitcoin-list@bluematt.me> wrote:
>> Ive been missing lately, when is 0.8 targeted for freeze?
>
> 0.8rc1 will probably happen when the core ultraprune/leveldb stuff is stable.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti.com
|
