1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
|
Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <me@ricmoo.com>) id 1Wsbhc-0007bU-9c
for bitcoin-development@lists.sourceforge.net;
Thu, 05 Jun 2014 17:43:48 +0000
X-ACL-Warn:
Received: from mail-ig0-f170.google.com ([209.85.213.170])
by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1Wsbha-0005pn-LF
for bitcoin-development@lists.sourceforge.net;
Thu, 05 Jun 2014 17:43:48 +0000
Received: by mail-ig0-f170.google.com with SMTP id h3so3175703igd.3
for <bitcoin-development@lists.sourceforge.net>;
Thu, 05 Jun 2014 10:43:40 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:content-type:mime-version:subject:from
:in-reply-to:date:cc:message-id:references:to;
bh=IsEQ6F7TrjVLLS6MtQMW8/IrEqMWCVyS8J+FrquOgt4=;
b=kGaF5pljGBRWz0g9M000VWmtHs+ZTo28I2ad6z/uniSp8azIKVQ8vzT9P2NWleHWYO
mMFOOBT8GHlP3k5L79i05Lq8Khog9w2pda6FkLcrEQeXfoFOAe04qTriJMkdMCNTcJwA
eawPZ/wmsrM1ccT9XRIbtSyqyBKf1PsYCsrYcLffNqtDw0/qhxWMXMwB7kQ8h3bc8uTY
HgCakOXNzXB0BaO2QprzfkSFaNniyjH2CBnYCaD6HXDbhCskN9bm856e6i6UmafgqVYf
loyHV7Mu9dMNEfcF77nK2I/H9MsyxQE3dxuzSKPPgrrsaBVh+Ly1AlWJfxstpDtJQ2lr
7Z/g==
X-Gm-Message-State: ALoCoQnRM75EjbUW9pzBJ87MljIchj70/sr5sZT3MiLOwIXGdMhhd56SfOqhVsFfP3UfHn3PH9wq
X-Received: by 10.50.153.8 with SMTP id vc8mr22633926igb.16.1401990220744;
Thu, 05 Jun 2014 10:43:40 -0700 (PDT)
Received: from [192.168.2.22] (bas5-toronto47-2925108301.dsl.bell.ca.
[174.89.156.77])
by mx.google.com with ESMTPSA id f9sm20018612igc.15.2014.06.05.10.43.39
for <multiple recipients>
(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
Thu, 05 Jun 2014 10:43:39 -0700 (PDT)
Content-Type: multipart/alternative;
boundary="Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46"
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\))
From: Richard Moore <me@ricmoo.com>
In-Reply-To: <CANEZrP1vVY1jnV8Kdg5OWEt=Rba2PPcs3ke4tMWc6fS4wGPpOw@mail.gmail.com>
Date: Thu, 5 Jun 2014 13:43:38 -0400
Message-Id: <2ADB9019-825E-410A-ADED-A7237CBC323C@ricmoo.com>
References: <34798C1C-FDA7-4A4C-B136-DBD4E59C254D@ricmoo.com>
<CANEZrP1vVY1jnV8Kdg5OWEt=Rba2PPcs3ke4tMWc6fS4wGPpOw@mail.gmail.com>
To: Mike Hearn <mike@plan99.net>
X-Mailer: Apple Mail (2.1878.2)
X-Spam-Score: 0.9 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Wsbha-0005pn-LF
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Future Feature Proposal - getgist
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 05 Jun 2014 17:43:48 -0000
--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=windows-1252
I was considering names like getcheckpoints() to use the terminology =
that already seemed to be in place, but they were too long :)
I have been using getheaders() in my thick client to quickly grab all =
the headers before downloading the full blocks since I can grab more at =
a time. Even with getblocks(), there is the case for a getgist() call. =
Right now you call getblocks(), which can take some time to get the =
corresponding inv(), at which time you can then start the call to =
getdata() as well as the next call to getblocks().
With a gist, for example of segment_count 50, you could call getgist(), =
then with the response, you could request 50 getblocks() each with a =
block_locator of 1 hash from the gist (and optimally the stop_hash of =
the next hash in the gist) to 50 different peers, providing 25,000 (50 x =
500) block hashes.
Currently:
>>> getblocks()
<<< inv()
>>> getdata()
<<< block(), block(), block(), =85 (x 500)
Saturates one peer, while leaving the rest un-used. Step 1 and 2 can be =
repeated and dispatched to different peers, but there is still the =
latency between the two calls.
Gist:
>>> getgist()
<<< inv()
>>> getblocks(), getblocks(), getblocks(), =85 (x segment_count, 1 per =
peer)
<<< inv(), inv(), inv(), =85 (x segment_count, 1 per peer)
>>> getdata(), getdata(), getdata(), =85 (x segment_count, 1 per peer)
<<< block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in =
per peer)
Each peer can be saturated.
I will try to run some experiments this weekend to get numbers as to =
whether there is actually any performance improvement using a gist, or =
whether the getdata(), block() latency ends up dominating the time =
anyways.
RicMoo
On Jun 4, 2014, at 11:42 PM, Mike Hearn <mike@plan99.net> wrote:
> Why do you want to optimise this? getheaders is used by SPV clients =
not full nodes. SPV clients like bitcoinj can and do simply ship with =
gist files in them, then getheaders from the last "checkpoint" (I wish =
I hadn't reused terminology like that but this is what bitcoinj calls =
them).
>=20
> In practice when I look at performance issues with current SPV =
clients, getheaders speed is not on my radar.
.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=
=B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA>
Richard Moore ~ Founder
Genetic Mistakes Software inc.
phone: (778) 882-6125
email: ricmoo@geneticmistakes.com
www: http://GeneticMistakes.com
--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=windows-1252
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"font-size: 13px;">I was considering names like getcheckpoints() =
to use the terminology that already seemed to be in place, but they were =
too long :)</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">I have been using getheaders() in my thick =
client to quickly grab all the headers before downloading the full =
blocks since I can grab more at a time. Even with getblocks(), there is =
the case for a getgist() call. Right now you call getblocks(), =
which can take some time to get the corresponding inv(), at which time =
you can then start the call to getdata() as well as the next call to =
getblocks().</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">With a gist, for example of segment_count 50, =
you could call getgist(), then with the response, you could request 50 =
getblocks() each with a block_locator of 1 hash from the gist (and =
optimally the stop_hash of the next hash in the gist) to 50 different =
peers, providing 25,000 (50 x 500) block hashes.</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">Currently:</div><div style=3D"font-size: =
13px;"><ol><li>>>> getblocks()</li><li><<< =
inv()</li><li>>>> getdata()</li><li><<< block(), =
block(), block(), =85 (x 500)</li></ol></div><div style=3D"font-size: =
13px;"><br></div><div style=3D"font-size: 13px;">Saturates one peer, =
while leaving the rest un-used. Step 1 and 2 can be repeated and =
dispatched to different peers, but there is still the latency between =
the two calls.</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">Gist:</div><div style=3D"font-size: =
13px;"><ol><li>>>> getgist()</li><li><<< =
inv()</li><li>>>> getblocks(), getblocks(), getblocks(), =85 (x =
segment_count, 1 per peer)</li><li><<< inv(), inv(), inv(), =85 =
(x segment_count, 1 per peer)</li><li>>>> getdata(), getdata(), =
getdata(), =85 (x segment_count, 1 per peer)</li><li><<< =
block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in per =
peer)</li></ol></div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">Each peer can be saturated.</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: 13px;">I =
will try to run some experiments this weekend to get numbers as to =
whether there is actually any performance improvement using a gist, or =
whether the getdata(), block() latency ends up dominating the time =
anyways.</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">RicMoo</div><div><div><br></div><div><br></div><div>On Jun 4, =
2014, at 11:42 PM, Mike Hearn <<a =
href=3D"mailto:mike@plan99.net">mike@plan99.net</a>> wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
dir=3D"ltr">Why do you want to optimise this? getheaders is used by SPV =
clients not full nodes. SPV clients like bitcoinj can and do simply ship =
with gist files in them, then getheaders from the last "checkpoint" =
(I wish I hadn't reused terminology like that but this is what =
bitcoinj calls them).<div>
<br></div><div>In practice when I look at performance issues with =
current SPV clients, getheaders speed is not on my radar.</div></div>
</blockquote></div><br><div apple-content-edited=3D"true">
<span class=3D"Apple-style-span" style=3D"border-collapse: separate; =
border-spacing: =
0px;">.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=
=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA><br><br>Richard Moore =
~ Founder<br>Genetic Mistakes Software inc.<br>phone: (778) =
882-6125<br>email: <a =
href=3D"mailto:ricmoo@geneticmistakes.com">ricmoo@geneticmistakes.com</a><=
br>www: <a =
href=3D"http://GeneticMistakes.com/">http://GeneticMistakes.com</a></span>=
</div>
<br></body></html>=
--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46--
|
