1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
|
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <imichaelmiers@gmail.com>) id 1TJoKs-0006w9-AV
for bitcoin-development@lists.sourceforge.net;
Thu, 04 Oct 2012 16:31:42 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.216.47 as permitted sender)
client-ip=209.85.216.47; envelope-from=imichaelmiers@gmail.com;
helo=mail-qa0-f47.google.com;
Received: from mail-qa0-f47.google.com ([209.85.216.47])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1TJoKm-0008M0-T3
for bitcoin-development@lists.sourceforge.net;
Thu, 04 Oct 2012 16:31:42 +0000
Received: by mail-qa0-f47.google.com with SMTP id i29so3150929qaf.13
for <bitcoin-development@lists.sourceforge.net>;
Thu, 04 Oct 2012 09:31:31 -0700 (PDT)
Received: by 10.229.38.17 with SMTP id z17mr2570460qcd.30.1349368291461; Thu,
04 Oct 2012 09:31:31 -0700 (PDT)
MIME-Version: 1.0
Sender: imichaelmiers@gmail.com
Received: by 10.49.30.40 with HTTP; Thu, 4 Oct 2012 09:31:11 -0700 (PDT)
In-Reply-To: <1349344554.31175.6.camel@mei>
References: <CAEC9zAbHO9y7Go4FhZFyOCKdm1js_eKCoz7iaOCh1Wu=B9OASw@mail.gmail.com>
<CAEC9zAYrMHHEyyTx1QVHoGSJU3fFypB0Hx4K-VFoUn0hp4Z7JA@mail.gmail.com>
<CAEC9zAbkWWcn5eRHLh6u+zyGu92E1Yq-9CQVPqE_6-QwMVs=4Q@mail.gmail.com>
<CA+8xBpdZDznkyWXn8AxdUDk0CJ0b2m1qpTgVjtPbs0ge+ODCSg@mail.gmail.com>
<CAEC9zAZR7xOUEJZz0f-Of0HTVAcL8uCw3tcR1s66Hg_kZbczRg@mail.gmail.com>
<1349344554.31175.6.camel@mei>
From: Ian Miers <imiers1@jhu.edu>
Date: Thu, 4 Oct 2012 12:31:11 -0400
X-Google-Sender-Auth: 8rxvDutk77sNxgvA_66n0gFaQtQ
Message-ID: <CAEC9zAYLks7sybY+HxJD9zjbp_nPyBn8ek8KkJcpat2mMQbLKw@mail.gmail.com>
To: Joel Joonatan Kaartinen <joel.kaartinen@gmail.com>
Content-Type: multipart/alternative; boundary=00504502d6a6af098804cb3e49fc
X-Spam-Score: -0.5 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(imichaelmiers[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1TJoKm-0008M0-T3
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] performance testing for bitcoin
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 04 Oct 2012 16:31:42 -0000
--00504502d6a6af098804cb3e49fc
Content-Type: text/plain; charset=ISO-8859-1
Agreed, you need to consider DoS attacks. I have that in my mind under
security and not performance. I'd imagine the test methodology would be
completely different since you no longer are attempting to simulate real
world traffic.
Ian
On Thu, Oct 4, 2012 at 5:55 AM, Joel Joonatan Kaartinen <
joel.kaartinen@gmail.com> wrote:
> For script evaluation benchmarking, I don't think just a good
> approximation of real-world traffic is enough. You really need to
> benchmark the worst case scenarios, otherwise you could be creating a
> DoS vulnerability.
>
> - Joel
>
> ke, 2012-10-03 kello 13:57 -0400, Ian Miers kirjoitti:
> > Script evaluation performance was what I was primarily concerned
> > with. I'm fooling around with adding some new instruction types.
> > The tricky part is that to test how that effects performance, you need
> > to be able to intersperse transactions with the new instructions with
> > existing ones. For accuracy, you'd like your simulated traffic to at
> > least approximate the real world traffic.
> >
> >
> >
> >
> > Also, is there any bench-marking / instrumentation in bitcoind ?
> >
> >
> > Ian
> > On Wed, Oct 3, 2012 at 1:43 PM, Jeff Garzik <jgarzik@exmulti.com>
> > wrote:
> > On Wed, Oct 3, 2012 at 1:38 PM, Ian Miers <imiers1@jhu.edu>
> > wrote:
> > > Whats the best way to get performance numbers for
> > modifications to bitcoin ?
> > > Profiling it while running on testnet might work, but that
> > would take a
> > > rather long time to get data.
> > > Is there anyway to speed this up if we only needed to
> > provide relative
> > > performance between tests. (in a sense a fast performance
> > regression test).
> >
> >
> > You have to be specific about what you're measuring, because
> > "performance" is vague.
> >
> > You can measure many aspects of blockchain performance by
> > importing
> > blocks via -loadblock=FILE.
> >
> > Other performance measurements like "how fast does a block
> > relay
> > through the network" cannot be as easily measured.
> >
> > --
> > Jeff Garzik
> > exMULTI, Inc.
> > jgarzik@exmulti.com
> >
> >
> >
> ------------------------------------------------------------------------------
> > Don't let slow site performance ruin your business. Deploy New Relic APM
> > Deploy New Relic app performance management and know exactly
> > what is happening inside your Ruby, Python, PHP, Java, and .NET app
> > Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> > http://p.sf.net/sfu/newrelic-dev2dev
> > _______________________________________________ Bitcoin-development
> mailing list Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
--00504502d6a6af098804cb3e49fc
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Agreed, you need to consider DoS attacks. =A0I have that in my mind under s=
ecurity and not performance. I'd imagine the test methodology would be =
completely different since you no longer are attempting to simulate real wo=
rld traffic.<div>
<br></div><div><br><div>Ian<br><br><div class=3D"gmail_quote">On Thu, Oct 4=
, 2012 at 5:55 AM, Joel Joonatan Kaartinen <span dir=3D"ltr"><<a href=3D=
"mailto:joel.kaartinen@gmail.com" target=3D"_blank">joel.kaartinen@gmail.co=
m</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">For script evaluation benchmarking, I don=
9;t think just a good<br>
approximation of real-world traffic is enough. You really need to<br>
benchmark the worst case scenarios, otherwise you could be creating a<br>
DoS vulnerability.<br>
<br>
- Joel<br>
<br>
ke, 2012-10-03 kello 13:57 -0400, Ian Miers kirjoitti:<br>
<div><div class=3D"h5">> Script evaluation performance was what I was pr=
imarily concerned<br>
> with. =A0I'm fooling around with adding some new instruction types=
.<br>
> The tricky part is that to test how that effects performance, you need=
<br>
> to be able to intersperse transactions with the new instructions with<=
br>
> existing ones. =A0For accuracy, you'd like your simulated traffic =
to at<br>
> least approximate the real world traffic.<br>
><br>
><br>
><br>
><br>
> Also, is there any bench-marking / instrumentation in bitcoind ?<br>
><br>
><br>
> Ian<br>
> On Wed, Oct 3, 2012 at 1:43 PM, Jeff Garzik <<a href=3D"mailto:jgar=
zik@exmulti.com">jgarzik@exmulti.com</a>><br>
> wrote:<br>
> =A0 =A0 =A0 =A0 On Wed, Oct 3, 2012 at 1:38 PM, Ian Miers <<a href=
=3D"mailto:imiers1@jhu.edu">imiers1@jhu.edu</a>><br>
> =A0 =A0 =A0 =A0 wrote:<br>
> =A0 =A0 =A0 =A0 > Whats the best way to get performance numbers for=
<br>
> =A0 =A0 =A0 =A0 modifications to bitcoin ?<br>
> =A0 =A0 =A0 =A0 > Profiling it while running on testnet might work,=
but that<br>
> =A0 =A0 =A0 =A0 would take a<br>
> =A0 =A0 =A0 =A0 > rather long time to get data.<br>
> =A0 =A0 =A0 =A0 > Is there anyway to speed this up =A0if we only ne=
eded to<br>
> =A0 =A0 =A0 =A0 provide =A0relative<br>
> =A0 =A0 =A0 =A0 > performance between tests. (in a sense a fast per=
formance<br>
> =A0 =A0 =A0 =A0 regression test).<br>
><br>
><br>
> =A0 =A0 =A0 =A0 You have to be specific about what you're measurin=
g, because<br>
> =A0 =A0 =A0 =A0 "performance" is vague.<br>
><br>
> =A0 =A0 =A0 =A0 You can measure many aspects of blockchain performance=
by<br>
> =A0 =A0 =A0 =A0 importing<br>
> =A0 =A0 =A0 =A0 blocks via -loadblock=3DFILE.<br>
><br>
> =A0 =A0 =A0 =A0 Other performance measurements like "how fast doe=
s a block<br>
> =A0 =A0 =A0 =A0 relay<br>
> =A0 =A0 =A0 =A0 through the network" cannot be as easily measured=
.<br>
><br>
> =A0 =A0 =A0 =A0 --<br>
> =A0 =A0 =A0 =A0 Jeff Garzik<br>
> =A0 =A0 =A0 =A0 exMULTI, Inc.<br>
> =A0 =A0 =A0 =A0 <a href=3D"mailto:jgarzik@exmulti.com">jgarzik@exmulti=
.com</a><br>
><br>
><br>
</div></div>> ----------------------------------------------------------=
--------------------<br>
> Don't let slow site performance ruin your business. Deploy New Rel=
ic APM<br>
> Deploy New Relic app performance management and know exactly<br>
> what is happening inside your Ruby, Python, PHP, Java, and .NET app<br=
>
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!<=
br>
> <a href=3D"http://p.sf.net/sfu/newrelic-dev2dev" target=3D"_blank">htt=
p://p.sf.net/sfu/newrelic-dev2dev</a><br>
> _______________________________________________ Bitcoin-development ma=
iling list <a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bit=
coin-development@lists.sourceforge.net</a> <a href=3D"https://lists.sourcef=
orge.net/lists/listinfo/bitcoin-development" target=3D"_blank">https://list=
s.sourceforge.net/lists/listinfo/bitcoin-development</a><br>
<br>
<br>
</blockquote></div><br></div></div>
--00504502d6a6af098804cb3e49fc--
|
