Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TJoKs-0006w9-AV for bitcoin-development@lists.sourceforge.net; Thu, 04 Oct 2012 16:31:42 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.216.47 as permitted sender) client-ip=209.85.216.47; envelope-from=imichaelmiers@gmail.com; helo=mail-qa0-f47.google.com; Received: from mail-qa0-f47.google.com ([209.85.216.47]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TJoKm-0008M0-T3 for bitcoin-development@lists.sourceforge.net; Thu, 04 Oct 2012 16:31:42 +0000 Received: by mail-qa0-f47.google.com with SMTP id i29so3150929qaf.13 for ; Thu, 04 Oct 2012 09:31:31 -0700 (PDT) Received: by 10.229.38.17 with SMTP id z17mr2570460qcd.30.1349368291461; Thu, 04 Oct 2012 09:31:31 -0700 (PDT) MIME-Version: 1.0 Sender: imichaelmiers@gmail.com Received: by 10.49.30.40 with HTTP; Thu, 4 Oct 2012 09:31:11 -0700 (PDT) In-Reply-To: <1349344554.31175.6.camel@mei> References: <1349344554.31175.6.camel@mei> From: Ian Miers Date: Thu, 4 Oct 2012 12:31:11 -0400 X-Google-Sender-Auth: 8rxvDutk77sNxgvA_66n0gFaQtQ Message-ID: To: Joel Joonatan Kaartinen Content-Type: multipart/alternative; boundary=00504502d6a6af098804cb3e49fc X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (imichaelmiers[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TJoKm-0008M0-T3 Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] performance testing for bitcoin X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:31:42 -0000 --00504502d6a6af098804cb3e49fc Content-Type: text/plain; charset=ISO-8859-1 Agreed, you need to consider DoS attacks. I have that in my mind under security and not performance. I'd imagine the test methodology would be completely different since you no longer are attempting to simulate real world traffic. Ian On Thu, Oct 4, 2012 at 5:55 AM, Joel Joonatan Kaartinen < joel.kaartinen@gmail.com> wrote: > For script evaluation benchmarking, I don't think just a good > approximation of real-world traffic is enough. You really need to > benchmark the worst case scenarios, otherwise you could be creating a > DoS vulnerability. > > - Joel > > ke, 2012-10-03 kello 13:57 -0400, Ian Miers kirjoitti: > > Script evaluation performance was what I was primarily concerned > > with. I'm fooling around with adding some new instruction types. > > The tricky part is that to test how that effects performance, you need > > to be able to intersperse transactions with the new instructions with > > existing ones. For accuracy, you'd like your simulated traffic to at > > least approximate the real world traffic. > > > > > > > > > > Also, is there any bench-marking / instrumentation in bitcoind ? > > > > > > Ian > > On Wed, Oct 3, 2012 at 1:43 PM, Jeff Garzik > > wrote: > > On Wed, Oct 3, 2012 at 1:38 PM, Ian Miers > > wrote: > > > Whats the best way to get performance numbers for > > modifications to bitcoin ? > > > Profiling it while running on testnet might work, but that > > would take a > > > rather long time to get data. > > > Is there anyway to speed this up if we only needed to > > provide relative > > > performance between tests. (in a sense a fast performance > > regression test). > > > > > > You have to be specific about what you're measuring, because > > "performance" is vague. > > > > You can measure many aspects of blockchain performance by > > importing > > blocks via -loadblock=FILE. > > > > Other performance measurements like "how fast does a block > > relay > > through the network" cannot be as easily measured. > > > > -- > > Jeff Garzik > > exMULTI, Inc. > > jgarzik@exmulti.com > > > > > > > ------------------------------------------------------------------------------ > > Don't let slow site performance ruin your business. Deploy New Relic APM > > Deploy New Relic app performance management and know exactly > > what is happening inside your Ruby, Python, PHP, Java, and .NET app > > Try New Relic at no cost today and get our sweet Data Nerd shirt too! > > http://p.sf.net/sfu/newrelic-dev2dev > > _______________________________________________ Bitcoin-development > mailing list Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > --00504502d6a6af098804cb3e49fc Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Agreed, you need to consider DoS attacks. =A0I have that in my mind under s= ecurity and not performance. I'd imagine the test methodology would be = completely different since you no longer are attempting to simulate real wo= rld traffic.


Ian

On Thu, Oct 4= , 2012 at 5:55 AM, Joel Joonatan Kaartinen <joel.kaartinen@gmail.co= m> wrote:
For script evaluation benchmarking, I don= 9;t think just a good
approximation of real-world traffic is enough. You really need to
benchmark the worst case scenarios, otherwise you could be creating a
DoS vulnerability.

- Joel

ke, 2012-10-03 kello 13:57 -0400, Ian Miers kirjoitti:
> Script evaluation performance was what I was pr= imarily concerned
> with. =A0I'm fooling around with adding some new instruction types= .
> The tricky part is that to test how that effects performance, you need=
> to be able to intersperse transactions with the new instructions with<= br> > existing ones. =A0For accuracy, you'd like your simulated traffic = to at
> least approximate the real world traffic.
>
>
>
>
> Also, is there any bench-marking / instrumentation in bitcoind ?
>
>
> Ian
> On Wed, Oct 3, 2012 at 1:43 PM, Jeff Garzik <jgarzik@exmulti.com>
> wrote:
> =A0 =A0 =A0 =A0 On Wed, Oct 3, 2012 at 1:38 PM, Ian Miers <imiers1@jhu.edu>
> =A0 =A0 =A0 =A0 wrote:
> =A0 =A0 =A0 =A0 > Whats the best way to get performance numbers for=
> =A0 =A0 =A0 =A0 modifications to bitcoin ?
> =A0 =A0 =A0 =A0 > Profiling it while running on testnet might work,= but that
> =A0 =A0 =A0 =A0 would take a
> =A0 =A0 =A0 =A0 > rather long time to get data.
> =A0 =A0 =A0 =A0 > Is there anyway to speed this up =A0if we only ne= eded to
> =A0 =A0 =A0 =A0 provide =A0relative
> =A0 =A0 =A0 =A0 > performance between tests. (in a sense a fast per= formance
> =A0 =A0 =A0 =A0 regression test).
>
>
> =A0 =A0 =A0 =A0 You have to be specific about what you're measurin= g, because
> =A0 =A0 =A0 =A0 "performance" is vague.
>
> =A0 =A0 =A0 =A0 You can measure many aspects of blockchain performance= by
> =A0 =A0 =A0 =A0 importing
> =A0 =A0 =A0 =A0 blocks via -loadblock=3DFILE.
>
> =A0 =A0 =A0 =A0 Other performance measurements like "how fast doe= s a block
> =A0 =A0 =A0 =A0 relay
> =A0 =A0 =A0 =A0 through the network" cannot be as easily measured= .
>
> =A0 =A0 =A0 =A0 --
> =A0 =A0 =A0 =A0 Jeff Garzik
> =A0 =A0 =A0 =A0 exMULTI, Inc.
> =A0 =A0 =A0 =A0 jgarzik@exmulti= .com
>
>
> ----------------------------------------------------------= --------------------
> Don't let slow site performance ruin your business. Deploy New Rel= ic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app > Try New Relic at no cost today and get our sweet Data Nerd shirt too!<= br> > htt= p://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________ Bitcoin-development ma= iling list Bit= coin-development@lists.sourceforge.net https://list= s.sourceforge.net/lists/listinfo/bitcoin-development



--00504502d6a6af098804cb3e49fc--