Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TLjRe-0005yK-Ac for bitcoin-development@lists.sourceforge.net; Tue, 09 Oct 2012 23:42:38 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.47 as permitted sender) client-ip=209.85.219.47; envelope-from=arklan.uthoslin@gmail.com; helo=mail-oa0-f47.google.com; Received: from mail-oa0-f47.google.com ([209.85.219.47]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TLjRd-0002tw-7x for bitcoin-development@lists.sourceforge.net; Tue, 09 Oct 2012 23:42:38 +0000 Received: by mail-oa0-f47.google.com with SMTP id h1so6166712oag.34 for ; Tue, 09 Oct 2012 16:42:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.78.228 with SMTP id e4mr4246903obx.77.1349826151520; Tue, 09 Oct 2012 16:42:31 -0700 (PDT) Received: by 10.76.152.97 with HTTP; Tue, 9 Oct 2012 16:42:31 -0700 (PDT) In-Reply-To: References: Date: Tue, 9 Oct 2012 17:42:31 -0600 Message-ID: From: Arklan Uth Oslin To: Jeff Garzik Content-Type: multipart/alternative; boundary=f46d0445170d45180104cba8e4e2 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (arklan.uthoslin[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TLjRd-0002tw-7x Cc: Bitcoin Development Subject: Re: [Bitcoin-development] On bitcoin testing X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 23:42:38 -0000 --f46d0445170d45180104cba8e4e2 Content-Type: text/plain; charset=ISO-8859-1 thanks for the great reply jeff. i'm going to get a virtual machine set up on my system later tonight so at the very least, i myself can start testing. steve - haven't heard from you in almost a week. I'd still really like to get a look at the test cases and such you set up. Arklan ---------- As long as there is light, the darkness holds no fear. And yet, even in the deepest black, there is life. - Arklan Uth Oslin I want to leave this world the same way I came into it: backwards and on fire. - Arklan Uth Oslin On Tue, Oct 9, 2012 at 5:12 PM, Jeff Garzik wrote: > Copying from a response posted to "Bitcoin software testing effort" > https://bitcointalk.org/index.php?topic=117487.0 as it is relevant to > a recent thread here... > > Any level of testing is useful and appreciated. Various types of > testing that are helpful: > > * "it works" testing: Simply run the latest Release Candidate (or > latest version, if released). Make sure all the basics work (for > whatever definition of "basics" you desire). This is the level most > accessible to casual users. > * Major features testing: Develop a short checklist of must-work > features, and organize volunteers to work together and go through that > checklist, item by item. Test each major feature on each major > platform. > * Stress and fuzz testing: Attempt to "stress" the system somehow, or > randomly corrupt bits of data. See what breaks. > * Regression testing: Record bugs fixed, and develop automated test > cases that successfully reproduce the bugs on older versions, and > verify newer versions remain fixed. > * Unit function testing: Rigorously exercise each C++ class to ensure > it behaves as expected at a micro level. > * Full peer automated testing: Automated testing of RPC and P2P > functions is non-existent, because of the difficulty in doing so. > Find a solution to this problem. > * Data-driven tests: If possible, write software-neutral, data-driven > tests. This enables clients other than the reference one (Satoshi > client) to be tested. Embed tests in testnet3 chain, if possible. > > > The community at large can be a big help simply by doing the first > item: download and run the Release Candidates and the latest version, > and report any problems. Even reporting success is fine by me, for > example: "Version 0.7.1 works for me on Windows 7/32-bit" posted on a > forum thread. > > It is always very difficult to organize any sort of testing regime > with open source volunteers that come and go. Each volunteer chooses > their level of involvement. Any amount of testing and test-case > writing, large or small, is helpful to bitcoin. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti.com > > > ------------------------------------------------------------------------------ > Don't let slow site performance ruin your business. Deploy New Relic APM > Deploy New Relic app performance management and know exactly > what is happening inside your Ruby, Python, PHP, Java, and .NET app > Try New Relic at no cost today and get our sweet Data Nerd shirt too! > http://p.sf.net/sfu/newrelic-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > --f46d0445170d45180104cba8e4e2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable thanks for the great reply jeff. i'm going to get a virtual machine set= up on my system later tonight so at the very least, i myself can start tes= ting.

steve - haven't heard from you in almost a wee= k. I'd still really like to get a look at the test cases and such you s= et up.
=A0
Arklan

----------
As long as there is light, the darkness holds no fear. And yet, even i= n the deepest black, there is life. - Arklan Uth Oslin
=A0
I want to leave this world the same way I came into it: backwards and = on fire. - Arklan Uth Oslin



On Tue, Oct 9, 2012 at 5:12 PM, Jeff Gar= zik <jgarzik@exmulti.com> wrote:
Copying from a response posted to "Bitcoin software testing effort&quo= t;
https://bitcointalk.org/index.php?topic=3D117487.0 as it is relev= ant to
a recent thread here...

Any level of testing is useful and appreciated. =A0Various types of
testing that are helpful:

* "it works" testing: =A0Simply run the latest Release Candidate = (or
latest version, if released). =A0Make sure all the basics work (for
whatever definition of "basics" you desire). =A0This is the level= most
accessible to casual users.
* Major features testing: =A0Develop a short checklist of must-work
features, and organize volunteers to work together and go through that
checklist, item by item. =A0Test each major feature on each major
platform.
* Stress and fuzz testing: =A0Attempt to "stress" the system some= how, or
randomly corrupt bits of data. =A0See what breaks.
* Regression testing: =A0Record bugs fixed, and develop automated test
cases that successfully reproduce the bugs on older versions, and
verify newer versions remain fixed.
* Unit function testing: =A0Rigorously exercise each C++ class to ensure it behaves as expected at a micro level.
* Full peer automated testing: =A0Automated testing of RPC and P2P
functions is non-existent, because of the difficulty in doing so.
Find a solution to this problem.
* Data-driven tests: If possible, write software-neutral, data-driven
tests. =A0This enables clients other than the reference one (Satoshi
client) to be tested. =A0Embed tests in testnet3 chain, if possible.


The community at large can be a big help simply by doing the first
item: =A0download and run the Release Candidates and the latest version, and report any problems. =A0Even reporting success is fine by me, for
example: "Version 0.7.1 works for me on Windows 7/32-bit" posted = on a
forum thread.

It is always very difficult to organize any sort of testing regime
with open source volunteers that come and go. =A0Each volunteer chooses
their level of involvement. =A0Any amount of testing and test-case
writing, large or small, is helpful to bitcoin.

--
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com

---------------------------------------------------------------------------= ---
Don't let slow site performance ruin your business. Deploy New Relic AP= M
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p= .sf.net/sfu/newrelic-dev2dev
_______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--f46d0445170d45180104cba8e4e2--