Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
	helo=mx.sourceforge.net)
	by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <arklan.uthoslin@gmail.com>) id 1TLjRe-0005yK-Ac
	for bitcoin-development@lists.sourceforge.net;
	Tue, 09 Oct 2012 23:42:38 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.219.47 as permitted sender)
	client-ip=209.85.219.47; envelope-from=arklan.uthoslin@gmail.com;
	helo=mail-oa0-f47.google.com; 
Received: from mail-oa0-f47.google.com ([209.85.219.47])
	by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1TLjRd-0002tw-7x
	for bitcoin-development@lists.sourceforge.net;
	Tue, 09 Oct 2012 23:42:38 +0000
Received: by mail-oa0-f47.google.com with SMTP id h1so6166712oag.34
	for <bitcoin-development@lists.sourceforge.net>;
	Tue, 09 Oct 2012 16:42:31 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.78.228 with SMTP id e4mr4246903obx.77.1349826151520; Tue,
	09 Oct 2012 16:42:31 -0700 (PDT)
Received: by 10.76.152.97 with HTTP; Tue, 9 Oct 2012 16:42:31 -0700 (PDT)
In-Reply-To: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com>
References: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com>
Date: Tue, 9 Oct 2012 17:42:31 -0600
Message-ID: <CAGg41SWAxhGwmMQCnzsBU_0Z=han2JvbTn3CaTFZxqaaTb+AHw@mail.gmail.com>
From: Arklan Uth Oslin <arklan.uthoslin@gmail.com>
To: Jeff Garzik <jgarzik@exmulti.com>
Content-Type: multipart/alternative; boundary=f46d0445170d45180104cba8e4e2
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	(arklan.uthoslin[at]gmail.com)
	-0.0 SPF_PASS               SPF: sender matches SPF record
	1.0 HTML_MESSAGE           BODY: HTML included in message
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1TLjRd-0002tw-7x
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] On bitcoin testing
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 09 Oct 2012 23:42:38 -0000

--f46d0445170d45180104cba8e4e2
Content-Type: text/plain; charset=ISO-8859-1

thanks for the great reply jeff. i'm going to get a virtual machine set up
on my system later tonight so at the very least, i myself can start testing.

steve - haven't heard from you in almost a week. I'd still really like to
get a look at the test cases and such you set up.

Arklan

----------
As long as there is light, the darkness holds no fear. And yet, even in the
deepest black, there is life. - Arklan Uth Oslin

I want to leave this world the same way I came into it: backwards and on
fire. - Arklan Uth Oslin



On Tue, Oct 9, 2012 at 5:12 PM, Jeff Garzik <jgarzik@exmulti.com> wrote:

> Copying from a response posted to "Bitcoin software testing effort"
> https://bitcointalk.org/index.php?topic=117487.0 as it is relevant to
> a recent thread here...
>
> Any level of testing is useful and appreciated.  Various types of
> testing that are helpful:
>
> * "it works" testing:  Simply run the latest Release Candidate (or
> latest version, if released).  Make sure all the basics work (for
> whatever definition of "basics" you desire).  This is the level most
> accessible to casual users.
> * Major features testing:  Develop a short checklist of must-work
> features, and organize volunteers to work together and go through that
> checklist, item by item.  Test each major feature on each major
> platform.
> * Stress and fuzz testing:  Attempt to "stress" the system somehow, or
> randomly corrupt bits of data.  See what breaks.
> * Regression testing:  Record bugs fixed, and develop automated test
> cases that successfully reproduce the bugs on older versions, and
> verify newer versions remain fixed.
> * Unit function testing:  Rigorously exercise each C++ class to ensure
> it behaves as expected at a micro level.
> * Full peer automated testing:  Automated testing of RPC and P2P
> functions is non-existent, because of the difficulty in doing so.
> Find a solution to this problem.
> * Data-driven tests: If possible, write software-neutral, data-driven
> tests.  This enables clients other than the reference one (Satoshi
> client) to be tested.  Embed tests in testnet3 chain, if possible.
>
>
> The community at large can be a big help simply by doing the first
> item:  download and run the Release Candidates and the latest version,
> and report any problems.  Even reporting success is fine by me, for
> example: "Version 0.7.1 works for me on Windows 7/32-bit" posted on a
> forum thread.
>
> It is always very difficult to organize any sort of testing regime
> with open source volunteers that come and go.  Each volunteer chooses
> their level of involvement.  Any amount of testing and test-case
> writing, large or small, is helpful to bitcoin.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti.com
>
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

--f46d0445170d45180104cba8e4e2
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

thanks for the great reply jeff. i&#39;m going to get a virtual machine set=
 up on my system later tonight so at the very least, i myself can start tes=
ting.<div><br></div><div>steve - haven&#39;t heard from you in almost a wee=
k. I&#39;d still really like to get a look at the test cases and such you s=
et up.<br clear=3D"all">
<div>=A0</div>
<div>Arklan<br><br>----------<br>
<div>As long as there is light, the darkness holds no fear. And yet, even i=
n the deepest black, there is life. - Arklan Uth Oslin</div></div>
<div>=A0</div>
<div>I want to leave this world the same way I came into it: backwards and =
on fire. - Arklan Uth Oslin</div><br>
<br><br><div class=3D"gmail_quote">On Tue, Oct 9, 2012 at 5:12 PM, Jeff Gar=
zik <span dir=3D"ltr">&lt;<a href=3D"mailto:jgarzik@exmulti.com" target=3D"=
_blank">jgarzik@exmulti.com</a>&gt;</span> wrote:<br><blockquote class=3D"g=
mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l=
eft:1ex">
Copying from a response posted to &quot;Bitcoin software testing effort&quo=
t;<br>
<a href=3D"https://bitcointalk.org/index.php?topic=3D117487.0" target=3D"_b=
lank">https://bitcointalk.org/index.php?topic=3D117487.0</a> as it is relev=
ant to<br>
a recent thread here...<br>
<br>
Any level of testing is useful and appreciated. =A0Various types of<br>
testing that are helpful:<br>
<br>
* &quot;it works&quot; testing: =A0Simply run the latest Release Candidate =
(or<br>
latest version, if released). =A0Make sure all the basics work (for<br>
whatever definition of &quot;basics&quot; you desire). =A0This is the level=
 most<br>
accessible to casual users.<br>
* Major features testing: =A0Develop a short checklist of must-work<br>
features, and organize volunteers to work together and go through that<br>
checklist, item by item. =A0Test each major feature on each major<br>
platform.<br>
* Stress and fuzz testing: =A0Attempt to &quot;stress&quot; the system some=
how, or<br>
randomly corrupt bits of data. =A0See what breaks.<br>
* Regression testing: =A0Record bugs fixed, and develop automated test<br>
cases that successfully reproduce the bugs on older versions, and<br>
verify newer versions remain fixed.<br>
* Unit function testing: =A0Rigorously exercise each C++ class to ensure<br=
>
it behaves as expected at a micro level.<br>
* Full peer automated testing: =A0Automated testing of RPC and P2P<br>
functions is non-existent, because of the difficulty in doing so.<br>
Find a solution to this problem.<br>
* Data-driven tests: If possible, write software-neutral, data-driven<br>
tests. =A0This enables clients other than the reference one (Satoshi<br>
client) to be tested. =A0Embed tests in testnet3 chain, if possible.<br>
<br>
<br>
The community at large can be a big help simply by doing the first<br>
item: =A0download and run the Release Candidates and the latest version,<br=
>
and report any problems. =A0Even reporting success is fine by me, for<br>
example: &quot;Version 0.7.1 works for me on Windows 7/32-bit&quot; posted =
on a<br>
forum thread.<br>
<br>
It is always very difficult to organize any sort of testing regime<br>
with open source volunteers that come and go. =A0Each volunteer chooses<br>
their level of involvement. =A0Any amount of testing and test-case<br>
writing, large or small, is helpful to bitcoin.<br>
<br>
--<br>
Jeff Garzik<br>
exMULTI, Inc.<br>
<a href=3D"mailto:jgarzik@exmulti.com">jgarzik@exmulti.com</a><br>
<br>
---------------------------------------------------------------------------=
---<br>
Don&#39;t let slow site performance ruin your business. Deploy New Relic AP=
M<br>
Deploy New Relic app performance management and know exactly<br>
what is happening inside your Ruby, Python, PHP, Java, and .NET app<br>
Try New Relic at no cost today and get our sweet Data Nerd shirt too!<br>
<a href=3D"http://p.sf.net/sfu/newrelic-dev2dev" target=3D"_blank">http://p=
.sf.net/sfu/newrelic-dev2dev</a><br>
_______________________________________________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
pment@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
</blockquote></div><br></div>

--f46d0445170d45180104cba8e4e2--