Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from <arklan.uthoslin@gmail.com>) id 1TLjRe-0005yK-Ac for bitcoin-development@lists.sourceforge.net; Tue, 09 Oct 2012 23:42:38 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.47 as permitted sender) client-ip=209.85.219.47; envelope-from=arklan.uthoslin@gmail.com; helo=mail-oa0-f47.google.com; Received: from mail-oa0-f47.google.com ([209.85.219.47]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TLjRd-0002tw-7x for bitcoin-development@lists.sourceforge.net; Tue, 09 Oct 2012 23:42:38 +0000 Received: by mail-oa0-f47.google.com with SMTP id h1so6166712oag.34 for <bitcoin-development@lists.sourceforge.net>; Tue, 09 Oct 2012 16:42:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.78.228 with SMTP id e4mr4246903obx.77.1349826151520; Tue, 09 Oct 2012 16:42:31 -0700 (PDT) Received: by 10.76.152.97 with HTTP; Tue, 9 Oct 2012 16:42:31 -0700 (PDT) In-Reply-To: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com> References: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com> Date: Tue, 9 Oct 2012 17:42:31 -0600 Message-ID: <CAGg41SWAxhGwmMQCnzsBU_0Z=han2JvbTn3CaTFZxqaaTb+AHw@mail.gmail.com> From: Arklan Uth Oslin <arklan.uthoslin@gmail.com> To: Jeff Garzik <jgarzik@exmulti.com> Content-Type: multipart/alternative; boundary=f46d0445170d45180104cba8e4e2 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (arklan.uthoslin[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TLjRd-0002tw-7x Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net> Subject: Re: [Bitcoin-development] On bitcoin testing X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: <bitcoin-development.lists.sourceforge.net> List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, <mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development> List-Post: <mailto:bitcoin-development@lists.sourceforge.net> List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, <mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe> X-List-Received-Date: Tue, 09 Oct 2012 23:42:38 -0000 --f46d0445170d45180104cba8e4e2 Content-Type: text/plain; charset=ISO-8859-1 thanks for the great reply jeff. i'm going to get a virtual machine set up on my system later tonight so at the very least, i myself can start testing. steve - haven't heard from you in almost a week. I'd still really like to get a look at the test cases and such you set up. Arklan ---------- As long as there is light, the darkness holds no fear. And yet, even in the deepest black, there is life. - Arklan Uth Oslin I want to leave this world the same way I came into it: backwards and on fire. - Arklan Uth Oslin On Tue, Oct 9, 2012 at 5:12 PM, Jeff Garzik <jgarzik@exmulti.com> wrote: > Copying from a response posted to "Bitcoin software testing effort" > https://bitcointalk.org/index.php?topic=117487.0 as it is relevant to > a recent thread here... > > Any level of testing is useful and appreciated. Various types of > testing that are helpful: > > * "it works" testing: Simply run the latest Release Candidate (or > latest version, if released). Make sure all the basics work (for > whatever definition of "basics" you desire). This is the level most > accessible to casual users. > * Major features testing: Develop a short checklist of must-work > features, and organize volunteers to work together and go through that > checklist, item by item. Test each major feature on each major > platform. > * Stress and fuzz testing: Attempt to "stress" the system somehow, or > randomly corrupt bits of data. See what breaks. > * Regression testing: Record bugs fixed, and develop automated test > cases that successfully reproduce the bugs on older versions, and > verify newer versions remain fixed. > * Unit function testing: Rigorously exercise each C++ class to ensure > it behaves as expected at a micro level. > * Full peer automated testing: Automated testing of RPC and P2P > functions is non-existent, because of the difficulty in doing so. > Find a solution to this problem. > * Data-driven tests: If possible, write software-neutral, data-driven > tests. This enables clients other than the reference one (Satoshi > client) to be tested. Embed tests in testnet3 chain, if possible. > > > The community at large can be a big help simply by doing the first > item: download and run the Release Candidates and the latest version, > and report any problems. Even reporting success is fine by me, for > example: "Version 0.7.1 works for me on Windows 7/32-bit" posted on a > forum thread. > > It is always very difficult to organize any sort of testing regime > with open source volunteers that come and go. Each volunteer chooses > their level of involvement. Any amount of testing and test-case > writing, large or small, is helpful to bitcoin. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti.com > > > ------------------------------------------------------------------------------ > Don't let slow site performance ruin your business. Deploy New Relic APM > Deploy New Relic app performance management and know exactly > what is happening inside your Ruby, Python, PHP, Java, and .NET app > Try New Relic at no cost today and get our sweet Data Nerd shirt too! > http://p.sf.net/sfu/newrelic-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > --f46d0445170d45180104cba8e4e2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable thanks for the great reply jeff. i'm going to get a virtual machine set= up on my system later tonight so at the very least, i myself can start tes= ting.<div><br></div><div>steve - haven't heard from you in almost a wee= k. I'd still really like to get a look at the test cases and such you s= et up.<br clear=3D"all"> <div>=A0</div> <div>Arklan<br><br>----------<br> <div>As long as there is light, the darkness holds no fear. And yet, even i= n the deepest black, there is life. - Arklan Uth Oslin</div></div> <div>=A0</div> <div>I want to leave this world the same way I came into it: backwards and = on fire. - Arklan Uth Oslin</div><br> <br><br><div class=3D"gmail_quote">On Tue, Oct 9, 2012 at 5:12 PM, Jeff Gar= zik <span dir=3D"ltr"><<a href=3D"mailto:jgarzik@exmulti.com" target=3D"= _blank">jgarzik@exmulti.com</a>></span> wrote:<br><blockquote class=3D"g= mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l= eft:1ex"> Copying from a response posted to "Bitcoin software testing effort&quo= t;<br> <a href=3D"https://bitcointalk.org/index.php?topic=3D117487.0" target=3D"_b= lank">https://bitcointalk.org/index.php?topic=3D117487.0</a> as it is relev= ant to<br> a recent thread here...<br> <br> Any level of testing is useful and appreciated. =A0Various types of<br> testing that are helpful:<br> <br> * "it works" testing: =A0Simply run the latest Release Candidate = (or<br> latest version, if released). =A0Make sure all the basics work (for<br> whatever definition of "basics" you desire). =A0This is the level= most<br> accessible to casual users.<br> * Major features testing: =A0Develop a short checklist of must-work<br> features, and organize volunteers to work together and go through that<br> checklist, item by item. =A0Test each major feature on each major<br> platform.<br> * Stress and fuzz testing: =A0Attempt to "stress" the system some= how, or<br> randomly corrupt bits of data. =A0See what breaks.<br> * Regression testing: =A0Record bugs fixed, and develop automated test<br> cases that successfully reproduce the bugs on older versions, and<br> verify newer versions remain fixed.<br> * Unit function testing: =A0Rigorously exercise each C++ class to ensure<br= > it behaves as expected at a micro level.<br> * Full peer automated testing: =A0Automated testing of RPC and P2P<br> functions is non-existent, because of the difficulty in doing so.<br> Find a solution to this problem.<br> * Data-driven tests: If possible, write software-neutral, data-driven<br> tests. =A0This enables clients other than the reference one (Satoshi<br> client) to be tested. =A0Embed tests in testnet3 chain, if possible.<br> <br> <br> The community at large can be a big help simply by doing the first<br> item: =A0download and run the Release Candidates and the latest version,<br= > and report any problems. =A0Even reporting success is fine by me, for<br> example: "Version 0.7.1 works for me on Windows 7/32-bit" posted = on a<br> forum thread.<br> <br> It is always very difficult to organize any sort of testing regime<br> with open source volunteers that come and go. =A0Each volunteer chooses<br> their level of involvement. =A0Any amount of testing and test-case<br> writing, large or small, is helpful to bitcoin.<br> <br> --<br> Jeff Garzik<br> exMULTI, Inc.<br> <a href=3D"mailto:jgarzik@exmulti.com">jgarzik@exmulti.com</a><br> <br> ---------------------------------------------------------------------------= ---<br> Don't let slow site performance ruin your business. Deploy New Relic AP= M<br> Deploy New Relic app performance management and know exactly<br> what is happening inside your Ruby, Python, PHP, Java, and .NET app<br> Try New Relic at no cost today and get our sweet Data Nerd shirt too!<br> <a href=3D"http://p.sf.net/sfu/newrelic-dev2dev" target=3D"_blank">http://p= .sf.net/sfu/newrelic-dev2dev</a><br> _______________________________________________<br> Bitcoin-development mailing list<br> <a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo= pment@lists.sourceforge.net</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development= " target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment</a><br> </blockquote></div><br></div> --f46d0445170d45180104cba8e4e2--