Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of mistfpga.net
	designates 208.91.199.213 as permitted sender)
	client-ip=208.91.199.213; envelope-from=steve@mistfpga.net;
	helo=us2.outbound.mailhostbox.com; 
Message-ID: <50673D69.5040105@mistfpga.net>
Date: Sat, 29 Sep 2012 19:26:49 +0100
From: steve <steve@mistfpga.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
	rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: Gavin Andresen <gavinandresen@gmail.com>
References: <5061F8CC.9070906@mistfpga.net>
	<1348605677.2284.2.camel@localhost.localdomain>
	<5062F4F8.6040504@mistfpga.net>
	<CA+s+GJBM4DwDoqT8RC0+SyrLYrLGZuGZSuoj7zbHunQa3kFoRA@mail.gmail.com>
	<506301AC.90101@mistfpga.net>
	<CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
	<50633F02.6030807@mistfpga.net>
	<CABsx9T119rT1NssV6YO8KPse3sEGHZJ-vzFFD3o=aAQ161g6-A@mail.gmail.com>
In-Reply-To: <CABsx9T119rT1NssV6YO8KPse3sEGHZJ-vzFFD3o=aAQ161g6-A@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
Precedence: list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Gavin,

Sorry for the delayed response, I wanted to take a couple of days to
reflect on your email.

On 26/09/2012 19:09, Gavin Andresen wrote:


And their are other methods too.


The GUI::Test package for perl will allow this to be greatly
automated. (I have done this before on the localisation of photoshop.)


this why we need detailed testscripts and plans.  so we know what has
and hasnt been done. The more boring the task the more work that needs
to go into testcase development.  This is the area I see as my
greatest failing last time.  I have a large number of virtual machines
and should have at least this work.  But we need very detailed
testcases.  with decent testplans just downloading the software,
syncing the block chain, syncing an existing wallet, rescanning the
blockchain and verifying the balance would cover a large number of
tests.  The idea behind having lots of very specific testcases is you
get to see what tests have not been run.


I understand your concern, however I have taken a couple of days to
reflect on this and I still strongly feel that in order to make sure
that this sticks, and is still useful in 1 years time we need to lay
proper foundations. Those foundations are not word documents,
spreadsheets, etc.  they are selecting the right tools for the job.

We can gain so much benefit from using 3rd party software.
(bettermeans would rock if it wasnt rotting)

I am sure you could do your coding work just using vi, but an sdk
makes it much easier and allows you to work in a more productive manner.

I have had a couple of off list emails with some testers and they also
feel that it is very important to make sure we have a sound foundation
(mantis is so much more than just a bug reporting tool, I see the bug
reporting functionality as secondary to the main test run
functionality - but it doesnt have to be mantis based, we do need
workflow and testcase software though - and proper software for this
is much better than just a massive google doc.) however I am checking
out some other software that has been recommended.  It will be very
hard to change 'the process' once we have something we are used too
(just look at the current resistance) I promise nothing will change
for the dev team.  But test does need other tools, and processes.

If you feel that strongly that I am going about this the wrong way, I
am happy to step back and let someone else sort it out (I will still
do all the testing I possibly can). I would feel that this would be a
real shame and we have the chance to setup requirements to
functionality to tests all with traceability. why not do it right from
the start?

I will open up my vps' somepoint over the next few days and you can
see what I mean. I will setup a fake git project, and sort out the
interactions.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQEcBAEBAgAGBQJQZz1pAAoJEFvEB9dQFvtQRLkIAJtPCkW1R9vmMPY9u4o+ET1t
w4pV/+W2PXo2p86HnljCIPLV/cua/1I/EJp7XR7s145Nj4KZUbzHGhvUUmwDOHW2
TGvJs+HO1bjsJfh4pWEb6PXcW3TguZxZSt5/rBAAI/5BeomSuRcZOdoV87D1xnK8
TSlgaseWrJcpKLO30/FQA3QnH/bjJ4OBmtHp8WaOtSnfww9Zbb5VYca37O15c2U4
2d0RUunDg1w2kRbkKjztxr3YasSOX+07Uvj4d5Lw7zgA0U93krNWVT1Ypo94dNJ7
6SyKi30UuqDdJ9XxZrMB/LBVNGOLlIBNWL++ocu5GFnOn9pnw57ZMBZM5g6YDpo=
=ekQ/
-----END PGP SIGNATURE-----