Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of mistfpga.net
	designates 208.91.199.212 as permitted sender)
	client-ip=208.91.199.212; envelope-from=steve@mistfpga.net;
	helo=us2.outbound.mailhostbox.com; 
Message-ID: <5069A89E.7090003@mistfpga.net>
Date: Mon, 01 Oct 2012 15:28:46 +0100
From: steve <steve@mistfpga.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
	rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: Arklan Uth Oslin <arklan.uthoslin@gmail.com>
References: <5061F8CC.9070906@mistfpga.net>
	<1348605677.2284.2.camel@localhost.localdomain>
	<5062F4F8.6040504@mistfpga.net>
	<CA+s+GJBM4DwDoqT8RC0+SyrLYrLGZuGZSuoj7zbHunQa3kFoRA@mail.gmail.com>
	<506301AC.90101@mistfpga.net>
	<CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
	<50633F02.6030807@mistfpga.net>
	<CABsx9T119rT1NssV6YO8KPse3sEGHZJ-vzFFD3o=aAQ161g6-A@mail.gmail.com>
	<50673D69.5040105@mistfpga.net>
	<CAGg41SV9nAA0ispDG4-QUnjAMprnyQrH-faGVn7buau0vj=PBw@mail.gmail.com>
In-Reply-To: <CAGg41SV9nAA0ispDG4-QUnjAMprnyQrH-faGVn7buau0vj=PBw@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
Precedence: list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/10/2012 14:52, Arklan Uth Oslin wrote:
> Hi guys.
> 
> So, as I mentioned on the bitcointalk.org forums thread about the 
> foundation, I want to get involved in the QA side of bitcoin 
> development. I've done functional testing in the video game
> industry for years.

Nice one, I worked in games for quite a few years. (before getting
into finance then pentesting) there are about 6 keen testers now.
maybe we should get a bitcoin-test mailing list, where we can discuss
stuff without disturbing the dev team.

> I've read all the messages in this thread, but I'm left unclear
> how I can most effectively and quickly being helping out. Could I
> get a bit of a directional nudge?

Great question... for me I feel structure is the most important thing
to sort out first.  However we desperately need detailed testcases for
the release of a new version. - Not too much on the change log stuff,
more on the noddy stuff (as gavin points out below), downloading and
making sure it works on a non dev machine, make sure the wallet isnt
overwritten, etc.) doing games qa I imagine this would be an ideal
place for you to start.  I have a MSDN and TechNet licence so if you
need some reference ms virtual machines I can help you out.

However we need some testcase software.  Please check out what was
done on bettermeans for the stuff I was planning out...

It details everything from recompense and testcases. bettermeans kinda
died a death though...

check out:

Bitcoin over all-
https://secure.bettermeans.com/projects/4180/wiki/Page_index
discussion
https://secure.bettermeans.com/projects/4180/boards

0.7
https://secure.bettermeans.com/projects/4256/boards
and
https://secure.bettermeans.com/projects/4256/wiki

I still have the testcases, but until we get some proper testcase
software I am loathed to publish them in a half arsed format. (they
worked well on bettermeans, then just vanished one day...) what
testcase software are you familiar with?

apart from that, what do you feel you can do for the project? how long
have you been involved in bitcoin?  It may well be worth reading up
all the dev stuff on the wiki so you can get you head around how the
bitcoin protocol is different from the daemon and qt client. What do
you think you can and will enjoy doing? What is your skill set in
regard to networking, crypto and operating systems. (not that you need
any, in any we still want and need you. :)

there really is room for you to do whatever role you want, and as
little or as much as you want - however funding is now a very tricky
issue. so much so that I am not sure I want anything to do with
it(distribution of coin based on work.). - I just paid for some logo
spec work out of my own pocket (for example).  I have some testers i
know irl who are willing to work for coin.

NOTE: This response has nothing to do with the bitcoin foundation.  I
am not a member of the foundation. I do not speak for them or even
probably with them. I am still trying to work out how much qa the
foundation should be responsible for, and/how it is supposed to work.
I think the games cert process would be ideal for this.  This however
this a discussion that probably wont have my involvement.  (personally
I believe that the foundation should publish requirements with example
code and testcases for each aspect of the reference client. (on
reference platforms - I do not expect many to agree with this though)

As a side note, what happens to the donations to the bitcoin testing
project? do they get moved over to the foundation? this question is
bigger than this email. as far as I know they are all on an address
Gavin holds. Actually I would like to be involved in any discussions
that would impact QA, does this mean I need to join the foundation or
just go lone wolf?

tbh I dont really understand foundations.  I always thought they were
just a tax dodge.

Sorry for the long message. :)

> 
> Arklan
> 
> ---------- As long as there is light, the darkness holds no fear.
> And yet, even in the deepest black, there is life. - Arklan Uth
> Oslin
> 
> I want to leave this world the same way I came into it: backwards
> and on fire. - Arklan Uth Oslin
> 
> 
> 
> On Sat, Sep 29, 2012 at 12:26 PM, steve <steve@mistfpga.net>
> wrote:
> 
> Hi Gavin,
> 
> Sorry for the delayed response, I wanted to take a couple of days
> to reflect on your email.
> 
> On 26/09/2012 19:09, Gavin Andresen wrote:
> 
> 
> And their are other methods too.
> 
> 
> 
> The GUI::Test package for perl will allow this to be greatly 
> automated. (I have done this before on the localisation of
> photoshop.)
> 
> 
> 
> this why we need detailed testscripts and plans.  so we know what
> has and hasnt been done. The more boring the task the more work
> that needs to go into testcase development.  This is the area I see
> as my greatest failing last time.  I have a large number of virtual
> machines and should have at least this work.  But we need very
> detailed testcases.  with decent testplans just downloading the
> software, syncing the block chain, syncing an existing wallet,
> rescanning the blockchain and verifying the balance would cover a
> large number of tests.  The idea behind having lots of very
> specific testcases is you get to see what tests have not been run.
> 
> 
> 
> I understand your concern, however I have taken a couple of days
> to reflect on this and I still strongly feel that in order to make
> sure that this sticks, and is still useful in 1 years time we need
> to lay proper foundations. Those foundations are not word
> documents, spreadsheets, etc.  they are selecting the right tools
> for the job.
> 
> We can gain so much benefit from using 3rd party software. 
> (bettermeans would rock if it wasnt rotting)
> 
> I am sure you could do your coding work just using vi, but an sdk 
> makes it much easier and allows you to work in a more productive
> manner.
> 
> I have had a couple of off list emails with some testers and they
> also feel that it is very important to make sure we have a sound
> foundation (mantis is so much more than just a bug reporting tool,
> I see the bug reporting functionality as secondary to the main test
> run functionality - but it doesnt have to be mantis based, we do
> need workflow and testcase software though - and proper software
> for this is much better than just a massive google doc.) however I
> am checking out some other software that has been recommended.  It
> will be very hard to change 'the process' once we have something we
> are used too (just look at the current resistance) I promise
> nothing will change for the dev team.  But test does need other
> tools, and processes.
> 
> If you feel that strongly that I am going about this the wrong way,
> I am happy to step back and let someone else sort it out (I will
> still do all the testing I possibly can). I would feel that this
> would be a real shame and we have the chance to setup requirements
> to functionality to tests all with traceability. why not do it
> right from the start?
> 
> I will open up my vps' somepoint over the next few days and you
> can see what I mean. I will setup a fake git project, and sort out
> the interactions.
> 
> 
>> 
>> 
>> ------------------------------------------------------------------------------
>>
>> 
How fast is your code?
>> 3 out of 4 devs don\\\'t know how their code performs in
>> production. Find out how slow your code is with AppDynamics
>> Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? 
>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html 
>> _______________________________________________ 
>> Bitcoin-development mailing list 
>> Bitcoin-development@lists.sourceforge.net 
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> 
> 
> 
> 
> ------------------------------------------------------------------------------
>
> 
Got visibility?
> Most devs has no idea what their production app looks like. Find
> out how fast your code is with AppDynamics Lite. 
> http://ad.doubleclick.net/clk;262219671;13503038;y? 
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> 
> 
> 
> _______________________________________________ Bitcoin-development
> mailing list Bitcoin-development@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQEcBAEBAgAGBQJQaaieAAoJEFvEB9dQFvtQUi0H/3Eh72DqxwBt6AeNos/hJNqQ
ZowMNFRupJQM301EJ7SPQmcnVuc3RF2Jw//ckpAqdpkqhHCgGO9HX/q+Ic2A9erQ
CfKbUOwQgqKuLQTZ8eT5UMh969Uw8PMazo711d2I23etU0VhSK/MvjB5IiYIq5fB
RJdCEluwvACaiftqfm4RA9HXXJGMnsZtlSWjVQCkg2FyYZKfpctEFdGCg6R8IaP7
ypyglR7TfygcjhUMHeWkQQCP2m2gBOHpx9H99uNBosF+6pLBhCiGUrwVkl8HUiiN
4YfhsvoRQ0BWNgvO9wH1H4EyyZtJsE7S926BNkhd4uewWGV/qe8gML+uJRstPDg=
=L66i
-----END PGP SIGNATURE-----