Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of mistfpga.net
	designates 208.91.199.215 as permitted sender)
	client-ip=208.91.199.215; envelope-from=steve@mistfpga.net;
	helo=us2.outbound.mailhostbox.com; 
Message-ID: <506B9197.2000202@mistfpga.net>
Date: Wed, 03 Oct 2012 02:15:03 +0100
From: steve <steve@mistfpga.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
	rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: Peter Vessenes <peter@coinlab.com>
References: <5061F8CC.9070906@mistfpga.net>
	<1348605677.2284.2.camel@localhost.localdomain>
	<5062F4F8.6040504@mistfpga.net>
	<CA+s+GJBM4DwDoqT8RC0+SyrLYrLGZuGZSuoj7zbHunQa3kFoRA@mail.gmail.com>
	<506301AC.90101@mistfpga.net>
	<CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
	<50633F02.6030807@mistfpga.net>
	<CABsx9T119rT1NssV6YO8KPse3sEGHZJ-vzFFD3o=aAQ161g6-A@mail.gmail.com>
	<50673D69.5040105@mistfpga.net>
	<CAGg41SV9nAA0ispDG4-QUnjAMprnyQrH-faGVn7buau0vj=PBw@mail.gmail.com>
	<5069A89E.7090003@mistfpga.net>
	<CAMGNxUugk3e7yhgjjSPO0CQEGe0Ucp3_Wb+JxJkwYv41t2MPYQ@mail.gmail.com>
In-Reply-To: <CAMGNxUugk3e7yhgjjSPO0CQEGe0Ucp3_Wb+JxJkwYv41t2MPYQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
Precedence: list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/10/2012 17:52, Peter Vessenes wrote:
> I'm a big proponent of a testing project.

I am very happy to hear this, however, your actual words are slightly
evasive. I do not expect you to be up to speed on this. Gavin started
a project called 'the bitcoin testing project' This project solicited
donations, about 80 coins last time I checked.  However these 80 odd
coins were donated to 'the bitcoin testing project' This would seem to
be an official bitcoin (both protocol and client) testing project.  I
signed up to work on this, and organise as much as i could of this.
for various reasons I did not manage to do the testing I wanted to on
0.7 i over committed myself.

Are the donations solicited for the 'bitcoin testing project) funds
going to be given to 'the foundation'?

Does the foundation support 'the bitcoin testing project'? does the
foundation have any involvement with 'the bitcoin testing project'?

> 
> I think if one could self organize that Gavin and team wanted to
> bless we could put up some BTC as bounties or funding. We won't
> have our heads around the foundation budget for a few more weeks,
> but self-organization is often slower than budgeting. :)

Im ready to go, more or less.  Please check out the links in my
previous emails. I have over 400 testcases (8 platforms * 50 release
tests) - Also I am not sure what you mean by bless, I take it that is
a euphemism for pay?

I have tried my hardest to get bettermeans to work, but it doesnt.  It
does show quite a lot of work that I have done though. If you were to
say to me, 'steve, by monday we need end to end, requirements based
testing' It would be done. (I have already spent over 4 months on
this)  Leaderless leadership is something I am having a hard time
with, bettermeans is excellent at this.  But I have found very little
in regards to voting and polling that integrates with the project in
an effortless way like bettermeans.

I understand that the budget from the foundation is something that
needs to be worked on and organised.  I offer my services in this area
(qa only).  I would be happy to submit my cv and refs for this, if
required.

I am now feeling frustrated and useless.  has my last 4 months of work
been for nothing? it feels like it.  I know I bang on about processes
but they are sorted, you can only attract talent like Arklan if he has
a process to follow. i feel like a broken record.

tl;dr version
1 - Will donations to the 'bitcoin testing project' as started by
gavin going to be given to the foundation?
2 - Is the work bill hees and myself going to be binned?
3 - I feel like I have the knowledge and drive to push this, but I
cant do it on my own.
4 - Is bill or I entitled to any of the cash raised for 'the bitcoin
testing project'
5 - Do I have to join the foundation to have a say in how the project
(testing) is done?
6 - sorry for being so mercenary, but am I going to receive any coin
for work I have done?
7 - It really probably is the time for a bitcoin-test list to appear.
 Is there anything I can do to make this happen?

> 
> This is just my opinion, but I would like very, very much to move
> the current specification into unit tests so that anyone could
> validate their alternate bitcoin implementation. This is a lot of
> work, some of which has been done, much of which hasn't.

have a look at the stuff in bettermeans.  I personally think we can go
a step further and publish guidelines (similar to RFC's and all the
tests that we would do against a ref client)

But I dont want to waste any more time on stuff that is going to be
ignored, life is short.

> 
> So, my two cents, plus an offer to bring it up at our next
> budgeting meeting.

I accept that offer. and I really appreciate it.  I have some more
questions I would like you to ask in regards to QA. (Gavin and I
skyped about this a while ago and we didnt really come to a
resolution, weworked out the problems though ;) )

I have an exceptionally detailed qa process (based off the game
certification process) - but I have gone on about this at length in
previous messages.

I thank you for your email and your involvement with this, but do you
think we are closer to getting stuff tested? call my bluff... Not one
person has asked for login details to my proposals - and i even have a
bugzilla version now.

I need to sleep.  sorry if i rambled.

nite nite,

steve

> 
> Peter
> 
> 
> On Mon, Oct 1, 2012 at 7:28 AM, steve <steve@mistfpga.net> wrote:
> 
> On 01/10/2012 14:52, Arklan Uth Oslin wrote:
>>>> Hi guys.
>>>> 
>>>> So, as I mentioned on the bitcointalk.org forums thread about
>>>> the foundation, I want to get involved in the QA side of
>>>> bitcoin development. I've done functional testing in the
>>>> video game industry for years.
> 
> Nice one, I worked in games for quite a few years. (before getting 
> into finance then pentesting) there are about 6 keen testers now. 
> maybe we should get a bitcoin-test mailing list, where we can
> discuss stuff without disturbing the dev team.
> 
>>>> I've read all the messages in this thread, but I'm left
>>>> unclear how I can most effectively and quickly being helping
>>>> out. Could I get a bit of a directional nudge?
> 
> Great question... for me I feel structure is the most important
> thing to sort out first.  However we desperately need detailed
> testcases for the release of a new version. - Not too much on the
> change log stuff, more on the noddy stuff (as gavin points out
> below), downloading and making sure it works on a non dev machine,
> make sure the wallet isnt overwritten, etc.) doing games qa I
> imagine this would be an ideal place for you to start.  I have a
> MSDN and TechNet licence so if you need some reference ms virtual
> machines I can help you out.
> 
> However we need some testcase software.  Please check out what was 
> done on bettermeans for the stuff I was planning out...
> 
> It details everything from recompense and testcases. bettermeans
> kinda died a death though...
> 
> check out:
> 
> Bitcoin over all- 
> https://secure.bettermeans.com/projects/4180/wiki/Page_index 
> discussion https://secure.bettermeans.com/projects/4180/boards
> 
> 0.7 https://secure.bettermeans.com/projects/4256/boards and 
> https://secure.bettermeans.com/projects/4256/wiki
> 
> I still have the testcases, but until we get some proper testcase 
> software I am loathed to publish them in a half arsed format.
> (they worked well on bettermeans, then just vanished one day...)
> what testcase software are you familiar with?
> 
> apart from that, what do you feel you can do for the project? how
> long have you been involved in bitcoin?  It may well be worth
> reading up all the dev stuff on the wiki so you can get you head
> around how the bitcoin protocol is different from the daemon and qt
> client. What do you think you can and will enjoy doing? What is
> your skill set in regard to networking, crypto and operating
> systems. (not that you need any, in any we still want and need you.
> :)
> 
> there really is room for you to do whatever role you want, and as 
> little or as much as you want - however funding is now a very
> tricky issue. so much so that I am not sure I want anything to do
> with it(distribution of coin based on work.). - I just paid for
> some logo spec work out of my own pocket (for example).  I have
> some testers i know irl who are willing to work for coin.
> 
> NOTE: This response has nothing to do with the bitcoin foundation.
> I am not a member of the foundation. I do not speak for them or
> even probably with them. I am still trying to work out how much qa
> the foundation should be responsible for, and/how it is supposed to
> work. I think the games cert process would be ideal for this.  This
> however this a discussion that probably wont have my involvement.
> (personally I believe that the foundation should publish
> requirements with example code and testcases for each aspect of the
> reference client. (on reference platforms - I do not expect many to
> agree with this though)
> 
> As a side note, what happens to the donations to the bitcoin
> testing project? do they get moved over to the foundation? this
> question is bigger than this email. as far as I know they are all
> on an address Gavin holds. Actually I would like to be involved in
> any discussions that would impact QA, does this mean I need to join
> the foundation or just go lone wolf?
> 
> tbh I dont really understand foundations.  I always thought they
> were just a tax dodge.
> 
> Sorry for the long message. :)
> 
>>>> 
>>>> Arklan
>>>> 
>>>> ---------- As long as there is light, the darkness holds no
>>>> fear. And yet, even in the deepest black, there is life. -
>>>> Arklan Uth Oslin
>>>> 
>>>> I want to leave this world the same way I came into it:
>>>> backwards and on fire. - Arklan Uth Oslin
>>>> 
>>>> 
>>>> 
>>>> On Sat, Sep 29, 2012 at 12:26 PM, steve <steve@mistfpga.net> 
>>>> wrote:
>>>> 
>>>> Hi Gavin,
>>>> 
>>>> Sorry for the delayed response, I wanted to take a couple of
>>>> days to reflect on your email.
>>>> 
>>>> On 26/09/2012 19:09, Gavin Andresen wrote:
>>>> 
>>>> 
>>>> And their are other methods too.
>>>> 
>>>> 
>>>> 
>>>> The GUI::Test package for perl will allow this to be greatly 
>>>> automated. (I have done this before on the localisation of 
>>>> photoshop.)
>>>> 
>>>> 
>>>> 
>>>> this why we need detailed testscripts and plans.  so we know
>>>> what has and hasnt been done. The more boring the task the
>>>> more work that needs to go into testcase development.  This
>>>> is the area I see as my greatest failing last time.  I have a
>>>> large number of virtual machines and should have at least
>>>> this work.  But we need very detailed testcases.  with decent
>>>> testplans just downloading the software, syncing the block
>>>> chain, syncing an existing wallet, rescanning the blockchain
>>>> and verifying the balance would cover a large number of
>>>> tests.  The idea behind having lots of very specific
>>>> testcases is you get to see what tests have not been run.
>>>> 
>>>> 
>>>> 
>>>> I understand your concern, however I have taken a couple of
>>>> days to reflect on this and I still strongly feel that in
>>>> order to make sure that this sticks, and is still useful in 1
>>>> years time we need to lay proper foundations. Those
>>>> foundations are not word documents, spreadsheets, etc.  they
>>>> are selecting the right tools for the job.
>>>> 
>>>> We can gain so much benefit from using 3rd party software. 
>>>> (bettermeans would rock if it wasnt rotting)
>>>> 
>>>> I am sure you could do your coding work just using vi, but an
>>>> sdk makes it much easier and allows you to work in a more
>>>> productive manner.
>>>> 
>>>> I have had a couple of off list emails with some testers and
>>>> they also feel that it is very important to make sure we have
>>>> a sound foundation (mantis is so much more than just a bug
>>>> reporting tool, I see the bug reporting functionality as
>>>> secondary to the main test run functionality - but it doesnt
>>>> have to be mantis based, we do need workflow and testcase
>>>> software though - and proper software for this is much better
>>>> than just a massive google doc.) however I am checking out
>>>> some other software that has been recommended.  It will be
>>>> very hard to change 'the process' once we have something we 
>>>> are used too (just look at the current resistance) I promise 
>>>> nothing will change for the dev team.  But test does need
>>>> other tools, and processes.
>>>> 
>>>> If you feel that strongly that I am going about this the
>>>> wrong way, I am happy to step back and let someone else sort
>>>> it out (I will still do all the testing I possibly can). I
>>>> would feel that this would be a real shame and we have the
>>>> chance to setup requirements to functionality to tests all
>>>> with traceability. why not do it right from the start?
>>>> 
>>>> I will open up my vps' somepoint over the next few days and
>>>> you can see what I mean. I will setup a fake git project, and
>>>> sort out the interactions.
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> 
> ------------------------------------------------------------------------------
>>>>>
>>>>>
>
> 
How fast is your code?
>>>>> 3 out of 4 devs don\\\'t know how their code performs in 
>>>>> production. Find out how slow your code is with
>>>>> AppDynamics Lite.
>>>>> http://ad.doubleclick.net/clk;262219672;13503038;z? 
>>>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>>>>
>>>>> 
_______________________________________________
>>>>> Bitcoin-development mailing list 
>>>>> Bitcoin-development@lists.sourceforge.net 
>>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>>
>>>>
>>>>
>>>>
>>>>
>
>>>>> 
-
------------------------------------------------------------------------------
>>>> 
>>>> 
> Got visibility?
>>>> Most devs has no idea what their production app looks like.
>>>> Find out how fast your code is with AppDynamics Lite. 
>>>> http://ad.doubleclick.net/clk;262219671;13503038;y? 
>>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Bitcoin-development mailing list
>>>> Bitcoin-development@lists.sourceforge.net 
>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>
>
>>>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>>
>> 
Got visibility?
>> Most devs has no idea what their production app looks like. Find
>> out how fast your code is with AppDynamics Lite. 
>> http://ad.doubleclick.net/clk;262219671;13503038;y? 
>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html 
>> _______________________________________________ 
>> Bitcoin-development mailing list 
>> Bitcoin-development@lists.sourceforge.net 
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> 
> 
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQEcBAEBAgAGBQJQa5GXAAoJEFvEB9dQFvtQUR4H/3o0aOyvF6u4Wi5UUxMtGWXD
Bduuag9jNY/GACFshYs365wWqJcK96GXc/svOx0l/TTv10tw9BboaMpKY2x9gWri
7DJ+mym1AorgHBlZUgZJy3kQrb5yUCR7MiYU0O28QAqK5aVoklBa6cGQJqQVRa2j
vm9UrKCOeMbHiNMjagLP3s1tcQxYJUwm8hEuALWjyqsDQ+Rk7j2ndBnfonJLTCDc
rIy6HhmfAGSCaXFzhH35Ydfl2hV6ASaUHtBSKke3bB/oJ/bA5XspmPPhCkjum4Lo
4b28i4/O8BTisAsr8N3FYQ17dmD2fu8/gmQt1Rl9MBfWBHPJ9YVIIittk+WhcI0=
=nER6
-----END PGP SIGNATURE-----