diff options
author | steve <steve@mistfpga.net> | 2012-09-26 13:28:40 +0100 |
---|---|---|
committer | bitcoindev <bitcoindev@gnusha.org> | 2012-09-26 12:29:36 +0000 |
commit | 49b6190088974ed7b2f3fae59867aacb2eb15c52 (patch) | |
tree | 9ef559a6f933c04fd95841a2ba526c44163f798f | |
parent | f57275f6358670fa4a2da968f5645349d6570933 (diff) | |
download | pi-bitcoindev-49b6190088974ed7b2f3fae59867aacb2eb15c52.tar.gz pi-bitcoindev-49b6190088974ed7b2f3fae59867aacb2eb15c52.zip |
Re: [Bitcoin-development] Bitcoin Testing Project
-rw-r--r-- | 69/f20be27732cd463a1a767e0f364c191cc33561 | 220 |
1 files changed, 220 insertions, 0 deletions
diff --git a/69/f20be27732cd463a1a767e0f364c191cc33561 b/69/f20be27732cd463a1a767e0f364c191cc33561 new file mode 100644 index 000000000..0761ccfbe --- /dev/null +++ b/69/f20be27732cd463a1a767e0f364c191cc33561 @@ -0,0 +1,220 @@ +Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] + helo=mx.sourceforge.net) + by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) + (envelope-from <steve@mistfpga.net>) id 1TGqkC-0008Fs-1h + for bitcoin-development@lists.sourceforge.net; + Wed, 26 Sep 2012 12:29:36 +0000 +Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of mistfpga.net + designates 208.91.199.220 as permitted sender) + client-ip=208.91.199.220; envelope-from=steve@mistfpga.net; + helo=us2.outbound.mailhostbox.com; +Received: from us2.outbound.mailhostbox.com ([208.91.199.220]) + by sog-mx-3.v43.ch3.sourceforge.com with esmtp (Exim 4.76) + id 1TGqkA-0006Gb-NU for bitcoin-development@lists.sourceforge.net; + Wed, 26 Sep 2012 12:29:36 +0000 +Received: from [10.10.10.55] (5ad2e75a.bb.sky.com [90.210.231.90]) + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + (Authenticated sender: steve@mistfpga.net) + by us2.outbound.mailhostbox.com (Postfix) with ESMTPSA id D56B4699585; + Wed, 26 Sep 2012 12:29:15 +0000 (GMT) +Message-ID: <5062F4F8.6040504@mistfpga.net> +Date: Wed, 26 Sep 2012 13:28:40 +0100 +From: steve <steve@mistfpga.net> +User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; + rv:15.0) Gecko/20120907 Thunderbird/15.0.1 +MIME-Version: 1.0 +To: Matt Corallo <bitcoin-list@bluematt.me> +References: <5061F8CC.9070906@mistfpga.net> + <1348605677.2284.2.camel@localhost.localdomain> +In-Reply-To: <1348605677.2284.2.camel@localhost.localdomain> +X-Enigmail-Version: 1.4.4 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 7bit +X-Scanned-By: MIMEDefang 2.72 on 172.16.214.9 +X-Spam-Score: -1.6 (-) +X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. + See http://spamassassin.org/tag/ for more details. + -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for + sender-domain + -0.0 SPF_PASS SPF: sender matches SPF record + -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from + author's domain + 0.1 DKIM_SIGNED Message has a DKIM or DK signature, + not necessarily valid + -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature +X-Headers-End: 1TGqkA-0006Gb-NU +Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>, + Bill Hees <billhees@gmail.com> +Subject: Re: [Bitcoin-development] Bitcoin Testing Project +X-BeenThere: bitcoin-development@lists.sourceforge.net +X-Mailman-Version: 2.1.9 +Precedence: list +List-Id: <bitcoin-development.lists.sourceforge.net> +List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, + <mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe> +List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development> +List-Post: <mailto:bitcoin-development@lists.sourceforge.net> +List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help> +List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, + <mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe> +X-List-Received-Date: Wed, 26 Sep 2012 12:29:36 -0000 + +-----BEGIN PGP SIGNED MESSAGE----- +Hash: SHA1 + +Hi Matt, + +Glad to have another ninja onboard :) + +On 25/09/2012 21:41, Matt Corallo wrote: +> Although Jenkins may not be the best system, we already have +> jenkins and pull-tester (which is a dumb python script I wrote to +> test all incoming pull requests from github). + +I have never heard of jenkins before. I need to do some more digging. +is this the right thing? + +https://wiki.jenkins-ci.org/display/JENKINS/Mantis+Plugin + +Mantis on the other hand, I know exceptionally well. I hate +duplication of work/data unless absolutely necessary. I will check +jenkins out (just out of interest what is it actually meant to do? the +website implies framework, but not what its for) + +So, currently there are 4 potential places for bugs to be reported +1 - jenkins (and unit tests) +2 - git +3 - mailing list +4 - forum (bitcointalk...) +5? - is there still the ability to add bugs via sourceforge? + +Adding to this doesnt make sense. Each one of these reporting methods +is for a different thing. I am not seeking to replace these (or even +unify them) I am looking for software that will take testcases and bug +reports against them [and allow for test campaigns]. Mantis is so +flexible and industry standard and if the jenkins plugin works... then +we can keep things as they are until they fit into better places. + +The reason I am so behind mantis as the backbone is it works with more +or less anything, and can easily modded to work with whatever people +are most comfortable with - however it is exceptionally powerful and +has had a constant stream of workflow improvements over the past few +years. + +> +> They both run the same set of scripts, namely those at +> https://github.com/TheBlueMatt/test-scripts (its pretty basic right +> now, but since it is on github, I was hoping someone would find +> the inspiration to add to it). + +I will check it out. I wrote a very basic script that wikified the +changelog, and linked to the changes and created wiki pages for the +testcases. have you seen the stuff I put on bettermeans? bits keep +vanishing then re appearing. + +This is the outline of the testing that I setup for 0.7 + +https://secure.bettermeans.com/projects/4256/wiki + +> +> I dont really care if we keep using jenkins, but I figure we might +> as well keep all the tests in one place? + +Yes, I would love to unify all build testing and testcases into one +place. I am still on the fence as to including unit tests into this. +However I do see 3 distinct type of testcases +1 - requirements based testcases (requirements based off the current +block chain rules - these are edge cases and known interoperability +issues) + +2 - Acceptance based testcases - these are testcases that should be +run for every build. Check out the General Acceptance Tests in the +wiki link for examples and testcases + +3 - Testcases for reference implementations of things (like multisig - +i see these working like the /test folder when you install a new perl +module) + +These three things alone are a massive task. and they still wont cover +everything. I would like to get the workflow so that people can +sponsor or donate to a specific campaign (eg a new feature is +implemented, people want it tested so can donate just for that +campaign [developing testcases, structure, requirements, etc]) + +Once this is done, I will get to do some exciting stuff (like writing +fuzzers, automation, etc) unfortunately I do not know python, only perl. + +> +> Anyway, I'm all for more testing (I'm always complaining about how +> we need more tests for stuff...). + +Nice, I love testing. I think we will get on :) + +And I would rather go for interoperability between testing rather than +rewriting it all. + +Cheers, + +steve + +> +> Matt +> +> On Tue, 2012-09-25 at 19:32 +0100, steve wrote: +>> Hi All, +>> +>> After the failure to get any real testing done for the 0.7 +>> release (all of which is my fault) I have decided to rejig +>> things. +>> +>> I am heavily into test driven development, and I have a strong +>> background in requirements management, and automation. +>> +>> I want to leave bettermeans behind, maybe we might be able to +>> keep the voting aspect of it, and link it into mantis. +>> +>> So, what I have been doing over the past few weeks is developing +>> a rudimentary requirements set, basic requirement tracking, tests +>> to prove/stress the requirements. +>> +>> The next most important thing is to get release/acceptance tests +>> done - these primarily focus on new stuff doesnt break old (ie +>> lose a wallet, etc) and needs no special requirements. +>> +>> To this end I have installed various opensource applications +>> (mantis, salomeTMF, bugzilla, etc) and am currently evaluating +>> the best workflow process. +>> +>> This was a much longer post, but decided against it. :) +>> +>> So, what I want to know is who wants to be a part helping me out +>> with all this? I am finalising the workflow flow diagrams and +>> they should be ready for inspection soon. +>> +>> Anyone interested in helping out/reviewing processes? even if it +>> is just some encouragement, it is all greatly appreciated. +>> +>> Drop me an email if you want access to the current setup and help +>> me review the different software for the bitcoin workflow +>> process. +>> +>> cheers, +>> +>> steve +> + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v2.0.17 (MingW32) +Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ + +iQEcBAEBAgAGBQJQYvT4AAoJEFvEB9dQFvtQlgkIAJX7JYel5RGmCsbptGdQrCnT +BR42tUwTg1t/NRUJ6RA8/Ou8lzallztQquShpLn4mZdQpoalvETdtAwcPnQKnaZb +M5inZE/IEq8WJM1y4YkHt3BLou4BJbjwncCNy1/jqcm6f2Oonrg7isVbDwY/7JlP +y/epm7XELS7NU4vVubBwQCunwvtsuydXRzuI812LiLXNqpXFMHvG2m8a2RajXE0/ +xW4lOMy/hUFzEgYRQWCTAru4Ts2x3Xt26NaEUh/uKvHLwBZJ4xbdu3gpupiPb4sI +bCHnVFOC7zoQKOAnfPkCMyvtyoqpzM9HW2+DWI51FoOz851Y2F36N3Fpk/2lii4= +=W5xI +-----END PGP SIGNATURE----- + + |