1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <steve@mistfpga.net>) id 1TGvfT-00008z-Pj
for bitcoin-development@lists.sourceforge.net;
Wed, 26 Sep 2012 17:45:03 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of mistfpga.net
designates 208.91.199.207 as permitted sender)
client-ip=208.91.199.207; envelope-from=steve@mistfpga.net;
helo=us2.outbound.mailhostbox.com;
Received: from us2.outbound.mailhostbox.com ([208.91.199.207])
by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76)
id 1TGvfS-0004RP-E5 for bitcoin-development@lists.sourceforge.net;
Wed, 26 Sep 2012 17:45:03 +0000
Received: from [10.10.10.55] (5ad2e75a.bb.sky.com [90.210.231.90])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
(Authenticated sender: steve@mistfpga.net)
by us2.outbound.mailhostbox.com (Postfix) with ESMTPSA id DAE621DF0658;
Wed, 26 Sep 2012 17:44:52 +0000 (GMT)
Message-ID: <50633F02.6030807@mistfpga.net>
Date: Wed, 26 Sep 2012 18:44:34 +0100
From: steve <steve@mistfpga.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: Mark Friedenbach <mark@monetize.io>
References: <5061F8CC.9070906@mistfpga.net>
<1348605677.2284.2.camel@localhost.localdomain>
<5062F4F8.6040504@mistfpga.net>
<CA+s+GJBM4DwDoqT8RC0+SyrLYrLGZuGZSuoj7zbHunQa3kFoRA@mail.gmail.com>
<506301AC.90101@mistfpga.net>
<CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
In-Reply-To: <CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
X-Enigmail-Version: 1.4.4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-CTCH-Spam: Unknown
X-CTCH-VOD: Unknown
X-CTCH-RefID: str=0001.0A020206.50633F16.00D7, ss=1, re=0.000, recu=0.000,
reip=0.000, cl=1, cld=1, fgs=0
X-Scanned-By: MIMEDefang 2.72 on 172.16.214.28
X-Spam-Score: -1.6 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
-0.0 SPF_PASS SPF: sender matches SPF record
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1TGvfS-0004RP-E5
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 26 Sep 2012 17:45:03 -0000
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 26/09/2012 17:06, Mark Friedenbach wrote:
> Running a concurrent Mantis tracker would be confusing and fragment
> the development pathway. We have an issue tracker; it's on github.
I think you misunderstand what I am proposing.
QA needs more than just an issue tracker. i have yet to find any
opensource software that integrates testcases, nor any method for
generating testplans, nor any method for linking testcases and plans
to requirements, that work with git.
We will need software for this (as well as workflow software) and it
is much easier to integrate this into mantis/bugzilla. They are both
so much more functional than Git.
both mantis and bugzilla have full two way functionality with git.
>
> What's being talked about here are two separate things. Jenkins is
> a continuous integration system. It can be configured to run the
> suite of unit tests, regression tests, and any kind of automated
> functional tests for every commit on github and every pull
> request.
well 3 but okay. Jenkins integrates both with mantis (and therefore a
testsuite, etc) and with git. I do not see why anything should be any
different. again I am not trying to change any current process, just
develop some new ones.
>
> Github is our issue tracker. Github, and only github, is where new
> issues should be reported (unless it's security related, in which
> case an email should be sent to the core devs directly).
You will only ever receive bug reports via git. How they are entered
should not be of concern.
There will be no space in mantis/zilla for bugs that are not related
to testcases.
>
> Certainly developers should be responsible for making sure that
> regression tests for bugs they fix make it either into the unit
> tests or Matt's functional test repository. QA should hold them
> accountable for that (re-opening tickets for bugs that have been
> fixed but without regression tests).
I feel very strongly that developers should not do regression testing
or any signoff testing on their own code. QA should do the testing. I
am 50/50 if they should write the testcases. the QA process should
make things easier for the dev team, not generate more work for them.
>
> The other thing we're talking about is coordinated release
> testing--getting release candidates in the hands of actual users
> and making sure that issues are reported. This is something that
> can't be automated as the point really is to pick up on things that
> the testing suite missed. You sound more qualified than me for
> coming up with a process, but in the end discovered issues should
> be reported to github, the final repository of issues that hold up
> Gavin from doing a release.
All the core development team will still use git. the extra software
is needed by test.
(And the third point was coming up with a suite of tests for 3rd party
developers to test their interoperability - this will having nothing
to do with git, or mantis. But the solution should be compatible with
mantis/zilla)
>
> Just my 0.002BTC Mark
>
> On Wed, Sep 26, 2012 at 6:22 AM, steve <steve@mistfpga.net> wrote:
>>
>> I think you might be misunderstanding a little. I am not trying
>> to replace the current system, I need to make sure that what I do
>> will be compatible with it (seamlessly so for the developer). I
>> do not want this to generate extra work for the development
>> team.
>>
>> However testing is a lot more than just bug reporting, dont get
>> me wrong bug reports are important, but so is running a testcase
>> and that testcase passing, especially if that testcase is linked
>> to the proof of a requirement. I am trying to develop a qa
>> environment that is conducive to testing and will allow the
>> testers to shine in all their glory :) and we need different
>> tools and methodologies.
>>
>> Git is too developer centric to be useful for organising testing.
>> - however there is a large amount of software that is compatible
>> with git, so the core development team only ever need to work
>> with git.
>>
>> The linking between a bug, the requirement, the fix, the retest,
>> and updating of regression testplan is vital. So is the ability
>> to organise testing campaigns and assigning tests, work units and
>> test relevant docs/scripts/ideas, etc.
>>
>> I hope this clears things up a bit?
>>
>> Cheers,
>>
>> steve
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iQEcBAEBAgAGBQJQYz8CAAoJEFvEB9dQFvtQjkMH/Apa95IRh21mfNIuyK8kOSdt
55tLoT9a6DFyF1IPTgjHQlPN/A0JCPy/p2rIEEL7XzWpCMu1zU8BzBNmJsxGAZJG
C0ue1eDEywKNFEMPTgQdebC2MbNSfUBA6lGJ5vijQlcXKoIuiV/LS7IMYh57T4u1
6Tc/SGypGe8kBLuFTihmIGH5uFS6arNGlcGgh+HRn+O4jKiAcw06lIoKh7S9Rj5e
bmkimvOfproCIZeNQfSJH1BfYZaVVsJ1ouVI7ch6ytFpKsZ622zYF0Iq3042kEEp
Fyqh9pDDNTJ/dwbyFpTx0WaxZySdZfZmQOCxFCAeLaCpop/nKeUnW5fy3i0sYno=
=rfHO
-----END PGP SIGNATURE-----
|
