1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
|
Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mark@monetize.io>) id 1TGuxE-0007Uy-JG
for bitcoin-development@lists.sourceforge.net;
Wed, 26 Sep 2012 16:59:20 +0000
Received: from mail-ob0-f175.google.com ([209.85.214.175])
by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1TGuxC-0005d6-4W
for bitcoin-development@lists.sourceforge.net;
Wed, 26 Sep 2012 16:59:20 +0000
Received: by obceq6 with SMTP id eq6so870125obc.34
for <bitcoin-development@lists.sourceforge.net>;
Wed, 26 Sep 2012 09:59:12 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=google.com; s=20120113;
h=mime-version:x-originating-ip:in-reply-to:references:date
:message-id:subject:from:to:cc:content-type:x-gm-message-state;
bh=LDieUc18vrgmqPlfzY6/gX1PEMKQ46bIf58vFYdoVeQ=;
b=nGrkqNlsNoN+cGNbB3+P8Q7C0shuF12Q2GP/emVjqYLzdIapQCj7zt7WsbU2SQ9DHH
k2Uk3I9d421op94H5jeqL2IeHaENo12YyyyQ+bIAgthiBnJYhXEwFPkannpVKJo6L88u
XxjnwHuuZg6d5zRy8IKY1boLHCS9PI8fWD5pumbcPZ1OhhrGWQKwLf8s8HvZ0fBFz8zU
2ixINiFLJPthfoPl6bKgzrN7WLibt+xstFdivcVip/7NwVnEiOgTLbbd7+NFSj7DAz1J
HEZHx3hze//FOMKsgaYF9GAcfKwDj9/2uyjg3mGEtz9G1EE0wevIadpvj6W8/bLnsDNv
wW2A==
MIME-Version: 1.0
Received: by 10.60.25.104 with SMTP id b8mr745981oeg.140.1348675601565; Wed,
26 Sep 2012 09:06:41 -0700 (PDT)
Received: by 10.76.170.194 with HTTP; Wed, 26 Sep 2012 09:06:41 -0700 (PDT)
X-Originating-IP: [50.0.36.26]
In-Reply-To: <506301AC.90101@mistfpga.net>
References: <5061F8CC.9070906@mistfpga.net>
<1348605677.2284.2.camel@localhost.localdomain>
<5062F4F8.6040504@mistfpga.net>
<CA+s+GJBM4DwDoqT8RC0+SyrLYrLGZuGZSuoj7zbHunQa3kFoRA@mail.gmail.com>
<506301AC.90101@mistfpga.net>
Date: Wed, 26 Sep 2012 09:06:41 -0700
Message-ID: <CACh7GpHFY_KUhhtk09H_oCzBtRh66artDCqz8pXNTh_ZzkAABg@mail.gmail.com>
From: Mark Friedenbach <mark@monetize.io>
To: steve <steve@mistfpga.net>
Content-Type: multipart/alternative; boundary=e89a8fb1fa0a26098404ca9d0235
X-Gm-Message-State: ALoCoQneV6TP+JNNaYqhg+jOtPpUFND0+wQfn+8u+6jJO6NapNGKTC7YE/rfstjnXDn0J3dkt0dz
X-Spam-Score: 1.0 (+)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
1.0 HTML_MESSAGE BODY: HTML included in message
X-Headers-End: 1TGuxC-0005d6-4W
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>,
Bill Hees <billhees@gmail.com>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 26 Sep 2012 16:59:20 -0000
--e89a8fb1fa0a26098404ca9d0235
Content-Type: text/plain; charset=UTF-8
Running a concurrent Mantis tracker would be confusing and fragment the
development pathway. We have an issue tracker; it's on github.
What's being talked about here are two separate things. Jenkins is a
continuous integration system. It can be configured to run the suite of
unit tests, regression tests, and any kind of automated functional tests
for every commit on github and every pull request.
Github is our issue tracker. Github, and only github, is where new issues
should be reported (unless it's security related, in which case an email
should be sent to the core devs directly).
Certainly developers should be responsible for making sure that regression
tests for bugs they fix make it either into the unit tests or Matt's
functional test repository. QA should hold them accountable for that
(re-opening tickets for bugs that have been fixed but without regression
tests).
The other thing we're talking about is coordinated release testing--getting
release candidates in the hands of actual users and making sure that issues
are reported. This is something that can't be automated as the point really
is to pick up on things that the testing suite missed. You sound more
qualified than me for coming up with a process, but in the end discovered
issues should be reported to github, the final repository of issues that
hold up Gavin from doing a release.
Just my 0.002BTC
Mark
On Wed, Sep 26, 2012 at 6:22 AM, steve <steve@mistfpga.net> wrote:
>
> I think you might be misunderstanding a little. I am not trying to
> replace the current system, I need to make sure that what I do will be
> compatible with it (seamlessly so for the developer). I do not want this
> to generate extra work for the development team.
>
> However testing is a lot more than just bug reporting, dont get me
> wrong bug reports are important, but so is running a testcase and that
> testcase passing, especially if that testcase is linked to the proof
> of a requirement. I am trying to develop a qa environment that is
> conducive to testing and will allow the testers to shine in all their
> glory :) and we need different tools and methodologies.
>
> Git is too developer centric to be useful for organising testing. -
> however there is a large amount of software that is compatible with
> git, so the core development team only ever need to work with git.
>
> The linking between a bug, the requirement, the fix, the retest, and
> updating of regression testplan is vital. So is the ability to
> organise testing campaigns and assigning tests, work units and test
> relevant docs/scripts/ideas, etc.
>
> I hope this clears things up a bit?
>
> Cheers,
>
> steve
>
--e89a8fb1fa0a26098404ca9d0235
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Running a concurrent Mantis tracker would be confusing and fragment the dev=
elopment pathway. We have an issue tracker; it's on github.<div><br></d=
iv><div>What's being talked about here are two separate things. Jenkins=
is a continuous integration system. It can be configured to run the suite =
of unit tests, regression tests, and any kind of automated functional tests=
for every commit on github and every pull request.</div>
<div><br></div><div>Github is our issue tracker. Github, and only github, i=
s where new issues should be reported (unless it's security related, in=
which case an email should be sent to the core devs directly).</div><div>
<br></div><div>Certainly developers should be responsible for making sure t=
hat regression tests for bugs they fix make it either into the unit tests o=
r Matt's functional test repository. QA should hold them accountable fo=
r that (re-opening tickets for bugs that have been fixed but without regres=
sion tests).</div>
<div><br></div><div>The other thing we're talking about is coordinated =
release testing--getting release candidates in the hands of actual users an=
d making sure that issues are reported. This is something that can't be=
automated as the point really is to pick up on things that the testing sui=
te missed. You sound more qualified than me for coming up with a process, b=
ut in the end discovered issues should be reported to github, the final rep=
ository of issues that hold up Gavin from doing a release.</div>
<div><br></div><div>Just my 0.002BTC</div><div>Mark</div><div><br><div clas=
s=3D"gmail_quote">On Wed, Sep 26, 2012 at 6:22 AM, steve <span dir=3D"ltr">=
<<a href=3D"mailto:steve@mistfpga.net" target=3D"_blank">steve@mistfpga.=
net</a>></span> wrote:<blockquote class=3D"gmail_quote" style=3D"margin:=
0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I think you might be misunderstanding a little. I am not trying to<br>
replace the current system, I need to make sure that what I do will be<br>
compatible with it (seamlessly so for the developer). I do not want this<br=
>
to generate extra work for the development team.<br>
<br>
However testing is a lot more than just bug reporting, dont get me<br>
wrong bug reports are important, but so is running a testcase and that<br>
testcase passing, especially if that testcase is linked to the proof<br>
of a requirement. I am trying to develop a qa environment that is<br>
conducive to testing and will allow the testers to shine in all their<br>
glory :) and we need different tools and methodologies.<br>
<br>
Git is too developer centric to be useful for organising testing. -<br>
however there is a large amount of software that is compatible with<br>
git, so the core development team only ever need to work with git.<br>
<br>
The linking between a bug, the requirement, the fix, the retest, and<br>
updating of regression testplan is vital. So is the ability to<br>
organise testing campaigns and assigning tests, work units and test<br>
relevant docs/scripts/ideas, etc.<br>
<br>
I hope this clears things up a bit?<br>
<br>
Cheers,<br>
<br>
steve<br></blockquote></div></div>
--e89a8fb1fa0a26098404ca9d0235--
|
