1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <gmaxwell@gmail.com>) id 1TH3r7-00014W-7L
for bitcoin-development@lists.sourceforge.net;
Thu, 27 Sep 2012 02:29:37 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.223.175 as permitted sender)
client-ip=209.85.223.175; envelope-from=gmaxwell@gmail.com;
helo=mail-ie0-f175.google.com;
Received: from mail-ie0-f175.google.com ([209.85.223.175])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1TH3r6-0008DR-Ct
for bitcoin-development@lists.sourceforge.net;
Thu, 27 Sep 2012 02:29:37 +0000
Received: by iebc13 with SMTP id c13so3381003ieb.34
for <bitcoin-development@lists.sourceforge.net>;
Wed, 26 Sep 2012 19:29:31 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.50.47.227 with SMTP id g3mr13010572ign.5.1348712971090; Wed,
26 Sep 2012 19:29:31 -0700 (PDT)
Received: by 10.64.34.4 with HTTP; Wed, 26 Sep 2012 19:29:30 -0700 (PDT)
In-Reply-To: <1348707206.1193.16.camel@localhost.localdomain>
References: <5061F8CC.9070906@mistfpga.net>
<1348605677.2284.2.camel@localhost.localdomain>
<5062F4F8.6040504@mistfpga.net>
<1348707206.1193.16.camel@localhost.localdomain>
Date: Wed, 26 Sep 2012 22:29:30 -0400
Message-ID: <CAAS2fgRbco9Qaq8AvkZFL9FhDe6C-dTH9KgyxFg-GXAOhYCEWA@mail.gmail.com>
From: Gregory Maxwell <gmaxwell@gmail.com>
To: Matt Corallo <bitcoin-list@bluematt.me>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.2 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(gmaxwell[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.4 AWL AWL: From: address is in the auto white-list
X-Headers-End: 1TH3r6-0008DR-Ct
Cc: Bitcoin Development List <bitcoin-development@lists.sourceforge.net>,
Bill Hees <billhees@gmail.com>
Subject: Re: [Bitcoin-development] Bitcoin Testing Project
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 27 Sep 2012 02:29:37 -0000
On Wed, Sep 26, 2012 at 8:53 PM, Matt Corallo <bitcoin-list@bluematt.me> wr=
ote:
> Jenkins currently just runs the test script after each new commit to
> bitcoin (and provides binaries to anyone who wants them), so its pretty
> basic (though jenkins has way more features than we use). The bitcoin
> one lives at http://jenkins.bluematt.me/
Jenkins is excellent at cycling through tests, while additional
external tools may bring some value they're not required. It's also
essential to automate all tests that we really care are run=E2=80=94 with o=
ur
small active development group and volunteer contributors the only
tests we can count on being run are the automated ones. Automated
tests included with the software=E2=80=94 or at least the source=E2=80=94 a=
re also the
only way to have a good chance of catching gnarly platform
interactions.
I think more than talking about testing I think we need is actual
testing. Code coverage from the current tests (e.g. bitcoin-test and a
testnet sync) is very unimpressive, and while coverage isn't some
magical silver bullet and does not, by itself, mean the tests are good
flaws in uncovered code can't be detected by the tests. We also lack
simple testing cycle documentation for people interested in testing
manually to walk through, etc. I think all the meta discussion is not
very useful until we actually have more substance to put into it.
Otherwise I fear we're just building an airport by painting stripes
and waiting for the planes to land...
If someone wants to help and would like a list of some of the testing
I think would be useful, ping me off-list and I can blast some
suggestions. But I assume that anyone who actually wants to work on
this isn't short of ideas, and at this point "work on what interests
you, report what interesting thing you accomplish or discover" is
probably a perfectly fine level of coordination.
|
