1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
|
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <turkeybreast@yahoo.com>) id 1UpcEv-000355-IU
for bitcoin-development@lists.sourceforge.net;
Thu, 20 Jun 2013 10:37:17 +0000
X-ACL-Warn:
Received: from nm18-vm0.bullet.mail.bf1.yahoo.com ([98.139.213.138])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256)
(Exim 4.76) id 1UpcEq-0005Yp-CZ
for bitcoin-development@lists.sourceforge.net;
Thu, 20 Jun 2013 10:37:17 +0000
Received: from [98.139.212.147] by nm18.bullet.mail.bf1.yahoo.com with NNFMP;
20 Jun 2013 10:37:05 -0000
Received: from [98.139.212.202] by tm4.bullet.mail.bf1.yahoo.com with NNFMP;
20 Jun 2013 10:37:05 -0000
Received: from [127.0.0.1] by omp1011.mail.bf1.yahoo.com with NNFMP;
20 Jun 2013 10:37:05 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 942963.70156.bm@omp1011.mail.bf1.yahoo.com
Received: (qmail 51059 invoked by uid 60001); 20 Jun 2013 10:37:05 -0000
X-YMail-OSG: PMWlfS0VM1kaz7noV9t7416HlBfI74NLAJfeqseGVdwoN_4
.cicUw7tQxkBTbJhWVEJUEcqGCtMn4H5naOr7Y5f0bcE.YnqVsGzziKxpDX3
JUz.ftPsNVH7ECEzrTY_iR5RKqa.wRR9mhe0TMVUyK09GrQBN8gF7VzgPBnO
6hafbVQhqaWM22UzVtADRAezC.PsC13seOVRw0uBErzykPlzwxqvL3u90dCQ
dE5AkI7fzDz_.VIyK5HmE_YeN5v5.vvs1_kmRq5GMlopPA5HLz_juErlsxqZ
yQRnfaQ_pRoYTFi.VrvFQ_0vCEc5IlxqicbB2S.Cj5r8CLGebXdeosfRsBwN
geDQnJxIj8oHsNVuKPscvXXeWiQrH0.ogownHUZ7SP7JwVuwEpYOSXFdMFYo
zpc76RKS1PJd_vg6zGjc2CRUBLNbn944x._BUvce7IWI6QBoLAGgtoc2aQxP
6oTitq4vcI..lP3VSabmiAwF.C.KB25pcaooOcSfNOv._CfEEGSB.lYkBT7X
jno5Yuhku3mPb3HxtvpCEW3N1w3c8T83g23oL7eC6d6UQAq282Yc-
Received: from [87.160.177.196] by web162706.mail.bf1.yahoo.com via HTTP;
Thu, 20 Jun 2013 03:37:05 PDT
X-Rocket-MIMEInfo: 002.001,
SSBkb24ndCBnZXQgd2h5IHRoaXMgaXMgc3VjaCBhIGNvbnRlbnRpb3VzIGNoYW5nZT8KCkJlZm9yZSBJIHdhcyBhYmxlIHRvIHVzZSBhc3NlcnRzIHRvIGNoZWNrIHRoZSBleHBlY3RlZCBsZW5ndGggb2YgbGVuZ3RoIG9mIG1lc3NhZ2VzIHBlciBwcm90b2NvbCB2ZXJzaW9uLCBJIGNvdWxkIHBhc3MgaW4gZHVtYiBpdGVyYXRvcnMgdGhhdCBqdXN0IHBhcnNlIHRoZSBieXRlIHN0cmVhbSBhbmQgSSBjb3VsZCBzZXJpYWxpemUgYW5kIGRlc2VyaWFsaXplIGEgbWVzc2FnZSB0byBjaGVjayB0aGUgcGFyc2VyIGkBMAEBAQE-
X-Mailer: YahooMailWebService/0.8.147.553
References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com>
<CANEZrP1MVKSNU92UrQ36sPCN0LNLKrVRXnfVwEZKhgoBFXatBA@mail.gmail.com>
<20130620090649.GA17765@vps7135.xlshosting.net>
<CANEZrP2cLR9sYZ59O-S6HNi9bKPtaY5yOkvVx+O8XAC792acZg@mail.gmail.com>
Message-ID: <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com>
Date: Thu, 20 Jun 2013 03:37:05 -0700 (PDT)
From: Turkey Breast <turkeybreast@yahoo.com>
To: "bitcoin-development@lists.sourceforge.net"
<bitcoin-development@lists.sourceforge.net>
In-Reply-To: <CANEZrP2cLR9sYZ59O-S6HNi9bKPtaY5yOkvVx+O8XAC792acZg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="-1963244382-266082894-1371724625=:50978"
X-Spam-Score: -0.4 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,
no trust [98.139.213.138 listed in list.dnswl.org]
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(turkeybreast[at]yahoo.com)
-1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain 1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1UpcEq-0005Yp-CZ
Subject: Re: [Bitcoin-development] Missing fRelayTxes in version
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: Turkey Breast <turkeybreast@yahoo.com>
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 20 Jun 2013 10:37:17 -0000
---1963244382-266082894-1371724625=:50978
Content-Type: text/plain; charset=us-ascii
I don't get why this is such a contentious change?
Before I was able to use asserts to check the expected length of length of messages per protocol version, I could pass in dumb iterators that just parse the byte stream and I could serialize and deserialize a message to check the parser is correct (in debug mode).
This 'simple' change causes all that behaviour to be lost. You can no longer just use iterators but must know the remaining length (or if you use std::distance, you can only use specific std containers - not just anything with an iterator and an operator++). You cannot check the deserialization process by serializing the deserialized message and comparing it to the original data (because the bool is always present in the serializer).
It's a bit stupid you call it buggy code when this behaviour has never been present in Bitcoin. The BIP doesn't introduce any unwanted side-effects and is a trivial reasonable change.
If you want optional fields then the proper way to do it, is to either set a flag in the Services field of the "version" message to indicate different formats for messages (i.e use this template structure for a message, not that one), introduce a new message (if the changes are big), to approve/improve Stefan's BIP 32 for custom services or to have a value in the byte stream indicating which fields are present (maybe a bitfield or so).
Using a quirk of an implementation is just bad form and sloppy coding. Optional fields should have their own mechanism that allows them to remain as optional fields between protocol version upgrades.
The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for the protocol version given by the connected node. Right now it makes no assumptions based on that which is a mistake (new clients can broadcast older version messages that don't have all the fields required). Probably the software should penalise hosts which do that.
What's the big deal to update the protocol version number from 70001 to 70002? It's not like we'll run out of integers. The field has now gone from optional to required now anyway - that's a behaviour change. It'd be good to enforce that. I see this as a bug.
________________________________
From: Mike Hearn <mike@plan99.net>
To: Pieter Wuille <pieter.wuille@gmail.com>
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>; Tamas Blummer <tamas@bitsofproof.com>
Sent: Thursday, June 20, 2013 11:17 AM
Subject: Re: [Bitcoin-development] Missing fRelayTxes in version
There's no problem, but there's no benefit either. It also locks us in to a potentially problematic guarantee - what if in future we want to have, say, two optional new pieces of data in two different messages. We don't want to require that if version > X then you have to implement all features up to and including that point.
Essentially the number of fields in a message is like a little version number, just for that message. It adds flexibility to keep it that way, and there's no downside, seeing as that bridge was already crossed and people with parsers that can't handle it need to fix their code anyway.
So I have a slight preference for keeping things the way they are, it keeps things flexible for future and costs nothing.
On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:
>> Sure but why not do that when there's an actual new field to add? Does
>> anyone have a proposal for a feature that needs a new version field at the
>> moment? There's no point changing the protocol now unless there's actually
>> a new field to add.
>>
>> Anyway I still don't see why anyone cares about this issue. The Bitcoin
>> protocol does not and never has required that all messages have a fixed
>> number of fields per version. Any parser written on the assumption it did
>> was just buggy. Look at how tx messages are relayed for the most obvious
>> example of that pattern in action - it's actually the raw byte stream
>> that's stored and relayed to ensure that fields added in new versions
>> aren't dropped during round-tripping. Old versions are supposed to preserve
>> fields from the future.
>
>Actually, that is not the same issue. What is being argued for here is that
>the version in the version message itself should indicate which fields are
>present, so a parser doesn't need to look at the length of the message. That
>seems like a minor but very reasonable request to me, and it's trivial to do.
>That doesn't mean that you may receive versions higher than what you know of,
>and thus messages with fields you don't know about. That doesn't matter, you
>can just ignore them.
>
>I see no problem with raising the protocol version number to indicate
>"all fields up to fRelayTxes are required, if the announced nVersion is above N".
>In fact, I believe (though haven't checked) all previous additions to the version
>message were accompanied with a protocol version (then: client version) increase
>as well.
>
>--
>Pieter
>
>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
---1963244382-266082894-1371724625=:50978
Content-Type: text/html; charset=us-ascii
<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>I don't get why this is such a contentious change?</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>Before I was able to use asserts to check the expected length of length of messages per protocol version, I could pass in dumb iterators that just parse the byte stream and I could serialize and deserialize a message to check the parser is correct (in debug mode).</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style:
normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>This 'simple' change causes all that behaviour to be lost. You can no longer just use iterators but must know the remaining length (or if you use std::distance, you can only use specific std containers - not just anything with an iterator and an operator++). You cannot check the deserialization process by serializing the deserialized message and comparing it to the original data (because the bool is always present in the serializer).</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style:
normal;"><span>It's a bit stupid you call it buggy code when this behaviour has never been present in Bitcoin. The BIP doesn't introduce any unwanted side-effects and is a trivial reasonable change.</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>If you want optional fields then the proper way to do it, is to either set a flag in the Services field of the "version" message to indicate different formats for messages (i.e use this template structure for a message, not that one), introduce a new message (if the changes are big), to approve/improve Stefan's BIP 32 for custom services or to have a value in the byte stream indicating which fields are present (maybe
a bitfield or so).</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>Using a quirk of an implementation is just bad form and sloppy coding. Optional fields should have their own mechanism that allows them to remain as optional fields between protocol version upgrades.<br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>The bitcoind software can probably be improved
too, by checking that the length of the version message is consistent for the protocol version given by the connected node. Right now it makes no assumptions based on that which is a mistake (new clients can broadcast older version messages that don't have all the fields required). Probably the software should penalise hosts which do that.</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><span>What's the big deal to update the protocol version number from 70001 to 70002? It's not like we'll run out of integers. The field has now gone from optional to required now anyway - that's a behaviour change. It'd be good to enforce that. I see this as a
bug.<br></span></div><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <hr size="1"> <font face="Arial" size="2"> <b><span style="font-weight:bold;">From:</span></b> Mike Hearn <mike@plan99.net><br> <b><span style="font-weight: bold;">To:</span></b> Pieter Wuille <pieter.wuille@gmail.com> <br><b><span style="font-weight: bold;">Cc:</span></b> Bitcoin Dev <bitcoin-development@lists.sourceforge.net>; Tamas Blummer <tamas@bitsofproof.com> <br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, June 20, 2013 11:17 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Bitcoin-development] Missing fRelayTxes in version<br> </font> </div> <div class="y_msg_container"><br>
<meta http-equiv="x-dns-prefetch-control" content="off"><div id="yiv650029406"><div dir="ltr">There's no problem, but there's no benefit either. It also locks us in to a potentially problematic guarantee - what if in future we want to have, say, two optional new pieces of data in two different messages. We don't want to require that if version > X then you have to implement all features up to and including that point.<div>
<br></div><div>Essentially the number of fields in a message is like a little version number, just for that message. It adds flexibility to keep it that way, and there's no downside, seeing as that bridge was already crossed and people with parsers that can't handle it need to fix their code anyway.</div>
<div><br></div><div>So I have a slight preference for keeping things the way they are, it keeps things flexible for future and costs nothing.</div><div><br></div></div><div class="yiv650029406gmail_extra"><br><br><div class="yiv650029406gmail_quote">
On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <span dir="ltr"><<a rel="nofollow" ymailto="mailto:pieter.wuille@gmail.com" target="_blank" href="mailto:pieter.wuille@gmail.com">pieter.wuille@gmail.com</a>></span> wrote:<br><blockquote class="yiv650029406gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="yiv650029406im">On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:<br>
> Sure but why not do that when there's an actual new field to add? Does<br>
> anyone have a proposal for a feature that needs a new version field at the<br>
> moment? There's no point changing the protocol now unless there's actually<br>
> a new field to add.<br>
><br>
> Anyway I still don't see why anyone cares about this issue. The Bitcoin<br>
> protocol does not and never has required that all messages have a fixed<br>
> number of fields per version. Any parser written on the assumption it did<br>
> was just buggy. Look at how tx messages are relayed for the most obvious<br>
> example of that pattern in action - it's actually the raw byte stream<br>
> that's stored and relayed to ensure that fields added in new versions<br>
> aren't dropped during round-tripping. Old versions are supposed to preserve<br>
> fields from the future.<br>
<br>
</div>Actually, that is not the same issue. What is being argued for here is that<br>
the version in the version message itself should indicate which fields are<br>
present, so a parser doesn't need to look at the length of the message. That<br>
seems like a minor but very reasonable request to me, and it's trivial to do.<br>
That doesn't mean that you may receive versions higher than what you know of,<br>
and thus messages with fields you don't know about. That doesn't matter, you<br>
can just ignore them.<br>
<br>
I see no problem with raising the protocol version number to indicate<br>
"all fields up to fRelayTxes are required, if the announced nVersion is above N".<br>
In fact, I believe (though haven't checked) all previous additions to the version<br>
message were accompanied with a protocol version (then: client version) increase<br>
as well.<br>
<span class="yiv650029406HOEnZb"><font color="#888888"><br>
--<br>
Pieter<br>
<br>
</font></span></blockquote></div><br></div>
</div><meta http-equiv="x-dns-prefetch-control" content="on"><br>------------------------------------------------------------------------------<br>This SF.net email is sponsored by Windows:<br><br>Build for Windows Store.<br><br><a href="http://p.sf.net/sfu/windows-dev2dev" target="_blank">http://p.sf.net/sfu/windows-dev2dev</a><br>_______________________________________________<br>Bitcoin-development mailing list<br><a ymailto="mailto:Bitcoin-development@lists.sourceforge.net" href="mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a><br><a href="https://lists.sourceforge.net/lists/listinfo/bitcoin-development" target="_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-development</a><br><br><br></div> </div> </div> </div></body></html>
---1963244382-266082894-1371724625=:50978--
|
