1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
|
Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <sirk390@gmail.com>) id 1SI2Si-00051u-6u
for bitcoin-development@lists.sourceforge.net;
Wed, 11 Apr 2012 18:40:12 +0000
Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.213.47 as permitted sender)
client-ip=209.85.213.47; envelope-from=sirk390@gmail.com;
helo=mail-yw0-f47.google.com;
Received: from mail-yw0-f47.google.com ([209.85.213.47])
by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-MD5:128)
(Exim 4.76) id 1SI2Sc-0000Qu-Jg
for bitcoin-development@lists.sourceforge.net;
Wed, 11 Apr 2012 18:40:12 +0000
Received: by yhjj56 with SMTP id j56so803530yhj.34
for <bitcoin-development@lists.sourceforge.net>;
Wed, 11 Apr 2012 11:40:01 -0700 (PDT)
Received: by 10.60.18.137 with SMTP id w9mr23761135oed.7.1334169601021; Wed,
11 Apr 2012 11:40:01 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.13.103 with HTTP; Wed, 11 Apr 2012 11:39:40 -0700 (PDT)
From: Christian Bodt <sirk390@gmail.com>
Date: Wed, 11 Apr 2012 20:39:40 +0200
Message-ID: <CA+XhJbpNYUyPm2Ymcpg3grbfGnfERCsUJNJuByEJbJLsMMmMbQ@mail.gmail.com>
To: bitcoin-development@lists.sourceforge.net
Content-Type: multipart/alternative; boundary=e89a8f83aa1723968304bd6b91ca
X-Spam-Score: -0.3 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(sirk390[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in
digit (sirk390[at]gmail.com)
1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1SI2Sc-0000Qu-Jg
Subject: [Bitcoin-development] Adding request/reply id in messages
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: sirk390@gmail.com
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 11 Apr 2012 18:40:12 -0000
--e89a8f83aa1723968304bd6b91ca
Content-Type: text/plain; charset=UTF-8
Hi,
I would like to discuss the following bitcoin protocol improvement proposal:
Adding request/reply id in all messages (in the message header,
based on what was done for the "checksum" field)
The original reason is that I found it very hard to implement robust
blockchain downloading as we are missing context information:
The download protocol relies on the other peer sending one (or more) "inv"
reponse(s) to "getblocks" and sending the "hashContinue".
But if the other peer doesn't send a response to getblock, send a partial
response to getblocks, mixes it with some unrelated blocks inventories
or doesn't send the "hashContinue" it is very hard to detect and handle
(e.g. ban the peer and switch to another). This could cause some DoS
attacks by misbehaving peers.
The problems are that 1/ we don't know how many "inv" messages to wait for
after "getblocks" and 2/ we don't know how to distinguish between result of
"getblocks" and spontaneous "inv" notifications.
The same is true for "addr" messages (it is both a notification and reply)
but this is less of a problem as a lack of response to getaddr
doesn't constitute a DoS.
The idea would be to add a new "requestid" field in messages and define the
following:
- verack sends back the requestid given in version.
- inv sends back the requestid given in getblocks or a special value
in case of a notification.
- addr sends back the requestid given in getaddr or a special value
in case of a notification.
- tx sends back the requestid given in getdata.
- block sends back the requestid given in getdata
- headers sends back the requestid given in getheaders.
- reply sends back the requestid given in checkorder/submitorder.
- pong sends back the requestid given in ping.
With the following rules:
- getblocks always results in a single"inv" reply (0-500 elements
allowed).
- getdata still result in multiple 'tx' or 'block' 's (as currently).
This is inspired by the common request/reply pattern (the id is sometimes
called "correlation id").
This would permit better asynchronous messaging (multiple request of the
same type in parallel. This could be useful for
getblocks/getdata/getheaders and future messages).
Historic note: It seems similar to the existing mechanism of
"CRequestTracker, hashReply & PushRequest" in the current bitcoin source
code except requestId would only be a 32bits instead of the 256bits
of hashReply .
I have implemented a prototype here: https://github.com/sirk390/bitcoin/
This could allow to remove "hashContinue" in a second step like this:
https://github.com/sirk390/bitcoin/tree/better_block_download
I'm discussing it now as I read the PONG BIP and saw some similar nonce
field added.
I was not sure if this nonce field would still be required if we had
request/reply ids .
What do you think?
Best Regards,
Christian Bodt, France
Note: I am working on a python implementation of bitcoin.
See https://github.com/sirk390/coinpy
--e89a8f83aa1723968304bd6b91ca
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<span style>Hi,</span><div style><br></div><div style>I would like to discu=
ss the following bitcoin protocol improvement proposal:</div><div style><br=
></div><div style>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Adding request/reply id=
in all messages (in the message header, based on what was done for the &qu=
ot;checksum" field)</div>
<div style><br></div><div style>The original reason is that I found it very=
hard to implement robust blockchain downloading as we are missing context =
information:</div><div style>The download protocol relies on the other peer=
sending one (or more) "inv" reponse(s) to "getblocks" =
and sending the "hashContinue".</div>
<div style>But if the other peer doesn't send a response to getblock, s=
end a partial response to getblocks, mixes it with some unrelated blocks in=
ventories or=C2=A0doesn't send the "hashContinue"=C2=A0it is =
very hard to detect and handle (e.g. ban the peer and switch to another). =
=C2=A0This could cause some DoS attacks by misbehaving peers.</div>
<div style>The problems are that 1/ we don't know how many "inv&qu=
ot; messages to wait for after "getblocks"=C2=A0and 2/ we don'=
;t know how to=C2=A0distinguish between result of "getblocks" and=
spontaneous "inv" notifications.</div>
<div style>The same is true for =C2=A0"addr" messages (it is both=
a notification and reply) but this is less of a problem as a lack of respo=
nse to getaddr doesn't=C2=A0constitute=C2=A0a DoS.</div><div style><br>=
</div><div style>
The idea would be to add a new "requestid" field in messages and =
define the following:</div><div style>=C2=A0 =C2=A0 =C2=A0 - verack sends b=
ack the=C2=A0requestid given in version.</div><div style>=C2=A0 =C2=A0 =C2=
=A0 - inv sends back the=C2=A0requestid=C2=A0given=C2=A0in getblocks or a s=
pecial value in case of a=C2=A0notification.</div>
<div style>=C2=A0 =C2=A0 =C2=A0 - addr sends back the=C2=A0requestid=C2=A0g=
iven=C2=A0in getaddr or a special value in case of a=C2=A0notification.</di=
v><span style>=C2=A0 =C2=A0 =C2=A0 - tx sends back the=C2=A0requestid=C2=A0=
given=C2=A0in getdata.=C2=A0</span><br style><div style>=C2=A0=C2=A0 =C2=
=A0 =C2=A0- block sends back the=C2=A0requestid=C2=A0given=C2=A0in getdata<=
/div>
<div style>=C2=A0=C2=A0 =C2=A0 =C2=A0- headers sends back the=C2=A0requesti=
d=C2=A0given=C2=A0in=C2=A0getheaders.</div><div style>=C2=A0=C2=A0 =C2=A0 =
=C2=A0- reply sends back the=C2=A0requestid=C2=A0given=C2=A0in=C2=A0checkor=
der/submitorder.</div><div style>=C2=A0 =C2=A0 =C2=A0 - pong sends back the=
requestid=C2=A0given=C2=A0in ping.</div>
<div style><br></div><div style>With the following rules:</div><div style>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0- getblocks always results in a single"inv&=
quot; reply (0-500 elements allowed).=C2=A0</div><div style>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0- getdata still result in multiple 'tx' or 'block&=
#39; 's (as currently).</div>
<div style><br></div><div style>This is inspired by the common=C2=A0request=
/reply=C2=A0pattern (the id is sometimes called "correlation id")=
.=C2=A0</div><div style>This would permit better=C2=A0asynchronous messagin=
g (multiple request of the same type in=C2=A0parallel. This could be useful=
for getblocks/getdata/getheaders and future messages).</div>
<div style><br></div><div style>Historic note: It seems similar to the exis=
ting mechanism of "CRequestTracker,=C2=A0hashReply & PushRequest&q=
uot; in the current bitcoin source code except=C2=A0requestId =C2=A0would o=
nly be a 32bits instead of the 256bits of=C2=A0hashReply=C2=A0.</div>
<div style><br></div><div style>I have implemented a prototype here:=C2=A0<=
a href=3D"https://github.com/sirk390/bitcoin/" target=3D"_blank" style=3D"c=
olor:rgb(17,85,204)">https://github.com/sirk390/bitcoin/</a></div><div styl=
e>This could allow to remove "hashContinue" in a second step like=
this:=C2=A0<a href=3D"https://github.com/sirk390/bitcoin/tree/better_block=
_download" target=3D"_blank" style=3D"color:rgb(17,85,204)">https://github.=
com/sirk390/bitcoin/tree/better_block_download</a></div>
<div style><br></div><div style>I'm discussing it now as I read the PON=
G BIP and saw some similar nonce field added.</div><div style>I was not sur=
e if this nonce field would still be required if we had request/reply ids .=
=C2=A0</div>
<div style><br></div><div style>What do you think?</div><div style><br></di=
v><div style>Best Regards,</div><div style>Christian Bodt, France</div><div=
style><div>Note: I am working on a python implementation of bitcoin.</div>
See=C2=A0<a href=3D"https://github.com/sirk390/coinpy" target=3D"_blank" st=
yle=3D"color:rgb(17,85,204)">https://github.com/sirk390/coinpy</a></div>
--e89a8f83aa1723968304bd6b91ca--
|
