1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <brianchoffman@gmail.com>) id 1WYHlT-0005Ra-R2
for bitcoin-development@lists.sourceforge.net;
Thu, 10 Apr 2014 16:23:47 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.192.175 as permitted sender)
client-ip=209.85.192.175; envelope-from=brianchoffman@gmail.com;
helo=mail-pd0-f175.google.com;
Received: from mail-pd0-f175.google.com ([209.85.192.175])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1WYHlS-000671-NG
for bitcoin-development@lists.sourceforge.net;
Thu, 10 Apr 2014 16:23:47 +0000
Received: by mail-pd0-f175.google.com with SMTP id x10so4070254pdj.20
for <bitcoin-development@lists.sourceforge.net>;
Thu, 10 Apr 2014 09:23:40 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.68.136.226 with SMTP id qd2mr5987684pbb.72.1397147020788;
Thu, 10 Apr 2014 09:23:40 -0700 (PDT)
Received: by 10.70.89.237 with HTTP; Thu, 10 Apr 2014 09:23:40 -0700 (PDT)
In-Reply-To: <CA+s+GJBxEC2MifJQY5-vn2zSOHo-UOm8B1vYHHOfuxq26=VscQ@mail.gmail.com>
References: <CANEZrP04O7EqB=TqyTiC7O1K2A9R0nKJ_ssANHKg=Byum8-LtA@mail.gmail.com>
<CA+s+GJDbYjwhpsV15a+7kCO_vTstEewVrwvqbnB=a5zOSwFC6Q@mail.gmail.com>
<CAAS2fgStmEpiUV4Yh-qqu6sZ+VZ5SiQPwp+QA=3X5zR52ia3OA@mail.gmail.com>
<CA+s+GJBxEC2MifJQY5-vn2zSOHo-UOm8B1vYHHOfuxq26=VscQ@mail.gmail.com>
Date: Thu, 10 Apr 2014 12:23:40 -0400
Message-ID: <CAADm4BDDJkS_xdjUn=2Yzs4B0RXTvpzpd5Z_kDRorzrn1HWSng@mail.gmail.com>
From: Brian Hoffman <brianchoffman@gmail.com>
To: Wladimir <laanwj@gmail.com>
Content-Type: multipart/alternative; boundary=e89a8f234a39df8cc204f6b2a33e
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(brianchoffman[at]gmail.com)
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,
no trust [209.85.192.175 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1WYHlS-000671-NG
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Chain pruning
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 10 Apr 2014 16:23:48 -0000
--e89a8f234a39df8cc204f6b2a33e
Content-Type: text/plain; charset=UTF-8
This is probably just noise, but what if nodes could compress and store
earlier transaction sets (archive sets) and serve them up conditionally. So
if there were let's say 100 archive sets of (10,000 blocks) you might have
5 open at any time when you're an active archive node while the others sit
on your disk compressed and unavailable to the network. This would allow
nodes to have all full transactions but conserve disk space and network
activity since they wouldn't ever respond about every possible transaction.
This could be based on a rotational request period, based on request count
or done periodically. Once their considered active they would be expected
to uncompress a set and make it available to the network. Clients would
have to piece together archive sets from different nodes, but if there
weren't enough archive nodes to cover the chain they could ratchet up the
amount of required open archive sets when your node was active.
I fully expect to have my idea trashed, but I'm dipping toes in the waters
of contribution.
On Thu, Apr 10, 2014 at 10:19 AM, Wladimir <laanwj@gmail.com> wrote:
>
> On Thu, Apr 10, 2014 at 2:10 PM, Gregory Maxwell <gmaxwell@gmail.com>wrote:
>
>> But sure I could see a fixed range as also being a useful contribution
>> though I'm struggling to figure out what set of constraints would
>> leave a node without following the consensus? Obviously it has
>> bandwidth if you're expecting to contribute much in serving those
>> historic blocks... and verifying is reasonably cpu cheap with fast
>> ecdsa code. Maybe it has a lot of read only storage?
>>
>
> The use case is that you could burn the node implementation + block data +
> a live operating system on a read-only medium. This could be set in stone
> for a long time.
>
> There would be no consensus code to keep up to date with protocol
> developments, because it doesn't take active part in it.
>
> I don't think it would be terribly useful right now, but it could be
> useful when nodes that host all history become rare. It'd allow
> distributing 'pieces of history' in a self-contained form.
>
>
>> I think it should be possible to express and use such a thing in the
>> protocol even if I'm currently unsure as to why you wouldn't do 100000
>> - 200000 _plus_ the most recent 144 that you were already keeping
>> around for reorgs.
>>
>
> Yes, it would be nice to at least be able to express it, if it doesn't
> make the protocol too finicky.
>
> In terms of peer selection, if the blocks you need aren't covered by
>> the nodes you're currently connected to I think you'd prefer to seek
>> node nodes which have the least rare-ness in the ranges they offer.
>> E.g. if you're looking for a block 50 from the tip, you're should
>> probably not prefer to fetch it from someone with blocks 100000-150000
>> if its one of only 100 nodes that has that range.
>>
>
> That makes sense.
>
> In general, if you want a block 50 from the tip, it would be best to
> request it from a node that only serves the last N (N>~50) blocks, and not
> a history node that could use the same bandwidth to serve earlier, rarer
> blocks to others.
>
> Wladimir
>
>
>
> ------------------------------------------------------------------------------
> Put Bad Developers to Shame
> Dominate Development with Jenkins Continuous Integration
> Continuously Automate Build, Test & Deployment
> Start a new project now. Try Jenkins in the cloud.
> http://p.sf.net/sfu/13600_Cloudbees
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
--e89a8f234a39df8cc204f6b2a33e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">This is probably just noise, but what if nodes could compr=
ess and store earlier transaction sets (archive sets) and serve them up con=
ditionally. So if there were let's say 100 archive sets of (10,000 bloc=
ks) you might have 5 open at any time when you're an active archive nod=
e while the others sit on your disk compressed and unavailable to the netwo=
rk. This would allow nodes to have all full transactions but conserve disk =
space and network activity since they wouldn't ever respond about every=
possible transaction.<div>
<br></div><div>This could be based on a rotational request period, based on=
request count or done periodically. Once their considered active they woul=
d be expected to uncompress a set and make it available to the network. Cli=
ents would have to piece together archive sets from different nodes, but if=
there weren't enough archive nodes to cover the chain they could ratch=
et up the amount of required open archive sets when your node was active.</=
div>
<div><br></div><div>I fully expect to have my idea trashed, but I'm dip=
ping toes in the waters of contribution.</div><div><br></div><div><br></div=
></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Thu=
, Apr 10, 2014 at 10:19 AM, Wladimir <span dir=3D"ltr"><<a href=3D"mailt=
o:laanwj@gmail.com" target=3D"_blank">laanwj@gmail.com</a>></span> wrote=
:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_extra">=
<br><div class=3D"gmail_quote">On Thu, Apr 10, 2014 at 2:10 PM, Gregory Max=
well <span dir=3D"ltr"><<a href=3D"mailto:gmaxwell@gmail.com" target=3D"=
_blank">gmaxwell@gmail.com</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">But sure I could see a fixed range as also b=
eing a useful contribution<br>
though I'm struggling to figure out what set of constraints would<br>
leave a node without following the consensus? =C2=A0 Obviously it has<br>
bandwidth if you're expecting to contribute much in serving those<br>
historic blocks... and verifying is reasonably cpu cheap with fast<br>
ecdsa code. =C2=A0 Maybe it has a lot of read only storage?<br></blockquote=
><div><br>The use case is that you could burn the node implementation + blo=
ck data + a live operating system on a read-only medium. This could be set =
in stone for a long time.<br>
<br></div><div>There would be no consensus code to keep up to date with pro=
tocol developments, because it doesn't take active part in it.<br></div=
><div><br></div><div>I don't think it would be terribly useful right no=
w, but it could be useful when nodes that host all history become rare. It&=
#39;d allow distributing 'pieces of history' in a self-contained fo=
rm.<br>
</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I think it should be possible to express and use such a thing in the<br>
protocol even if I'm currently unsure as to why you wouldn't do 100=
000<br>
- 200000 =C2=A0_plus_ the most recent 144 that you were already keeping<br>
around for reorgs.<br></blockquote><div><br></div><div>Yes, it would be nic=
e to at least be able to express it, if it doesn't make the protocol to=
o finicky.<br><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0=
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In terms of peer selection, if the blocks you need aren't covered by<br=
>
the nodes you're currently connected to I think you'd prefer to see=
k<br>
node nodes which have the least rare-ness in the ranges they offer.<br>
E.g. if you're looking for a block 50 from the tip, =C2=A0you're sh=
ould<br>
probably not prefer to fetch it from someone with blocks 100000-150000<br>
if its one of only 100 nodes that has that range.<br></blockquote><div><br>=
</div><div>That makes sense. <br><br>In general, if you want a block 50 fro=
m the tip, it would be best to request it from a node that only serves the =
last N (N>~50) blocks, and not a history node that could use the same ba=
ndwidth to serve earlier, rarer blocks to others.<span class=3D"HOEnZb"><fo=
nt color=3D"#888888"><br>
</font></span></div><span class=3D"HOEnZb"><font color=3D"#888888"><div><br=
></div><div>Wladimir <br></div></font></span></div><br></div></div>
<br>-----------------------------------------------------------------------=
-------<br>
Put Bad Developers to Shame<br>
Dominate Development with Jenkins Continuous Integration<br>
Continuously Automate Build, Test & Deployment<br>
Start a new project now. Try Jenkins in the cloud.<br>
<a href=3D"http://p.sf.net/sfu/13600_Cloudbees" target=3D"_blank">http://p.=
sf.net/sfu/13600_Cloudbees</a><br>_________________________________________=
______<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
pment@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div><br></div>
--e89a8f234a39df8cc204f6b2a33e--
|
