1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
|
Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
helo=mx.sourceforge.net)
by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <moon@justmoon.de>) id 1RcO8g-0000bS-Nu
for bitcoin-development@lists.sourceforge.net;
Sun, 18 Dec 2011 21:19:22 +0000
X-ACL-Warn:
Received: from sulfur.webpack.hosteurope.de ([217.115.142.104])
by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76)
id 1RcO8e-0000MH-Ip for bitcoin-development@lists.sourceforge.net;
Sun, 18 Dec 2011 21:19:22 +0000
Received: from 84-72-69-153.dclient.hispeed.ch ([84.72.69.153]
helo=[192.168.0.21]); authenticated
by sulfur.webpack.hosteurope.de running ExIM with esmtpsa
(TLSv1:AES256-SHA:256)
id 1RcO8Y-0000hw-9X; Sun, 18 Dec 2011 22:19:14 +0100
Message-ID: <4EEE58CA.5090902@justmoon.de>
Date: Sun, 18 Dec 2011 22:19:06 +0100
From: Stefan Thomas <moon@justmoon.de>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
rv:8.0) Gecko/20111105 Thunderbird/8.0
MIME-Version: 1.0
To: bitcoin-development@lists.sourceforge.net
References: <CABr1YTebhitO4g-SarZ7H=aoG9a8zW1wd0rfR32o8i0vODbLJw@mail.gmail.com>
<82659F61-0449-47BB-88DC-497E0D02F8A1@ceptacle.com>
<CALxbBHUXEJLRDZ=RS1vuvkm7rDjFUPir0sU__f6TJXiTTQxWzA@mail.gmail.com>
In-Reply-To: <CALxbBHUXEJLRDZ=RS1vuvkm7rDjFUPir0sU__f6TJXiTTQxWzA@mail.gmail.com>
Content-Type: multipart/alternative;
boundary="------------060705060006050107070606"
X-bounce-key: webpack.hosteurope.de;moon@justmoon.de;1324243160;b0d68d12;
X-Spam-Score: 0.5 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
1.0 HTML_MESSAGE BODY: HTML included in message
-0.5 AWL AWL: From: address is in the auto white-list
X-Headers-End: 1RcO8e-0000MH-Ip
Subject: Re: [Bitcoin-development] Protocol extensions
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Sun, 18 Dec 2011 21:19:22 -0000
This is a multi-part message in MIME format.
--------------060705060006050107070606
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Hey Chris,
> The storage would be distributed, messages are routed on behalf of
> others, which makes finding the origin of the query hard to find
> (think Tor)
This type of intermediate routing makes Tor slow
<https://svn.torproject.org/svn/projects/roadmaps/2009-03-11-performance.pdf>.
Bitcoin does not and imho should not make anonymity guarantees. Many
users do not need them.
Let those who want anonymity connect through Tor, Freenet, etc. It's
easy to add anonymity via an extra layer, but it is impossible to add
performance on top of a slow system.
That's really the only thing I wanted to point out - if you do DHTs,
focus on performance, not anonymity. :)
Cheers,
Stefan
On 12/17/2011 2:37 PM, Christian Decker wrote:
> A while back I had proposed a similar idea to the DHT, although my
> main goal was to reduce the need for broadcasts.
>
> My idea was to structure the network in a hypercube and use prefixes
> to address different parts of the network, and use those prefixes also
> to find the location where an item (transaction, block, ...) should be
> stored. Each vertex in the hypercube is a small, highly connected,
> cluster of nodes. The storage would be distributed, messages are
> routed on behalf of others, which makes finding the origin of the
> query hard to find (think Tor), each node would have to store only
> O(log(p)) items, with p being the prefix length, maximum number of
> hops is equal to the dimension of the hypercube O(log(n)).
>
> Newly created transaction will be sent directly to the location
> they'll be stored and miners retrieve new transactions at regular
> intervals. It might increase delays to the confirmations, but it
> reduces the number of broadcasts and storage requirements on nodes
> greatly.
>
> Regards,
> Chris
>
>
> On Sat, Dec 17, 2011 at 2:13 PM, Michael Grønager
> <gronager@ceptacle.com <mailto:gronager@ceptacle.com>> wrote:
>
> Hey Eric,
>
> Two comments.
>
> 1.
> The ability to query for transactions belonging to pubkeys or
> bitcoin addresses is supported today by several implementations:
> * blockexplorer.com <http://blockexplorer.com>
> * bitcoin-js
> * my own libBTC (will more on this soon)
>
> To query for transactions you need to use json-rpc and not the
> bitcoin protocol, however. But still the purpose is the same: to
> be able to build thin clients that can rely on a server for
> storing the blockchain and keeping connected on the p2p network.
>
> The reason for not having these queries part of the standard
> protocol (I think) are as they breaks anonymity, and that you
> would actually encourage people to participate in the p2p.
>
> 2. The second part you mention, to some how move the storage of
> the blockchain into a DHT based storage would be quite nice. The
> benefit of this is that it could be a way to integrate the smaller
> clients into the network without breaking the anonymity. But it
> should be thought out quite carefully. Further, if each client
> only store a fraction of the blockchain we should work out what
> fraction that need to be in order to ensure a similar service
> level. I would be happy to work with you on this.
>
> Cheers,
>
> Michael
>
> On 17/12/2011, at 08:41, Eric Lombrozo wrote:
>
> > Hey, guys.
> >
> > I haven't posted here before so I'll introduce myself. My name's
> Eric,
> > I've been developing cryptocurrency-related
> > software for several months now, I've implemented some libraries for
> > dealing with core bitcoin datastructures, made
> > some custom builds of bitcoind and interfaced it with a few apps
> I've written.
> >
> > In doing so, I've come to appreciate just how little of the
> potential
> > for the bitcoin protocol is being exploited right now...
> > not only in terms of the script features but in terms of the
> potential
> > commands and node types that could exist.
> >
> > For instance, the protocol spec at
> > https://en.bitcoin.it/wiki/Protocol_specification only has 16
> commands
> > listed and
> > only one service type...despite having a full 12 bytes for a command
> > code and a full eight bytes for a services
> > type.
> >
> > The fact that only one node service type is specified is
> probably due
> > to the fact that the satoshi client was written
> > to be a standalone monolithic app that took care of all the
> essential
> > needs for a network of peers.
> > i.e. block chain storage/management, transaction
> signing/verification,
> > key generation/wallet management, block mining, etc...
> > However, I think there's an urgent need for breaking up all these
> > different tasks into separate components that can run as independent
> > services on different types of devices.
> >
> > One of the big issues I'm dealing with now pertains to block chain
> > storage. As of right now, it is implemented as sequential
> > disk files using Berkeley DB in the satoshi client. Then you have
> > other projects that have been using SQL tables, etc...
> > But I believe the direction this really needs to move towards is
> some
> > sort of distributed hash table...and the database queries
> > should be performed using the bitcoin protocol itself. Perhaps
> adding
> > a few more commands. As things stand right now,
> > the only way to query for transactions or blocks is by their
> hash. And
> > once a transaction gets incorporated into a block and
> > removed from the transaction pool, one can no longer query it by the
> > transaction hash without stepping outside the bitcoin protocol.
> > We need access to the disk file that stores the blocks whether it be
> > via Berkeley DB or SQL or whatever.
> >
> > I propose an extension to the bitcoin protocol to provide
> methods for
> > performing more sophisticated queries, such as "Give me
> > an inventory of transactions involving this particular public
> key" or
> > "Give me an inventory all transactions in the last n blocks with
> > unredeemed outputs." This could be done by adding a few more
> commands.
> >
> > Furthermore, I propose a new network services type for nodes that
> > serve as block chain/transaction pool storage.
> >
> > Of couse, any peer that wishes to verify the integrity of the block
> > chain would still have to download at the very least
> > all the block headers...and to be completely sure, also all the
> blocks
> > themselves...and verify everything. But it would be
> > very nice to be able to run thin services that can rely on other
> > network peers to do this work. It is still possible to attain
> > a high level of confidence in the integrity by querying multiple
> peers
> > for similar objects and comparing. It is also possible
> > to run your own dedicated block chain storage servers which you
> trust.
> >
> > There are other ideas I have for other types of services, too.
> >
> > Anyhow, I'm just throwing this out there...if anyone's
> interested I'd
> > love to develop these ideas further and help put together some
> > specs.
> >
> > -Eric Lombrozo
> >
> >
> ------------------------------------------------------------------------------
> > Learn Windows Azure Live! Tuesday, Dec 13, 2011
> > Microsoft is holding a special Learn Windows Azure training
> event for
> > developers. It will provide a great way to learn Windows Azure
> and what it
> > provides. You can attend the event by watching it streamed LIVE
> online.
> > Learn more at http://p.sf.net/sfu/ms-windowsazure
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists.sourceforge.net
> <mailto:Bitcoin-development@lists.sourceforge.net>
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live! Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and
> what it
> provides. You can attend the event by watching it streamed LIVE
> online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> <mailto:Bitcoin-development@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live! Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
>
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--------------060705060006050107070606
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hey Chris,<br>
<br>
<blockquote type="cite">The storage would be distributed, messages
are routed on behalf of others, which makes finding the origin of
the query hard to find (think Tor)</blockquote>
<br>
This type of intermediate routing makes Tor <a
href="https://svn.torproject.org/svn/projects/roadmaps/2009-03-11-performance.pdf">slow</a>.
Bitcoin does not and imho should not make anonymity guarantees. Many
users do not need them.<br>
<br>
Let those who want anonymity connect through Tor, Freenet, etc. It's
easy to add anonymity via an extra layer, but it is impossible to
add performance on top of a slow system.<br>
<br>
That's really the only thing I wanted to point out - if you do DHTs,
focus on performance, not anonymity. :)<br>
<br>
Cheers,<br>
<br>
Stefan<br>
<br>
On 12/17/2011 2:37 PM, Christian Decker wrote:
<blockquote
cite="mid:CALxbBHUXEJLRDZ=RS1vuvkm7rDjFUPir0sU__f6TJXiTTQxWzA@mail.gmail.com"
type="cite">A while back I had proposed a similar idea to the DHT,
although my main goal was to reduce the need for broadcasts.<br>
<br>
My idea was to structure the network in a hypercube and use
prefixes to address different parts of the network, and use those
prefixes also to find the location where an item (transaction,
block, ...) should be stored. Each vertex in the hypercube is a
small, highly connected, cluster of nodes. The storage would be
distributed, messages are routed on behalf of others, which makes
finding the origin of the query hard to find (think Tor), each
node would have to store only O(log(p)) items, with p being the
prefix length, maximum number of hops is equal to the dimension of
the hypercube O(log(n)).<br>
<br>
Newly created transaction will be sent directly to the location
they'll be stored and miners retrieve new transactions at regular
intervals. It might increase delays to the confirmations, but it
reduces the number of broadcasts and storage requirements on nodes
greatly.<br>
<br>
Regards,<br>
Chris<br clear="all">
<br>
<br>
<div class="gmail_quote">On Sat, Dec 17, 2011 at 2:13 PM, Michael
Grønager <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:gronager@ceptacle.com">gronager@ceptacle.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Eric,<br>
<br>
Two comments.<br>
<br>
1.<br>
The ability to query for transactions belonging to pubkeys or
bitcoin addresses is supported today by several
implementations:<br>
* <a moz-do-not-send="true" href="http://blockexplorer.com"
target="_blank">blockexplorer.com</a><br>
* bitcoin-js<br>
* my own libBTC (will more on this soon)<br>
<br>
To query for transactions you need to use json-rpc and not the
bitcoin protocol, however. But still the purpose is the same:
to be able to build thin clients that can rely on a server for
storing the blockchain and keeping connected on the p2p
network.<br>
<br>
The reason for not having these queries part of the standard
protocol (I think) are as they breaks anonymity, and that you
would actually encourage people to participate in the p2p.<br>
<br>
2. The second part you mention, to some how move the storage
of the blockchain into a DHT based storage would be quite
nice. The benefit of this is that it could be a way to
integrate the smaller clients into the network without
breaking the anonymity. But it should be thought out quite
carefully. Further, if each client only store a fraction of
the blockchain we should work out what fraction that need to
be in order to ensure a similar service level. I would be
happy to work with you on this.<br>
<br>
Cheers,<br>
<br>
Michael<br>
<div class="HOEnZb">
<div class="h5"><br>
On 17/12/2011, at 08:41, Eric Lombrozo wrote:<br>
<br>
> Hey, guys.<br>
><br>
> I haven't posted here before so I'll introduce
myself. My name's Eric,<br>
> I've been developing cryptocurrency-related<br>
> software for several months now, I've implemented
some libraries for<br>
> dealing with core bitcoin datastructures, made<br>
> some custom builds of bitcoind and interfaced it with
a few apps I've written.<br>
><br>
> In doing so, I've come to appreciate just how little
of the potential<br>
> for the bitcoin protocol is being exploited right
now...<br>
> not only in terms of the script features but in terms
of the potential<br>
> commands and node types that could exist.<br>
><br>
> For instance, the protocol spec at<br>
> <a moz-do-not-send="true"
href="https://en.bitcoin.it/wiki/Protocol_specification"
target="_blank">https://en.bitcoin.it/wiki/Protocol_specification</a>
only has 16 commands<br>
> listed and<br>
> only one service type...despite having a full 12
bytes for a command<br>
> code and a full eight bytes for a services<br>
> type.<br>
><br>
> The fact that only one node service type is specified
is probably due<br>
> to the fact that the satoshi client was written<br>
> to be a standalone monolithic app that took care of
all the essential<br>
> needs for a network of peers.<br>
> i.e. block chain storage/management, transaction
signing/verification,<br>
> key generation/wallet management, block mining,
etc...<br>
> However, I think there's an urgent need for breaking
up all these<br>
> different tasks into separate components that can run
as independent<br>
> services on different types of devices.<br>
><br>
> One of the big issues I'm dealing with now pertains
to block chain<br>
> storage. As of right now, it is implemented as
sequential<br>
> disk files using Berkeley DB in the satoshi client.
Then you have<br>
> other projects that have been using SQL tables,
etc...<br>
> But I believe the direction this really needs to move
towards is some<br>
> sort of distributed hash table...and the database
queries<br>
> should be performed using the bitcoin protocol
itself. Perhaps adding<br>
> a few more commands. As things stand right now,<br>
> the only way to query for transactions or blocks is
by their hash. And<br>
> once a transaction gets incorporated into a block and<br>
> removed from the transaction pool, one can no longer
query it by the<br>
> transaction hash without stepping outside the bitcoin
protocol.<br>
> We need access to the disk file that stores the
blocks whether it be<br>
> via Berkeley DB or SQL or whatever.<br>
><br>
> I propose an extension to the bitcoin protocol to
provide methods for<br>
> performing more sophisticated queries, such as "Give
me<br>
> an inventory of transactions involving this
particular public key" or<br>
> "Give me an inventory all transactions in the last n
blocks with<br>
> unredeemed outputs." This could be done by adding a
few more commands.<br>
><br>
> Furthermore, I propose a new network services type
for nodes that<br>
> serve as block chain/transaction pool storage.<br>
><br>
> Of couse, any peer that wishes to verify the
integrity of the block<br>
> chain would still have to download at the very least<br>
> all the block headers...and to be completely sure,
also all the blocks<br>
> themselves...and verify everything. But it would be<br>
> very nice to be able to run thin services that can
rely on other<br>
> network peers to do this work. It is still possible
to attain<br>
> a high level of confidence in the integrity by
querying multiple peers<br>
> for similar objects and comparing. It is also
possible<br>
> to run your own dedicated block chain storage servers
which you trust.<br>
><br>
> There are other ideas I have for other types of
services, too.<br>
><br>
> Anyhow, I'm just throwing this out there...if
anyone's interested I'd<br>
> love to develop these ideas further and help put
together some<br>
> specs.<br>
><br>
> -Eric Lombrozo<br>
><br>
>
------------------------------------------------------------------------------<br>
> Learn Windows Azure Live! Tuesday, Dec 13, 2011<br>
> Microsoft is holding a special Learn Windows Azure
training event for<br>
> developers. It will provide a great way to learn
Windows Azure and what it<br>
> provides. You can attend the event by watching it
streamed LIVE online.<br>
> Learn more at <a moz-do-not-send="true"
href="http://p.sf.net/sfu/ms-windowsazure"
target="_blank">http://p.sf.net/sfu/ms-windowsazure</a><br>
> _______________________________________________<br>
> Bitcoin-development mailing list<br>
> <a moz-do-not-send="true"
href="mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a><br>
> <a moz-do-not-send="true"
href="https://lists.sourceforge.net/lists/listinfo/bitcoin-development"
target="_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-development</a><br>
<br>
<br>
<br>
------------------------------------------------------------------------------<br>
Learn Windows Azure Live! Tuesday, Dec 13, 2011<br>
Microsoft is holding a special Learn Windows Azure
training event for<br>
developers. It will provide a great way to learn Windows
Azure and what it<br>
provides. You can attend the event by watching it streamed
LIVE online.<br>
Learn more at <a moz-do-not-send="true"
href="http://p.sf.net/sfu/ms-windowsazure"
target="_blank">http://p.sf.net/sfu/ms-windowsazure</a><br>
_______________________________________________<br>
Bitcoin-development mailing list<br>
<a moz-do-not-send="true"
href="mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a><br>
<a moz-do-not-send="true"
href="https://lists.sourceforge.net/lists/listinfo/bitcoin-development"
target="_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-development</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at <a class="moz-txt-link-freetext" href="http://p.sf.net/sfu/ms-windowsazure">http://p.sf.net/sfu/ms-windowsazure</a></pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Bitcoin-development mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a>
<a class="moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/bitcoin-development">https://lists.sourceforge.net/lists/listinfo/bitcoin-development</a>
</pre>
</blockquote>
<br>
</body>
</html>
--------------060705060006050107070606--
|