summaryrefslogtreecommitdiff
path: root/60/9bfe0d25d9f867f1555979818c69d2b2fdd48b
blob: 560f9d5e1dbacbc4936619d43fc8c3d0c469e49f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
Return-Path: <vitteaymeric@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id B974798C
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 20 Apr 2017 23:42:07 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-wm0-f53.google.com (mail-wm0-f53.google.com [74.125.82.53])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A8DB6140
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 20 Apr 2017 23:42:05 +0000 (UTC)
Received: by mail-wm0-f53.google.com with SMTP id o81so4335234wmb.1
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 20 Apr 2017 16:42:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=subject:to:references:from:message-id:date:user-agent:mime-version
	:in-reply-to; bh=fAdYePRRNWVsGnnZTrmMk6oBkn0aZilnY4hXBsFPkUI=;
	b=EZBw8mhlo2D8rscwFfZfM1zq16gX7XvwDb2Wb8mZp8ZDSPA41kgLd4JwPDGmc6ueMK
	Dv3LqxsB3uduyMOvO6NNzeQJfIODvjwHS1MFHfyRAh5yk+UeAogjQR45AuJ+3Jvz9Uqy
	dR5/AYf+Vv8waX66qA2PhexBSgoJRL10JAweBUxQPwm+gGTi9jv0DLd1dQElb/UBHSS8
	WvQ9kDibUmFIKs9ODrSZNAg/hPAnc+0gpXFnSK0qblszeddRE1Q4i5Na4w5kHC71mAAx
	5t5dC8C42NioAq4AYnmkvtIPh7Jx6CrSUBmzcDxi4CJNZdhxf5B2b6TlwKzlT99UkSTj
	ToYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to;
	bh=fAdYePRRNWVsGnnZTrmMk6oBkn0aZilnY4hXBsFPkUI=;
	b=NiUXhgZrChaji7EE0vO1qEcbrV2HOykO/9pWydIUSe9IZy0kQy1pkGjTjEVd24Ntv4
	bjK3keIPJSpZIYd+CcNv/8Aw5IojIOoQXpEZa7T7J6J4ckQIH6Cs3RHVThyYpg86vwO6
	fKH+aUuQ/WOIvHXWalg0w8EV+VRKppSaErTuX4v75KrHDkaKj9Uib45KNlgYbK6tLaS+
	xM/x3zd37FbVbo3rcza2QurAJQFFQNaEWKRYuhzdYrOdfMZJova+EVU9ANcbGZeHrwH5
	yLxl/6rLYSBdkxva/bdbZ9psbYEHGViT8sY0yMzBHDeeBSG5oCjrmIWMPy33VUXnppXj
	Mowg==
X-Gm-Message-State: AN3rC/7VID002+AMXAkNCTN9MDV8fPjU9ClYRBqhiXzPzZ8OAE/Y6564
	2s5FneLHpDVJMGMs
X-Received: by 10.28.56.1 with SMTP id f1mr5854761wma.20.1492731724093;
	Thu, 20 Apr 2017 16:42:04 -0700 (PDT)
Received: from [192.168.1.10] (ANice-654-1-153-20.w86-205.abo.wanadoo.fr.
	[86.205.80.20]) by smtp.googlemail.com with ESMTPSA id
	v7sm4016824wrb.68.2017.04.20.16.42.02
	for <bitcoin-dev@lists.linuxfoundation.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Thu, 20 Apr 2017 16:42:03 -0700 (PDT)
To: bitcoin-dev@lists.linuxfoundation.org
References: <CAFVRnypbQQ-vsSLqv48cYaqTCty4R1DmFRqfAvxe4mAqyQNXxQ@mail.gmail.com>
	<CAJN5wHW=p+q+DT9R=uheLxOjKBX=xcB+fOZR2KACgJO9SdXypw@mail.gmail.com>
	<CAJowKg++8GD3gE15pdwe0Bj-L0A6MAzG0_uTSLASaRT9yVb1aQ@mail.gmail.com>
From: Aymeric Vitte <vitteaymeric@gmail.com>
Message-ID: <eab58efa-7fbb-6b3b-8982-17f4d83b594b@gmail.com>
Date: Fri, 21 Apr 2017 01:42:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 6.3; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <CAJowKg++8GD3gE15pdwe0Bj-L0A6MAzG0_uTSLASaRT9yVb1aQ@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary="------------AA5ADC5A396E498A58089402"
X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
	RCVD_IN_DNSWL_NONE, 
	RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] Small Nodes: A Better Alternative to Pruned Nodes
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Apr 2017 23:42:07 -0000

This is a multi-part message in MIME format.
--------------AA5ADC5A396E498A58089402
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit

??? what do you mean? (https://www.soyoustart.com/fr/serveurs-essential/)


Le 20/04/2017 à 17:50, Erik Aronesty via bitcoin-dev a écrit :
> Try to find 1TB dedicated server hosting ...
>
> If you want to set up an ecommerce site somewhere besides your living
> room, storage costs are still a concern.
>
> On Mon, Apr 17, 2017 at 3:11 AM, Danny Thorpe via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>
>     1TB HDD is now available for under $40 USD.  How is the 100GB
>     storage requirement preventing anyone from setting up full nodes?
>
>     On Apr 16, 2017 11:55 PM, "David Vorick via bitcoin-dev"
>     <bitcoin-dev@lists.linuxfoundation.org
>     <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>
>         *Rationale:*
>
>         A node that stores the full blockchain (I will use the term
>         archival node) requires over 100GB of disk space, which I
>         believe is one of the most significant barriers to more people
>         running full nodes. And I believe the ecosystem would benefit
>         substantially if more users were running full nodes.
>
>         The best alternative today to storing the full blockchain is
>         to run a pruned node, which keeps only the UTXO set and throws
>         away already verified blocks. The operator of the pruned node
>         is able to enjoy the full security benefits of a full node,
>         but is essentially leeching the network, as they performed a
>         large download likely without contributing anything back.
>
>         This puts more pressure on the archival nodes, as the archival
>         nodes need to pick up the slack and help new nodes bootstrap
>         to the network. As the pressure on archival nodes grows, fewer
>         people will be able to actually run archival nodes, and the
>         situation will degrade. The situation would likely become
>         problematic quickly if bitcoin-core were to ship with the
>         defaults set to a pruned node.
>
>         Even further, the people most likely to care about saving
>         100GB of disk space are also the people least likely to care
>         about some extra bandwidth usage. For datacenter nodes, and
>         for nodes doing lots of bandwidth, the bandwidth is usually
>         the biggest cost of running the node. For home users however,
>         as long as they stay under their bandwidth cap, the bandwidth
>         is actually free. Ideally, new nodes would be able to
>         bootstrap from nodes that do not have to pay for their
>         bandwidth, instead of needing to rely on a decreasing
>         percentage of heavy-duty archival nodes.
>
>         I have (perhaps incorrectly) identified disk space consumption
>         as the most significant factor in your average user choosing
>         to run a pruned node or a lite client instead of a full node.
>         The average user is not typically too worried about bandwidth,
>         and is also not typically too worried about initial blockchain
>         download time. But the 100GB hit to your disk space can be a
>         huge psychological factor, especially if your hard drive only
>         has 500GB available in the first place, and 250+ GB is already
>         consumed by other files you have.
>
>         I believe that improving the disk usage situation would
>         greatly benefit decentralization, especially if it could be
>         done without putting pressure on archival nodes.
>
>         *Small Nodes Proposal:*
>
>         I propose an alternative to the pruned node that does not put
>         undue pressure on archival nodes, and would be acceptable and
>         non-risky to ship as a default in bitcoin-core. For lack of a
>         better name, I'll call this new type of node a 'small node'.
>         The intention is that bitcoin-core would eventually ship
>         'small nodes' by default, such that the expected amount of
>         disk consumption drops from today's 100+ GB to less than 30 GB.
>
>         My alternative proposal has the following properties:
>
>         + Full nodes only need to store ~20% of the blockchain
>         + With very high probability, a new node will be able to
>         recover the entire blockchain by connecting to 6 random small
>         node peers.
>         + An attacker that can eliminate a chosen+ 95% of the full
>         nodes running today will be unable to prevent new nodes from
>         downloading the full blockchain, even if the attacker is also
>         able to eliminate all archival nodes. (assuming all nodes
>         today were small nodes instead of archival nodes)
>
>         Method:
>
>         A small node will pick an index [5, 256). This index is that
>         node's permanent index. When storing a block, instead of
>         storing the full block, the node will use Reed-Solomon coding
>         to erasure code the block using a 5-of-256 scheme. The result
>         will be 256 pieces that are 20% of the size of the block each.
>         The node picks the piece that corresponds to its index, and
>         stores that instead. (Indexes 0-4 are reserved for archival
>         nodes - explained later)
>
>         The node is now storing a fragment of every block. Alone, this
>         fragment cannot be used to recover any piece of the
>         blockchain. However, when paired with any 5 unique fragments
>         (fragments of the same index will not be unique), the full
>         block can be recovered.
>
>         Nodes can optionally store more than 1 fragment each. At 5
>         fragments, the node becomes a full archival node, and the
>         chosen indexes should be 0-4. This is advantageous for the
>         archival node as the encoded data for the first 5 indexes will
>         actually be identical to the block itself - there is no
>         computational overhead for selecting the first indexes. There
>         is also no need to choose random indexes, because the full
>         block can be recovered no matter which indexes are chosen.
>
>         When connecting to new peers, the indexes of each peer needs
>         to be known. Once peers totaling 5 unique indexes are
>         discovered, blockchain download can begin. Connecting to just
>         5 small node peers provides a >95% chance of getting 5
>         uniques, with exponentially improving odds of success as you
>         connect to more peers. Connecting to a single archive node
>         guarantees that any gaps can be filled.
>
>         A good encoder should be able to turn a block into a 5-of-256
>         piece set in under 10 milliseconds using a single core on a
>         standard consumer desktop. This should not slow down initial
>         blockchain download substantially, though the overhead is more
>         than a rounding error.
>
>         *DoS Prevention:*
>
>         A malicious node may provide garbage data instead of the
>         actual piece. Given just the garbage data and 4 other correct
>         pieces, it is impossible (best I know anyway) to tell which
>         piece is the garbage piece.
>
>         One option in this case would be to seek out an archival node
>         that could verify the correctness of the pieces, and identify
>         the malicious node.
>
>         Another option would be to have the small nodes store a
>         cryptographic checksum of each piece. Obtaining the
>         cryptographic checksum for all 256 pieces would incur a
>         nontrivial amount of hashing (post segwit, as much as 100MB of
>         extra hashing per block), and would require an additional ~4kb
>         of storage per block. The hashing overhead here may be
>         prohibitive.
>
>         Another solution would be to find additional pieces and
>         brute-force combinations of 5 until a working combination was
>         discovered. Though this sounds nasty, it should take less than
>         five seconds of computation to find the working combination
>         given 5 correct pieces and 2 incorrect pieces. This
>         computation only needs to be performed once to identify the
>         malicious peers.
>
>         I also believe that alternative erasure coding schemes exist
>         which actually are able to identify the bad pieces given
>         sufficient good pieces, however I don't know if they have the
>         same computational performance as the best Reed-Solomon coding
>         implementations.
>
>         *Deployment:*
>
>         Small nodes are completely useless unless the critical mass of
>         5 pieces can be obtained. The first version that supports
>         small node block downloads should default everyone to an
>         archival node (meaning indexes 0-4 are used)
>
>         Once there are enough small-node-enabled archive nodes, the
>         default can be switched so that nodes only have a single index
>         by default. In the first few days, when there are only a few
>         small nodes, the previously-deployed archival nodes can help
>         fill in the gaps, and the small nodes can be useful for
>         blockchain download right away.
>
>         ----------------------------------
>
>         This represents a non-trivial amount of code, but I believe
>         that the result would be a non-trivial increase in the
>         percentage of users running full nodes, and a healthier
>         overall network.
>
>         _______________________________________________
>         bitcoin-dev mailing list
>         bitcoin-dev@lists.linuxfoundation.org
>         <mailto:bitcoin-dev@lists.linuxfoundation.org>
>         https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>         <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>
>
>
>     _______________________________________________
>     bitcoin-dev mailing list
>     bitcoin-dev@lists.linuxfoundation.org
>     <mailto:bitcoin-dev@lists.linuxfoundation.org>
>     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms


--------------AA5ADC5A396E498A58089402
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p>??? what do you mean?
      (<a class="moz-txt-link-freetext" href="https://www.soyoustart.com/fr/serveurs-essential/">https://www.soyoustart.com/fr/serveurs-essential/</a>)<br>
    </p>
    <br>
    <div class="moz-cite-prefix">Le 20/04/2017 à 17:50, Erik Aronesty
      via bitcoin-dev a écrit :<br>
    </div>
    <blockquote
cite="mid:CAJowKg++8GD3gE15pdwe0Bj-L0A6MAzG0_uTSLASaRT9yVb1aQ@mail.gmail.com"
      type="cite">
      <div dir="ltr">Try to find 1TB dedicated server hosting ... <br>
        <br>
        If you want to set up an ecommerce site somewhere besides your
        living room, storage costs are still a concern.<br>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Mon, Apr 17, 2017 at 3:11 AM, Danny
          Thorpe via bitcoin-dev <span dir="ltr">&lt;<a
              moz-do-not-send="true"
              href="mailto:bitcoin-dev@lists.linuxfoundation.org"
              target="_blank">bitcoin-dev@lists.linuxfoundation.org</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="auto">1TB HDD is now available for under <span
                class="m_2752087275107902543money">$40</span> USD.  How
              is the 100GB storage requirement preventing anyone from
              setting up full nodes?</div>
            <div class="gmail_extra"><br>
              <div class="gmail_quote">
                <div>
                  <div class="h5">On Apr 16, 2017 11:55 PM, "David
                    Vorick via bitcoin-dev" &lt;<a
                      moz-do-not-send="true"
                      href="mailto:bitcoin-dev@lists.linuxfoundation.org"
                      target="_blank">bitcoin-dev@lists.<wbr>linuxfoundation.org</a>&gt;
                    wrote:<br type="attribution">
                  </div>
                </div>
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div>
                    <div class="h5">
                      <div dir="ltr">
                        <div>
                          <div><b>Rationale:</b><br>
                          </div>
                          <div><br>
                            A node that stores the full blockchain (I
                            will use the term archival node) requires
                            over 100GB of disk space, which I believe is
                            one of the most significant barriers to more
                            people running full nodes. And I believe the
                            ecosystem would benefit substantially if
                            more users were running full nodes.<br>
                            <br>
                          </div>
                          The best alternative today to storing the full
                          blockchain is to run a pruned node, which
                          keeps only the UTXO set and throws away
                          already verified blocks. The operator of the
                          pruned node is able to enjoy the full security
                          benefits of a full node, but is essentially
                          leeching the network, as they performed a
                          large download likely without contributing
                          anything back.<br>
                          <br>
                        </div>
                        <div>This puts more pressure on the archival
                          nodes, as the archival nodes need to pick up
                          the slack and help new nodes bootstrap to the
                          network. As the pressure on archival nodes
                          grows, fewer people will be able to actually
                          run archival nodes, and the situation will
                          degrade. The situation would likely become
                          problematic quickly if bitcoin-core were to
                          ship with the defaults set to a pruned node.<br>
                          <br>
                        </div>
                        <div>Even further, the people most likely to
                          care about saving 100GB of disk space are also
                          the people least likely to care about some
                          extra bandwidth usage. For datacenter nodes,
                          and for nodes doing lots of bandwidth, the
                          bandwidth is usually the biggest cost of
                          running the node. For home users however, as
                          long as they stay under their bandwidth cap,
                          the bandwidth is actually free. Ideally, new
                          nodes would be able to bootstrap from nodes
                          that do not have to pay for their bandwidth,
                          instead of needing to rely on a decreasing
                          percentage of heavy-duty archival nodes.<br>
                          <br>
                        </div>
                        <div>I have (perhaps incorrectly) identified
                          disk space consumption as the most significant
                          factor in your average user choosing to run a
                          pruned node or a lite client instead of a full
                          node. The average user is not typically too
                          worried about bandwidth, and is also not
                          typically too worried about initial blockchain
                          download time. But the 100GB hit to your disk
                          space can be a huge psychological factor,
                          especially if your hard drive only has 500GB
                          available in the first place, and 250+ GB is
                          already consumed by other files you have.<br>
                          <br>
                        </div>
                        <div>I believe that improving the disk usage
                          situation would greatly benefit
                          decentralization, especially if it could be
                          done without putting pressure on archival
                          nodes.<br>
                        </div>
                        <div><br>
                        </div>
                        <div><b>Small Nodes Proposal:</b><br>
                          <br>
                        </div>
                        <div>I propose an alternative to the pruned node
                          that does not put undue pressure on archival
                          nodes, and would be acceptable and non-risky
                          to ship as a default in bitcoin-core. For lack
                          of a better name, I'll call this new type of
                          node a 'small node'. The intention is that
                          bitcoin-core would eventually ship 'small
                          nodes' by default, such that the expected
                          amount of disk consumption drops from today's
                          100+ GB to less than 30 GB.<br>
                          <br>
                        </div>
                        <div>My alternative proposal has the following
                          properties:<br>
                          <br>
                        </div>
                        <div>+ Full nodes only need to store ~20% of the
                          blockchain<br>
                        </div>
                        <div>+ With very high probability, a new node
                          will be able to recover the entire blockchain
                          by connecting to 6 random small node peers.<br>
                        </div>
                        <div>+ An attacker that can eliminate a chosen+
                          95% of the full nodes running today will be
                          unable to prevent new nodes from downloading
                          the full blockchain, even if the attacker is
                          also able to eliminate all archival nodes.
                          (assuming all nodes today were small nodes
                          instead of archival nodes)<br>
                          <br>
                        </div>
                        <div>Method:<br>
                          <br>
                        </div>
                        <div>A small node will pick an index [5, 256).
                          This index is that node's permanent index.
                          When storing a block, instead of storing the
                          full block, the node will use Reed-Solomon
                          coding to erasure code the block using a
                          5-of-256 scheme. The result will be 256 pieces
                          that are 20% of the size of the block each.
                          The node picks the piece that corresponds to
                          its index, and stores that instead. (Indexes
                          0-4 are reserved for archival nodes -
                          explained later)<br>
                          <br>
                        </div>
                        <div>The node is now storing a fragment of every
                          block. Alone, this fragment cannot be used to
                          recover any piece of the blockchain. However,
                          when paired with any 5 unique fragments
                          (fragments of the same index will not be
                          unique), the full block can be recovered.<br>
                          <br>
                        </div>
                        <div>Nodes can optionally store more than 1
                          fragment each. At 5 fragments, the node
                          becomes a full archival node, and the chosen
                          indexes should be 0-4. This is advantageous
                          for the archival node as the encoded data for
                          the first 5 indexes will actually be identical
                          to the block itself - there is no
                          computational overhead for selecting the first
                          indexes. There is also no need to choose
                          random indexes, because the full block can be
                          recovered no matter which indexes are chosen.<br>
                          <br>
                        </div>
                        <div>When connecting to new peers, the indexes
                          of each peer needs to be known. Once peers
                          totaling 5 unique indexes are discovered,
                          blockchain download can begin. Connecting to
                          just 5 small node peers provides a &gt;95%
                          chance of getting 5 uniques, with
                          exponentially improving odds of success as you
                          connect to more peers. Connecting to a single
                          archive node guarantees that any gaps can be
                          filled.<br>
                          <br>
                        </div>
                        <div>A good encoder should be able to turn a
                          block into a 5-of-256 piece set in under 10
                          milliseconds using a single core on a standard
                          consumer desktop. This should not slow down
                          initial blockchain download substantially,
                          though the overhead is more than a rounding
                          error.<br>
                        </div>
                        <div><br>
                        </div>
                        <div><b>DoS Prevention:</b><br>
                          <br>
                        </div>
                        <div>A malicious node may provide garbage data
                          instead of the actual piece. Given just the
                          garbage data and 4 other correct pieces, it is
                          impossible (best I know anyway) to tell which
                          piece is the garbage piece.<br>
                          <br>
                        </div>
                        <div>One option in this case would be to seek
                          out an archival node that could verify the
                          correctness of the pieces, and identify the
                          malicious node.<br>
                          <br>
                        </div>
                        <div>Another option would be to have the small
                          nodes store a cryptographic checksum of each
                          piece. Obtaining the cryptographic checksum
                          for all 256 pieces would incur a nontrivial
                          amount of hashing (post segwit, as much as
                          100MB of extra hashing per block), and would
                          require an additional ~4kb of storage per
                          block. The hashing overhead here may be
                          prohibitive.<br>
                          <br>
                        </div>
                        <div>Another solution would be to find
                          additional pieces and brute-force combinations
                          of 5 until a working combination was
                          discovered. Though this sounds nasty, it
                          should take less than five seconds of
                          computation to find the working combination
                          given 5 correct pieces and 2 incorrect pieces.
                          This computation only needs to be performed
                          once to identify the malicious peers.<br>
                          <br>
                        </div>
                        <div>I also believe that alternative erasure
                          coding schemes exist which actually are able
                          to identify the bad pieces given sufficient
                          good pieces, however I don't know if they have
                          the same computational performance as the best
                          Reed-Solomon coding implementations.<br>
                        </div>
                        <br>
                        <div><b>Deployment:</b><br>
                          <br>
                        </div>
                        <div>Small nodes are completely useless unless
                          the critical mass of 5 pieces can be obtained.
                          The first version that supports small node
                          block downloads should default everyone to an
                          archival node (meaning indexes 0-4 are used)<br>
                          <br>
                        </div>
                        <div>Once there are enough small-node-enabled
                          archive nodes, the default can be switched so
                          that nodes only have a single index by
                          default. In the first few days, when there are
                          only a few small nodes, the
                          previously-deployed archival nodes can help
                          fill in the gaps, and the small nodes can be
                          useful for blockchain download right away.<br>
                          <br>
                          ------------------------------<wbr>----<br>
                          <br>
                        </div>
                        <div>This represents a non-trivial amount of
                          code, but I believe that the result would be a
                          non-trivial increase in the percentage of
                          users running full nodes, and a healthier
                          overall network.<br>
                        </div>
                      </div>
                      <br>
                    </div>
                  </div>
                  ______________________________<wbr>_________________<br>
                  bitcoin-dev mailing list<br>
                  <a moz-do-not-send="true"
                    href="mailto:bitcoin-dev@lists.linuxfoundation.org"
                    target="_blank">bitcoin-dev@lists.linuxfoundat<wbr>ion.org</a><br>
                  <a moz-do-not-send="true"
                    href="https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev"
                    rel="noreferrer" target="_blank">https://lists.linuxfoundation.<wbr>org/mailman/listinfo/bitcoin-d<wbr>ev</a><br>
                  <br>
                </blockquote>
              </div>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            bitcoin-dev mailing list<br>
            <a moz-do-not-send="true"
              href="mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.<wbr>linuxfoundation.org</a><br>
            <a moz-do-not-send="true"
              href="https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev"
              rel="noreferrer" target="_blank">https://lists.linuxfoundation.<wbr>org/mailman/listinfo/bitcoin-<wbr>dev</a><br>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
bitcoin-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfoundation.org</a>
<a class="moz-txt-link-freetext" href="https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev">https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev</a>
</pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Zcash wallets made simple: <a class="moz-txt-link-freetext" href="https://github.com/Ayms/zcash-wallets">https://github.com/Ayms/zcash-wallets</a>
Bitcoin wallets made simple: <a class="moz-txt-link-freetext" href="https://github.com/Ayms/bitcoin-wallets">https://github.com/Ayms/bitcoin-wallets</a>
Get the torrent dynamic blocklist: <a class="moz-txt-link-freetext" href="http://peersm.com/getblocklist">http://peersm.com/getblocklist</a>
Check the 10 M passwords list: <a class="moz-txt-link-freetext" href="http://peersm.com/findmyass">http://peersm.com/findmyass</a>
Anti-spies and private torrents, dynamic blocklist: <a class="moz-txt-link-freetext" href="http://torrent-live.org">http://torrent-live.org</a>
Peersm : <a class="moz-txt-link-freetext" href="http://www.peersm.com">http://www.peersm.com</a>
torrent-live: <a class="moz-txt-link-freetext" href="https://github.com/Ayms/torrent-live">https://github.com/Ayms/torrent-live</a>
node-Tor : <a class="moz-txt-link-freetext" href="https://www.github.com/Ayms/node-Tor">https://www.github.com/Ayms/node-Tor</a>
GitHub : <a class="moz-txt-link-freetext" href="https://www.github.com/Ayms">https://www.github.com/Ayms</a></pre>
  </body>
</html>

--------------AA5ADC5A396E498A58089402--