1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
|
Return-Path: <leo@LeoWandersleb.de>
Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136])
by lists.linuxfoundation.org (Postfix) with ESMTP id 2D42BC0001
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 28 Feb 2021 03:48:43 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by smtp3.osuosl.org (Postfix) with ESMTP id 1AC326F4B8
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 28 Feb 2021 03:48:43 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -0.148
X-Spam-Level:
X-Spam-Status: No, score=-0.148 tagged_above=-999 required=5
tests=[BAYES_40=-0.001, DKIM_INVALID=0.1, DKIM_SIGNED=0.1,
KHOP_HELO_FCRDNS=0.001, NICE_REPLY_A=-0.35, SPF_HELO_NONE=0.001,
SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=neutral
reason="invalid (public key: not available)" header.d=leowandersleb.de
Received: from smtp3.osuosl.org ([127.0.0.1])
by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id DgSr1-rxpfrE
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 28 Feb 2021 03:48:41 +0000 (UTC)
X-Greylist: delayed 00:07:27 by SQLgrey-1.8.0
Received: from geekbox.info (mx01.geekbox.info [95.216.118.16])
by smtp3.osuosl.org (Postfix) with ESMTPS id AE1CF6F4A1
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 28 Feb 2021 03:48:40 +0000 (UTC)
Received: from [192.168.0.6] (pc-180-175-104-200.cm.vtr.net [200.104.175.180])
(Authenticated sender: leo)
by geekbox.info (Postfix) with ESMTPSA id EB3CC7000E8
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 28 Feb 2021 03:41:10 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=leowandersleb.de;
s=mail; t=1614483671;
bh=siAMY4VuDYminTEQrvZyiTluxbJnsHdIV3mudl7uTqk=;
h=Subject:To:References:From:Date:In-Reply-To;
b=pf3Xal3lxu57EfFlM8p71uYTv9N9YRYL6D8zsZNpuvYjsQ/k2nfqKua2u+gJP6uG3
fZKW3Mvh+ejPrfh4UVJJqi6WdZTHJXKmT9nwGNA97Dt4+7Zr1cxbsaE6C/R7BouwM+
GCE10dvDVcjpPRKNktRKX3c+oAYjXNbhMv8rGKj8=
To: bitcoin-dev@lists.linuxfoundation.org
References: <CALeFGL1WSSA69ARvJW3di-UC_gz7NV9q7=6zd7s=CHnmttdQFg@mail.gmail.com>
<CAJx8jdz3uOCpwed3MZkf1ghkvaZMfy-+vvOCVZdvhz2KAn38DQ@mail.gmail.com>
From: Leo Wandersleb <leo@LeoWandersleb.de>
Autocrypt: addr=leo@LeoWandersleb.de; prefer-encrypt=mutual; keydata=
mQENBE3ePdoBCADZmTvPyledh9wmNRE+i5k7dUmegSmBpe1kMgzlt3nxwzyE6l3wzT9wen8E
clmA4ZCLn77DjxF3Dg9X0sRio9No5r4u6luI0CzM7axvTi+5DWU25b+JGGdhKTMkyfyKBZdT
4xvjLzW9tT7upUuAh9nTcz9MBZS3vhKB1SqiBUdYqj33+2erIQzaYMIZ/crR6FbgDT5dI3gg
2YKJn4rdFy9saA0qngoDPkyU9u5TYtFk8zOreQrFXmXpLiptoxRolF9tqHyn51H4uV8ggIGc
xJWRW/i2vBFHSM6ZD7PceWCRU5ehf3h/wat1xtn/H7AB+wcywkYrdg1TLifftwcKYBlzABEB
AAG0JUxlbyBXYW5kZXJzbGViIDxMZW9ATGVvV2FuZGVyc2xlYi5kZT6JATgEEwECACIFAk3e
PdoCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEGuaHwy3wggSrqQH/Amh6+kYnIUM
8X5yicpXd37h/Nqnrs6KIE0nYCZe03tkQytndbWXmf2pBth6CEzqrS+wivU7L8dPdiFQtdEW
dDe8bjJ993TsVSyYrPHv549vD4DbjtB8WS7KEkMhhl+uVaC07YXNwOTAkshv4l3G2ll6dmBn
LDeF8kSU+DZ1eNzUYHjuUcQYl0XAK5wE7rVpzZUlR0Z5bKNm8QmUPgrZPu6cVRZ6tOD0EMR6
3dcGI5x0fB35gEUasgNYxQ/g2arh66Pg45olKT1bTcrTJYrE8IePElSCIFDrzrZMhStY3TbQ
qI3T0vsFGzEeU9M6LmKV8aii4g8o7af0kr4vYS3Nsp65AQ0ETd492gEIAMnahwe+MNliWgpF
B+UfrwzfgiLjtizH307HYB67THS4h2+MoBJUAqrV7CQkZ8kMYLBl4D+hvxrXDYte6wTb0kcY
X4NTGof8EDBPxUERYJlWbTbLccbA8d3ia++aTtqd9yRHobK8AO4R22cslYV/nQChHc+1/NFV
8V1O67U7XpnxQMUzYIhhvVcUwj+8bw/F3zPzS03PWnDv+gmwHt63w72Dth+P0F93/rI+OiEA
kC/7//1ngQ7auqOeWVdqR8Z1s7tI4kxsVkJfqj3wVu+qaso8FpjNGs888xH8xhgXlll8APWj
V9Rl3GtpRTdobzNh58jUfGmHh5bcRrYTrtldDwkAEQEAAYkBHwQYAQIACQUCTd492gIbDAAK
CRBrmh8Mt8IIEi3MB/0UlpEORi7qBOH9egn/8E9fXj3QGpmQ7IAvzvLjkp3cArerBhObhGJ+
lZ679/9KZ4FxWLDStzGJH3thQstZjcUZzLwTP1sNdY+uPkvOqrzclfPK13z5hJhORD8W0brB
UptTvyZthIxNIbY4uSbrvaAJmcPFi3cdKOjZz9XyH8AAxXcWGUjQuqxEc0GuM8h8asaUJesB
bE4J+HPZXcAOC3D+qIqvM438Nbb8Pu7yYWrdq5kmbDMg2nPHXUOOovEsXWBGjWEiXWVUei75
5Fumqg5mz4P8/Ry3DwZBuHN3Op77gXogvyeE/B5O6dZzWWpSqBIS6MQcLD1divwi/2vNS4c4
Message-ID: <b895f2e4-513f-0c0d-91ac-52af055f332c@LeoWandersleb.de>
Date: Sun, 28 Feb 2021 00:41:06 -0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <CAJx8jdz3uOCpwed3MZkf1ghkvaZMfy-+vvOCVZdvhz2KAn38DQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Mailman-Approved-At: Sun, 28 Feb 2021 08:18:08 +0000
Subject: Re: [bitcoin-dev] A design for Probabilistic Partial Pruning
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sun, 28 Feb 2021 03:48:43 -0000
Only headers need to be downloaded sequentially so downloading relevant blocks
from one node is totally possible with gaps in between.
On 2/27/21 4:10 AM, Igor Cota via bitcoin-dev wrote:
> Hi Keagan,
>
> I had a very similar idea. The only difference being for the node to decide on
> a range of blocks to keep beforehand, rather than making the decision
> block-by-block like you suggest.
>
> I felt the other nodes would be better served by ranges due to the sequential
> nature of IBD. Perhaps this would be computationally lighter as well.
>
> I also encourage you to read Ryosuke Abe's paper [1] that proposes a DHT
> scheme to solve this same problem.
>
> Cheers,
> Igor
>
> [1] https://arxiv.org/abs/1902.02174
>
> On Fri, 26 Feb 2021 at 21:57, Keagan McClelland via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>
> Hi all,
>
> I've been thinking for quite some time about the problem of pruned nodes
> and ongoing storage costs for full nodes. One of the things that strikes
> me as odd is that we only really have two settings.
>
> A. Prune everything except the most recent blocks, down to the cache size
> B. Keep everything since genesis
>
> From my observations and conversations with various folks in the
> community, they would like to be able to run a "partially" pruned node to
> help bear the load of bootstrapping other nodes and helping with data
> redundancy in the network, but would prefer to not dedicate hundreds of
> Gigabytes of storage space to the cause.
>
> This led me to the idea that a node could randomly prune some of the
> blocks from history if it passed some predicate. A rough sketch of this
> would look as follows.
>
> 1. At node startup, it would generate a random seed, this would be unique
> to the node but not necessary that it be cryptographically secure.
> 2. In the node configuration it would also carry a "threshold" expressed
> as some percentage of blocks it wanted to keep.
> 3. As IBD occurs, based off of the threshold, the block hash, and the
> node's unique seed, the node would either decide to prune the data or keep
> it. The uniqueness of the node's hash should ensure that no block is
> systematically overrepresented in the set of nodes choosing this storage
> scheme.
> 4. Once the node's IBD is complete it would advertise this as a peer
> service, advertising its seed and threshold, so that nodes could
> deterministically deduce which of its peers had which blocks.
>
> The goals are to increase data redundancy in a way that more uniformly
> shares the load across nodes, alleviating some of the pressure of full
> archive nodes on the IBD problem. I am working on a draft BIP for this
> proposal but figured I would submit it as a high level idea in case anyone
> had any feedback on the initial design before I go into specification
> levels of detail.
>
> If you have thoughts on
>
> A. The protocol design itself
> B. The barriers to put this kind of functionality into Core
>
> I would love to hear from you,
>
> Cheers,
> Keagan
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
> --
> *Igor Cota*
> Codex Apertus d.o.o.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
|
