1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
|
Return-Path: <jim.posen@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 7D9CD147A
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 10 Jun 2018 23:07:23 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-qt0-f173.google.com (mail-qt0-f173.google.com
[209.85.216.173])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6CAAC762
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 10 Jun 2018 23:07:21 +0000 (UTC)
Received: by mail-qt0-f173.google.com with SMTP id y20-v6so18696666qto.8
for <bitcoin-dev@lists.linuxfoundation.org>;
Sun, 10 Jun 2018 16:07:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
bh=F4gyOt0UlKhKYKAMuiC1zcX7GtcCviMPsAvEsUL1c/k=;
b=BfDdks37QizRaLNO2O28lXcxMj558kkI5I2zyTsvspZV/AfIih/GTAESJXsKh2XK7K
OIv20F0j4Vy0BVaom+zC3H5XkShRaZ98jXlClYFrOQ4Ffyj8e65VxbVQllb3MsKaykKl
zFyHkrKTPSjoBJoa/mTPfBvkRtToTSNwEZh/HbUQKEcqLYiowMN5G3DJ0lcZ19BnlQZc
82tvviIAZnfFUCUJJz4cebwzXwRnABi2caU2I5OuTVJ1mGr7VkvvTfQmM35nLpZPBELs
Ya3STCu+mjMzQggIJSOsA3tOq3xPtqZhMCM3T+hUvjoL5C4zDYn66T9Jk4z7Y/uBtOdH
RNtQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to;
bh=F4gyOt0UlKhKYKAMuiC1zcX7GtcCviMPsAvEsUL1c/k=;
b=SFyDqcEP+ncJy+XllRwzN4CNMyScqzUsi3rXPJqRz0bpQ3KoO3jX6S3FLR+4q2IJdk
rlUCMQSFDjOaykBgHlgQ2yKqspIC2QrMo9g8GrUx3/+UOwSqMtOWUsUyUkdmCKqjl2U1
hJXg6xQVmlgPf8Voy7wW1Cik5zovfm7pZimBG57X/KA8OCoNJ8GhRCJ30m68kRnY5iNS
P7DKAtVMQa8le4ePMaPxN+lnJBqcDxBnMXhEzwFYV94IscPHKnkHv7uEwI31Oo+Ekgcc
/cKz3V/SzLPvfW9cMjSEe0Bv5kuLmHvibp/adqVymgVthZHvf1ChryGimB78hkTdDtQ+
jcMQ==
X-Gm-Message-State: APt69E3ZcwXLyzwlfY/NZfZhids8vRG4fx1gLbaiEb1bulgJjWT3uGrb
gmw8jdaM9x3LM6/tSPZp0Pgg6EWZOks+4xaOwnBwdw==
X-Google-Smtp-Source: ADUXVKI23umF2WChjKsQUWTNnW5VQ5pRVYTrhekQB5c/TuexApYFFuDVAw4mfue5sm+6hvhi8G8JwtWynGAtQIAkG+U=
X-Received: by 2002:ac8:2836:: with SMTP id
51-v6mr14332101qtq.131.1528672040252;
Sun, 10 Jun 2018 16:07:20 -0700 (PDT)
MIME-Version: 1.0
References: <CAApLimjfPKDxmiy_SHjuOKbfm6HumFPjc9EFKvw=3NwZO8JcmQ@mail.gmail.com>
<CAAS2fgTHTK8Dve9xHW9yULa1yObWtmwmeKKcD_BMjON=RAw8Sg@mail.gmail.com>
<E5BD6DC6-281B-46E5-ABD3-71B2D5549902@sprovoost.nl>
In-Reply-To: <E5BD6DC6-281B-46E5-ABD3-71B2D5549902@sprovoost.nl>
From: Jim Posen <jim.posen@gmail.com>
Date: Sun, 10 Jun 2018 16:07:07 -0700
Message-ID: <CADZtCSiPkah3sX5yw_29mcnFxFm_SrgzFxOuaQwffMHhRDaobg@mail.gmail.com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000f00327056e51b1ee"
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
X-Mailman-Approved-At: Sun, 10 Jun 2018 23:15:36 +0000
Subject: Re: [bitcoin-dev] UHS: Full-node security without maintaining a
full UTXO set
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jun 2018 23:07:23 -0000
--000000000000f00327056e51b1ee
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I generally like the direction of this proposal in terms of allowing full
nodes to run with a different storage/bandwidth tradeoff. Cory, were this
implemented, would you expect Core to support both operating modes (full
UTXO set and UHS) depending on user configuration, or would UHS be
mandatory?
Also, given that Bram Cohen's TXO bitfield proposal was an inspiration for
this, could you comment on why the UHS is preferable to that approach? An
alternative that goes even further in the direction of more bandwidth, less
storage, would be for nodes to simply maintain a Merkle Mountain Range over
all TXOs in order of creation and a spentness bitfield. Blocks could be
requested with the prev outputs and a Merkle proof linking them into the
MMR root. Since the Merkle proof is deterministic, it could be computed by
archive nodes and miners and saved alongside the block data for relay.
Another benefit of this is the TXO MMR root may be independently useful if
committed into the coinbase transaction.
On Thu, Jun 7, 2018 at 7:02 AM Sjors Provoost via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> eMMC storage, which low end devices often use, come in 2x increments.
> Running a pruned full node on 8 GB is difficult if not impossible (the UT=
XO
> set peaked at 3.5 GB in January, but a full node stores additional stuff)=
.
>
> However, 16 GB is only =E2=82=AC10 more expensive and presumably standard=
by the
> time this would be rolled out.
>
> On AWS every GB of SSD storage avoided saves $1 per year, not end of the
> world stuff, but not negligible either. Outbound traffic costs $0.10 / GB
> (ignoring free allowance), so when uploading 200 GB per year, the 5% woul=
d
> offset $1 of storage cost savings.
>
> The above seems marginal, probably not worth it unless there=E2=80=99s re=
ally no
> downside.
>
> What I find attractive about this proposal is the ability to squeeze more
> out of limited RAM (typically only 1 or 2 GB on these low end devices). I=
=E2=80=99d
> have to test Cory=E2=80=99s branch to see if that actually matters in pra=
ctice.
>
> It=E2=80=99s also useful to distinguish benefits during initial sync from=
ongoing
> operation. The former I=E2=80=99ve almost given up on for low end device=
s (can
> take weeks), in favor of doing it on a faster computer and copying the
> result. The latter needs far less RAM, so perhaps this proposal doesn=E2=
=80=99t
> help much there, but that would be useful to measure.
>
> Did you try the recent SHA256 optimizations on your branch?
>
> Sjors
>
> > Op 17 mei 2018, om 18:56 heeft Gregory Maxwell via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> het volgende geschreven:
> >
> > On Wed, May 16, 2018 at 4:36 PM, Cory Fields via bitcoin-dev
> > <bitcoin-dev@lists.linuxfoundation.org> wrote:
> >> Tl;dr: Rather than storing all unspent outputs, store their hashes.
> >
> > My initial thoughts are it's not _completely_ obvious to me that a 5%
> > ongoing bandwidth increase is actually a win to get something like a
> > 40% reduction in the size of a pruned node (and less than a 1%
> > reduction in an archive node) primarily because I've not seen size of
> > a pruned node cited as a usage limiting factor basically anywhere. I
> > would assume it is a win but wouldn't be shocked to see a careful
> > analysis that concluded it wasn't.
> >
> > But perhaps more interestingly, I think the overhead is not really 5%,
> > but it's 5% measured in the context of the phenomenally inefficient tx
> > mechanisms ( https://bitcointalk.org/index.php?topic=3D1377345.0 ).
> > Napkin math on the size of a txn alone tells me it's more like a 25%
> > increase if you just consider size of tx vs size of
> > tx+scriptpubkeys,amounts. If I'm not missing something there, I think
> > that would get in into a very clear not-win range.
> >
> > On the positive side is that it doesn't change the blockchain
> > datastructure, so it's something implementations could do without
> > marrying the network to it forever.
> >
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
--000000000000f00327056e51b1ee
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I generally like the direction of this proposal in terms o=
f allowing full nodes to run with a different storage/bandwidth tradeoff. C=
ory, were this implemented, would you expect Core to support both operating=
modes (full UTXO set and UHS) depending on user configuration, or would UH=
S be mandatory?<div><br></div><div>Also, given that Bram Cohen's TXO bi=
tfield proposal was an inspiration for this, could you comment on why the U=
HS is preferable to that approach? An alternative that goes even further in=
the direction of more bandwidth, less storage, would be for nodes to simpl=
y maintain a Merkle Mountain Range over all TXOs in order of creation and a=
spentness bitfield. Blocks could be requested with the prev outputs and a =
Merkle proof linking them into the MMR root. Since the Merkle proof is dete=
rministic, it could be computed by archive nodes and miners and saved along=
side the block data for relay. Another benefit of this is the TXO MMR root =
may be independently useful if committed into the coinbase transaction.</di=
v></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Thu, Jun 7, 2018=
at 7:02 AM Sjors Provoost via bitcoin-dev <<a href=3D"mailto:bitcoin-de=
v@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfoundation.org</a>> =
wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex">eMMC storage, which low end =
devices often use, come in 2x increments. Running a pruned full node on 8 G=
B is difficult if not impossible (the UTXO set peaked at 3.5 GB in January,=
but a full node stores additional stuff).<br>
<br>
However, 16 GB is only =E2=82=AC10 more expensive and presumably standard b=
y the time this would be rolled out.<br>
<br>
On AWS every GB of SSD storage avoided saves $1 per year, not end of the wo=
rld stuff, but not negligible either. Outbound traffic costs $0.10 / GB (ig=
noring free allowance), so when uploading 200 GB per year, the 5% would off=
set $1 of storage cost savings.<br>
<br>
The above seems marginal, probably not worth it unless there=E2=80=99s real=
ly no downside.<br>
<br>
What I find attractive about this proposal is the ability to squeeze more o=
ut of limited RAM (typically only 1 or 2 GB on these low end devices). I=E2=
=80=99d have to test Cory=E2=80=99s branch to see if that actually matters =
in practice.<br>
<br>
It=E2=80=99s also useful to distinguish benefits during initial sync from o=
ngoing operation. The former I=E2=80=99ve almost given up on for=C2=A0 low =
end devices (can take weeks), in favor of doing it on a faster computer and=
copying the result. The latter needs far less RAM, so perhaps this proposa=
l doesn=E2=80=99t help much there, but that would be useful to measure.<br>
<br>
Did you try the recent SHA256 optimizations on your branch?<br>
<br>
Sjors<br>
<br>
> Op 17 mei 2018, om 18:56 heeft Gregory Maxwell via bitcoin-dev <<a =
href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bit=
coin-dev@lists.linuxfoundation.org</a>> het volgende geschreven:<br>
> <br>
> On Wed, May 16, 2018 at 4:36 PM, Cory Fields via bitcoin-dev<br>
> <<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D=
"_blank">bitcoin-dev@lists.linuxfoundation.org</a>> wrote:<br>
>> Tl;dr: Rather than storing all unspent outputs, store their hashes=
.<br>
> <br>
> My initial thoughts are it's not _completely_ obvious to me that a=
5%<br>
> ongoing bandwidth increase is actually a win to get something like a<b=
r>
> 40% reduction in the size of a pruned node (and less than a 1%<br>
> reduction in an archive node) primarily because I've not seen size=
of<br>
> a pruned node cited as a usage limiting factor basically anywhere. I<b=
r>
> would assume it is a win but wouldn't be shocked to see a careful<=
br>
> analysis that concluded it wasn't.<br>
> <br>
> But perhaps more interestingly, I think the overhead is not really 5%,=
<br>
> but it's 5% measured in the context of the phenomenally inefficien=
t tx<br>
> mechanisms ( <a href=3D"https://bitcointalk.org/index.php?topic=3D1377=
345.0" rel=3D"noreferrer" target=3D"_blank">https://bitcointalk.org/index.p=
hp?topic=3D1377345.0</a> ).<br>
> Napkin math on the size of a txn alone tells me it's more like a 2=
5%<br>
> increase if you just consider size of tx vs size of<br>
> tx+scriptpubkeys,amounts.=C2=A0 If I'm not missing something there=
, I think<br>
> that would get in into a very clear not-win range.<br>
> <br>
> On the positive side is that it doesn't change the blockchain<br>
> datastructure, so it's something implementations could do without<=
br>
> marrying the network to it forever.<br>
> <br>
<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div>
--000000000000f00327056e51b1ee--
|
