1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
|
Return-Path: <rx@awsomnet.org>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id F2DEF957
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 3 Jan 2017 23:06:27 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f175.google.com (mail-io0-f175.google.com
[209.85.223.175])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 33197171
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 3 Jan 2017 23:06:27 +0000 (UTC)
Received: by mail-io0-f175.google.com with SMTP id h133so205242727ioe.3
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 03 Jan 2017 15:06:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=awsomnet-org.20150623.gappssmtp.com; s=20150623;
h=mime-version:in-reply-to:references:from:date:message-id:subject:to
:cc; bh=YSlA875jsHyKdX+3QkkfsaK+IJ3iJmoTMlqb3esLTR8=;
b=v8A874wuDXbD0iPTXz9HGKtRGdOsM8yLKNqQ62msYfA07Uuh37/Qd07cYeXWulR2Kw
2Ai0/sg5hCEQOPhZJ3IkWDp82I0gwoqF9B33RiOPkPEWDLsWYK0VgIcaLdkwZx9rW9GF
xzxLeFdLzvEUvq9QYYjYiASB1U1Y6OTg6tMsIlrSry3oquVJspTYHWKpj0hR89JCNhQU
Di+ke0JUAJBHxX4j2STvl4Dfqn3JrBvAtvWZ5Wg4uHwhrumtpW0ISAIbx6lqugoPTErz
4XlMqno+Zcn1T8N+QzsMCy/mSzvgU8WbYSHuinlV8fc3pAni43qeOofHX3ykkeDYJ58n
hqcg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to:cc;
bh=YSlA875jsHyKdX+3QkkfsaK+IJ3iJmoTMlqb3esLTR8=;
b=EnRyFMCoR2mADtC1jCKWvBJow/iKiCXWduMspTZ90IMOlfHWgKhF5FhmGQagNe4nco
n74jhtkwXIhm+inLz88Zp5mpZRRhu2uUzGq3nlVDtnrRNQSX49NKcESx4pnbXr3TZykK
rFas9h8aX+8217WV5UvF+WJ61cr8VVCZNYZgUuEVJQF9yUW+Ufl3vdusve2TYDNJnlGy
MstwZxQ3R6gQYJMIuoyhoE2Up8qv67KIJLF7Y+xgM5NKuaLPkbtNaPJYO/EDbkXGMYol
L101HTsDKSnG85CfisY0y8r6MmsnHNBb6Zu8Gf1UrMdOD/7I4OjUH6/JnCC7vjDancM0
BLtQ==
X-Gm-Message-State: AIkVDXJhVfYv6VpFyfFpGNKjvMHf3YsELGC6KeCyTXG/kXla5fRUKvLR/gZD2GcgIdkSU+qi7kNmQiK3SZ4L+w==
X-Received: by 10.107.166.84 with SMTP id p81mr34183386ioe.15.1483484786532;
Tue, 03 Jan 2017 15:06:26 -0800 (PST)
MIME-Version: 1.0
Received: by 10.36.208.207 with HTTP; Tue, 3 Jan 2017 15:06:26 -0800 (PST)
X-Originating-IP: [73.234.153.245]
In-Reply-To: <CACq0ZD7XT_h8ADptKA0uBT7617fvvgh3uGndkc08RZUSQM2yQg@mail.gmail.com>
References: <71d822e413ac457a530e1c367811cc24@cock.lu>
<77b6dd25-0603-a0bd-6a9e-38098e5cb19d@jonasschnelli.ch>
<74aeb4760316b59a3db56c0d16d11f28@cock.lu>
<CACq0ZD7XT_h8ADptKA0uBT7617fvvgh3uGndkc08RZUSQM2yQg@mail.gmail.com>
From: adiabat <rx@awsomnet.org>
Date: Tue, 3 Jan 2017 18:06:26 -0500
Message-ID: <CAKEeUhiQiUA_E6JF22foV11-WnGZH+kEzfUhROm=gvVN1qMr4A@mail.gmail.com>
To: Aaron Voisine <voisine@gmail.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a11415270bb45c8054538b790
X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, HTML_MESSAGE, RCVD_IN_DNSWL_NONE,
RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] Committed bloom filters for improved wallet
performance and SPV security
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Jan 2017 23:06:28 -0000
--001a11415270bb45c8054538b790
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Mempool transactions have their place, but "unconfirmed" and "SPV" don't
belong together. Only a full node can tell if a transaction may get
confirmed, or is nonsense. Unfortunately all the light / SPV wallets I
know of show mempool transactions, which makes it hard to go back... (e.g.
"why doesn't your software show 0-conf! your wallet is broken!", somewhat
akin to people complaining about RBF)
So, this is easy, just don't worry about mempool filtering. Why are light
clients looking at the mempool anyway? Maybe if there were some way to
provide SPV proofs of all inputs, but that's a bit of a mess for full nodes
to do.
Without mempool filtering, I think the committed bloom filters would be a
great improvement over the current bloom filter setup, especially for
lightning network use cases (with lightning, not finding out about a
transaction can make you lose money). I want to work on it and may be able
to at some point as it's somewhat related to lightning.
Also, if you're running a light client, and storing the filters the way you
store block headers, there's really no reason to go all the way back to
height 0. You can start grabbing headers at some point a while ago, before
your set of keys was generated. I think it'd be very worth it even with
GB-scale disk usage.
-Tadge
On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> Unconfirmed transactions are incredibly important for real world use.
> Merchants for instance are willing to accept credit card payments of
> thousands of dollars and ship the goods despite the fact that the
> transaction can be reversed up to 60 days later. There is a very large co=
st
> to losing the ability to have instant transactions in many or even most
> situations. This cost is typically well above the fraud risk.
>
> It's important to recognize that bitcoin serves a wide variety of use
> cases with different profiles for time sensitivity and fraud risk.
>
> Aaron
>
> On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev <bitcoin-dev@lists=
.
> linuxfoundation.org> wrote:
>
>> The concept combined with the weak blocks system where miners commit
>>
>> to potential transaction inclusion with fractional difficulty blocks
>>
>> is possible. I'm not personally convinced that unconfirmed transaction
>>
>> display in a wallet is worth the privacy trade-off. The user has very
>>
>> little to gain from this knowledge until the txn is in a block.
>>
>>
>>
>>
>>
>> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
>>
>> > Hi
>>
>> >> We introduce several concepts that rework the lightweight Bitcoin
>>
>> >> client model in a manner which is secure, efficient and privacy
>>
>> >> compatible.
>>
>> >>
>>
>> >> The BFD can be used verbatim in replacement of BIP37, where the filte=
r
>>
>> >> can be cached between clients without needing to be recomputed. It ca=
n
>>
>> >> also be used by normal pruned nodes to do re-scans locally of their
>>
>> >> wallet without needing to have the block data available to scan, or
>>
>> >> without reading the entire block chain from disk.
>>
>> > I started exploring the potential of BFD after this specification.
>>
>> >
>>
>> > What would be the preferred/recommended way to handle 0-conf/mempool
>>
>> > filtering =E2=80=93 if & once BDF would have been deployed (any type,
>>
>> > semi-trusted oracles or protocol-level/softfork)?
>>
>> >
>>
>> > From the user-experience perspective, this is probably pretty importan=
t
>>
>> > (otherwise the experience will be that incoming funds can take serval
>>
>> > minutes to hours until they appear).
>>
>> > Using BIP37 bloom filters just for mempool filtering would obviously
>>
>> > result in the same unwanted privacy-setup.
>>
>> >
>>
>> > </jonas>
>>
>> >
>>
>> >
>>
>> > _______________________________________________
>>
>> > bitcoin-dev mailing list
>>
>> > bitcoin-dev@lists.linuxfoundation.org
>>
>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>> _______________________________________________
>>
>> bitcoin-dev mailing list
>>
>> bitcoin-dev@lists.linuxfoundation.org
>>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
--001a11415270bb45c8054538b790
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><span style=3D"font-size:12.8px">Mempool transactions have=
their place, but "unconfirmed" and "SPV" don't bel=
ong together.=C2=A0 Only a full node can tell if a transaction may get conf=
irmed, or is nonsense.=C2=A0 Unfortunately all the light / SPV wallets I kn=
ow of show mempool transactions, which makes it hard to go back... (e.g. &q=
uot;why doesn't your software show 0-conf! your wallet is broken!"=
, somewhat akin to people complaining about RBF)</span><div><span style=3D"=
font-size:12.8px"><br></span></div><div><span style=3D"font-size:12.8px">So=
, this is easy, just don't worry about mempool filtering.=C2=A0 Why are=
light clients looking at the mempool anyway?=C2=A0 Maybe if there were som=
e way to provide SPV proofs of all inputs, but that's a bit of a mess f=
or full nodes to do.<br></span><div><span style=3D"font-size:12.8px"><br>Wi=
thout mempool filtering, I think the committed bloom filters would be a gre=
at improvement over the current bloom filter setup, especially for lightnin=
g network use cases (with lightning, not finding out about a transaction ca=
n make you lose money).=C2=A0 I want to work on it and may be able to at so=
me point as it's somewhat related to lightning.</span></div><div><span =
style=3D"font-size:12.8px"><br></span></div><div><span style=3D"font-size:1=
2.8px">Also, if you're running a light client, and storing the filters =
the way you store block headers, there's really no reason to go all the=
way back to height 0.=C2=A0 You can start grabbing headers at some point a=
while ago, before your set of keys was generated.=C2=A0 I think it'd b=
e very worth it even with GB-scale disk usage.</span></div><div><span style=
=3D"font-size:12.8px"><br></span></div><div><span style=3D"font-size:12.8px=
">-Tadge</span></div><div><span style=3D"font-size:12.8px"><br></span></div=
></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On T=
ue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev <span dir=3D"ltr"=
><<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_bl=
ank">bitcoin-dev@lists.linuxfoundation.org</a>></span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex"><div>Unconfirmed transactions are incredibly import=
ant for real world use. Merchants for instance are willing to accept credit=
card payments of thousands of dollars and ship the goods despite the fact =
that the transaction can be reversed up to 60 days later. There is a very l=
arge cost to losing the ability to have instant transactions in many or eve=
n most situations. This cost is typically well above the fraud risk.=C2=A0<=
/div><div><br></div><div>It's important to recognize that bitcoin serve=
s a wide variety of use cases with different profiles for time sensitivity =
and fraud risk.</div><div><br></div><div>Aaron</div><div class=3D"HOEnZb"><=
div class=3D"h5"><div><br><div class=3D"gmail_quote"><div>On Tue, Jan 3, 20=
17 at 12:41 PM bfd--- via bitcoin-dev <<a href=3D"mailto:bitcoin-dev@lis=
ts.linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.<wbr>linuxfound=
ation.org</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The conc=
ept combined with the weak blocks system where miners commit<br class=3D"m_=
582444580811563830gmail_msg"><br>to potential transaction inclusion with fr=
actional difficulty blocks<br class=3D"m_582444580811563830gmail_msg"><br>i=
s possible. I'm not personally convinced that unconfirmed transaction<b=
r class=3D"m_582444580811563830gmail_msg"><br>display in a wallet is worth =
the privacy trade-off. The user has very<br class=3D"m_582444580811563830gm=
ail_msg"><br>little to gain from this knowledge until the txn is in a block=
.<br class=3D"m_582444580811563830gmail_msg"><br><br class=3D"m_58244458081=
1563830gmail_msg"><br><br class=3D"m_582444580811563830gmail_msg"><br>On 20=
17-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:<br class=3D"m_5824445=
80811563830gmail_msg"><br>> Hi<br class=3D"m_582444580811563830gmail_msg=
"><br>>> We introduce several concepts that rework the lightweight Bi=
tcoin<br class=3D"m_582444580811563830gmail_msg"><br>>> client model =
in a manner which is secure, efficient and privacy<br class=3D"m_5824445808=
11563830gmail_msg"><br>>> compatible.<br class=3D"m_58244458081156383=
0gmail_msg"><br>>><br class=3D"m_582444580811563830gmail_msg"><br>>=
;> The BFD can be used verbatim in replacement of BIP37, where the filte=
r<br class=3D"m_582444580811563830gmail_msg"><br>>> can be cached bet=
ween clients without needing to be recomputed. It can<br class=3D"m_5824445=
80811563830gmail_msg"><br>>> also be used by normal pruned nodes to d=
o re-scans locally of their<br class=3D"m_582444580811563830gmail_msg"><br>=
>> wallet without needing to have the block data available to scan, o=
r<br class=3D"m_582444580811563830gmail_msg"><br>>> without reading t=
he entire block chain from disk.<br class=3D"m_582444580811563830gmail_msg"=
><br>> I started exploring the potential of BFD after this specification=
.<br class=3D"m_582444580811563830gmail_msg"><br>><br class=3D"m_5824445=
80811563830gmail_msg"><br>> What would be the preferred/recommended way =
to handle 0-conf/mempool<br class=3D"m_582444580811563830gmail_msg"><br>>=
; filtering =E2=80=93 if & once BDF would have been deployed (any type,=
<br class=3D"m_582444580811563830gmail_msg"><br>> semi-trusted oracles o=
r protocol-level/softfork)?<br class=3D"m_582444580811563830gmail_msg"><br>=
><br class=3D"m_582444580811563830gmail_msg"><br>> From the user-expe=
rience perspective, this is probably pretty important<br class=3D"m_5824445=
80811563830gmail_msg"><br>> (otherwise the experience will be that incom=
ing funds can take serval<br class=3D"m_582444580811563830gmail_msg"><br>&g=
t; minutes to hours until they appear).<br class=3D"m_582444580811563830gma=
il_msg"><br>> Using BIP37 bloom filters just for mempool filtering would=
obviously<br class=3D"m_582444580811563830gmail_msg"><br>> result in th=
e same unwanted privacy-setup.<br class=3D"m_582444580811563830gmail_msg"><=
br>><br class=3D"m_582444580811563830gmail_msg"><br>> </jonas><=
br class=3D"m_582444580811563830gmail_msg"><br>><br class=3D"m_582444580=
811563830gmail_msg"><br>><br class=3D"m_582444580811563830gmail_msg"><br=
>> ______________________________<wbr>_________________<br class=3D"m_58=
2444580811563830gmail_msg"><br>> bitcoin-dev mailing list<br class=3D"m_=
582444580811563830gmail_msg"><br>> <a href=3D"mailto:bitcoin-dev@lists.l=
inuxfoundation.org" class=3D"m_582444580811563830gmail_msg" target=3D"_blan=
k">bitcoin-dev@lists.<wbr>linuxfoundation.org</a><br class=3D"m_58244458081=
1563830gmail_msg"><br>> <a href=3D"https://lists.linuxfoundation.org/mai=
lman/listinfo/bitcoin-dev" rel=3D"noreferrer" class=3D"m_582444580811563830=
gmail_msg" target=3D"_blank">https://lists.linuxfoundation.<wbr>org/mailman=
/listinfo/bitcoin-<wbr>dev</a><br class=3D"m_582444580811563830gmail_msg"><=
br>______________________________<wbr>_________________<br class=3D"m_58244=
4580811563830gmail_msg"><br>bitcoin-dev mailing list<br class=3D"m_58244458=
0811563830gmail_msg"><br><a href=3D"mailto:bitcoin-dev@lists.linuxfoundatio=
n.org" class=3D"m_582444580811563830gmail_msg" target=3D"_blank">bitcoin-de=
v@lists.<wbr>linuxfoundation.org</a><br class=3D"m_582444580811563830gmail_=
msg"><br><a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitc=
oin-dev" rel=3D"noreferrer" class=3D"m_582444580811563830gmail_msg" target=
=3D"_blank">https://lists.linuxfoundation.<wbr>org/mailman/listinfo/bitcoin=
-<wbr>dev</a><br class=3D"m_582444580811563830gmail_msg"><br></blockquote><=
/div></div>
</div></div><br>______________________________<wbr>_________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
<wbr>linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.<wbr>org=
/mailman/listinfo/bitcoin-<wbr>dev</a><br>
<br></blockquote></div><br></div>
--001a11415270bb45c8054538b790--
|
