Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id ED4D8B5F for ; Wed, 4 Jan 2017 00:36:36 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f176.google.com (mail-qk0-f176.google.com [209.85.220.176]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A9F23FB for ; Wed, 4 Jan 2017 00:36:35 +0000 (UTC) Received: by mail-qk0-f176.google.com with SMTP id u25so381521052qki.2 for ; Tue, 03 Jan 2017 16:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=4LZe89/T+ZW4wrH4JbksAVLB1MdKUcSbiCDU0Jnvw7k=; b=eZhKe/+gk+6OpRJK1KHLCtnWUlQIGjF5MKh9DlmFYmyqeIizGv6Bwk4QpCZ/hjgdIb 5mqUapBoWNTj9rE06UE9OTeMfBDWkZImBsDYyIxs1dNJOt4R1mk48B2jkDXyr0VhHf+C 0SKFdMKY5FRF/b1kCX5+ouR5hyy93UJ1n47dtpGu9mOO+1vc+0jYFBy3W6Uwk9S70WNe mRPRxGwRM30tosYbSGNWnuFnE5E0eYJD4W9caTgGUMVgfsZThXG6/dyQ6BfG75eZABvQ M2etC1AVXJvO324py+eq8S5Hhu43X8FTvEJyXeQohd3Is2vb7xi97f8MfPyYMMshW9oF NfSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4LZe89/T+ZW4wrH4JbksAVLB1MdKUcSbiCDU0Jnvw7k=; b=RtN9oy1z0fK6L5zlpY1u89kVbgeCKgsUzirtRlhlOunDYP3knS6K1mBmRaoJ4fjhEX 4y08S6hj/g0Qj48EjZdLBFkRaUh6cIRL9W+Kz+MsLVqcF3ESCvYSXoa/ZZoQNhXf7jVc +78hbYOElO04QWidT083ZpheCwg4ixDichxxyX/CxzPcqe+9lMDWLuDKV9/KSVwCUzCW DNlH8bl7gdJZTLRU5/zl54q+0OX5u4PNTWL6FXI4FQaf6lE5RrJIiFn4MOj1vwbKxW1z 3FWV7j2w9kVVIsIGAKCBOxXPiZBIAvXgBNdtw+iu3ap1gVP1DBb8/hds7n6oxdflTq4o hGAw== X-Gm-Message-State: AIkVDXL1DPXDWKQYNe+deL4r52grR0OcdZ3FIpS3Oq5SlgV9NLe44ylwrkQ2OYuF9DlqT3BsqvnVjBY4m2lMbQ== X-Received: by 10.55.44.193 with SMTP id s184mr68981856qkh.278.1483490194891; Tue, 03 Jan 2017 16:36:34 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.140.5 with HTTP; Tue, 3 Jan 2017 16:36:34 -0800 (PST) In-Reply-To: <22b7d05fb2b8a7a0f1c2fa0b6b375f7e@cock.lu> References: <71d822e413ac457a530e1c367811cc24@cock.lu> <77b6dd25-0603-a0bd-6a9e-38098e5cb19d@jonasschnelli.ch> <74aeb4760316b59a3db56c0d16d11f28@cock.lu> <22b7d05fb2b8a7a0f1c2fa0b6b375f7e@cock.lu> From: Aaron Voisine Date: Tue, 3 Jan 2017 16:36:34 -0800 Message-ID: To: bfd@cock.lu Content-Type: multipart/alternative; boundary=001a114f4d9a182776054539fae2 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2017 00:36:37 -0000 --001a114f4d9a182776054539fae2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Knowing that a transaction is property formatted and that it has been broadcast to the gossip network is useful in many situations. You're only thinking about whether you can know a transaction is valid and/or settled. This is not the only possible useful information in actual real world use. Any situation where credit card transactions are accepted today for instance, it is useful to know that a transaction has been initiated, even though it can be reversed at any time up to 60 days later. Aaron Voisine co-founder and CEO breadwallet On Tue, Jan 3, 2017 at 4:10 PM, wrote: > Unfortunately a non validating SPV wallet has absolutely no idea if > the information about an unconfirmed transaction they are seeing is > anything but properly formatted. They are connecting to an easily > manipulated, sybil attacked, and untrusted network and then asking > them for financial information. Seeing an unconfirmed transaction in a > wallet that's not also fully validating is at best meaningless. > > > On 2017-01-03 15:46, Aaron Voisine wrote: > >> If the sender doesn't control the receiver's network connection, then >> the information the receiver gains by watching the mempool is if the >> transaction has propagated across the bitcoin network. This is useful >> to know in all kinds of situations. >> >> Aaron Voisine >> co-founder and CEO >> breadwallet [2] >> >> On Tue, Jan 3, 2017 at 3:06 PM, adiabat wrote: >> >> Mempool transactions have their place, but "unconfirmed" and "SPV" >>> don't belong together. Only a full node can tell if a transaction >>> may get confirmed, or is nonsense. Unfortunately all the light / >>> SPV wallets I know of show mempool transactions, which makes it hard >>> to go back... (e.g. "why doesn't your software show 0-conf! your >>> wallet is broken!", somewhat akin to people complaining about RBF) >>> >>> So, this is easy, just don't worry about mempool filtering. Why are >>> light clients looking at the mempool anyway? Maybe if there were >>> some way to provide SPV proofs of all inputs, but that's a bit of a >>> mess for full nodes to do. >>> >>> Without mempool filtering, I think the committed bloom filters would >>> be a great improvement over the current bloom filter setup, >>> especially for lightning network use cases (with lightning, not >>> finding out about a transaction can make you lose money). I want to >>> work on it and may be able to at some point as it's somewhat related >>> to lightning. >>> >>> Also, if you're running a light client, and storing the filters the >>> way you store block headers, there's really no reason to go all the >>> way back to height 0. You can start grabbing headers at some point >>> a while ago, before your set of keys was generated. I think it'd be >>> very worth it even with GB-scale disk usage. >>> >>> -Tadge >>> >>> On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev >>> wrote: >>> >>> Unconfirmed transactions are incredibly important for real world >>> use. Merchants for instance are willing to accept credit card >>> payments of thousands of dollars and ship the goods despite the fact >>> that the transaction can be reversed up to 60 days later. There is a >>> very large cost to losing the ability to have instant transactions >>> in many or even most situations. This cost is typically well above >>> the fraud risk. >>> >>> It's important to recognize that bitcoin serves a wide variety of >>> use cases with different profiles for time sensitivity and fraud >>> risk. >>> >>> Aaron >>> >>> On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev >>> wrote: >>> The concept combined with the weak blocks system where miners commit >>> >>> to potential transaction inclusion with fractional difficulty blocks >>> >>> is possible. I'm not personally convinced that unconfirmed >>> transaction >>> >>> display in a wallet is worth the privacy trade-off. The user has >>> very >>> >>> little to gain from this knowledge until the txn is in a block. >>> >>> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote: >>> >>> Hi >>>> >>> >>> We introduce several concepts that rework the lightweight Bitcoin >>>>> >>>> >>> client model in a manner which is secure, efficient and privacy >>>>> >>>> >>> compatible. >>>>> >>>> >>> >>>>> >>> The BFD can be used verbatim in replacement of BIP37, where the >>>>> >>>> filter >>> >>> can be cached between clients without needing to be recomputed. >>>>> >>>> It can >>> >>> also be used by normal pruned nodes to do re-scans locally of >>>>> >>>> their >>> >>> wallet without needing to have the block data available to scan, >>>>> >>>> or >>> >>> without reading the entire block chain from disk. >>>>> >>>> >>> I started exploring the potential of BFD after this specification. >>>> >>> >>> >>>> >>> What would be the preferred/recommended way to handle >>>> >>> 0-conf/mempool >>> >>> filtering =E2=80=93 if & once BDF would have been deployed (any type, >>>> >>> >>> semi-trusted oracles or protocol-level/softfork)? >>>> >>> >>> >>>> >>> From the user-experience perspective, this is probably pretty >>>> >>> important >>> >>> (otherwise the experience will be that incoming funds can take >>>> >>> serval >>> >>> minutes to hours until they appear). >>>> >>> >>> Using BIP37 bloom filters just for mempool filtering would >>>> >>> obviously >>> >>> result in the same unwanted privacy-setup. >>>> >>> >>> >>>> >>> >>>> >>> >>> >>>> >>> >>>> >>> _______________________________________________ >>>> >>> >>> bitcoin-dev mailing list >>>> >>> >>> bitcoin-dev@lists.linuxfoundation.org >>>> >>> >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>>> >>> >>> _______________________________________________ >>> >>> bitcoin-dev mailing list >>> >>> bitcoin-dev@lists.linuxfoundation.org >>> >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>> >> >> >> >> Links: >> ------ >> [1] https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> [2] http://breadwallet.com >> > --001a114f4d9a182776054539fae2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Knowing that a transaction is property formatted and that = it has been broadcast to the gossip network is useful in many situations. Y= ou're only thinking about whether you can know a transaction is valid a= nd/or settled. This is not the only possible useful information in actual r= eal world use. Any situation where credit card transactions are accepted to= day for instance, it is useful to know that a transaction has been initiate= d, even though it can be reversed at any time up to 60 days later.

Aaron Voisine
co-founder and CEO
breadwallet

On Tue, Jan 3, 2017 at 4:10 PM, <bfd@cock.lu> wrote:
Unfortunately a non = validating SPV wallet has absolutely no idea if
the information about an unconfirmed transaction they are seeing is
anything but properly formatted. They are connecting to an easily
manipulated, sybil attacked, and untrusted network and then asking
them for financial information. Seeing an unconfirmed transaction in a
wallet that's not also fully validating is at best meaningless.


On 2017-01-03 15:46, Aaron Voisine wrote:
If the sender doesn't control the receiver's network connection, th= en
the information the receiver gains by watching the mempool is if the
transaction has propagated across the bitcoin network. This is useful
to know in all kinds of situations.

Aaron Voisine
co-founder and CEO
 breadwallet [2]
Mempool transactions have their place, but "unconfirmed" and &quo= t;SPV"
don't belong together.=C2=A0 Only a full node can tell if a transaction=
may get confirmed, or is nonsense.=C2=A0 Unfortunately all the light /
SPV wallets I know of show mempool transactions, which makes it hard
to go back... (e.g. "why doesn't your software show 0-conf! your wallet is broken!", somewhat akin to people complaining about RBF)

So, this is easy, just don't worry about mempool filtering.=C2=A0 Why a= re
light clients looking at the mempool anyway?=C2=A0 Maybe if there were
some way to provide SPV proofs of all inputs, but that's a bit of a
mess for full nodes to do.

Without mempool filtering, I think the committed bloom filters would
be a great improvement over the current bloom filter setup,
especially for lightning network use cases (with lightning, not
finding out about a transaction can make you lose money).=C2=A0 I want to work on it and may be able to at some point as it's somewhat related to lightning.

Also, if you're running a light client, and storing the filters the
way you store block headers, there's really no reason to go all the
way back to height 0.=C2=A0 You can start grabbing headers at some point a while ago, before your set of keys was generated.=C2=A0 I think it'd = be
very worth it even with GB-scale disk usage.

-Tadge

On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:

Unconfirmed transactions are incredibly important for real world
use. Merchants for instance are willing to accept credit card
payments of thousands of dollars and ship the goods despite the fact
that the transaction can be reversed up to 60 days later. There is a
very large cost to losing the ability to have instant transactions
in many or even most situations. This cost is typically well above
the fraud risk.

It's important to recognize that bitcoin serves a wide variety of
use cases with different profiles for time sensitivity and fraud
risk.

Aaron

On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
The concept combined with the weak blocks system where miners commit

to potential transaction inclusion with fractional difficulty blocks

is possible. I'm not personally convinced that unconfirmed
transaction

display in a wallet is worth the privacy trade-off. The user has
very

little to gain from this knowledge until the txn is in a block.

On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:

Hi

We introduce several concepts that rework the lightweight Bitcoin

client model in a manner which is secure, efficient and privacy

compatible.



The BFD can be used verbatim in replacement of BIP37, where the
filter

can be cached between clients without needing to be recomputed.
It can

also be used by normal pruned nodes to do re-scans locally of
their

wallet without needing to have the block data available to scan,
or

without reading the entire block chain from disk.

I started exploring the potential of BFD after this specification.



What would be the preferred/recommended way to handle
0-conf/mempool

filtering =E2=80=93 if & once BDF would have been deployed (any type,

semi-trusted oracles or protocol-level/softfork)?



From the user-experience perspective, this is probably pretty
important

(otherwise the experience will be that incoming funds can take
serval

minutes to hours until they appear).

Using BIP37 bloom filters just for mempool filtering would
obviously

result in the same unwanted privacy-setup.



</jonas>





_______________________________________________

bitcoin-dev mailing list

= bitcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev [1]

_______________________________________________

bitcoin-dev mailing list

= bitcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.<= wbr>org/mailman/listinfo/bitcoin-dev [1]

_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.<= wbr>org/mailman/listinfo/bitcoin-dev [1]



Links:
------
[1] https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[2] http://breadwallet.com

--001a114f4d9a182776054539fae2--