path: root/82/995d3a3942489316a380cb8d377b39c519feef

Return-Path: <laolu32@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id F329F2D
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Fri,  8 Jun 2018 05:03:18 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E36EB734
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Fri,  8 Jun 2018 05:03:17 +0000 (UTC)
Received: by mail-wm0-f52.google.com with SMTP id 69-v6so1133678wmf.3
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 07 Jun 2018 22:03:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:references:in-reply-to:from:date:message-id:subject:to; 
	bh=AJdMCpfqkfnHGPDr85TwIVMcRYD/GS7+KGCYk2mKR4M=;
	b=ThYZgKsWojwa7Ob04h2+CdCcw2WsvPfEBoNDjgTAyFWSI4E/RaxaiRg2VSPopHppRv
	G36/68IlPJdRv9KVDFiodCx/Eqs6+ejtp7/er4yZsbHDULxqx9kihqm6bR8LMbGYFNWd
	jTmnZVVUjbSHenfQLvh7FddMQ79jVdNLQNgPxcvkhbKa/yCrFdA3IKw7f6yQisGTaCL6
	aj6JQi29F43RUV7gSMZ+waH26uo3zZjXa9e40MXVCrdDGBsWIOmHJK3OIny5giAw+LOR
	qynGMKh5JXmfJwnRAv6dlnKHCzA682pTZubM3mryoe6rCD3YugkP8JOnBCEpBn2sb1GS
	8evw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:references:in-reply-to:from:date
	:message-id:subject:to;
	bh=AJdMCpfqkfnHGPDr85TwIVMcRYD/GS7+KGCYk2mKR4M=;
	b=eZXOkICRnMiiB/6WPlCG+AtDMLUQxa3D7oDmuxkgf8T5YsGOJKkqnSqpx5mn7kEsus
	ULIYtz0ywrcJdAKa6FEndau+55cuG0Nok3OhQY/6VT3ICap/hIKQ6kLwcaH0B3voAG41
	yqfKfeKiI5LDrGKYUEqz1FUM0Mi+jTexxgwQdAu/7VDS7qQ2Ba3FIOw7+eNuLYveGXc7
	V/bJaPeqoKFNn/D9ucKuLmebRxhGNiUD3j2+6CGUmnkrKriBtjlzz+6sGGKKDa2OEqDQ
	70gc799d9zQSsZOG8rq75/rqPOqgc8Z1+q1zmu2IYLJe2OHy3uaguASwn3WBao6SrvyB
	QZRQ==
X-Gm-Message-State: APt69E10cRWkfJd+WyGdJ8niAH6ys1RvExL1EyIxEiPp68OSIxesAfDo
	jtpXSx8JjTGR0iSd0zaSRNiU02wHdZdaSgHqevg=
X-Google-Smtp-Source: ADUXVKLFFl1GPyVqgdpqnxfp2M84MftlWRBveTHLz3Xuoi/lkvhB99FuvYPkeEZSKlUWPAf6dmIBXRQtX672WitluNY=
X-Received: by 2002:a50:d311:: with SMTP id
	g17-v6mr5291138edh.160.1528434196334; 
	Thu, 07 Jun 2018 22:03:16 -0700 (PDT)
MIME-Version: 1.0
References: <7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com>
	<F87D7069-0FDC-4572-B02B-398A2A455935@gmail.com>
	<CAAS2fgT716PiP0ucoASxryM9y+s9H2z06Z0ToaP1xT3BozAtNw@mail.gmail.com>
	<CADZtCSguto2z6Z9CykymxnCokqo1G=sW0Ov0ht+KcD+KMnYyow@mail.gmail.com>
	<CAO3Pvs-YDzfRqmyJ85wTH0ciccjCvkm5stGyP_tVGGna=PMv3A@mail.gmail.com>
	<CAO3Pvs9p5COiS_7Jbj1r2iAKTEdXUcnVTRzL27c3=CeuB9WDTQ@mail.gmail.com>
	<CAAS2fgSyVi0d_ixp-auRPPzPfFeffN=hsWhWT5=EzDO3O+Ue1g@mail.gmail.com>
	<CAO3Pvs_0qCZbRCfL8EJw6gzWjZeXWcJrtg27g_SJ7+PkYTHg6A@mail.gmail.com>
	<CAAS2fgTs+aKyiL8Kg_AZk=Mdh6896MEg=KHa6ANAZO7unsGEsg@mail.gmail.com>
	<CADZtCShyYbgKk2zsKzQniqDw--XKfYWTk3Hk3o50V=MgT6zeuQ@mail.gmail.com>
	<20180602124157.744x7j4u7dqtaa43@email>
	<343A3542-3103-42E9-95B7-640DFE958FFA@gmail.com>
	<CAAS2fgQDdJpzPR9Ve81hhyqU+MO7Ryy125fzK-iv=sfwwORDCw@mail.gmail.com>
	<37BECD1A-7515-4081-85AC-871B9FB57772@gmail.com>
	<CAPg+sBjXbwTKW+qbGwJgau-Q2-uJC6N1JH8hH4KThv0Ah3WuqA@mail.gmail.com>
In-Reply-To: <CAPg+sBjXbwTKW+qbGwJgau-Q2-uJC6N1JH8hH4KThv0Ah3WuqA@mail.gmail.com>
From: Olaoluwa Osuntokun <laolu32@gmail.com>
Date: Thu, 7 Jun 2018 22:03:04 -0700
Message-ID: <CAO3Pvs9BQ2Dc9GCuJNxko_34Jx5kSOd8jxYkfpMW2E_1EOBEuQ@mail.gmail.com>
To: Pieter Wuille <pieter.wuille@gmail.com>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="00000000000055d0a0056e1a515d"
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,
	HTML_MESSAGE,RCVD_IN_DNSWL_NONE autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2018 05:03:19 -0000

--00000000000055d0a0056e1a515d
Content-Type: text/plain; charset="UTF-8"

Hi sipa,

> The advantage of (a) is that it can be verified against a full block
without
> access to the outputs being spent by it
>
> The advantage of (b) is that it is more compact (scriot reuse, and outputs
> spent within the same block as they are created).

Thanks for this breakdown. I think you've accurately summarized the sole
remaining discussing point in this thread.

As someone who's written and reviews code integrating the proposal all the
way up the stack (from node to wallet, to application), IMO, there's no
immediate cost to deferring the inclusion/creation of a filter that includes
prev scripts (b) instead of the outpoint as the "regular" filter does now.
Switching to prev script in the _short term_ would be costly for the set of
applications already deployed (or deployed in a minimal or flag flip gated
fashion) as the move from prev script to outpoint is a cascading one that
impacts wallet operation, rescans, HD seed imports, etc.

Maintaining the outpoint also allows us to rely on a "single honest peer"
security model in the short term. In the long term the main barrier to
committing the filters isn't choosing what to place in the filters (as once
you have the gcs code, adding/removing elements is a minor change), but the
actual proposal to add new consensus enforced commitments to Bitcoin in the
first place. Such a proposal would need to be generalized enough to allow
several components to be committed, likely have versioning, and also provide
the necessary extensibility to allow additional items to be committed in the
future. To my knowledge no such soft-fork has yet been proposed in a serious
manner, although we have years of brainstorming on the topic. The timeline
of the drafting, design, review, and deployment of such a change would
likely be measures in years, compared to the immediate deployment of the
current p2p filter model proposed in the BIP.

As a result, I see no reason to delay the p2p filter deployment (with the
outpoint) in the short term, as the long lead time a soft-fork to add
extensible commitments to Bitcoin would give application+wallet authors
ample time to switch to the new model. Also there's no reason that full-node
wallets which wish to primarily use the filters for rescan purposes can't
just construct them locally for this particular use case independent of
what's currently deployed on the p2p network.

Finally, I've addressed the remaining comments on my PR modifying the BIP
from my last message.

-- Laolu

On Sat, Jun 2, 2018 at 11:12 PM Pieter Wuille via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Sat, Jun 2, 2018, 22:56 Tamas Blummer via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Lighter but SPV secure nodes (filter committed) would help the network
>> (esp. Layer 2) to grow mesh like, but add more user that blindly follow POW.
>>
>> On longer term most users' security will be determined by either trusted
>> hubs or POW.
>> I do not know which is worse, but we should at least offer the choice to
>> the user, therefore commit filters.
>>
>
> I don't think that's the point of discussion here. Of course, in order to
> have filters that verifiably don't lie by omission, the filters need to be
> committed to by blocks.
>
> The question is what data that filter should contain.
>
> There are two suggestions:
> (a) The scriptPubKeys of the block's outputs, and prevouts of the block's
> inputs.
> (b) The scriptPubKeys of the block's outputs, and scriptPubKeys of outputs
> being spent by the block's inputs.
>
> The advantage of (a) is that it can be verified against a full block
> without access to the outputs being spent by it. This allows light clients
> to ban nodes that give them incorrect filters, but they do need to actually
> see the blocks (partially defeating the purpose of having filters in the
> first place).
>
> The advantage of (b) is that it is more compact (scriot reuse, and outputs
> spent within the same block as they are created). It also had the advantage
> of being more easily usable for scanning of a wallet's transactions. Using
> (a) for that in some cases may need to restart and refetch when an output
> is discovered, to go test for its spending (whose outpoint is not known
> ahead of time). Especially when fetching multiple filters at a time this
> may be an issue.
>
> I think both of these potentially good arguments. However, once a
> committed filter exists, the advantage of (a) goes away completely -
> validation of committed filters is trivial and can be done without needing
> the full blocks in the first place.
>
> So I think the question is do we aim for an uncommitted (a) first and a
> committed (b) later, or go for (b) immediately?
>
> Cheers,
>
> --
> Pieter
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--00000000000055d0a0056e1a515d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div>Hi sipa,</div><div><br></div><div>&gt; The advant=
age of (a) is that it can be verified against a full block without</div><di=
v>&gt; access to the outputs being spent by it</div><div>&gt;=C2=A0</div><d=
iv>&gt; The advantage of (b) is that it is more compact (scriot reuse, and =
outputs</div><div>&gt; spent within the same block as they are created).</d=
iv><div><br></div><div>Thanks for this breakdown. I think you&#39;ve accura=
tely summarized the sole</div><div>remaining discussing point in this threa=
d.</div><div><br></div><div>As someone who&#39;s written and reviews code i=
ntegrating the proposal all the</div><div>way up the stack (from node to wa=
llet, to application), IMO, there&#39;s no</div><div>immediate cost to defe=
rring the inclusion/creation of a filter that includes</div><div>prev scrip=
ts (b) instead of the outpoint as the &quot;regular&quot; filter does now.<=
/div><div>Switching to prev script in the _short term_ would be costly for =
the set of</div><div>applications already deployed (or deployed in a minima=
l or flag flip gated</div><div>fashion) as the move from prev script to out=
point is a cascading one that</div><div>impacts wallet operation, rescans, =
HD seed imports, etc.</div><div><br></div><div>Maintaining the outpoint als=
o allows us to rely on a &quot;single honest peer&quot;</div><div>security =
model in the short term. In the long term the main barrier to</div><div>com=
mitting the filters isn&#39;t choosing what to place in the filters (as onc=
e</div><div>you have the gcs code, adding/removing elements is a minor chan=
ge), but the</div><div>actual proposal to add new consensus enforced commit=
ments to Bitcoin in the</div><div>first place. Such a proposal would need t=
o be generalized enough to allow</div><div>several components to be committ=
ed, likely have versioning, and also provide</div><div>the necessary extens=
ibility to allow additional items to be committed in the</div><div>future. =
To my knowledge no such soft-fork has yet been proposed in a serious</div><=
div>manner, although we have years of brainstorming on the topic. The timel=
ine</div><div>of the drafting, design, review, and deployment of such a cha=
nge would</div><div>likely be measures in years, compared to the immediate =
deployment of the</div><div>current p2p filter model proposed in the BIP.=
=C2=A0</div><div><br></div><div>As a result, I see no reason to delay the p=
2p filter deployment (with the</div><div>outpoint) in the short term, as th=
e long lead time a soft-fork to add</div><div>extensible commitments to Bit=
coin would give application+wallet authors</div><div>ample time to switch t=
o the new model. Also there&#39;s no reason that full-node</div><div>wallet=
s which wish to primarily use the filters for rescan purposes can&#39;t</di=
v><div>just construct them locally for this particular use case independent=
 of</div><div>what&#39;s currently deployed on the p2p network.</div><div><=
br></div><div>Finally, I&#39;ve addressed the remaining comments on my PR m=
odifying the BIP</div><div>from my last message.=C2=A0</div><div><br></div>=
<div>-- Laolu</div><div><br></div><div class=3D"gmail_quote"><div dir=3D"lt=
r">On Sat, Jun 2, 2018 at 11:12 PM Pieter Wuille via bitcoin-dev &lt;<a hre=
f=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxf=
oundation.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div d=
ir=3D"auto"><div><div class=3D"gmail_quote"><div dir=3D"ltr">On Sat, Jun 2,=
 2018, 22:56 Tamas Blummer via bitcoin-dev &lt;<a href=3D"mailto:bitcoin-de=
v@lists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfound=
ation.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Lighter =
but SPV secure nodes (filter committed) would help the network (esp. Layer =
2) to grow mesh like, but add more user that blindly follow POW.<br>
<br>
On longer term most users&#39; security will be determined by either truste=
d hubs or POW.<br>
I do not know which is worse, but we should at least offer the choice to th=
e user, therefore commit filters.<br></blockquote></div></div><div dir=3D"a=
uto"><br></div></div><div dir=3D"auto"><div dir=3D"auto">I don&#39;t think =
that&#39;s the point of discussion here. Of course, in order to have filter=
s that verifiably don&#39;t lie by omission, the filters need to be committ=
ed to by blocks.</div><div dir=3D"auto"><br></div><div dir=3D"auto">The que=
stion is what data that filter should contain.</div><div dir=3D"auto"><br><=
/div><div dir=3D"auto">There are two suggestions:</div><div dir=3D"auto">(a=
) The scriptPubKeys of the block&#39;s outputs, and prevouts of the block&#=
39;s inputs.</div><div dir=3D"auto">(b) The scriptPubKeys of the block&#39;=
s outputs, and scriptPubKeys of outputs being spent by the block&#39;s inpu=
ts.</div><div dir=3D"auto"><br></div><div dir=3D"auto">The advantage of (a)=
 is that it can be verified against a full block without access to the outp=
uts being spent by it. This allows light clients to ban nodes that give the=
m incorrect filters, but they do need to actually see the blocks (partially=
 defeating the purpose of having filters in the first place).</div><div dir=
=3D"auto"><br></div><div dir=3D"auto">The advantage of (b) is that it is mo=
re compact (scriot reuse, and outputs spent within the same block as they a=
re created). It also had the advantage of being more easily usable for scan=
ning of a wallet&#39;s transactions. Using (a) for that in some cases may n=
eed to restart and refetch when an output is discovered, to go test for its=
 spending (whose outpoint is not known ahead of time). Especially when fetc=
hing multiple filters at a time this may be an issue.</div><div dir=3D"auto=
"><br></div><div dir=3D"auto">I think both of these potentially good argume=
nts. However, once a committed filter exists, the advantage of (a) goes awa=
y completely - validation of committed filters is trivial and can be done w=
ithout needing the full blocks in the first place.</div><div dir=3D"auto"><=
br></div><div dir=3D"auto">So I think the question is do we aim for an unco=
mmitted (a) first and a committed (b) later, or go for (b) immediately?</di=
v><div dir=3D"auto"><br></div><div dir=3D"auto">Cheers,</div></div><div dir=
=3D"auto"><div dir=3D"auto"><br></div><div dir=3D"auto">--=C2=A0</div><div =
dir=3D"auto">Pieter</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div=
 class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
</blockquote></div></div></div>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div></div>

--00000000000055d0a0056e1a515d--