MIME-Version: 1.0
In-Reply-To: <CAAS2fgR3QRHeHEjjOS1ckEkL-h7=Na56G12hYW9Bmy9WEMduvg@mail.gmail.com>
References: <d43c6082-1b2c-c95b-5144-99ad0021ea6c@mattcorallo.com>
	<CAAS2fgRF-MhOvpFY6c_qAPzNMo3GQ28RExdSbOV6Q6Oy2iWn1A@mail.gmail.com>
	<22d375c7-a032-8691-98dc-0e6ee87a4b08@mattcorallo.com>
	<CAAS2fgR3QRHeHEjjOS1ckEkL-h7=Na56G12hYW9Bmy9WEMduvg@mail.gmail.com>
From: Jim Posen <jim.posen@gmail.com>
Date: Thu, 17 May 2018 13:19:17 -0700
Message-ID: <CADZtCShLmH_k-UssNWahUNHgHvWQQ1y638LwaOfnJEipwjbiYg@mail.gmail.com>
To: Gregory Maxwell <greg@xiph.org>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000cb58c4056c6c8c87"
Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size
Precedence: list

--000000000000cb58c4056c6c8c87
Content-Type: text/plain; charset="UTF-8"

>
> I think lite clients cross checking is something which very likely
> will never be implemented by anyone, and probably not stay working
> (due to under-usage) if it is implemented.  This thought is driven by
> three things  (1) the bandwidth overhead of performing the check, (2)
> thinking about the network-interacting-state-machine complexity of it,
> and by the multitude of sanity checks that lite clients already don't
> implement (e.g. when a lite client noticed a split tip it could ask
> peers for the respective blocks and check at least the stateless
> checks, but none has ever done that), and...
>

In my opinion, it's overly pessimistic to design the protocol in an
insecure way because some light clients historically have taken shortcuts.
If the protocol can provide clients the option of getting additional
security, it should.

On the general topic, Peter makes a good point that in many cases filtering
by txid of spending transaction may be preferable to filtering by outpoint
spend, which has the nice benefit that there are obviously fewer txs in a
block than txins. This wouldn't work for malleable transactions though.

I'm open to the idea of splitting the basic filter into three separate
filters based on data type, but there are some bandwidth concerns. First,
the GCS encoding gets better compression with a greater number of elements,
though as I recall in my analysis, that starts to tail off at ~1000
elements per filter with P=20, in which case it's not as much of a concern
given current block sizes. The other is that clients need to download
and/or store the filter header chain for each filter type, which are 32
bytes each per block. So if a client is expected to download all three
filter types anyway, or even two of three, it's less efficient in these
terms. It would be possible though to split the filters themselves, but
still have the basic filter header cover all three filters. This would mean
that full nodes could not support just a subset of the basic filters --
they'd have to compute all of them to compute the filter header.

--000000000000cb58c4056c6c8c87
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex">I think lite clients cross checking is something=
 which very likely<br>
will never be implemented by anyone, and probably not stay working<br>
(due to under-usage) if it is implemented.=C2=A0 This thought is driven by<=
br>
three things=C2=A0 (1) the bandwidth overhead of performing the check, (2)<=
br>
thinking about the network-interacting-state-<wbr>machine complexity of it,=
<br>
and by the multitude of sanity checks that lite clients already don&#39;t<b=
r>
implement (e.g. when a lite client noticed a split tip it could ask<br>
peers for the respective blocks and check at least the stateless<br>
checks, but none has ever done that), and...<br></blockquote><div><br></div=
><div><span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-=
size:small;font-style:normal;font-variant-ligatures:normal;font-variant-cap=
s:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent=
:0px;text-transform:none;white-space:normal;word-spacing:0px;background-col=
or:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:ini=
tial;float:none;display:inline">In my opinion, it&#39;s overly pessimistic =
to design the protocol in an insecure way because some light clients histor=
ically have taken shortcuts. If the protocol can provide clients the option=
 of getting additional security, it should.</span><br class=3D"gmail-Apple-=
interchange-newline">=C2=A0</div><div>On the general topic, Peter makes a g=
ood point that in many cases filtering by txid of spending transaction may =
be preferable to filtering by outpoint spend, which has the nice benefit th=
at there are obviously fewer txs in a block than txins. This wouldn&#39;t w=
ork for malleable transactions though.</div><div><br></div><div>I&#39;m ope=
n to the idea of splitting the basic filter into three separate filters bas=
ed on data type, but there are some bandwidth concerns. First, the GCS enco=
ding gets better compression with a greater number of elements, though as I=
 recall in my analysis, that starts to tail off at ~1000 elements per filte=
r with P=3D20, in which case it&#39;s not as much of a concern given curren=
t block sizes. The other is that clients need to download and/or store the =
filter header chain for each filter type, which are 32 bytes each per block=
. So if a client is expected to download all three filter types anyway, or =
even two of three, it&#39;s less efficient in these terms. It would be poss=
ible though to split the filters themselves, but still have the basic filte=
r header cover all three filters. This would mean that full nodes could not=
 support just a subset of the basic filters -- they&#39;d have to compute a=
ll of them to compute the filter header.</div></div></div></div>

--000000000000cb58c4056c6c8c87--