MIME-Version: 1.0
Sender: gmaxwell@gmail.com
In-Reply-To: <F87D7069-0FDC-4572-B02B-398A2A455935@gmail.com>
References: <d43c6082-1b2c-c95b-5144-99ad0021ea6c@mattcorallo.com>
	<CAAS2fgRF-MhOvpFY6c_qAPzNMo3GQ28RExdSbOV6Q6Oy2iWn1A@mail.gmail.com>
	<22d375c7-a032-8691-98dc-0e6ee87a4b08@mattcorallo.com>
	<CAAS2fgR3QRHeHEjjOS1ckEkL-h7=Na56G12hYW9Bmy9WEMduvg@mail.gmail.com>
	<CADZtCShLmH_k-UssNWahUNHgHvWQQ1y638LwaOfnJEipwjbiYg@mail.gmail.com>
	<CAAS2fgQLCN_cuZ-3QPjCLfYOtHfEk=SenTn5=y9LfGzJxLPR3Q@mail.gmail.com>
	<CADZtCSjYr6VMBVQ=rx44SgRWcFSXhVXUZJB=rHMh4X78Z2eY1A@mail.gmail.com>
	<CAO3Pvs9K3n=OzVQ06XGQvzNC+Aqp9S60kWM9VRPA8hWTJ3u9BQ@mail.gmail.com>
	<c23a5346-9f99-44f0-abbf-d7e7979bf1d8@gmail.com>
	<CAO3Pvs_MA4TtgCCu1NgCBjK2bZRN+rKnGQJN6m4yTrViBXRiPA@mail.gmail.com>
	<CAD3i26BibcaMdbQv-j+Egz_1y0GuhzepBp5ATNpj=Qv8hi1TVA@mail.gmail.com>
	<CADZtCShAYpbN=4qNoX5c8yd1j08+mEZzG8gZwcHrj2suY0mb9w@mail.gmail.com>
	<CADZtCShYnM3A949H18V2+BArA-K9J+cDkd=rX8xRn0+0js5CwA@mail.gmail.com>
	<CAAS2fgTXS5Tains7dfe_Rc9JxR6M=NuFW9UtieRELm+6N2uNog@mail.gmail.com>
	<CAFfwr8F+ghYb2HYEgC7Lh7Z-ytNE7EABr6cxiVXYhWLk-TPO7A@mail.gmail.com>
	<CADZtCShDzPK_jqeOrK4XBoB2uriU9c9T8Dm7By-8ew3XOoAeQg@mail.gmail.com>
	<7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com>
	<F87D7069-0FDC-4572-B02B-398A2A455935@gmail.com>
From: Gregory Maxwell <greg@xiph.org>
Date: Mon, 28 May 2018 19:24:09 +0000
Message-ID: <CAAS2fgT716PiP0ucoASxryM9y+s9H2z06Z0ToaP1xT3BozAtNw@mail.gmail.com>
To: Tamas Blummer <tamas.blummer@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size
Precedence: list

Is there an application that requires watching for output scripts that
doesn't also require watching for input scrips (or, less efficiently,
input outpoints)?

Any of the wallet application that I'm aware of need to see coins
being spent as well as created, else they may try to spend already
spent coins. If we're not aware of any applications that wouldnt use
both, there isn't much reason to separate them and if input scripts
are used instead of input outpoints there is additional savings from
combining them (due to the same scripts being spent as were created in
the block-- due to reuse and chaining).

I still am of the belief, based on Matt's argument, that there is no
use for txid what-so-ever (instead just watch for an outpoint).


On Mon, May 28, 2018 at 6:28 PM, Tamas Blummer <tamas.blummer@gmail.com> wr=
ote:
> Forgot to mention: The link I sent is to a branch that is patched to prod=
uce the filter stats.
> This is the main project and the BIP158 implementation: https://github.co=
m/rust-bitcoin/rust-bitcoin-spv/blob/master/src/blockfilter.rs
>
> Tamas Blummer
>
>> On May 28, 2018, at 20:18, Tamas Blummer <tamas.blummer@gmail.com> wrote=
:
>>
>> Hi Jim,
>>
>> A =E2=80=9Cbasic=E2=80=9D combined filter would mean up to 0.5 GB filter=
 data per month (with 100% segwith use). Considering that 1 GB is the usual=
 data quota for an entry level mobile phone contract, this could be a too h=
igh barrier for adoption.
>>
>> I repeated your calculations and produced a slightly different graph tha=
t shows the fraction of cummulative filter size to cummulative blockchain s=
ize. This is less noisy but otherwise confirms your measurement.
>>
>> I think that the data supports separation of filters as a combined filte=
r does not seem to come with significant savings. (basic  size ~=3D txid + =
input points + output scripts sizes)
>>
>> My calculations are repeatable with:
>>
>> https://github.com/tamasblummer/rust-bitcoin-spv/blob/blockfilterstats/s=
rc/bin/blockfilterstats.rs
>>
>> that is using a Rust implementation of an SPV client built on top of oth=
er libraries we work on in the rust-bitcoin GitHub community (https://githu=
b.com/rust-bitcoin). Yes, this is a shameles plug for the project hoping to=
 attract more developer.
>>
>> Tamas Blummer
>>
>>
>> <filters.png>
>>
>>> On May 24, 2018, at 05:48, Jim Posen via bitcoin-dev <bitcoin-dev@lists=
.linuxfoundation.org> wrote:
>>>
>>> Greg, I've attached a graph including the input scripts.
>>>
>>> In the top graph, we can see how the input script filter compares to th=
e input outpoint filter. It is definitely smaller as a result of address re=
use. The bottom graph shows the ratio over time of combining the input prev=
 script and output script filters vs keeping them separate. In more recent =
blocks, it appears that there are decreasing savings.
>>>
>>
>