Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.220.177 as permitted sender)
	client-ip=209.85.220.177; envelope-from=voisine@gmail.com;
	helo=mail-qk0-f177.google.com; 
MIME-Version: 1.0
In-Reply-To: <CAFdHNGh=eGCwoMF36Siup-h6aSQtE0mvxCfk+OQRJb-37pds9w@mail.gmail.com>
References: <CAFdHNGgtgWGu8gnnJfM0EcVn2m_Wff5HPwAe-9FBvjR++q0Q-Q@mail.gmail.com>
	<CACq0ZD5=EunMZJJMKfFUGkR=Ye_8nmV0qLkJJ997gbWk1MTC9w@mail.gmail.com>
	<CAFdHNGh=eGCwoMF36Siup-h6aSQtE0mvxCfk+OQRJb-37pds9w@mail.gmail.com>
Date: Wed, 10 Jun 2015 14:18:26 -0700
Message-ID: <CACq0ZD5O2Yt_XTzhTPm6tEEdA1OQinbeTfJi7cQE-=N=GWXTxQ@mail.gmail.com>
From: Aaron Voisine <voisine@gmail.com>
To: Nathan Wilcox <nathan@leastauthority.com>
Content-Type: multipart/alternative; boundary=001a113a5d526c8fe90518306a1e
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
Precedence: list

--001a113a5d526c8fe90518306a1e
Content-Type: text/plain; charset=UTF-8

> Sounds plausible, except SPV protocols would need to include this
coinbase txn if it's going to help SPV clients.

Yes you'd either need a way to add those transactions to the bloom filter,
or add/modify a p2p message to request it specifically.

> when you mention Sybil attack, I don't quite follow.

I just mean that someone could spin up a bunch of malicious p2p nodes that
lied about mempool data. It's a bit worse for SPV clients since they can't
verify that unconfirmed transactions are valid.

> I had previously believed fees were fairly static presently,

I actually just added it the other day after getting blockcypher to include
it in their api. The current release is still using a hard coded fee rate.

Aaron Voisine
co-founder and CEO
breadwallet.com

On Wed, Jun 10, 2015 at 1:00 PM, Nathan Wilcox <nathan@leastauthority.com>
wrote:

> On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine <voisine@gmail.com> wrote:
>
>> It could be done by agreeing on a data format and encoding it in an
>> op_return output in the coinbase transaction. If it catches on it could
>> later be enforced with a soft fork.
>>
>>
> Sounds plausible, except SPV protocols would need to include this coinbase
> txn if it's going to help SPV clients. (Until a softfork is activated, SPV
> clients should not rely on this encoding, since until that time the results
> can be fabricated by individual miners.)
>
>
>> For real up-to-the-minute fee calculations you're also going to want to
>> look at the current mempool, how many transactions are waiting, what fees
>> they're paying, etc, but of course that information is susceptible to sybil
>> attack.
>>
>
> Hm, when you mention Sybil attack, I don't quite follow.
>
> When a client relies on any report of a mempool [*], this is already
> outside the realm of locally-verifiable SPV information, so they are
> already susceptible to the service making false claims. If that's
> acceptable (and in many cases it may be) then this whole mechanism is moot,
> because the client can ask the service for fee statistics for past blocks.
>
>
>> In practice what we're doing for now is using services like blockcypher
>> who's business is improving reliability of zero-conf to tell us what
>> fee-per-kb is needed, and then putting a hard coded range around it to
>> protect against the service being compromised.
>>
>
> This is interesting for me, because I had previously believed fees were
> fairly static presently, and also because I like hearing about real life
> wallet implementations.
>
> So if this "SPV Fee Stats" feature were added, a wallet might rely on an
> API for timely stats (aka "block height < 1") then verify that the API
> isn't lying after doing SPV verification of fee stats for confirmed blocks.
>
>
> This is also the kind of thing being done for exchange rate data which is
>> probably the bigger security risk until bitcoin becomes the standard unit
>> of account for the planet.
>>
>>
> That makes sense, although there's no SPV equivalent for exchange data.
>
>
> Aaron Voisine
>> co-founder and CEO
>> breadwallet.com
>>
>> On Wed, Jun 10, 2015 at 10:37 AM, Nathan Wilcox <
>> nathan@leastauthority.com> wrote:
>>
>>> [I'm currently wading through bitcoin-development. I'm still about a
>>> month behind, so I apologize in advance for any noisy redundancy in this
>>> post.]
>>>
>>> While reading about blocksize, I've just finished Mike Hearn's blog post
>>> describing expected systemic behavior as actual blocks approach the current
>>> limit (with or without non-protocol-changing implementation improvements):
>>>
>>> https://medium.com/@octskyward/crash-landing-f5cc19908e32
>>>
>>>
>>> One detail Mike uses to argue against the "fee's will save us" line of
>>> reasoning is that wallets have no good way to learn fee information.
>>>
>>> So, here's a proposal to fix that: put fee and (and perhaps block size,
>>> UTXO, etc...) statistics into the locally-verifiable data available to SPV
>>> clients (ie: block headers).
>>>
>>>
>>> It's easy to imagine a hard fork that places details like per-block
>>> total fees, transaction count, fee variance, UTXO delta, etc... in a each
>>> block header. This would allow SPV clients to rely on this data with the
>>> same PoW-backed assurances as all other header data.
>>>
>>> This mechanism seems valuable regardless of the outcome of blocksize
>>> debate. So long as fees are interesting or important, SPV clients should
>>> know about them. (Same for other stats such as UTXO count.)
>>>
>>> Upgrading the protocol without a hard-fork may be possible and is left
>>> as an exercise for the reader. ;-)
>>>
>>> --
>>> Nathan Wilcox
>>> Least Authoritarian
>>>
>>> email: nathan@leastauthority.com
>>> twitter: @least_nathan
>>> PGP: 11169993 / AAAC 5675 E3F7 514C 67ED  E9C9 3BFE 5263 1116 9993
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>>
>>
>
>
> --
> Nathan Wilcox
> Least Authoritarian
>
> email: nathan@leastauthority.com
> twitter: @least_nathan
> PGP: 11169993 / AAAC 5675 E3F7 514C 67ED  E9C9 3BFE 5263 1116 9993
>

--001a113a5d526c8fe90518306a1e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>&gt;=C2=A0<span style=3D"font-size:13px">Sounds plaus=
ible, except SPV protocols would need to include this coinbase txn if it&#3=
9;s going to help SPV clients.</span></div><div><span style=3D"font-size:13=
px"><br></span></div><div><span style=3D"font-size:13px">Yes you&#39;d eith=
er need a way to add those transactions to the bloom filter, or add/modify =
a p2p message to request it specifically.</span></div><div><br></div>&gt;=
=C2=A0<span style=3D"font-size:13px">when you mention Sybil attack, I don&#=
39;t quite follow.</span><div><span style=3D"font-size:13px"><br></span></d=
iv><div>I just mean that someone could spin up a bunch of malicious p2p nod=
es that lied about mempool data. It&#39;s a bit worse for SPV clients since=
 they can&#39;t verify that unconfirmed transactions are valid.</div><div><=
br></div><div>&gt;=C2=A0<span style=3D"font-size:13px">I had previously bel=
ieved fees were fairly static presently,</span></div><div><span style=3D"fo=
nt-size:13px"><br></span></div><div><span style=3D"font-size:13px">I actual=
ly just added it the other day after getting blockcypher to include it in t=
heir api. The current release is still using a hard coded fee rate.</span><=
/div><div class=3D"gmail_extra"><br><div><div class=3D"gmail_signature"><di=
v dir=3D"ltr"><div><div dir=3D"ltr"><div>Aaron Voisine</div><div>co-founder=
 and CEO<br><a href=3D"http://breadwallet.com" target=3D"_blank">breadwalle=
t.com</a></div></div></div></div></div></div>
<br><div class=3D"gmail_quote">On Wed, Jun 10, 2015 at 1:00 PM, Nathan Wilc=
ox <span dir=3D"ltr">&lt;<a href=3D"mailto:nathan@leastauthority.com" targe=
t=3D"_blank">nathan@leastauthority.com</a>&gt;</span> wrote:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex"><div dir=3D"ltr"><span class=3D"">On Wed, Jun 10, 2015 =
at 1:19 PM, Aaron Voisine <span dir=3D"ltr">&lt;<a href=3D"mailto:voisine@g=
mail.com" target=3D"_blank">voisine@gmail.com</a>&gt;</span> wrote:<br></sp=
an><div class=3D"gmail_extra"><div class=3D"gmail_quote"><span class=3D""><=
blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
 #ccc solid;padding-left:1ex"><div dir=3D"ltr">It could be done by agreeing=
 on a data format and encoding it in an op_return output in the coinbase tr=
ansaction. If it catches on it could later be enforced with a soft fork.<di=
v><br></div></div></blockquote><div><br></div></span><div>Sounds plausible,=
 except SPV protocols would need to include this coinbase txn if it&#39;s g=
oing to help SPV clients. (Until a softfork is activated, SPV clients shoul=
d not rely on this encoding, since until that time the results can be fabri=
cated by individual miners.)<br></div><span class=3D""><div>=C2=A0</div><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #=
ccc solid;padding-left:1ex"><div dir=3D"ltr"><div></div><div>For real up-to=
-the-minute fee calculations you&#39;re also going to want to look at the c=
urrent mempool, how many transactions are waiting, what fees they&#39;re pa=
ying, etc, but of course that information is susceptible to sybil attack.</=
div></div></blockquote><div><br></div></span><div>Hm, when you mention Sybi=
l attack, I don&#39;t quite follow.<br></div><div><br></div><div>When a cli=
ent relies on any report of a mempool [*], this is already outside the real=
m of locally-verifiable SPV information, so they are already susceptible to=
 the service making false claims. If that&#39;s acceptable (and in many cas=
es it may be) then this whole mechanism is moot, because the client can ask=
 the service for fee statistics for past blocks.<br><br></div><span class=
=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><br></div><div>I=
n practice what we&#39;re doing for now is using services like blockcypher =
who&#39;s business is improving reliability of zero-conf to tell us what fe=
e-per-kb is needed, and then putting a hard coded range around it to protec=
t against the service being compromised.</div></div></blockquote><div><br><=
/div></span><div>This is interesting for me, because I had previously belie=
ved fees were fairly static presently, and also because I like hearing abou=
t real life wallet implementations. <br><br>So if this &quot;SPV Fee Stats&=
quot; feature were added, a wallet might rely on an API for timely stats (a=
ka &quot;block height &lt; 1&quot;) then verify that the API isn&#39;t lyin=
g after doing SPV verification of fee stats for confirmed blocks.<br></div>=
<span class=3D""><div>=C2=A0<br><br></div><blockquote class=3D"gmail_quote"=
 style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><d=
iv dir=3D"ltr"><div>This is also the kind of thing being done for exchange =
rate data which is probably the bigger security risk until bitcoin becomes =
the standard unit of account for the planet.</div><div class=3D"gmail_extra=
"><div><div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><br></div></div></d=
iv></div></div></div></div></div></blockquote><div><br></div></span><div>Th=
at makes sense, although there&#39;s no SPV equivalent for exchange data.<b=
r><br><br></div><div><div class=3D"h5"><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div =
dir=3D"ltr"><div class=3D"gmail_extra"><div><div><div dir=3D"ltr"><div><div=
 dir=3D"ltr"><div>Aaron Voisine</div><div>co-founder and CEO<br><a href=3D"=
http://breadwallet.com" target=3D"_blank">breadwallet.com</a></div></div></=
div></div></div></div>
<br><div class=3D"gmail_quote"><div><div>On Wed, Jun 10, 2015 at 10:37 AM, =
Nathan Wilcox <span dir=3D"ltr">&lt;<a href=3D"mailto:nathan@leastauthority=
.com" target=3D"_blank">nathan@leastauthority.com</a>&gt;</span> wrote:<br>=
</div></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bo=
rder-left:1px #ccc solid;padding-left:1ex"><div><div><div dir=3D"ltr"><div>=
[I&#39;m currently wading through bitcoin-development. I&#39;m still about =
a month behind, so I apologize in advance for any noisy redundancy in this =
post.]<br><br></div>While reading about blocksize, I&#39;ve just finished M=
ike Hearn&#39;s blog post describing expected systemic behavior as actual b=
locks approach the current limit (with or without non-protocol-changing imp=
lementation improvements):<br><div><br><a href=3D"https://medium.com/@octsk=
yward/crash-landing-f5cc19908e32" target=3D"_blank">https://medium.com/@oct=
skyward/crash-landing-f5cc19908e32</a><br><br><br></div><div>One detail Mik=
e uses to argue against the &quot;fee&#39;s will save us&quot; line of reas=
oning is that wallets have no good way to learn fee information.<br><br></d=
iv><div>So, here&#39;s a proposal to fix that: put fee and (and perhaps blo=
ck size, UTXO, etc...) statistics into the locally-verifiable data availabl=
e to SPV clients (ie: block headers).<br><br></div><br><div>It&#39;s easy t=
o imagine a hard fork that places details like per-block total fees, transa=
ction count, fee variance, UTXO delta, etc... in a each block header. This =
would allow SPV clients to rely on this data with the same PoW-backed assur=
ances as all other header data.<br><br></div><div><div>This mechanism seems=
 valuable regardless of the outcome of=20
blocksize debate. So long as fees are interesting or important, SPV=20
clients should know about them. (Same for other stats such as UTXO=20
count.)<br><br></div><div>Upgrading the protocol without a hard-fork may be=
 possible and is left as an exercise for the reader. ;-)<span><font color=
=3D"#888888"><br></font></span></div><span><font color=3D"#888888"><br>-- <=
br><div>Nathan Wilcox<br>Least Authoritarian<br><br>email: <a href=3D"mailt=
o:nathan@leastauthority.com" target=3D"_blank">nathan@leastauthority.com</a=
><br>twitter: @least_nathan<br>PGP: 11169993 / AAAC 5675 E3F7 514C 67ED =C2=
=A0E9C9 3BFE 5263 1116 9993<br></div>
</font></span></div></div>
<br></div></div>-----------------------------------------------------------=
-------------------<br>
<br>_______________________________________________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net" target=3D"_bla=
nk">Bitcoin-development@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div></div></div><div><div class=3D"h5"><br><br clear=3D"all"=
><br>-- <br><div>Nathan Wilcox<br>Least Authoritarian<br><br>email: <a href=
=3D"mailto:nathan@leastauthority.com" target=3D"_blank">nathan@leastauthori=
ty.com</a><br>twitter: @least_nathan<br>PGP: 11169993 / AAAC 5675 E3F7 514C=
 67ED =C2=A0E9C9 3BFE 5263 1116 9993<br></div>
</div></div></div></div>
</blockquote></div><br></div></div>

--001a113a5d526c8fe90518306a1e--