path: root/3a/e1c4ca07787e8ca045b810b286d308205dad50

Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
	helo=mx.sourceforge.net)
	by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <tier.nolan@gmail.com>) id 1V3XFm-0001WA-2L
	for bitcoin-development@lists.sourceforge.net;
	Sun, 28 Jul 2013 20:07:42 +0000
Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.160.65 as permitted sender)
	client-ip=209.85.160.65; envelope-from=tier.nolan@gmail.com;
	helo=mail-pb0-f65.google.com; 
Received: from mail-pb0-f65.google.com ([209.85.160.65])
	by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1V3XFj-0008T1-Vt
	for bitcoin-development@lists.sourceforge.net;
	Sun, 28 Jul 2013 20:07:41 +0000
Received: by mail-pb0-f65.google.com with SMTP id un1so2019956pbc.0
	for <bitcoin-development@lists.sourceforge.net>;
	Sun, 28 Jul 2013 13:07:34 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.68.170.37 with SMTP id aj5mr64844840pbc.79.1375042054146;
	Sun, 28 Jul 2013 13:07:34 -0700 (PDT)
Received: by 10.70.78.232 with HTTP; Sun, 28 Jul 2013 13:07:34 -0700 (PDT)
In-Reply-To: <CAPaL=UVc0CGvvam0tdxw+4Y8XwSw0Awz8ifv64HYgORJ7zLztg@mail.gmail.com>
References: <CAE-z3OX+Uzw_diW97yKWGzVMBFZHq2t+w15jNSdGMGqwyJ65yQ@mail.gmail.com>
	<20130724094255.GB12568@savin>
	<CAE-z3OWBoMz6jC36Bp+dGqj6jHiWj1d1zGtH+iBbLcCt_6kNJw@mail.gmail.com>
	<CAPaL=UVc0CGvvam0tdxw+4Y8XwSw0Awz8ifv64HYgORJ7zLztg@mail.gmail.com>
Date: Sun, 28 Jul 2013 21:07:34 +0100
Message-ID: <CAE-z3OVTyR1cn31kHJWd=ZUQCwcgX8UdSeZsTMRTeDuDLctz3w@mail.gmail.com>
From: Tier Nolan <tier.nolan@gmail.com>
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Content-Type: multipart/alternative; boundary=047d7ba970a6304fcb04e297edd8
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	(tier.nolan[at]gmail.com)
	-0.0 SPF_PASS               SPF: sender matches SPF record
	1.0 HTML_MESSAGE           BODY: HTML included in message
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1V3XFj-0008T1-Vt
Subject: Re: [Bitcoin-development] Distributing low POW headers
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Sun, 28 Jul 2013 20:07:42 -0000

--047d7ba970a6304fcb04e297edd8
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Jul 28, 2013 at 7:42 PM, John Dillon
<john.dillon892@googlemail.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On Wed, Jul 24, 2013 at 11:55 AM, Tier Nolan <tier.nolan@gmail.com> wrote:
> > Distributing headers with 1/64 of the standard POW means that a header
> would
> > be broadcast approximately once every 9 seconds (assuming a 10 minute
> block
> > time).  This was picked because sending 80 byte headers every 9 seconds
> > shouldn't represent much load on the network.
>
> As Peter said, "much" should be quantified.
>

It has the same statistic properties as normal blocks just 64 times faster.

Even if there is a new block 30 seconds after the previous one, that
doesn't cause a burst of 64 low POW block headers in the 30 second window.
They are all statistically independent hashing attempts.


> Sounds like you are changing economics and requiring miners to have even
> better
> network connections. This is not a thing to do lightly and it probably a
> bad
> idea.
>

No, it just breaks ties.  In most cases there would be only 1 contender
block, so all miners are equal.

If 10% of blocks were ties/orphans, then only 1% of blocks would be a 3-way
tie.  That probably overestimates the orphan rate.

This means the miner has to download 2 blocks 10% of the time and 3 blocks
1% of the time.

However, even then, half the network wouldn't have to download the 2nd
block of the tie, since they happened to get the winner first.  This means
5% extra bandwidth on average.

16 low POW headers at 9 seconds per header is more than 2 minutes for a
miner to switch to the other contender.

A miner would only lose out if he doesn't notice that block he is mining
against is not getting built on by anyone else.

He needs to download both tied blocks so that he can switch, but he has 2
minutes to actually switch.

I understand Pieter Wuille is working on letting Bitcoin propagate and make
> use
> of pure block headers, a step towards SPV and partial UTXO mode.
>

That would need to happen before low POW ones are broadcast.  There is a
basic set of rules in the first post.

At the moment, the client only provides headers when asked, but never
broadcasts them.


> Orphan measurement would be very useful for a lot of reasons, how about you
> think about that first?


I think distributing the low POW headers on an advisory basis a reasonable
first step.  However, just broadcasting the headers is a zeroth step.

Miners would probably break ties towards the block that seems to be getting
the most hashing anyway.

I think for orphan rate, the best is to have a system to link to orphans.
This would add the POW of the orphan to the main chain's total.

Unfortunately adding fields to the header is hard.  It could be done as a
coinbase extra-nonce thing.  A better option would be if the merkle tree
could include non-transactions.

The merkle root could be replaced by hash(auxiliary header).  This has the
advantage of not impacting ASIC miners.

Broadcasting all headers would at least allow clients to count orphans,
even if they aren't integrated into the block chain.

It wouldn't have the potential data rate issues either
> and should be a very simple change.


I don't think the data rate is really that high.  It would be 80 bytes
every 9 seconds, or 9 bytes per second.

Blocks are 500kB every 10 minutes, or 853 bytes per second.


> Just set some threshold relative to the
> height of the best block where you will not further propagate and orphan
> block(header) and prior to that limit do so freely. I believe the change
> would
> be 100% compatible with the P2P protocol as it is based on inventories.
>

Right absolutely.  Headers of blocks that add to the block tree within
recent history should be forwarded.

The inv system would need to be tweaked, since it can only say block and
transaction.

A block header field would allow the node to say that it only has the
header.  Alternatively, it would reply with a header message to the
getblocks message.

--047d7ba970a6304fcb04e297edd8
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On S=
un, Jul 28, 2013 at 7:42 PM, John Dillon <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:john.dillon892@googlemail.com" target=3D"_blank">john.dillon892@googl=
email.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left:1px solid rgb(204,204,204);padding-left:1ex">-----BEGIN PGP SIGNED MES=
SAGE-----<br>
Hash: SHA256<br>
<div class=3D"im"><br>
On Wed, Jul 24, 2013 at 11:55 AM, Tier Nolan &lt;<a href=3D"mailto:tier.nol=
an@gmail.com">tier.nolan@gmail.com</a>&gt; wrote:<br>
&gt; Distributing headers with 1/64 of the standard POW means that a header=
 would<br>
&gt; be broadcast approximately once every 9 seconds (assuming a 10 minute =
block<br>
&gt; time). =A0This was picked because sending 80 byte headers every 9 seco=
nds<br>
&gt; shouldn&#39;t represent much load on the network.<br>
<br>
</div>As Peter said, &quot;much&quot; should be quantified.<br></blockquote=
><div><br></div><div>It has the same statistic properties as normal blocks =
just 64 times faster.<br><br></div><div>Even if there is a new block 30 sec=
onds after the previous one, that doesn&#39;t cause a burst of 64 low POW b=
lock headers in the 30 second window.=A0 They are all statistically indepen=
dent hashing attempts.<br>
</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Sounds like you are changing economics and requiring miners to have even be=
tter<br>
network connections. This is not a thing to do lightly and it probably a ba=
d<br>
idea.<br></blockquote><div><br></div><div>No, it just breaks ties.=A0 In mo=
st cases there would be only 1 contender block, so all miners are equal.<br=
><br></div><div>If 10% of blocks were ties/orphans, then only 1% of blocks =
would be a 3-way tie.=A0 That probably overestimates the orphan rate. <br>
<br>This means the miner has to download 2 blocks 10% of the time and 3 blo=
cks 1% of the time.<br><br>However, even then, half the network wouldn&#39;=
t have to download the 2nd=20
block of the tie, since they happened to get the winner first.=A0 This mean=
s 5% extra=20
bandwidth on average.<br><br></div><div>16 low POW headers at 9 seconds per=
 header is more than 2 minutes for a miner to switch to the other contender=
.<br><br></div><div>A miner would only lose out if he doesn&#39;t notice th=
at block he is mining against is not getting built on by anyone else.<br>
<br></div><div>He needs to download both tied blocks so that he can switch,=
 but he has 2 minutes to actually switch.<br></div><div><br></div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">

I understand Pieter Wuille is working on letting Bitcoin propagate and make=
 use<br>
of pure block headers, a step towards SPV and partial UTXO mode.<br></block=
quote><div><br></div><div>That would need to happen before low POW ones are=
 broadcast.=A0 There is a basic set of rules in the first post.<br><br>At t=
he moment, the client only provides headers when asked, but never broadcast=
s them.<br>
</div><div>=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0=
px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Orphan measurement would be very useful for a lot of reasons, how about you=
<br>
think about that first? </blockquote><div><br></div><div>I think distributi=
ng the low POW headers on an advisory basis a reasonable first step.=A0 How=
ever, just broadcasting the headers is a zeroth step.<br><br></div><div>
Miners would probably break ties towards the block that seems to be getting=
 the most hashing anyway.<br><br></div><div>I think for orphan rate, the be=
st is to have a system to link to orphans.=A0 This would add the POW of the=
 orphan to the main chain&#39;s total.<br>
<br></div><div>Unfortunately adding fields to the header is hard.=A0 It cou=
ld be done as a coinbase extra-nonce thing.=A0 A better option would be if =
the merkle tree could include non-transactions.<br><br></div><div>The merkl=
e root could be replaced by hash(auxiliary header).=A0 This has the advanta=
ge of not impacting ASIC miners.<br>
<br></div><div>Broadcasting all headers would at least allow clients to cou=
nt orphans, even if they aren&#39;t integrated into the block chain.<br></d=
iv><br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex">
It wouldn&#39;t have the potential data rate issues either<br>
and should be a very simple change.</blockquote><div><br></div><div>I don&#=
39;t think the data rate is really that high.=A0 It would be 80 bytes every=
 9 seconds, or 9 bytes per second.<br><br></div><div>Blocks are 500kB every=
 10 minutes, or 853 bytes per second.<br>
</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Just=
 set some threshold relative to the<br>
height of the best block where you will not further propagate and orphan<br=
>
block(header) and prior to that limit do so freely. I believe the change wo=
uld<br>
be 100% compatible with the P2P protocol as it is based on inventories.<br>=
</blockquote><div><br></div><div>Right absolutely.=A0 Headers of blocks tha=
t add to the block tree within recent history should be forwarded.<br><br>
The inv system would need to be tweaked, since it can only say block and tr=
ansaction.<br></div></div><br></div><div class=3D"gmail_extra">A block head=
er field would allow the node to say that it only has the header.=A0 Altern=
atively, it would reply with a header message to the getblocks message.<br>
</div></div>

--047d7ba970a6304fcb04e297edd8--