MIME-Version: 1.0
In-Reply-To: <CAAS2fgROykenGH43_FXun+q4u+=94tKqnRW=kQQ2AQFeXKdW=Q@mail.gmail.com>
References: <CADZtCShywq_s9ZK3oBBTUdCgjrTxbyeb-p7QrhJJ3mwRAECCMA@mail.gmail.com>
	<CAAS2fgROykenGH43_FXun+q4u+=94tKqnRW=kQQ2AQFeXKdW=Q@mail.gmail.com>
From: Suhas Daftuar <sdaftuar@gmail.com>
Date: Tue, 12 Dec 2017 16:07:11 -0500
Message-ID: <CAFp6fsHwdAbcYVdPzuDdD1OV7ibCN-ebMBi113m5-JOoJtccPQ@mail.gmail.com>
To: Gregory Maxwell <greg@xiph.org>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="94eb2c1916c6e08d0705602b085a"
Subject: Re: [bitcoin-dev] "Compressed" headers stream
Precedence: list

--94eb2c1916c6e08d0705602b085a
Content-Type: text/plain; charset="UTF-8"

Hi,

First, thanks for resurrecting this, I agree this is worth pursuing.

On Mon, Dec 11, 2017 at 4:04 PM, Gregory Maxwell via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
> Nbits _never_ needs to be sent even with other consensus rules because
> its more or less necessarily a strict function of the prior headers.
> This still holds in every clone of Bitcoin I'm aware of; sending it
> with the first header in a group probably makes sense so it can be
> checked independently.
>

I think it would be nice, though, to not require the consensus-correct
calculation of nBits in order to process p2p messages.  For instance, I
think there's a use for nBits at the p2p layer for calculating the work on
a chain, which can be used as an anti-DoS measure, even without verifying
that the difficulty adjustments are following the consensus rules.

Moreover I think it's a bit messy if the p2p layer depends on intricate
consensus rules in order to reconstruct a message -- either we'd need to
interact with existing consensus logic in a potentially new way, or we'd
reimplement the same logic in the p2p layer, neither of which is very
desirable imo.

But I think we should be able to get nearly all the benefit just by
including nBits in any messages where the value is ambiguous; ie we include
it with the first header in a message, and whenever it changes from the
previous header's nBits.

I would rather not change the serialization of existing messages,
> nodes are going to have to support speaking both messages for a long
> time, and I think we already want a different protocol flow for
> headers fetching in any case.
>

I agree with this.  Specifically the way I envisioned this working is that
we could introduce a new 'cmpctheaders'/'getcmpcthdrs' message pair for
syncing using this new message type, while leaving the existing
'headers'/'getheaders' messages unchanged.  So when communicating with
upgraded peers, we'd never use 'getheaders' messages, and we'd only use
'headers' messages for potentially announcing new blocks.

Of course, we'll have to support the existing protocol for a long time.
But one downside I've discovered from deploying BIP 130, which introduced
block announcements via headers messages, is that overloading a 'headers'
message to be either a block announcement or a response to a 'getheaders'
message results in p2p-handling logic which is more complicated than it
needs to be.  So splitting off the headers chain-sync functionality to a
new message pair seems like a nice side-effect benefit, in the long run.

--94eb2c1916c6e08d0705602b085a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">Hi,<=
/div><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">First,=
 thanks for resurrecting this, I agree this is worth pursuing.<br></div><di=
v class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">On Mon, Dec 11=
, 2017 at 4:04 PM, Gregory Maxwell via bitcoin-dev <span dir=3D"ltr">&lt;<a=
 href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bi=
tcoin-dev@lists.linuxfounda<wbr>tion.org</a>&gt;</span> wrote:<br><blockquo=
te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so=
lid;padding-left:1ex"><br>
Nbits _never_ needs to be sent even with other consensus rules because<br>
its more or less necessarily a strict function of the prior headers.<br>
This still holds in every clone of Bitcoin I&#39;m aware of; sending it<br>
with the first header in a group probably makes sense so it can be<br>
checked independently.<br></blockquote><div><br></div><div>I think it would=
 be nice, though, to not require the consensus-correct calculation of nBits=
 in order to process p2p messages.=C2=A0 For instance, I think there&#39;s =
a use for nBits at the p2p layer for calculating the work on a chain, which=
 can be used as an anti-DoS measure, even without verifying that the diffic=
ulty adjustments are following the consensus rules.</div><div><br></div><di=
v>Moreover I think it&#39;s a bit messy if the p2p layer depends on intrica=
te consensus rules in order to reconstruct a message -- either we&#39;d nee=
d to interact with existing consensus logic in a potentially new way, or we=
&#39;d reimplement the same logic in the p2p layer, neither of which is ver=
y desirable imo.</div><div><br></div><div>But I think we should be able to =
get nearly all the benefit just by including nBits in any messages where th=
e value is ambiguous; ie we include it with the first header in a message, =
and whenever it changes from the previous header&#39;s nBits.</div><div><br=
></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-=
left:1px #ccc solid;padding-left:1ex">I would rather not change the seriali=
zation of existing messages,<br>
nodes are going to have to support speaking both messages for a long<br>
time, and I think we already want a different protocol flow for<br>
headers fetching in any case.<br>
</blockquote><div><br></div><div>I agree with this.=C2=A0 Specifically the =
way I envisioned this working is that we could introduce a new &#39;cmpcthe=
aders&#39;/&#39;getcmpcthdrs&#39; message pair for syncing using this new m=
essage type, while leaving the existing &#39;headers&#39;/&#39;getheaders&#=
39; messages unchanged.=C2=A0 So when communicating with upgraded peers, we=
&#39;d never use &#39;getheaders&#39; messages, and we&#39;d only use &#39;=
headers&#39; messages for potentially announcing new blocks.</div><div><br>=
</div><div>Of course, we&#39;ll have to support the existing protocol for a=
 long time.=C2=A0 But one downside I&#39;ve discovered from deploying BIP 1=
30, which introduced block announcements via headers messages, is that over=
loading a &#39;headers&#39; message to be either a block announcement or a =
response to a &#39;getheaders&#39; message results in p2p-handling logic wh=
ich is more complicated than it needs to be.=C2=A0 So splitting off the hea=
ders chain-sync functionality to a new message pair seems like a nice side-=
effect benefit, in the long run.</div><div><br></div></div></div></div>

--94eb2c1916c6e08d0705602b085a--