MIME-Version: 1.0
Sender: gmaxwell@gmail.com
In-Reply-To: <CAAS2fgS3uG=4vgFuObPKA_5MstoGm4AabO=60fhV3EU_0dvejg@mail.gmail.com>
References: <CADabwBBrrPM2f9h_sgxY12tg=FUvKKCcnCC8ixnct93YL9uEFQ@mail.gmail.com>
	<CAAS2fgS3uG=4vgFuObPKA_5MstoGm4AabO=60fhV3EU_0dvejg@mail.gmail.com>
From: Gregory Maxwell <greg@xiph.org>
Date: Mon, 28 Aug 2017 17:12:15 +0000
Message-ID: <CAAS2fgRvSZm-NVLU++0WmWoaYbpX1R0Fqmv_Jf7a_RsqzXfOog@mail.gmail.com>
To: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: text/plain; charset="UTF-8"
Subject: [bitcoin-dev] Fwd:  "Compressed" headers stream
Precedence: list

On Mon, Aug 28, 2017 at 3:50 PM, Riccardo Casatta via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi everyone,
>
> the Bitcoin headers are probably the most condensed and important piece of
> data in the world, their demand is expected to grow.
>
> When sending a stream of continuous block headers, a common case in IBD and
> in disconnected clients, I think there is a possible optimization of the
> transmitted data:
> The headers after the first could avoid transmitting the previous hash cause
> the receiver could compute it by double hashing the previous header (an
> operation he needs to do anyway to verify PoW).
> In a long stream, for example 2016 headers, the savings in bandwidth are
> about 32/80 ~= 40%
> without compressed headers 2016*80=161280 bytes
> with compressed headers 80+2015*48=96800 bytes
>
> What do you think?

You are leaving a lot of bytes on the table.

The bits field can only change every 2016 blocks (4 bytes per header),
the timestamp can not be less than the median of the last 11 and is
usually only a small amount over the last one (saves 2 bytes per
header), the block version is usually one of the last few (save 3
bytes per header).

But all these things improvements are just a constant factor. I think
you want the compact SPV proofs described in the appendix of the
sidechains whitepaper which creates log scaling proofs.