Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 46131AB8 for ; Mon, 28 Aug 2017 17:12:18 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f177.google.com (mail-ua0-f177.google.com [209.85.217.177]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 255F141D for ; Mon, 28 Aug 2017 17:12:17 +0000 (UTC) Received: by mail-ua0-f177.google.com with SMTP id 104so3439195uas.1 for ; Mon, 28 Aug 2017 10:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=Co61hHYk4L01bYLISfpQVO/tcRPFa3i24q4UQXfe+lU=; b=kDnfHAQu+UTXC8EfDykeRhQC1ShhMRF4rbQwbMXsTYfTDKpNeOZSyZrAE7d69qNlZx zWMv/PojxNakq7nePiS/O4Fr1LFmg2Jdw7+dGPlZEHEh2jTc1MxYwX95h4UuNrYZV8Fo h5CieRKyPIgzyd9rvePzeW8zPCigculUTnpLS3ntDH0PicHWAwpnToghq3QOfDBAdPZj TD26fD6gV5UebjHbH4L3hVv4dmr43RDrT9iPGIFJrZyA+ZXrFrKJzXoZMsBxc0TNQ45p 0jachBlYiEWtVhGg4N+3nGUMvJqUCWPZRcMzrtg+mi4p7cB+v+CJ+gt+0LFP4SBTm2te l2fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=Co61hHYk4L01bYLISfpQVO/tcRPFa3i24q4UQXfe+lU=; b=VbMa67oa2rNbhXoqCRVyy8ZjnOu9/xcgSljPwalxCQgBQVPdAv1QBQl/wOz7Hn9XHO wUXsvB8GkL3STj8eohRxaLj1/UwFZvCNQo/6AC+0C5eMIyhpcGHG1PRrlmkjLJ4I4ChS 6iYVmQnee0bDc3kryMdnzSeNngeGcgtHeqNnx+Ne42ph4U3Zn4iERnfXGFD5lhV3XEjx cJyU8KWSrQLr4pMvOAfvn7T/1rUXDY5f4mrAhwBO2TYQWlDPbwbHzDgOErTbJAyk9Atx hrbON25+Wvmor9VY9UYomyDKixCG+rxsSsaRRiXDpa7WOtKXUC9uc8j/jCQkvz7ZpNPH et9A== X-Gm-Message-State: AHPjjUj/1IdSSqxLrRV72MhBhN4kci2JaJADwZwSJ6+NJ28WSZ6Jx8F9 gOf+R4kFdSys3p5wACm9k2RW6jYWGw== X-Received: by 10.176.79.218 with SMTP id t26mr813361uah.18.1503940336136; Mon, 28 Aug 2017 10:12:16 -0700 (PDT) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 10.103.176.71 with HTTP; Mon, 28 Aug 2017 10:12:15 -0700 (PDT) In-Reply-To: References: From: Gregory Maxwell Date: Mon, 28 Aug 2017 17:12:15 +0000 X-Google-Sender-Auth: as1W06rnpeUPQ802xosJMxuBEpg Message-ID: To: Bitcoin Dev Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=0.5 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [bitcoin-dev] Fwd: "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Aug 2017 17:12:18 -0000 On Mon, Aug 28, 2017 at 3:50 PM, Riccardo Casatta via bitcoin-dev wrote: > Hi everyone, > > the Bitcoin headers are probably the most condensed and important piece of > data in the world, their demand is expected to grow. > > When sending a stream of continuous block headers, a common case in IBD and > in disconnected clients, I think there is a possible optimization of the > transmitted data: > The headers after the first could avoid transmitting the previous hash cause > the receiver could compute it by double hashing the previous header (an > operation he needs to do anyway to verify PoW). > In a long stream, for example 2016 headers, the savings in bandwidth are > about 32/80 ~= 40% > without compressed headers 2016*80=161280 bytes > with compressed headers 80+2015*48=96800 bytes > > What do you think? You are leaving a lot of bytes on the table. The bits field can only change every 2016 blocks (4 bytes per header), the timestamp can not be less than the median of the last 11 and is usually only a small amount over the last one (saves 2 bytes per header), the block version is usually one of the last few (save 3 bytes per header). But all these things improvements are just a constant factor. I think you want the compact SPV proofs described in the appendix of the sidechains whitepaper which creates log scaling proofs.