Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E6D9B98A for ; Mon, 28 Aug 2017 16:25:24 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-io0-f175.google.com (mail-io0-f175.google.com [209.85.223.175]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F10FA196 for ; Mon, 28 Aug 2017 16:25:22 +0000 (UTC) Received: by mail-io0-f175.google.com with SMTP id n71so3255224iod.1 for ; Mon, 28 Aug 2017 09:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uNp0PMXH5fiVEXgCgnSg39vGLwP6hvHPF58Drzbt3ME=; b=A2NDrYsMrYR8TWxabOzYYRTs9P4m0WqyPMD8YqfX46iWdafF127TVMV1Z68GlFaVTk wdtqBRjjS06v5krv4dBXXAKtPlLX/7hlk78nzwS1ZDuiFrjt/Zgc5dzwVg3z2nXfeZsd sOI6a0YpthrwwNwnYsZK0EXQOL/E5HYtLstkM45qPQTXgwkLW3uVUqVlBcbUBEi1eStW 3xqO8BHx3lQ2u0k1YryB6VWceQaHvWJ3gGnPep7mNiJc1+LIIYDHeLmo6WY9RdNANJI9 frzvqzpib9A39W7E/0Cj5wnnY401rz8dDOneNdC4pnH9/V3rGaRTypQPaJKvWmPNiRnc sd+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uNp0PMXH5fiVEXgCgnSg39vGLwP6hvHPF58Drzbt3ME=; b=lESf8ywMGOVfDCphSil+wMFR+kmjJO87Xl8Klo1oO6dDoFscCp7bAuXrsDreSW3LaC 3uvZD5gGdhlkaFLyY5jRJmd+RbDLwpie32d5+QeRglQ8iqUxQ09zZ40i/ttm6yfUq6jH w2gEkevUrz5ePQDo8CC2AE286yZ/8Og/Y0mZ+38miYcMamCHu7N0L1yyHE/mkMJhWfmG 7DtoUM+BFAe+K61eP6IfNGqbife5gNMPcAMa04XQbGl7/lJpqywjEaEbZnuUnHnjQx1Q yqtniHu41sIl+GHWwyN+WYdK11TarCIMrqufzZpCBXaCCZh4V8BfyPXQZaBvlw62b6mh 5FsQ== X-Gm-Message-State: AHYfb5hhqlZCcE4h/etVVNDrXknHz9GmWky1ZYzorfgIt0Dhisfnz5Pk Iqosf5A9OMD33XxWNXLxcX4m4D/aDw== X-Received: by 10.107.59.23 with SMTP id i23mr993153ioa.345.1503937522194; Mon, 28 Aug 2017 09:25:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.120.29 with HTTP; Mon, 28 Aug 2017 09:25:01 -0700 (PDT) In-Reply-To: References: From: Riccardo Casatta Date: Mon, 28 Aug 2017 18:25:01 +0200 Message-ID: To: Greg Sanders Content-Type: multipart/alternative; boundary="94eb2c05e2ecc660f50557d2bd0b" X-Spam-Status: No, score=0.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Mon, 28 Aug 2017 16:27:32 +0000 Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Aug 2017 16:25:25 -0000 --94eb2c05e2ecc660f50557d2bd0b Content-Type: text/plain; charset="UTF-8" 2017-08-28 18:13 GMT+02:00 Greg Sanders : > Is there any reason to believe that you need Bitcoin "full security" at > all for timestamping? > This is a little bit out of the main topic of the email which is the savings in bandwidth in transmitting headers, any comment about that? P.S. As a personal experience timestamping is nowadays used to prove date and integrity of private databases containing a lot of value, so yes, in that cases I will go with Bitcoin "full security" > > On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> Hi everyone, >> >> the Bitcoin headers are probably the most condensed and important piece >> of data in the world, their demand is expected to grow. >> >> When sending a stream of continuous block headers, a common case in IBD >> and in disconnected clients, I think there is a possible optimization of >> the transmitted data: >> The headers after the first could avoid transmitting the previous hash >> cause the receiver could compute it by double hashing the previous header >> (an operation he needs to do anyway to verify PoW). >> In a long stream, for example 2016 headers, the savings in bandwidth are >> about 32/80 ~= 40% >> without compressed headers 2016*80=161280 bytes >> with compressed headers 80+2015*48=96800 bytes >> >> What do you think? >> >> >> In OpenTimestamps calendars we are going to use this compression to give >> lite-client a reasonable secure proofs (a full node give higher security >> but isn't feasible in all situations, for example for in-browser >> verification) >> To speed up sync of a new client Electrum starts with the download of a >> file ~36MB containing >> the first 477637 headers. >> For this kind of clients could be useful a common http API with fixed >> position chunks to leverage http caching. For example /headers/2016/0 >> returns the headers from the genesis to the 2015 header included while >> /headers/2016/1 gives the headers from the 2016th to the 4031. >> Other endpoints could have chunks of 20160 blocks or 201600 such that >> with about 10 http requests a client could fast sync the headers >> >> >> -- >> Riccardo Casatta - @RCasatta >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> > -- Riccardo Casatta - @RCasatta --94eb2c05e2ecc660f50557d2bd0b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

= 2017-08-28 18:13 GMT+02:00 Greg Sanders <gsanders87@gmail.com>:
Is there any reason to believe that you need Bitcoin "full security&= quot; at all for timestamping?

This i= s a little bit out of the main topic of the email which is the savings in b= andwidth in transmitting headers, any comment about that?


P.S. As a personal experience timestamping is nowada= ys used to prove date and integrity of private databases containing a lot o= f value, so yes, in that cases I will go with Bitcoin "full security&q= uot;
=C2=A0

On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoi= n-dev <bitcoin-dev@lists.linuxfoundation.org&= gt; wrote:
Hi everyone,

the Bitcoin headers are probably the most condensed a= nd important piece of data in the world, their demand is expected to grow.<= /div>

When sending a stream of continuous block headers,= a common case in IBD and in disconnected clients, I think there is a possi= ble optimization of the transmitted data:
The headers after the f= irst could avoid transmitting the previous hash cause the receiver could co= mpute it by double hashing the previous header (an operation he needs to do= anyway to verify PoW).
In a long stream, for example 2016 header= s, the savings in bandwidth are about 32/80 ~=3D 40%
without comp= ressed headers 2016*80=3D161280 bytes
with compressed headers 80+= 2015*48=3D96800 bytes

What do you think?


In OpenTimestamps calendars we are going to = use this compression to give lite-client a reasonable secure proofs (a full= node give higher security but isn't feasible in all situations, for ex= ample for in-browser verification)
To speed up sync of a new clie= nt Electrum starts with the download of a file=C2=A0~36MB containing= the first 477637 headers.=C2=A0
For this kind of clients could b= e useful a common http API with fixed position chunks to leverage http cach= ing. For example /headers/2016/0 returns the headers from the genesis to th= e 2015 header included while /headers/2016/1 gives the headers from the 201= 6th to the 4031.
Other endpoints could have chunks of 20160 block= s or 201600 such that with about 10 http requests a client could fast sync = the headers


--
Riccardo Casatta - @RCasatta

_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev





--
Riccardo Casatta - @RCasatta
--94eb2c05e2ecc660f50557d2bd0b--