1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
|
Return-Path: <riccardo.casatta@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id E6D9B98A
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 16:25:24 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f175.google.com (mail-io0-f175.google.com
[209.85.223.175])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F10FA196
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 16:25:22 +0000 (UTC)
Received: by mail-io0-f175.google.com with SMTP id n71so3255224iod.1
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 09:25:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:in-reply-to:references:from:date:message-id:subject:to
:cc; bh=uNp0PMXH5fiVEXgCgnSg39vGLwP6hvHPF58Drzbt3ME=;
b=A2NDrYsMrYR8TWxabOzYYRTs9P4m0WqyPMD8YqfX46iWdafF127TVMV1Z68GlFaVTk
wdtqBRjjS06v5krv4dBXXAKtPlLX/7hlk78nzwS1ZDuiFrjt/Zgc5dzwVg3z2nXfeZsd
sOI6a0YpthrwwNwnYsZK0EXQOL/E5HYtLstkM45qPQTXgwkLW3uVUqVlBcbUBEi1eStW
3xqO8BHx3lQ2u0k1YryB6VWceQaHvWJ3gGnPep7mNiJc1+LIIYDHeLmo6WY9RdNANJI9
frzvqzpib9A39W7E/0Cj5wnnY401rz8dDOneNdC4pnH9/V3rGaRTypQPaJKvWmPNiRnc
sd+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to:cc;
bh=uNp0PMXH5fiVEXgCgnSg39vGLwP6hvHPF58Drzbt3ME=;
b=lESf8ywMGOVfDCphSil+wMFR+kmjJO87Xl8Klo1oO6dDoFscCp7bAuXrsDreSW3LaC
3uvZD5gGdhlkaFLyY5jRJmd+RbDLwpie32d5+QeRglQ8iqUxQ09zZ40i/ttm6yfUq6jH
w2gEkevUrz5ePQDo8CC2AE286yZ/8Og/Y0mZ+38miYcMamCHu7N0L1yyHE/mkMJhWfmG
7DtoUM+BFAe+K61eP6IfNGqbife5gNMPcAMa04XQbGl7/lJpqywjEaEbZnuUnHnjQx1Q
yqtniHu41sIl+GHWwyN+WYdK11TarCIMrqufzZpCBXaCCZh4V8BfyPXQZaBvlw62b6mh
5FsQ==
X-Gm-Message-State: AHYfb5hhqlZCcE4h/etVVNDrXknHz9GmWky1ZYzorfgIt0Dhisfnz5Pk
Iqosf5A9OMD33XxWNXLxcX4m4D/aDw==
X-Received: by 10.107.59.23 with SMTP id i23mr993153ioa.345.1503937522194;
Mon, 28 Aug 2017 09:25:22 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.2.120.29 with HTTP; Mon, 28 Aug 2017 09:25:01 -0700 (PDT)
In-Reply-To: <CAB3F3DuK-5Bs-NunBVBnNbAT3SCVBZEqJqRHUHsSZhCVeO8xEQ@mail.gmail.com>
References: <CADabwBBrrPM2f9h_sgxY12tg=FUvKKCcnCC8ixnct93YL9uEFQ@mail.gmail.com>
<CAB3F3DuK-5Bs-NunBVBnNbAT3SCVBZEqJqRHUHsSZhCVeO8xEQ@mail.gmail.com>
From: Riccardo Casatta <riccardo.casatta@gmail.com>
Date: Mon, 28 Aug 2017 18:25:01 +0200
Message-ID: <CADabwBDQ=aJuW7fGcU2h-yYKxfj5A0Vx6DNHEM=_ppMrdA_mcw@mail.gmail.com>
To: Greg Sanders <gsanders87@gmail.com>
Content-Type: multipart/alternative; boundary="94eb2c05e2ecc660f50557d2bd0b"
X-Spam-Status: No, score=0.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,
RCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
X-Mailman-Approved-At: Mon, 28 Aug 2017 16:27:32 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] "Compressed" headers stream
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Aug 2017 16:25:25 -0000
--94eb2c05e2ecc660f50557d2bd0b
Content-Type: text/plain; charset="UTF-8"
2017-08-28 18:13 GMT+02:00 Greg Sanders <gsanders87@gmail.com>:
> Is there any reason to believe that you need Bitcoin "full security" at
> all for timestamping?
>
This is a little bit out of the main topic of the email which is the
savings in bandwidth in transmitting headers, any comment about that?
P.S. As a personal experience timestamping is nowadays used to prove date
and integrity of private databases containing a lot of value, so yes, in
that cases I will go with Bitcoin "full security"
>
> On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Hi everyone,
>>
>> the Bitcoin headers are probably the most condensed and important piece
>> of data in the world, their demand is expected to grow.
>>
>> When sending a stream of continuous block headers, a common case in IBD
>> and in disconnected clients, I think there is a possible optimization of
>> the transmitted data:
>> The headers after the first could avoid transmitting the previous hash
>> cause the receiver could compute it by double hashing the previous header
>> (an operation he needs to do anyway to verify PoW).
>> In a long stream, for example 2016 headers, the savings in bandwidth are
>> about 32/80 ~= 40%
>> without compressed headers 2016*80=161280 bytes
>> with compressed headers 80+2015*48=96800 bytes
>>
>> What do you think?
>>
>>
>> In OpenTimestamps calendars we are going to use this compression to give
>> lite-client a reasonable secure proofs (a full node give higher security
>> but isn't feasible in all situations, for example for in-browser
>> verification)
>> To speed up sync of a new client Electrum starts with the download of a
>> file <https://headers.electrum.org/blockchain_headers> ~36MB containing
>> the first 477637 headers.
>> For this kind of clients could be useful a common http API with fixed
>> position chunks to leverage http caching. For example /headers/2016/0
>> returns the headers from the genesis to the 2015 header included while
>> /headers/2016/1 gives the headers from the 2016th to the 4031.
>> Other endpoints could have chunks of 20160 blocks or 201600 such that
>> with about 10 http requests a client could fast sync the headers
>>
>>
>> --
>> Riccardo Casatta - @RCasatta <https://twitter.com/RCasatta>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
--
Riccardo Casatta - @RCasatta <https://twitter.com/RCasatta>
--94eb2c05e2ecc660f50557d2bd0b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">=
2017-08-28 18:13 GMT+02:00 Greg Sanders <span dir=3D"ltr"><<a href=3D"ma=
ilto:gsanders87@gmail.com" target=3D"_blank">gsanders87@gmail.com</a>></=
span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8=
ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr=
">Is there any reason to believe that you need Bitcoin "full security&=
quot; at all for timestamping?</div></blockquote><div><br></div><div>This i=
s a little bit out of the main topic of the email which is the savings in b=
andwidth in transmitting headers, any comment about that?</div><div><br></d=
iv><div><br></div><div>P.S. As a personal experience timestamping is nowada=
ys used to prove date and integrity of private databases containing a lot o=
f value, so yes, in that cases I will go with Bitcoin "full security&q=
uot;</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex=
"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote"><div><div class=
=3D"gmail-h5">On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoi=
n-dev <span dir=3D"ltr"><<a href=3D"mailto:bitcoin-dev@lists.linuxfounda=
tion.org" target=3D"_blank">bitcoin-dev@lists.<wbr>linuxfoundation.org</a>&=
gt;</span> wrote:<br></div></div><blockquote class=3D"gmail_quote" style=3D=
"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-le=
ft:1ex"><div><div class=3D"gmail-h5"><div dir=3D"ltr"><div>Hi everyone,</di=
v><div><br></div><div>the Bitcoin headers are probably the most condensed a=
nd important piece of data in the world, their demand is expected to grow.<=
/div><div><br></div><div>When sending a stream of continuous block headers,=
a common case in IBD and in disconnected clients, I think there is a possi=
ble optimization of the transmitted data:</div><div>The headers after the f=
irst could avoid transmitting the previous hash cause the receiver could co=
mpute it by double hashing the previous header (an operation he needs to do=
anyway to verify PoW).</div><div>In a long stream, for example 2016 header=
s, the savings in bandwidth are about 32/80 ~=3D 40%</div><div>without comp=
ressed headers 2016*80=3D161280 bytes</div><div>with compressed headers 80+=
2015*48=3D96800 bytes</div><div><br></div><div>What do you think?</div><div=
><br></div><div><br></div><div>In OpenTimestamps calendars we are going to =
use this compression to give lite-client a reasonable secure proofs (a full=
node give higher security but isn't feasible in all situations, for ex=
ample for in-browser verification)</div><div>To speed up sync of a new clie=
nt Electrum starts with the download of a <a href=3D"https://headers.electr=
um.org/blockchain_headers" target=3D"_blank">file</a>=C2=A0~36MB containing=
the first 477637 headers.=C2=A0</div><div>For this kind of clients could b=
e useful a common http API with fixed position chunks to leverage http cach=
ing. For example /headers/2016/0 returns the headers from the genesis to th=
e 2015 header included while /headers/2016/1 gives the headers from the 201=
6th to the 4031.</div><div>Other endpoints could have chunks of 20160 block=
s or 201600 such that with about 10 http requests a client could fast sync =
the headers</div><span class=3D"gmail-m_-937381146659249381HOEnZb"><font co=
lor=3D"#888888"><br clear=3D"all"><div><br></div>-- <br><div class=3D"gmail=
-m_-937381146659249381m_6184025682653505407gmail_signature"><div dir=3D"ltr=
">Riccardo Casatta - <a href=3D"https://twitter.com/RCasatta" target=3D"_bl=
ank">@RCasatta</a></div></div>
</font></span></div>
<br></div></div>______________________________<wbr>_________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundat<wbr>ion.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.<wbr>org=
/mailman/listinfo/bitcoin-d<wbr>ev</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature"><div dir=3D"ltr">Riccardo Casatta - <a href=3D"https:/=
/twitter.com/RCasatta" target=3D"_blank">@RCasatta</a></div></div>
</div></div>
--94eb2c05e2ecc660f50557d2bd0b--
|
