1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
|
Return-Path: <gsanders87@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 026E6A85
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 16:13:35 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 06BE1F6
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 16:13:32 +0000 (UTC)
Received: by mail-wm0-f48.google.com with SMTP id y71so6232391wmd.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 28 Aug 2017 09:13:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:in-reply-to:references:from:date:message-id:subject:to;
bh=ezrit99F0ScZrwNtrAD0CKVRcL3JdrJH/1moi1rJ9aA=;
b=i4VnhmmlUCcwOxvcOwtUB65qIYhYS+S5nsCixTdvknZuwDwXztjPkC81jczLSBaJxx
sHBUYdIOICqIv+jkVPVqGvuHr+3izF764r8+upJeuZ4g+/b2J7bVDAIP0QjSkWw5AnBo
p6lRifayUPSkNSnRw+O368oB/OAOn+V1Y3A5CftZmWnMPErcoJspudK/tuf6arlLUfJB
+Wi0WuL7DrqTUS5W2tte6NCpzVEjbA5h2iJ3HwLver29uBMwhFdBVHFv+K1bEVXAgY9s
Sgmn6MD0YjRGdm7lH5vSWS9qjpKsJ60Em/PefrdCS3Nhrb8VXSMnQQSluv4FbrLkaZok
by+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to;
bh=ezrit99F0ScZrwNtrAD0CKVRcL3JdrJH/1moi1rJ9aA=;
b=hjETjCqxWoDix1cKZ2ebQ1uMtEQPdNqMRjkHsPzrg23Q071/zGN23tLkXlDhadDX7J
6aYZbDcy8kAmPAGMHthbFtDwTWtYU1mRCJ7rq/eTwgmKnS+lOZNfBudvCTDa9hFS27QQ
VA/AKRbYyEWHpZGkZt9QZA3tQjnfqsJYKOl2z9/jCHDqv8A8NZikWUen6zxqmGlvdUV0
hlrTmg1liQyifhh9deJqTEBIfjVjzA1nPrvMnVR/VAksffxbDcvZQBeO5/o6z9zOOdbD
kqUD3FDgHBlDD3Dkp6c+HgPDKrJOGv3tPORsilOmj0AVcZ4boivAYr87nEL/5Im6vB8N
+oJQ==
X-Gm-Message-State: AHYfb5gYyyuvOD/vGyCokf2FY/9mbwcw3Bf35CbCJjORpeKUc5+rJuA8
UlZOJ8yFuXhvYKHrSJSVqJWZb3N5cQ==
X-Received: by 10.80.146.57 with SMTP id i54mr938929eda.170.1503936811635;
Mon, 28 Aug 2017 09:13:31 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.80.129.163 with HTTP; Mon, 28 Aug 2017 09:13:11 -0700 (PDT)
In-Reply-To: <CADabwBBrrPM2f9h_sgxY12tg=FUvKKCcnCC8ixnct93YL9uEFQ@mail.gmail.com>
References: <CADabwBBrrPM2f9h_sgxY12tg=FUvKKCcnCC8ixnct93YL9uEFQ@mail.gmail.com>
From: Greg Sanders <gsanders87@gmail.com>
Date: Mon, 28 Aug 2017 12:13:11 -0400
Message-ID: <CAB3F3DuK-5Bs-NunBVBnNbAT3SCVBZEqJqRHUHsSZhCVeO8xEQ@mail.gmail.com>
To: Riccardo Casatta <riccardo.casatta@gmail.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="f403045c2a666c19d10557d293bf"
X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
DKIM_VALID_AU,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,
RCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] "Compressed" headers stream
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Aug 2017 16:13:35 -0000
--f403045c2a666c19d10557d293bf
Content-Type: text/plain; charset="UTF-8"
Is there any reason to believe that you need Bitcoin "full security" at all
for timestamping?
On Mon, Aug 28, 2017 at 11:50 AM, Riccardo Casatta via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi everyone,
>
> the Bitcoin headers are probably the most condensed and important piece of
> data in the world, their demand is expected to grow.
>
> When sending a stream of continuous block headers, a common case in IBD
> and in disconnected clients, I think there is a possible optimization of
> the transmitted data:
> The headers after the first could avoid transmitting the previous hash
> cause the receiver could compute it by double hashing the previous header
> (an operation he needs to do anyway to verify PoW).
> In a long stream, for example 2016 headers, the savings in bandwidth are
> about 32/80 ~= 40%
> without compressed headers 2016*80=161280 bytes
> with compressed headers 80+2015*48=96800 bytes
>
> What do you think?
>
>
> In OpenTimestamps calendars we are going to use this compression to give
> lite-client a reasonable secure proofs (a full node give higher security
> but isn't feasible in all situations, for example for in-browser
> verification)
> To speed up sync of a new client Electrum starts with the download of a
> file <https://headers.electrum.org/blockchain_headers> ~36MB containing
> the first 477637 headers.
> For this kind of clients could be useful a common http API with fixed
> position chunks to leverage http caching. For example /headers/2016/0
> returns the headers from the genesis to the 2015 header included while
> /headers/2016/1 gives the headers from the 2016th to the 4031.
> Other endpoints could have chunks of 20160 blocks or 201600 such that with
> about 10 http requests a client could fast sync the headers
>
>
> --
> Riccardo Casatta - @RCasatta <https://twitter.com/RCasatta>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
--f403045c2a666c19d10557d293bf
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Is there any reason to believe that you need Bitcoin "=
;full security" at all for timestamping?</div><div class=3D"gmail_extr=
a"><br><div class=3D"gmail_quote">On Mon, Aug 28, 2017 at 11:50 AM, Riccard=
o Casatta via bitcoin-dev <span dir=3D"ltr"><<a href=3D"mailto:bitcoin-d=
ev@lists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoun=
dation.org</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=
=3D"ltr"><div>Hi everyone,</div><div><br></div><div>the Bitcoin headers are=
probably the most condensed and important piece of data in the world, thei=
r demand is expected to grow.</div><div><br></div><div>When sending a strea=
m of continuous block headers, a common case in IBD and in disconnected cli=
ents, I think there is a possible optimization of the transmitted data:</di=
v><div>The headers after the first could avoid transmitting the previous ha=
sh cause the receiver could compute it by double hashing the previous heade=
r (an operation he needs to do anyway to verify PoW).</div><div>In a long s=
tream, for example 2016 headers, the savings in bandwidth are about 32/80 ~=
=3D 40%</div><div>without compressed headers 2016*80=3D161280 bytes</div><d=
iv>with compressed headers 80+2015*48=3D96800 bytes</div><div><br></div><di=
v>What do you think?</div><div><br></div><div><br></div><div>In OpenTimesta=
mps calendars we are going to use this compression to give lite-client a re=
asonable secure proofs (a full node give higher security but isn't feas=
ible in all situations, for example for in-browser verification)</div><div>=
To speed up sync of a new client Electrum starts with the download of a <a =
href=3D"https://headers.electrum.org/blockchain_headers" target=3D"_blank">=
file</a>=C2=A0~36MB containing the first 477637 headers.=C2=A0</div><div>Fo=
r this kind of clients could be useful a common http API with fixed positio=
n chunks to leverage http caching. For example /headers/2016/0 returns the =
headers from the genesis to the 2015 header included while /headers/2016/1 =
gives the headers from the 2016th to the 4031.</div><div>Other endpoints co=
uld have chunks of 20160 blocks or 201600 such that with about 10 http requ=
ests a client could fast sync the headers</div><span class=3D"HOEnZb"><font=
color=3D"#888888"><br clear=3D"all"><div><br></div>-- <br><div class=3D"m_=
6184025682653505407gmail_signature"><div dir=3D"ltr">Riccardo Casatta - <a =
href=3D"https://twitter.com/RCasatta" target=3D"_blank">@RCasatta</a></div>=
</div>
</font></span></div>
<br>______________________________<wbr>_________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
<wbr>linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.<wbr>org=
/mailman/listinfo/bitcoin-<wbr>dev</a><br>
<br></blockquote></div><br></div>
--f403045c2a666c19d10557d293bf--
|
