1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
|
Return-Path: <riccardo.casatta@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 3794DBC5
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 30 Mar 2018 08:06:46 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f177.google.com (mail-io0-f177.google.com
[209.85.223.177])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 86F615F4
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 30 Mar 2018 08:06:45 +0000 (UTC)
Received: by mail-io0-f177.google.com with SMTP id b20so10375982iof.5
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 30 Mar 2018 01:06:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:in-reply-to:references:from:date:message-id:subject:to
:cc; bh=fZfnW+2HObHwVYtPkWki6fTNBbXEBNmg0IYRPJ2Fq9U=;
b=K/5qbvPCvX6zyEwnjRxN35kf3Uw9odUjqJExKd5ob2LFbLdz+bvVRqz8gL+TvPBGn9
DRmT4Fo3sOJUqOWH/4vfo6ixzJdWLmrc191vcxOwILdscFaYJkju3qR9rzpGDrLWIekj
5fSLZPtH5Wke0Z+B6Kbj8U5PkuGv3bX6ywYOtEyBQVS86eTl0q84JUL+WMgYsy8YJQ4C
cSmsjFzb7T9O9uAcGBxe9672zYbjrlMngHgMceNp5EHBvxnPe5XWuMknC8omMa33zkEn
LVN2bGd6r+PJWNkg9qSmULccJhUS2aEXh5QLAyX73x+jnhXI8zDK5ntVUYDvwN9zWE4b
C7Ig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to:cc;
bh=fZfnW+2HObHwVYtPkWki6fTNBbXEBNmg0IYRPJ2Fq9U=;
b=Ar3YzCc4+4VqxrFRh8a+HbjkzPl6Bh0cSSibxeEATMLkVUhfHgTCG0Hivemv2D2v5N
3pVtqlocrGZOma95rJWX+TeRpAsXwQbzC1ttFRdLr3KR3xpRExE5oMlRPc/S58/pP/0P
Pgb8BgGNMk4eRsaof2nn2ltrq+joqlUowI/IXYXFEprh+jJD4b/VF3HD1dDU2UwzgwRz
BQOtSH/vxTkK/jU5FfTCAIDDZEtcluEB1gpTCNEYRZ2KH/XK4NEwp/5kbMlvaZwTnhx5
+qhOQ7502BLVq+iQzpkInxyenmcVQJc4b0P66L7ch3xmFS3/tWyZKNJkyaQHbk9F3FNe
7TZw==
X-Gm-Message-State: AElRT7EJPcuBqrBkpsZ/V7oWvDFqM2WS7y6b8/qaT34YSUB5BpXGBpRs
uxL2H/q9B19vQmU48ueRYHrHYuqohbR2TPk+e/8=
X-Google-Smtp-Source: AG47ELsLktS/R/aU9LZXiONyWK9lTfejQx03AiiZw8Ft/a5zu713p9gBBSO/URA8gmqNTPGN/PB8e37ldB37Wg5vTWs=
X-Received: by 10.107.11.204 with SMTP id 73mr55020905iol.25.1522397204750;
Fri, 30 Mar 2018 01:06:44 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.2.46.32 with HTTP; Fri, 30 Mar 2018 01:06:24 -0700 (PDT)
In-Reply-To: <20180330061418.GA6017@erisian.com.au>
References: <CADZtCSg7+x-sg-ysgacXobRexOVwT+k9fr6a9S-6xU2w8f8m3A@mail.gmail.com>
<CADabwBAjTRdVqsL+V=YdQ+kr8+LtSPOeSXUJOzKoPNdKEbAOWQ@mail.gmail.com>
<CADZtCSjmQfBZoaO=MCyRoEn-AYe4A=1kDhxSVxVMw+O4k7YJfQ@mail.gmail.com>
<20180330061418.GA6017@erisian.com.au>
From: Riccardo Casatta <riccardo.casatta@gmail.com>
Date: Fri, 30 Mar 2018 10:06:24 +0200
Message-ID: <CADabwBDiT2zNPHZ2=OyvCVrSY3Km2oyTnhRCHyMNsjW2vLMmOg@mail.gmail.com>
To: Anthony Towns <aj@erisian.com.au>
Content-Type: multipart/alternative; boundary="001a113de7a498a89f05689cb8c5"
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
X-Mailman-Approved-At: Sun, 01 Apr 2018 14:27:14 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Optimized Header Sync
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Mar 2018 08:06:46 -0000
--001a113de7a498a89f05689cb8c5
Content-Type: text/plain; charset="UTF-8"
Yes, I think the checkpoints and the compressed headers streams should be
handled in chunks of 2016 headers and queried by chunk number instead of
height, falling back to current method if the chunk is not full yet.
This is cache friendly and allows to avoid bit 0 and bit 1 in the bitfield
(because they are always 1 after the first header in the chunk of 2016).
2018-03-30 8:14 GMT+02:00 Anthony Towns <aj@erisian.com.au>:
> On Thu, Mar 29, 2018 at 05:50:30PM -0700, Jim Posen via bitcoin-dev wrote:
> > Taken a step further though, I'm really interested in treating the
> checkpoints
> > as commitments to chain work [...]
>
> In that case, shouldn't the checkpoints just be every 2016 blocks and
> include the corresponding bits value for that set of blocks?
>
> That way every node commits to (approximately) how much work their entire
> chain has by sending something like 10kB of data (currently), and you
> could verify the deltas in each node's chain's target by downloading the
> 2016 headers between those checkpoints (~80kB with the proposed compact
> encoding?) and checking the timestamps and proof of work match both the
> old target and the new target from adjacent checkpoints.
>
> (That probably still works fine even if there's a hardfork that allows
> difficulty to adjust more frequently: a bits value at block n*2016 will
> still enforce *some* lower limit on how much work blocks n*2016+{1..2016}
> will have to contribute; so will still allow you to estimate how much work
> will have been done, it may just be less precise than the estimate you
> could
> generate now)
>
> Cheers,
> aj
>
>
--
Riccardo Casatta - @RCasatta <https://twitter.com/RCasatta>
--001a113de7a498a89f05689cb8c5
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>Yes, I think the checkpoints and the compressed heade=
rs streams should be handled in chunks of 2016 headers and queried by chunk=
number instead of height, falling back to current method if the chunk is n=
ot full yet.</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_=
extra">This is cache friendly and allows to avoid bit 0 and bit 1 in the bi=
tfield (because they are always 1 after the first header in the chunk of 20=
16).</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><=
div class=3D"gmail_quote">2018-03-30 8:14 GMT+02:00 Anthony Towns <span dir=
=3D"ltr"><<a href=3D"mailto:aj@erisian.com.au" target=3D"_blank">aj@eris=
ian.com.au</a>></span>:<br><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=3D=
"">On Thu, Mar 29, 2018 at 05:50:30PM -0700, Jim Posen via bitcoin-dev wrot=
e:<br>
> Taken a step further though, I'm really interested in treating the=
checkpoints<br>
</span>> as commitments to chain work [...]<br>
<br>
In that case, shouldn't the checkpoints just be every 2016 blocks and<b=
r>
include the corresponding bits value for that set of blocks?<br>
<br>
That way every node commits to (approximately) how much work their entire<b=
r>
chain has by sending something like 10kB of data (currently), and you<br>
could verify the deltas in each node's chain's target by downloadin=
g the<br>
2016 headers between those checkpoints (~80kB with the proposed compact<br>
encoding?) and checking the timestamps and proof of work match both the<br>
old target and the new target from adjacent checkpoints.<br>
<br>
(That probably still works fine even if there's a hardfork that allows<=
br>
difficulty to adjust more frequently: a bits value at block n*2016 will<br>
still enforce *some* lower limit on how much work blocks n*2016+{1..2016}<b=
r>
will have to contribute; so will still allow you to estimate how much work<=
br>
will have been done, it may just be less precise than the estimate you coul=
d<br>
generate now)<br>
<br>
Cheers,<br>
aj<br>
<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr">Ri=
ccardo Casatta - <a href=3D"https://twitter.com/RCasatta" target=3D"_blank"=
>@RCasatta</a></div></div>
</div></div>
--001a113de7a498a89f05689cb8c5--
|
