1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
|
Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <gavinandresen@gmail.com>) id 1Yy07x-0001wU-NQ
for bitcoin-development@lists.sourceforge.net;
Thu, 28 May 2015 15:53:49 +0000
Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.215.46 as permitted sender)
client-ip=209.85.215.46; envelope-from=gavinandresen@gmail.com;
helo=mail-la0-f46.google.com;
Received: from mail-la0-f46.google.com ([209.85.215.46])
by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1Yy07w-0007aS-54
for bitcoin-development@lists.sourceforge.net;
Thu, 28 May 2015 15:53:49 +0000
Received: by labko7 with SMTP id ko7so31747879lab.2
for <bitcoin-development@lists.sourceforge.net>;
Thu, 28 May 2015 08:53:41 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.152.6.196 with SMTP id d4mr3574662laa.40.1432828421529; Thu,
28 May 2015 08:53:41 -0700 (PDT)
Received: by 10.25.90.75 with HTTP; Thu, 28 May 2015 08:53:41 -0700 (PDT)
In-Reply-To: <16096345.A1MpJQQkRW@crushinator>
References: <16096345.A1MpJQQkRW@crushinator>
Date: Thu, 28 May 2015 11:53:41 -0400
Message-ID: <CABsx9T3-zxCAagAS0megd06xvG5n-3tUL9NUK9TT3vt7XNL9Tg@mail.gmail.com>
From: Gavin Andresen <gavinandresen@gmail.com>
To: Matt Whitlock <bip@mattwhitlock.name>
Content-Type: multipart/alternative; boundary=089e0141a71816e6320517265d30
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(gavinandresen[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Yy07w-0007aS-54
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Proposed alternatives to the 20MB step
function
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 28 May 2015 15:53:49 -0000
--089e0141a71816e6320517265d30
Content-Type: text/plain; charset=UTF-8
On Fri, May 8, 2015 at 3:20 AM, Matt Whitlock <bip@mattwhitlock.name> wrote:
> Between all the flames on this list, several ideas were raised that did
> not get much attention. I hereby resubmit these ideas for consideration and
> discussion.
>
> - Perhaps the hard block size limit should be a function of the actual
> block sizes over some trailing sampling period. For example, take the
> median block size among the most recent 2016 blocks and multiply it by 1.5.
> This allows Bitcoin to scale up gradually and organically, rather than
> having human beings guessing at what is an appropriate limit.
>
A lot of people like this idea, or something like it. It is nice and
simple, which is really important for consensus-critical code.
With this rule in place, I believe there would be more "fee pressure"
(miners would be creating smaller blocks) today. I created a couple of
histograms of block sizes to infer what policy miners are ACTUALLY
following today with respect to block size:
Last 1,000 blocks:
http://bitcoincore.org/~gavin/sizes_last1000.html
Notice a big spike at 750K -- the default size for Bitcoin Core.
This graph might be misleading, because transaction volume or fees might
not be high enough over the last few days to fill blocks to whatever limit
miners are willing to mine.
So I graphed a time when (according to statoshi.info) there WERE a lot of
transactions waiting to be confirmed:
http://bitcoincore.org/~gavin/sizes_357511.html
That might also be misleading, because it is possible there were a lot of
transactions waiting to be confirmed because miners who choose to create
small blocks got lucky and found more blocks than normal. In fact, it
looks like that is what happened: more smaller-than-normal blocks were
found, and the memory pool backed up.
So: what if we had a dynamic maximum size limit based on recent history?
The average block size is about 400K, so a 1.5x rule would make the max
block size 600K; miners would definitely be squeezing out transactions /
putting pressure to increase transaction fees. Even a 2x rule (implying
800K max blocks) would, today, be squeezing out transactions / putting
pressure to increase fees.
Using a median size instead of an average means the size can increase or
decrease more quickly. For example, imagine the rule is "median of last
2016 blocks" and 49% of miners are producing 0-size blocks and 51% are
producing max-size blocks. The median is max-size, so the 51% have total
control over making blocks bigger. Swap the roles, and the median is
min-size.
Because of that, I think using an average is better-- it means the max size
will change (up or down) more slowly.
I also think 2016 blocks is too long, because transaction volumes change
quicker than that. An average over 144 blocks (last 24 hours) would be
better able to handle increased transaction volume around major holidays,
and would also be able to react more quickly if an economically irrational
attacker attempted to flood the network with fee-paying transactions.
So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.
There are a couple of other changes I'd pair with that consensus change:
+ Make the default mining policy for Bitcoin Core neutral-- have its target
block size be the average size, so miners that don't care will "go along
with the people who do care."
+ Use something like Greg's formula for size instead of bytes-on-the-wire,
to discourage bloating the UTXO set.
---------
When I've proposed (privately, to the other core committers) some dynamic
algorithm the objection has been "but that gives miners complete control
over the max block size."
I think that worry is unjustified right now-- certainly, until we have
size-independent new block propagation there is an incentive for miners to
keep their blocks small, and we see miners creating small blocks even when
there are fee-paying transactions waiting to be confirmed.
I don't even think it will be a problem if/when we do have size-independent
new block propagation, because I think the combination of the random timing
of block-finding plus a dynamic limit as described above will create a
healthy system.
If I'm wrong, then it seems to me the miners will have a very strong
incentive to, collectively, impose whatever rules are necessary (maybe a
soft-fork to put a hard cap on block size) to make the system healthy again.
--
--
Gavin Andresen
--089e0141a71816e6320517265d30
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On F=
ri, May 8, 2015 at 3:20 AM, Matt Whitlock <span dir=3D"ltr"><<a href=3D"=
mailto:bip@mattwhitlock.name" target=3D"_blank">bip@mattwhitlock.name</a>&g=
t;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);borde=
r-left-style:solid;padding-left:1ex">Between all the flames on this list, s=
everal ideas were raised that did not get much attention. I hereby resubmit=
these ideas for consideration and discussion.<br>
<br>
- Perhaps the hard block size limit should be a function of the actual bloc=
k sizes over some trailing sampling period. For example, take the median bl=
ock size among the most recent 2016 blocks and multiply it by 1.5. This all=
ows Bitcoin to scale up gradually and organically, rather than having human=
beings guessing at what is an appropriate limit.<br></blockquote><div><br>=
</div><div>A lot of people like this idea, or something like it. It is nice=
and simple, which is really important for consensus-critical code.</div><d=
iv><br></div><div>With this rule in place, I believe there would be more &q=
uot;fee pressure" (miners would be creating smaller blocks) today. I c=
reated a couple of histograms of block sizes to infer what policy miners ar=
e ACTUALLY following today with respect to block size:</div><div><br></div>=
<div>Last 1,000 blocks:</div><div>=C2=A0=C2=A0<a href=3D"http://bitcoincore=
.org/~gavin/sizes_last1000.html">http://bitcoincore.org/~gavin/sizes_last10=
00.html</a></div><div><br></div><div>Notice a big spike at 750K -- the defa=
ult size for Bitcoin Core.</div><div>This graph might be misleading, becaus=
e transaction volume or fees might not be high enough over the last few day=
s to fill blocks to whatever limit miners are willing to mine.<br></div><di=
v><br></div><div>So I graphed a time when (according to <a href=3D"http://s=
tatoshi.info">statoshi.info</a>) there WERE a lot of transactions waiting t=
o be confirmed:</div><div>=C2=A0 =C2=A0<a href=3D"http://bitcoincore.org/~g=
avin/sizes_357511.html">http://bitcoincore.org/~gavin/sizes_357511.html</a>=
</div><div><br></div><div>That might also be misleading, because it is poss=
ible there were a lot of transactions waiting to be confirmed because miner=
s who choose to create small blocks got lucky and found more blocks than no=
rmal.=C2=A0 In fact, it looks like that is what happened: more smaller-than=
-normal blocks were found, and the memory pool backed up.</div><div><br></d=
iv><div>So: what if we had a dynamic maximum size limit based on recent his=
tory?</div><div><br></div><div>The average block size is about 400K, so a 1=
.5x rule would make the max block size 600K; miners would definitely be squ=
eezing out transactions / putting pressure to increase transaction fees. Ev=
en a 2x rule (implying 800K max blocks) would, today, be squeezing out tran=
sactions / putting pressure to increase fees.</div><div><br></div><div>Usin=
g a median size instead of an average means the size can increase or decrea=
se more quickly. For example, imagine the rule is "median of last 2016=
blocks" and 49% of miners are producing 0-size blocks and 51% are pro=
ducing max-size blocks. The median is max-size, so the 51% have total contr=
ol over making blocks bigger.=C2=A0 Swap the roles, and the median is min-s=
ize.</div><div><br></div><div>Because of that, I think using an average is =
better-- it means the max size will change (up or down) more slowly.</div><=
div><br></div><div>I also think 2016 blocks is too long, because transactio=
n volumes change quicker than that. An average over 144 blocks (last 24 hou=
rs) would be better able to handle increased transaction volume around majo=
r holidays, and would also be able to react more quickly if an economically=
irrational attacker attempted to flood the network with fee-paying transac=
tions.</div><div><br></div><div>So my straw-man proposal would be: =C2=A0ma=
x size 2x average size over last 144 blocks, calculated at every block.</di=
v><div><br></div><div>There are a couple of other changes I'd pair with=
that consensus change:</div><div><br></div><div>+ Make the default mining =
policy for Bitcoin Core neutral-- have its target block size be the average=
size, so miners that don't care will "go along with the people wh=
o do care."</div><div><br></div><div>+ Use something like Greg's f=
ormula for size instead of bytes-on-the-wire, to discourage bloating the UT=
XO set.</div><div><br></div><div><br></div><div>---------</div><div><br></d=
iv><div>When I've proposed (privately, to the other core committers) so=
me dynamic algorithm the objection has been "but that gives miners com=
plete control over the max block size."</div><div><br></div><div>I thi=
nk that worry is unjustified right now-- certainly, until we have size-inde=
pendent new block propagation there is an incentive for miners to keep thei=
r blocks small, and we see miners creating small blocks even when there are=
fee-paying transactions waiting to be confirmed.</div><div><br></div><div>=
I don't even think it will be a problem if/when we do have size-indepen=
dent new block propagation, because I think the combination of the random t=
iming of block-finding plus a dynamic limit as described above will create =
a healthy system.</div><div><br></div><div>If I'm wrong, then it seems =
to me the miners will have a very strong incentive to, collectively, impose=
whatever rules are necessary (maybe a soft-fork to put a hard cap on block=
size) to make the system healthy again.</div></div><div><br></div><div><br=
></div>-- <br><div class=3D"gmail_signature">--<br>Gavin Andresen<br></div>=
<div class=3D"gmail_signature"><br></div>
</div></div>
--089e0141a71816e6320517265d30--
|