1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
|
Return-Path: <morcos@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id A610F19DC
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 5 Oct 2015 18:45:59 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f182.google.com (mail-io0-f182.google.com
[209.85.223.182])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 94EA615C
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 5 Oct 2015 18:45:58 +0000 (UTC)
Received: by iofh134 with SMTP id h134so196474403iof.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 05 Oct 2015 11:45:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:content-type; bh=FB49BKFI+5qHeNnMNydGhoAnfEaD/Qhg+zZAJdJJAw4=;
b=x3RXWnVCf8aom6oANEz7HRF9yO90oCsEHdfgV0cdS4mKJxpyclmVlxSMIbO0+Ir0G2
1KVm4MPgC96OWYhLRUMbytrw4JHSgg3DRRaRGE2Alx0eIO4dRqi8dfxCbbNH82UC/kf/
FM6ZKDHsU/XRCyW5wyhG7Zfbv1mEIIXRJ7QbhD9Xjod1CFNJ7VZqrY4A+EiqiAnxe4bD
gffhf+Oph12JAhAPhdgmLkDMTS/lcCphdUuUHvzO2a4Cgzy9Mp0ei2YD8Q+fYewImSTe
4NxAcO2pKGpX/9tZFzYOAZSuWaD8jSdulPfNVVZgwNAHT0FCcWl8wrpHnKz7PdQVjKnG
g3Jw==
MIME-Version: 1.0
X-Received: by 10.107.25.71 with SMTP id 68mr28013099ioz.46.1444070757976;
Mon, 05 Oct 2015 11:45:57 -0700 (PDT)
Received: by 10.64.106.103 with HTTP; Mon, 5 Oct 2015 11:45:57 -0700 (PDT)
In-Reply-To: <CAPWm=eW-g9F5YZ9EdqXGzpzvs2mQJ8N5wKG15Ofz4cWGaHQ0BQ@mail.gmail.com>
References: <CAPWm=eWuvC8zYM_ipAnaQttKQQG2Vas6np_bAFkxG31eR5w=xQ@mail.gmail.com>
<55D77A7F.40402@mattcorallo.com>
<CAJN5wHVzzo-dD6FFyaydEDm27HK2OkWxC0o0Pxcy-N9wTfv8Gw@mail.gmail.com>
<CAPWm=eW-g9F5YZ9EdqXGzpzvs2mQJ8N5wKG15Ofz4cWGaHQ0BQ@mail.gmail.com>
Date: Mon, 5 Oct 2015 14:45:57 -0400
Message-ID: <CAPWm=eVVdyYxePrXur17P=FdMpUvNmByz30hey5=R46PQPhf-Q@mail.gmail.com>
From: Alex Morcos <morcos@gmail.com>
To: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a113ff20e8f550405215fec09
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW
autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] Proposed new policy for transactions that depend
on other unconfirmed transactions
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 05 Oct 2015 18:45:59 -0000
--001a113ff20e8f550405215fec09
Content-Type: text/plain; charset=UTF-8
I'd like to propose updates to the new policy limits on unconfirmed
transaction chains.
The existing limits in master and scheduled for release in 0.12 are:
Ancestor packages = 100 txs and 900kb total size
Descendant packages = 1000 txs and 2500kb total size
Before 0.12 is released I would like to propose a significant reduction in
these limits. In the course of analyzing algorithms for mempool limiting,
it became clear that large packages of unconfirmed transactions were the
primary vector for mempool clogging or relay fee boosting attacks. Feedback
from the initial proposed limits was that they were too generous anyway.
The proposed new limits are:
Ancestor packages = 25 txs and 100kb total size
Descendant packages = 25 txs and 100kb total size
Based on historical transaction data, the most restrictive of these limits
is the 25 transaction count on descendant packages. Over the period of
April and May of this year (before stress tests), 5.8% of transactions
would have violated this limit alone. Applying all the limits together
would have affected 6.1% of transactions.
Please keep in mind these are policy limits that affect transactions which
depend on other unconfirmed transactions only. They are not a change to
consensus rules and do not affect how many chained txs a valid block may
contain. Furthermore, any transaction that was unable to be relayed due to
these limits need only wait for some of its unconfirmed ancestors to be
included in a block and then it could be successfully broadcast. This is
unlikely to affect the total time from creation to inclusion in a block.
Finally, these limits are command line arguments that can easily be changed
on an individual node basis in Bitcoin Core.
Please give your feedback if you know of legitimate use cases that would be
hindered by these limits.
Thanks,
Alex
On Mon, Sep 21, 2015 at 11:02 AM, Alex Morcos <morcos@gmail.com> wrote:
> Thanks for everyone's review. These policy changes have been merged in to
> master in 6654 <https://github.com/bitcoin/bitcoin/pull/6654>, which just
> implements these limits and no mempool limiting yet. The default ancestor
> package size limit is 900kb not 1MB.
>
> Yes I think these limits are generous, but they were designed to be as
> generous as was computationally feasible so they were unobjectionable
> (since the existing policy was no limits). This does not preclude future
> changes to policy that would reduce these limits.
>
>
>
>
>
> On Fri, Aug 21, 2015 at 3:52 PM, Danny Thorpe <danny.thorpe@gmail.com>
> wrote:
>
>> The limits Alex proposed are generous (bordering on obscene!), but
>> dropping that down to allowing only two levels of chained unconfirmed
>> transactions is too tight.
>>
>> Use case: Brokered asset transfers may require sets of transactions with
>> a dependency tree depth of 3 to be published together. ( N seller txs, 1
>> broker bridge tx, M buyer txs )
>>
>> If the originally proposed depth limit of 100 does not provide a
>> sufficient cap on memory consumption or loop/recursion depth, a depth limit
>> of 10 would provide plenty of headroom for this 3 level use case and
>> similar patterns.
>>
>> -Danny
>>
>> On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> I dont see any problem with such limits. Though, hell, if you limited
>>> entire tx dependency trees (ie transactions and all required unconfirmed
>>> transactions for them) to something like 10 txn, maximum two levels
>>> deep, I also wouldnt have a problem.
>>>
>>> Matt
>>>
>>> On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote:
>>> > Hi everyone,
>>> >
>>> >
>>> > I'd like to propose a new set of requirements as a policy on when to
>>> > accept new transactions into the mempool and relay them. This policy
>>> > would affect transactions which have as inputs other transactions which
>>> > are not yet confirmed in the blockchain.
>>> >
>>> > The motivation for this policy is 6470
>>> > <https://github.com/bitcoin/bitcoin/pull/6470> which aims to limit the
>>> > size of a mempool. As discussed in that pull
>>> > <https://github.com/bitcoin/bitcoin/pull/6470#issuecomment-125324736>,
>>> > once the mempool is full a new transaction must be able to pay not only
>>> > for the transaction it would evict, but any dependent transactions that
>>> > would be removed from the mempool as well. In order to make sure this
>>> > is always feasible, I'm proposing 4 new policy limits.
>>> >
>>> > All limits are command line configurable.
>>> >
>>> > The first two limits are required to make sure no chain of transactions
>>> > will be too large for the eviction code to handle:
>>> >
>>> > Max number of descendant txs : No transaction shall be accepted if it
>>> > would cause another transaction in the mempool to have too many
>>> > descendant transactions (all of which would have to be evicted if the
>>> > ancestor transaction was evicted). Default: 1000
>>> >
>>> > Max descendant size : No transaction shall be accepted if it would
>>> cause
>>> > another transaction in the mempool to have the total size of all its
>>> > descendant transactions be too great. Default : maxmempool / 200 =
>>> 2.5MB
>>> >
>>> > The third limit is required to make sure calculating the state required
>>> > for sorting and limiting the mempool and enforcing the first 2 limits
>>> is
>>> > computationally feasible:
>>> >
>>> > Max number of ancestor txs: No transaction shall be accepted if it has
>>> > too many ancestor transactions which are not yet confirmed (ie, in the
>>> > mempool). Default: 100
>>> >
>>> > The fourth limit is required to maintain the pre existing policy goal
>>> > that all transactions in the mempool should be mineable in the next
>>> block.
>>> >
>>> > Max ancestor size: No transaction shall be accepted if the total size
>>> of
>>> > all its unconfirmed ancestor transactions is too large. Default: 1MB
>>> >
>>> > (All limits include the transaction itself.)
>>> >
>>> > For reference, these limits would have affected less than 2% of
>>> > transactions entering the mempool in April or May of this year. During
>>> > the period of 7/6 through 7/14, while the network was under stress
>>> test,
>>> > as many as 25% of the transactions would have been affected.
>>> >
>>> > The code to implement the descendant package tracking and new policy
>>> > limits can be found in 6557
>>> > <https://github.com/bitcoin/bitcoin/pull/6557> which is built off of
>>> 6470.
>>> >
>>> > Thanks,
>>> > Alex
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > bitcoin-dev mailing list
>>> > bitcoin-dev@lists.linuxfoundation.org
>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>> >
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>
>>
>
--001a113ff20e8f550405215fec09
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I'd like to propose updates to the new policy limits o=
n unconfirmed transaction chains.=C2=A0<div><br></div><div>The existing lim=
its in master and scheduled for release in 0.12 are:=C2=A0</div><div>Ancest=
or packages =3D 100 txs and 900kb total size</div><div>Descendant packages =
=3D 1000 txs and 2500kb total size=C2=A0</div><div><br></div><div>Before 0.=
12 is released I would like to propose a significant reduction in these lim=
its. In the course of analyzing algorithms for mempool limiting, it became =
clear that large packages of unconfirmed transactions were the primary vect=
or for mempool clogging or relay fee boosting attacks. Feedback from the in=
itial proposed limits was that they were too generous anyway.=C2=A0</div><d=
iv><br></div><div>The proposed new limits are:=C2=A0</div><div>Ancestor pac=
kages =3D 25 txs and 100kb total size</div><div>Descendant packages =3D 25 =
txs and 100kb total size=C2=A0</div><div><br></div><div>Based on historical=
transaction data, the most restrictive of these limits is the 25 transacti=
on count on descendant packages. Over the period of April and May of this y=
ear (before stress tests), 5.8% of transactions would have violated this li=
mit alone. Applying all the limits together would have affected 6.1% of tra=
nsactions.=C2=A0</div><div><br></div><div>Please keep in mind these are pol=
icy limits that affect transactions which depend on other unconfirmed trans=
actions only. They are not a change to consensus rules and do not affect ho=
w many chained txs a valid block may contain. Furthermore, any transaction =
that was unable to be relayed due to these limits need only wait for some o=
f its unconfirmed ancestors to be included in a block and then it could be =
successfully broadcast. This is unlikely to affect the total time from crea=
tion to inclusion in a block. Finally, these limits are command line argume=
nts that can easily be changed on an individual node basis in Bitcoin Core.=
=C2=A0</div><div><br></div><div>Please give your feedback if you know of le=
gitimate use cases that would be hindered by these limits.=C2=A0</div><div>=
<br></div><div>Thanks,=C2=A0</div><div>Alex</div></div><div class=3D"gmail_=
extra"><br><div class=3D"gmail_quote">On Mon, Sep 21, 2015 at 11:02 AM, Ale=
x Morcos <span dir=3D"ltr"><<a href=3D"mailto:morcos@gmail.com" target=
=3D"_blank">morcos@gmail.com</a>></span> wrote:<br><blockquote class=3D"=
gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-=
left:1ex"><div dir=3D"ltr">Thanks for everyone's review.=C2=A0 These po=
licy changes have been merged in to master in <a href=3D"https://github.com=
/bitcoin/bitcoin/pull/6654" target=3D"_blank">6654</a>, which just implemen=
ts these limits and no mempool limiting yet.=C2=A0 The default ancestor pac=
kage size limit is 900kb not 1MB.<div><br></div><div>Yes I think these limi=
ts are generous, but they were designed to be as generous as was computatio=
nally feasible so they were unobjectionable (since the existing policy was =
no limits).=C2=A0 This does not preclude future changes to policy that woul=
d reduce these limits.</div><div><br></div><div><br></div><div><br></div><d=
iv><br></div></div><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">On Fri, Aug 21, 2015 at 3:52 PM, =
Danny Thorpe <span dir=3D"ltr"><<a href=3D"mailto:danny.thorpe@gmail.com=
" target=3D"_blank">danny.thorpe@gmail.com</a>></span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex"><div dir=3D"ltr">The limits Alex proposed are gener=
ous (bordering on obscene!), but dropping that down to allowing only two le=
vels of chained unconfirmed transactions is too tight. =C2=A0<div><br></div=
><div>Use case: Brokered asset transfers may require sets of transactions w=
ith a dependency tree depth of 3 to be published together. ( N seller txs, =
1 broker bridge tx, M buyer txs )</div><div><br></div><div>If the originall=
y proposed depth limit of 100 does not provide a sufficient cap on memory c=
onsumption or loop/recursion depth, a depth limit of 10 would provide plent=
y of headroom for this 3 level use case and similar patterns.</div><span><f=
ont color=3D"#888888"><div><br></div><div>-Danny</div></font></span></div><=
div><div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Fri, =
Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev <span dir=3D"ltr">&l=
t;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank=
">bitcoin-dev@lists.linuxfoundation.org</a>></span> wrote:<br><blockquot=
e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol=
id;padding-left:1ex">I dont see any problem with such limits. Though, hell,=
if you limited<br>
entire tx dependency trees (ie transactions and all required unconfirmed<br=
>
transactions for them) to something like 10 txn, maximum two levels<br>
deep, I also wouldnt have a problem.<br>
<br>
Matt<br>
<br>
On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote:<br>
> Hi everyone,<br>
><br>
><br>
> I'd like to propose a new set of requirements as a policy on when =
to<br>
> accept new transactions into the mempool and relay them.=C2=A0 This po=
licy<br>
> would affect transactions which have as inputs other transactions whic=
h<br>
> are not yet confirmed in the blockchain.<br>
><br>
> The motivation for this policy is 6470<br>
> <<a href=3D"https://github.com/bitcoin/bitcoin/pull/6470" rel=3D"no=
referrer" target=3D"_blank">https://github.com/bitcoin/bitcoin/pull/6470</a=
>> which aims to limit the<br>
> size of a mempool.=C2=A0 As discussed in that pull<br>
> <<a href=3D"https://github.com/bitcoin/bitcoin/pull/6470#issuecomme=
nt-125324736" rel=3D"noreferrer" target=3D"_blank">https://github.com/bitco=
in/bitcoin/pull/6470#issuecomment-125324736</a>>,<br>
> once the mempool is full a new transaction must be able to pay not onl=
y<br>
> for the transaction it would evict, but any dependent transactions tha=
t<br>
> would be removed from the mempool as well.=C2=A0 In order to make sure=
this<br>
> is always feasible, I'm proposing 4 new policy limits.<br>
><br>
> All limits are command line configurable.<br>
><br>
> The first two limits are required to make sure no chain of transaction=
s<br>
> will be too large for the eviction code to handle:<br>
><br>
> Max number of descendant txs : No transaction shall be accepted if it<=
br>
> would cause another transaction in the mempool to have too many<br>
> descendant transactions (all of which would have to be evicted if the<=
br>
> ancestor transaction was evicted).=C2=A0 Default: 1000<br>
><br>
> Max descendant size : No transaction shall be accepted if it would cau=
se<br>
> another transaction in the mempool to have the total size of all its<b=
r>
> descendant transactions be too great.=C2=A0 Default : maxmempool / 200=
=C2=A0 =3D=C2=A0 2.5MB<br>
><br>
> The third limit is required to make sure calculating the state require=
d<br>
> for sorting and limiting the mempool and enforcing the first 2 limits =
is<br>
> computationally feasible:<br>
><br>
> Max number of ancestor txs:=C2=A0 No transaction shall be accepted if =
it has<br>
> too many ancestor transactions which are not yet confirmed (ie, in the=
<br>
> mempool). Default: 100<br>
><br>
> The fourth limit is required to maintain the pre existing policy goal<=
br>
> that all transactions in the mempool should be mineable in the next bl=
ock.<br>
><br>
> Max ancestor size: No transaction shall be accepted if the total size =
of<br>
> all its unconfirmed ancestor transactions is too large.=C2=A0 Default:=
1MB<br>
><br>
> (All limits include the transaction itself.)<br>
><br>
> For reference, these limits would have affected less than 2% of<br>
> transactions entering the mempool in April or May of this year.=C2=A0 =
During<br>
> the period of 7/6 through 7/14, while the network was under stress tes=
t,<br>
> as many as 25% of the transactions would have been affected.<br>
><br>
> The code to implement the descendant package tracking and new policy<b=
r>
> limits can be found in 6557<br>
> <<a href=3D"https://github.com/bitcoin/bitcoin/pull/6557" rel=3D"no=
referrer" target=3D"_blank">https://github.com/bitcoin/bitcoin/pull/6557</a=
>> which is built off of 6470.<br>
><br>
> Thanks,<br>
> Alex<br>
><br>
><br>
><br>
> _______________________________________________<br>
> bitcoin-dev mailing list<br>
> <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_bl=
ank">bitcoin-dev@lists.linuxfoundation.org</a><br>
> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-=
dev" rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org=
/mailman/listinfo/bitcoin-dev</a><br>
><br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
--001a113ff20e8f550405215fec09--
|