Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 68E041266 for ; Mon, 21 Sep 2015 15:02:33 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-io0-f179.google.com (mail-io0-f179.google.com [209.85.223.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 71C201E1 for ; Mon, 21 Sep 2015 15:02:21 +0000 (UTC) Received: by iofb144 with SMTP id b144so123052480iof.1 for ; Mon, 21 Sep 2015 08:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=c0LQyXNN4SxGrwtBmQMF7nT0LQTvCQcgpE/Q7A/rM3s=; b=bKji9KJ8sZzWYnI+1wyw7sT/U6EMkLBSvHBeSvwSmDgEg+FhHYL3RYb01sxbVgwRBm g7o2XRy5NqEj+Sh4EJJEfTZ456eSSgodakFVYh6VDq+0AaPJEvNWOgKl6hkQwSq+A6BT +bSBbjevyi+3AMFksdmRan42/5mHKeziEIUKH7jUZ5rMSxus50dfGnwRAZKSXVwLjKhk ZnHFBXIYcmpuvaJyb/hLsrjffP2i29U2D/fhFNF8CDPYZj1RIVlcfio5IWieJap94Wuu DuvkxK8aoAwPCJH1olMwL0MAY4QIXVTHSiMc8bbq6mYJaIMWwVw3OtXvNmD6o341ek3I Yobw== MIME-Version: 1.0 X-Received: by 10.107.25.71 with SMTP id 68mr25024795ioz.46.1442847740785; Mon, 21 Sep 2015 08:02:20 -0700 (PDT) Received: by 10.64.106.103 with HTTP; Mon, 21 Sep 2015 08:02:20 -0700 (PDT) In-Reply-To: References: <55D77A7F.40402@mattcorallo.com> Date: Mon, 21 Sep 2015 11:02:20 -0400 Message-ID: From: Alex Morcos To: Danny Thorpe Content-Type: multipart/alternative; boundary=001a113ff20e0dfbad0520432b10 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 15:02:33 -0000 --001a113ff20e0dfbad0520432b10 Content-Type: text/plain; charset=UTF-8 Thanks for everyone's review. These policy changes have been merged in to master in 6654 , which just implements these limits and no mempool limiting yet. The default ancestor package size limit is 900kb not 1MB. Yes I think these limits are generous, but they were designed to be as generous as was computationally feasible so they were unobjectionable (since the existing policy was no limits). This does not preclude future changes to policy that would reduce these limits. On Fri, Aug 21, 2015 at 3:52 PM, Danny Thorpe wrote: > The limits Alex proposed are generous (bordering on obscene!), but > dropping that down to allowing only two levels of chained unconfirmed > transactions is too tight. > > Use case: Brokered asset transfers may require sets of transactions with a > dependency tree depth of 3 to be published together. ( N seller txs, 1 > broker bridge tx, M buyer txs ) > > If the originally proposed depth limit of 100 does not provide a > sufficient cap on memory consumption or loop/recursion depth, a depth limit > of 10 would provide plenty of headroom for this 3 level use case and > similar patterns. > > -Danny > > On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> I dont see any problem with such limits. Though, hell, if you limited >> entire tx dependency trees (ie transactions and all required unconfirmed >> transactions for them) to something like 10 txn, maximum two levels >> deep, I also wouldnt have a problem. >> >> Matt >> >> On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote: >> > Hi everyone, >> > >> > >> > I'd like to propose a new set of requirements as a policy on when to >> > accept new transactions into the mempool and relay them. This policy >> > would affect transactions which have as inputs other transactions which >> > are not yet confirmed in the blockchain. >> > >> > The motivation for this policy is 6470 >> > which aims to limit the >> > size of a mempool. As discussed in that pull >> > , >> > once the mempool is full a new transaction must be able to pay not only >> > for the transaction it would evict, but any dependent transactions that >> > would be removed from the mempool as well. In order to make sure this >> > is always feasible, I'm proposing 4 new policy limits. >> > >> > All limits are command line configurable. >> > >> > The first two limits are required to make sure no chain of transactions >> > will be too large for the eviction code to handle: >> > >> > Max number of descendant txs : No transaction shall be accepted if it >> > would cause another transaction in the mempool to have too many >> > descendant transactions (all of which would have to be evicted if the >> > ancestor transaction was evicted). Default: 1000 >> > >> > Max descendant size : No transaction shall be accepted if it would cause >> > another transaction in the mempool to have the total size of all its >> > descendant transactions be too great. Default : maxmempool / 200 = >> 2.5MB >> > >> > The third limit is required to make sure calculating the state required >> > for sorting and limiting the mempool and enforcing the first 2 limits is >> > computationally feasible: >> > >> > Max number of ancestor txs: No transaction shall be accepted if it has >> > too many ancestor transactions which are not yet confirmed (ie, in the >> > mempool). Default: 100 >> > >> > The fourth limit is required to maintain the pre existing policy goal >> > that all transactions in the mempool should be mineable in the next >> block. >> > >> > Max ancestor size: No transaction shall be accepted if the total size of >> > all its unconfirmed ancestor transactions is too large. Default: 1MB >> > >> > (All limits include the transaction itself.) >> > >> > For reference, these limits would have affected less than 2% of >> > transactions entering the mempool in April or May of this year. During >> > the period of 7/6 through 7/14, while the network was under stress test, >> > as many as 25% of the transactions would have been affected. >> > >> > The code to implement the descendant package tracking and new policy >> > limits can be found in 6557 >> > which is built off of >> 6470. >> > >> > Thanks, >> > Alex >> > >> > >> > >> > _______________________________________________ >> > bitcoin-dev mailing list >> > bitcoin-dev@lists.linuxfoundation.org >> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > > --001a113ff20e0dfbad0520432b10 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for everyone's review.=C2=A0 These policy chang= es have been merged in to master in 6654, which just implements these limits and no mempo= ol limiting yet.=C2=A0 The default ancestor package size limit is 900kb not= 1MB.

Yes I think these limits are generous, but they we= re designed to be as generous as was computationally feasible so they were = unobjectionable (since the existing policy was no limits).=C2=A0 This does = not preclude future changes to policy that would reduce these limits.
=





On Fri, Aug 21, 2015 at 3:= 52 PM, Danny Thorpe <danny.thorpe@gmail.com> wrote:
=
The limits Alex proposed ar= e generous (bordering on obscene!), but dropping that down to allowing only= two levels of chained unconfirmed transactions is too tight. =C2=A0
Use case: Brokered asset transfers may require sets of transac= tions with a dependency tree depth of 3 to be published together. ( N selle= r txs, 1 broker bridge tx, M buyer txs )

If the or= iginally proposed depth limit of 100 does not provide a sufficient cap on m= emory consumption or loop/recursion depth, a depth limit of 10 would provid= e plenty of headroom for this 3 level use case and similar patterns.
<= span class=3D"HOEnZb">

-Danny

On Fri, Aug 21, 2015 at 12:= 22 PM, Matt Corallo via bitcoin-dev <bitcoin-dev@lists= .linuxfoundation.org> wrote:
https://github.com/bitcoin/bitcoin/pull/6470> which aims to limit the
> size of a mempool.=C2=A0 As discussed in that pull
> <https://github.com/bitco= in/bitcoin/pull/6470#issuecomment-125324736>,
> once the mempool is full a new transaction must be able to pay not onl= y
> for the transaction it would evict, but any dependent transactions tha= t
> would be removed from the mempool as well.=C2=A0 In order to make sure= this
> is always feasible, I'm proposing 4 new policy limits.
>
> All limits are command line configurable.
>
> The first two limits are required to make sure no chain of transaction= s
> will be too large for the eviction code to handle:
>
> Max number of descendant txs : No transaction shall be accepted if it<= br> > would cause another transaction in the mempool to have too many
> descendant transactions (all of which would have to be evicted if the<= br> > ancestor transaction was evicted).=C2=A0 Default: 1000
>
> Max descendant size : No transaction shall be accepted if it would cau= se
> another transaction in the mempool to have the total size of all its > descendant transactions be too great.=C2=A0 Default : maxmempool / 200= =C2=A0 =3D=C2=A0 2.5MB
>
> The third limit is required to make sure calculating the state require= d
> for sorting and limiting the mempool and enforcing the first 2 limits = is
> computationally feasible:
>
> Max number of ancestor txs:=C2=A0 No transaction shall be accepted if = it has
> too many ancestor transactions which are not yet confirmed (ie, in the=
> mempool). Default: 100
>
> The fourth limit is required to maintain the pre existing policy goal<= br> > that all transactions in the mempool should be mineable in the next bl= ock.
>
> Max ancestor size: No transaction shall be accepted if the total size = of
> all its unconfirmed ancestor transactions is too large.=C2=A0 Default:= 1MB
>
> (All limits include the transaction itself.)
>
> For reference, these limits would have affected less than 2% of
> transactions entering the mempool in April or May of this year.=C2=A0 = During
> the period of 7/6 through 7/14, while the network was under stress tes= t,
> as many as 25% of the transactions would have been affected.
>
> The code to implement the descendant package tracking and new policy > limits can be found in 6557
> <https://github.com/bitcoin/bitcoin/pull/6557> which is built off of 6470.
>
> Thanks,
> Alex
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev
>
_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev


--001a113ff20e0dfbad0520432b10--