Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E68509FA for ; Thu, 30 Mar 2017 09:34:34 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-yb0-f171.google.com (mail-yb0-f171.google.com [209.85.213.171]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 77D43AC for ; Thu, 30 Mar 2017 09:34:33 +0000 (UTC) Received: by mail-yb0-f171.google.com with SMTP id m133so6628005ybb.1 for ; Thu, 30 Mar 2017 02:34:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=6F8CF7NkJMq1mXAeK/rCAIUADwIeXppeKbDeaGhjmzQ=; b=QigIiZFMifFiJCzEJPj3dn2FXcTOLkKhRlNZjrCZ4tZLoIwy/PnL3NQV+YxS5vrftA pvZ/R+rh6WFJ7wSqMHHI0x0q5w0WniyCj4k7+Dopgjuev1pkCmpw/6KBiTe+N1OqAp6x ZoMMBhWiBZh0XjLLTIGQCYnZcXeVlliNIr4ipIoG4pOd4CuV+0vt3/UhpprXIlT6FLvi rDiNBjyOXODDuG2/4ErvlOcaFr1P4+3q87mjOMiUPK0pGs5FifDeo0ZYclZvRzgINuoK xGHs6Goo1kTSFnMJXy4kDUESskEZNM650h28iqR5bgJEHQFZs9A5Zb+QcDT6CcQrHkaH E5Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=6F8CF7NkJMq1mXAeK/rCAIUADwIeXppeKbDeaGhjmzQ=; b=fNNK00YMXPy+CkenzATVoMLS1VtUu3N/YzzGjZTtk4UW6RVVcKxN6SimJmQDjVER7c pZ9CtfqE5I4R+AQhI88Bwggfs0XTg/zG+X+saEWgeSWuoEpQYg5exbiycupIXXsbRM35 f1u5msFreC+EdffsUifkjJW60FUDDsOYJIUx8tD561olMEuMorKJiraG9qN0Kmvb8Fw9 0wfwn6IbTHDXlwrTVsfUDMNZ79MDDAPMvLqCM2btzLtAa1nxIxNI937S2eOWyB6dA/lc J15BQDeUkrsiIYtpAm3gjZLCmHds9C27xrvnw/AzIoo8RM2rV/pXej/fCRsPlFRaF67z hwSA== X-Gm-Message-State: AFeK/H3a710B2UZlCQpMcBiwRokOHzDgS5Bujzl2fQbUWiK3Gg82LK7oV1YUM6tJXxIG3PXRe0CcBall8ymEGQ== X-Received: by 10.37.111.136 with SMTP id k130mr6560541ybc.156.1490866472359; Thu, 30 Mar 2017 02:34:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.123.135 with HTTP; Thu, 30 Mar 2017 02:34:31 -0700 (PDT) Received: by 10.37.123.135 with HTTP; Thu, 30 Mar 2017 02:34:31 -0700 (PDT) In-Reply-To: References: From: Natanael Date: Thu, 30 Mar 2017 11:34:31 +0200 Message-ID: To: Bitcoin Dev Content-Type: multipart/alternative; boundary=001a1148b8467e1edd054bef66e5 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [bitcoin-dev] Block size adjustment idea - expedience fees + difficulty scaling proportional to block size (+ fee pool) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Mar 2017 09:34:35 -0000 --001a1148b8467e1edd054bef66e5 Content-Type: text/plain; charset=UTF-8 I had these following ideas as I was thinking about how to allow larger blocks without incentivizing the creation of excessively large blocks. I don't know how much of this is novel, I'd appreciate if anybody could link to any relevant prior art. I'm making no claims on this, anything novel in here is directly released into public domain. In short, I'm trying to rely on simple but direct incentives to improve the behavior of Bitcoin. Feedback requested. Some simulations requested, see below if you're willing to help. Any questions are welcome. --- Expedience fees. Softfork compatible. You want to really make sure your transaction gets processed quickly? Transactions could have a second fee type, a specially labeled anyone-can-spend output with an op_return value defining a "best-before" block number and some function describing the decline of the fee value for every future block, such that before block N the miners can claim the full expedience fee + the standard fee (if any), between block N+1 and N+X the miner can claim a reduced expedience fee + standard fee, afterwards only the standard fee. When a transaction is processed late such that not the full expedience fee can be claimed, the remainder of the expedience fee output is returned to the specified address among the inputs/outputs (could be something like in#3 for the address used by the 3rd UTXO input). This would have to be done for all remaining expedience fees within the last transaction in the block, inserted there by the miner. These additional UTXO:s does increase overhead somewhat, but hopefully not by too much. If we're going to modify the transaction syntax eventually, then we could take the chance to design for this to reduce overhead. My current best idea for how to handle returned expedience fees in multiuser transactions (coinjoin, etc) is to donate it to an agreed upon address. For recurring donation addresses (the fee pool included!), this reduces the number of return UTXO:s in the fee processing transaction. The default client policy may be to split the entire fee across an expedience fee and a fee pool donation, where the donation part becomes larger the later the transaction gets processed. This is expected to slow down the average inclusion speed of already delayed transactions, but they remain profitable to include. The dynamics here is simple, a miner is incentivized to process a transaction with an expedience fee before a standard fee of the same value-per-bit in order to not reduce the total value of the available fees of all standing transactions they can process. The longer they wait, the less total fees available. Sidenote: a steady stream of expedience fees reduces the profitability of block withholding attacks (!), at some threshold it should make it entirely unprofitable vs standard mining. This is due to the increased risk of losing valuable expedience fees added after you finished your first block (as the available value will be reduced in your block #2, vs what other miners can claim while still mining on that previous block). (Can somebody verify this with simulations?) --- Fee pool. Softfork compatible. We want to smooth out fee payments too for the future when the subsidy drops, to prevent deliberate forking to steal fees. We can introduce a designated P2SH anyone-can-spend fee pool address. The miner can never claim the full fees from his block or claim the full amount in the pool, only some percentage of both. The remainder goes back into the pool (this might be done at the end of the same expedience fee processing transaction described above). Anybody can deliberately pay to the pool. The fee pool is intended to act as a "buffer" such that it remains profitable to not try to steal fees but to just mine normally, even during relatively extreme fee value variance (consider the end of a big international shopping weekend). The fee value claimed by the miners between blocks is allowed to vary, but we want to avoid order-of-magnitude size variation (10x). We do however want the effect of expedience fees to have an impact. Perhaps some logarithmic function can smooth it out? Forcing larger fees to be distributed over longer time periods? --- Block size dependent difficulty scaling. Hardfork required. Larger blocks means greater difficulty - but it doesn't scale linearly, rather a little less than linearly. That means miners can take a penalty in difficulty to claim a greater number of high fee transactions in the same amount of time (effectively increasing "block size bitrate"), increasing their profits. When such profitable fees aren't available, they have to reduce block size. In other words, the users literally pay miners to increase block size (or don't pay, which reduces it). (Sidenote: I am in favor of combining this with the idea of a 32 MB max blocksize (or larger), with softforked scheduled lower size caps (perhaps starting at 4 MB max) that grows according to a schedule. This reduces the risk of rapidly increasing load before we have functional second layer scaling in place.) In order for a miner to profit from adding additional transactions, their fees must exceed the calculated cost of the resulting difficulty penalty to make it worth it to create a larger block. Such loads are expected during international shopping weekends. With only a few available high value transactions the incentive becomes the reverse, to create a smaller block with lower difficulty to faster claim those fees. To keep the average 10 minute block rate and to let this mechanism shift the "block size bitrate" as according to the fee justified block size changes, we set an Expected blocksize value that changes over time, and we change the difficulty target into the Standard difficulty target, where each block must reach a Scaled difficulty target . In terms of math we do something like this: Scaled difficulty = Standard difficulty * f(blocksize), where f would likely be some logarithmic function, and blocksize is defined in terms of units of Expected blocksize (a block 1.5x larger than Expected blocksize gets a value of 1.5). When we retarget the Standard difficulty and Expected blocksize we do this: Standard difficulty = Network hashrate per 10 minutes (approximately same as before, but now we take the Scaled difficulty of the last period's previous blocks into consideration) Standard blocksize = Recent average effective block bitrate = (sum of recent (weighted!) block sizes / length of timeperiod) / number of blocks in a retargeting period. Thus, generating larger blocks drives up the long term standard block bitrate, smaller blocks reduces it, in both cases we strive to average 1 block per 10 minutes. Combining this with expedience fees makes it even more effective; There's always a cutoff for where a miner stops including unprocessed transactions and let the rest remain for the next block. For standard fees, this would result in a fairly static block size and transactions backlog. With expedience fees your transaction can bypass standard fees with same value-per-bit, as explained above, because otherwise the miners reduces the value of their future expected fees. The more people that do this, the greater incentive to not delay transactions and instead increase the blocksize. (Can somebody help with the math here? I want simulations of this.) (Sidenote: I'm in favor of RBF, replace-by-fee. This makes the above work much more smoothly. Anybody relying on the security of unconfirmed transactions for any significant value *have to* rely on some kind of incentive protected multisignature transaction, including LN type second layer schemes. The other option is just not secure.) If load is low then you can add a high expedience fee to incentivize the creation of a smaller block with your transaction, since difficulty will be reduced for the smaller block. This means the miner has a higher chance of beating the competition. Adding additional lower fee transactions may reduce his average value-per-bit to become less profitable. Miners simply aim to maximize their fees-per-bit, while also paying as little as possible in mining costs. To make this work as intended for those willing to explicitly pay to reduce block size, one could tag such an expedience fee with a maximum allowed blocksize (where the fee will be claimed in such a smaller block if it is the more profitable option), such that it won't be countered by others making more high expedience fees to increase blocksize. Note: I'm not particularly in favor of this idea, just mentioning the possibility. --001a1148b8467e1edd054bef66e5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I had these following ideas as I was thinking about how t= o allow larger blocks without incentivizing the creation of excessively lar= ge blocks. I don't know how much of this is novel, I'd appreciate i= f anybody could link to any relevant prior art. I'm making no claims on= this, anything novel in here is directly released into public domain.=C2= =A0

In short, I'm trying t= o rely on simple but direct incentives to improve the behavior of Bitcoin.= =C2=A0

Feedback requested.= Some simulations requested, see below if you're willing to help. Any q= uestions are welcome.=C2=A0

---

Expedience fees.= Softfork compatible.=C2=A0

You want to really make sure your transaction gets processed quickly? T= ransactions could have a second fee type, a specially labeled anyone-can-sp= end output with an op_return value defining a "best-before" block= number and some function describing the decline of the fee value for every= future block, such that before block N the miners can claim the full exped= ience fee + the standard fee (if any), between block N+1 and N+X the miner = can claim a reduced expedience fee + standard fee, afterwards only the stan= dard fee.=C2=A0

When a t= ransaction is processed late such that not the full expedience fee can be c= laimed, the remainder of the expedience fee output is returned to the speci= fied address among the inputs/outputs (could be something like in#3 for the= address used by the 3rd UTXO input). This would have to be done for all re= maining expedience fees within the last transaction in the block, inserted = there by the miner.=C2=A0

These additional UTXO:s does increase overhead somewhat, but hopefully no= t by too much. If we're going to modify the transaction syntax eventual= ly, then we could take the chance to design for this to reduce overhead.=C2= =A0

My current best idea= for how to handle returned expedience fees in multiuser transactions (coin= join, etc) is to donate it to an agreed upon address. For recurring donatio= n addresses (the fee pool included!), this reduces the number of return UTX= O:s in the fee processing transaction.=C2=A0

The default client policy may be to split the entire f= ee across an expedience fee and a fee pool donation, where the donation par= t becomes larger the later the transaction gets processed. This is expected= to slow down the average inclusion speed of already delayed transactions, = but they remain profitable to include.=C2=A0

The dynamics here is simple, a miner is incentivized t= o process a transaction with an expedience fee before a standard fee of the= same value-per-bit in order to not reduce the total value of the available= fees of all standing transactions they can process. The longer they wait, = the less total fees available.=C2=A0

Sidenote: a steady stream of expedience fees reduces the profi= tability of block withholding attacks (!), at some threshold it should make= it entirely unprofitable vs standard mining. This is due to the increased = risk of losing valuable expedience fees added after you finished your first= block (as the available value will be reduced in your block #2, vs what ot= her miners can claim while still mining on that previous block).=C2=A0
(Can somebody verify this with simulations?)=C2=A0
=

---
Fee pool. Softfork compatible.=C2=A0

We want to smooth out fee payments t= oo for the future when the subsidy drops, to prevent deliberate forking to = steal fees. We can introduce a designated P2SH anyone-can-spend fee pool ad= dress. The miner can never claim the full fees from his block or claim the = full amount in the pool, only some percentage of both. The remainder goes b= ack into the pool (this might be done at the end of the same expedience fee= processing transaction described above). Anybody can deliberately pay to t= he pool.=C2=A0

The fee p= ool is intended to act as a "buffer" such that it remains profita= ble to not try to steal fees but to just mine normally, even during relativ= ely extreme fee value variance (consider the end of a big international sho= pping weekend).=C2=A0

Th= e fee value claimed by the miners between blocks is allowed to vary, but we= want to avoid order-of-magnitude size variation (10x). We do however want = the effect of expedience fees to have an impact. Perhaps some logarithmic f= unction can smooth it out? Forcing larger fees to be distributed over longe= r time periods?=C2=A0

--= -

Block size dependent d= ifficulty scaling. Hardfork required.=C2=A0

Larger blocks means greater difficulty - but it doesn&#= 39;t scale linearly, rather a little less than linearly. That means miners = can take a penalty in difficulty to claim a greater number of high fee tran= sactions in the same amount of time (effectively increasing "block siz= e bitrate"), increasing their profits. When such profitable fees aren&= #39;t available, they have to reduce block size.
In other words, the users literally pay miners to = increase block size (or don't pay, which reduces it).=C2=A0

(Sidenote: I am in favor of combini= ng this with the idea of a 32 MB max blocksize (or larger), with softforked= scheduled lower size caps (perhaps starting at 4 MB max) that grows accord= ing to a schedule. This reduces the risk of rapidly increasing load before = we have functional second layer scaling in place.)=C2=A0

In order for a miner to profit from adding= additional transactions, their fees must exceed the calculated cost of the= resulting difficulty penalty to make it worth it to create a larger block.= Such loads are expected during international shopping weekends.=C2=A0
With only a few available high value transactions the in= centive becomes the reverse, to create a smaller block with lower difficult= y to faster claim those fees.

To keep the average 10 minute block rate and to let this mechanism sh= ift the "block size bitrate" as according to the fee justified bl= ock size changes, we set an Expected blocksize value that changes over time= , and we change the difficulty target into the Standard difficulty target, = where each block must reach a Scaled difficulty target .=C2=A0

In terms of math we do something lik= e this:
Scaled difficulty =3D Standard difficulty * = f(blocksize), where f would likely be some logarithmic function, and blocks= ize is defined in terms of units of Expected blocksize (a block 1.5x larger= than Expected blocksize gets a value of 1.5).=C2=A0

When we retarget the Standard difficulty and E= xpected blocksize we do this:=C2=A0
Standard difficu= lty =3D Network hashrate per 10 minutes (approximately same as before, but = now we take the Scaled difficulty of the last period's previous blocks = into consideration)
Standard blocksize =3D Recent av= erage effective block bitrate =3D (sum of recent (weighted!) block sizes / = length of timeperiod) / number of blocks in a retargeting period.=C2=A0

Thus, generating larger blo= cks drives up the long term standard block bitrate, smaller blocks reduces = it, in both cases we strive to average 1 block per 10 minutes.=C2=A0
<= div dir=3D"auto">
Combining this with expedience= fees makes it even more effective;

There's always a cutoff for where a miner stops including u= nprocessed transactions and let the rest remain for the next block. For sta= ndard fees, this would result in a fairly static block size and transaction= s backlog.=C2=A0
With expedience fees your transacti= on can bypass standard fees with same value-per-bit, as explained above, be= cause otherwise the miners reduces the value of their future expected fees.= The more people that do this, the greater incentive to not delay transacti= ons and instead increase the blocksize. (Can somebody help with the math he= re? I want simulations of this.)=C2=A0

(Sidenote: I'm in favor of RBF, replace-by-fee. This mak= es the above work much more smoothly. Anybody relying on the security of un= confirmed transactions for any significant value *have to* rely on some kin= d of incentive protected multisignature transaction, including LN type seco= nd layer schemes. The other option is just not secure.)=C2=A0

If load is low then you can add a hig= h expedience fee to incentivize the creation of a smaller block with your t= ransaction, since difficulty will be reduced for the smaller block. This me= ans the miner has a higher chance of beating the competition. Adding additi= onal lower fee transactions may reduce his average value-per-bit to become = less profitable.=C2=A0

M= iners simply aim to maximize their fees-per-bit, while also paying as littl= e as possible in mining costs.=C2=A0

To make this work as intended for those willing to explicitly = pay to reduce block size, one could tag such an expedience fee with a maxim= um allowed blocksize (where the fee will be claimed in such a smaller block= if it is the more profitable option), such that it won't be countered = by others making more high expedience fees to increase blocksize. Note: I&#= 39;m not particularly in favor of this idea, just mentioning the possibilit= y.=C2=A0

--001a1148b8467e1edd054bef66e5--