Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1YqKgq-0002RE-86 for bitcoin-development@lists.sourceforge.net; Thu, 07 May 2015 12:14:08 +0000 Received-SPF: softfail (sog-mx-2.v43.ch3.sourceforge.com: transitioning domain of hashingit.com does not designate 89.145.69.228 as permitted sender) client-ip=89.145.69.228; envelope-from=dave@hashingit.com; helo=heron.directrouter.co.uk; Received: from heron.directrouter.co.uk ([89.145.69.228]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1YqKgn-0004QK-Gm for bitcoin-development@lists.sourceforge.net; Thu, 07 May 2015 12:14:08 +0000 Received: from host109-152-69-18.range109-152.btcentralplus.com ([109.152.69.18]:56030 helo=[192.168.1.82]) by heron.directrouter.co.uk with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85) (envelope-from ) id 1YqKP8-000LMd-0C; Thu, 07 May 2015 11:55:50 +0000 Content-Type: multipart/alternative; boundary="Apple-Mail=_D00D50D0-E09A-40EF-AC54-6B001E41379D" Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) From: Dave Hudson In-Reply-To: Date: Thu, 7 May 2015 12:55:49 +0100 Message-Id: <5049F137-E123-47F6-9D24-FE51B92629FF@hashingit.com> References: <554A91BE.6060105@bluematt.me> To: =?utf-8?Q?Jorge_Tim=C3=B3n?= X-Mailer: Apple Mail (2.2098) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - heron.directrouter.co.uk X-AntiAbuse: Original Domain - lists.sourceforge.net X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - hashingit.com X-Get-Message-Sender-Via: heron.directrouter.co.uk: authenticated_id: dave@hashingit.com X-Source: X-Source-Args: X-Source-Dir: X-Spam-Score: 2.0 (++) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1YqKgn-0004QK-Gm Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Block Size Increase X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2015 12:14:08 -0000 --Apple-Mail=_D00D50D0-E09A-40EF-AC54-6B001E41379D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 7 May 2015, at 11:52, Jorge Tim=C3=B3n wrote: >=20 > On Thu, May 7, 2015 at 11:25 AM, Mike Hearn wrote: >> I observed to Wladimir and Gavin in private that this timeline meant = a change to the block size was unlikely to get into 0.11, leaving only = 0.12, which would give everyone only a few months to upgrade in order to = fork the chain by the end of the winter growth season. That seemed = tight. >=20 > Can you please elaborate on what terrible things will happen if we > don't increase the block size by winter this year? > I assume that you are expecting full blocks by then, have you used any > statistical technique to come up with that date or is it just your > guess? > Because I love wild guesses and mine is that full 1 MB blocks will not > happen until June 2017. I've been looking at this problem for quite a while (Gavin cited some of = my work a few days ago) so thought I'd chime in with a few thoughts = (some of which I've not published). I believe the major problem here is = that this isn't just an engineering decision; the reaction of the miners = will actually determine the success or failure of any course of action. = In fact any decision forced upon them may backfire if they collectively = take exception to it. It's worth bearing in mind that most of the hash = rate is now under the control of relatively large companies, many of = whom have investors who are expecting to see returns; it probably isn't = sufficient to just expect them to "do the right thing". We're seeing plenty of full 1M byte blocks already and have been for = months. Typically whenever we have one of the large inter-block gaps = then these are often followed by one (and sometimes several) completely = full blocks (full by the definition of whatever the miner wanted to use = as a size limit). The problem with this particular discussion is that there are quite a = few "knowns" but an equally large number of "unknowns". Let's look at = them: Known: There has been a steady trend towards the mean block size getting = larger. See = https://blockchain.info/charts/avg-block-size?timespan=3Dall&showDataPoint= s=3Dfalse&daysAverageString=3D7&show_header=3Dtrue&scale=3D0&address=3D = Known: Now the trend was definitely increasing quite quickly last year = but for the last few months has been slowing down, however we did see = pretty much a 2x increase in mean block sizes in 2014. Known: For most of 2015 we've actually been seeing that rate slow quite = dramatically, but the total numbers of transactions are still rising so = we're seeing mean transaction sizes have been reducing, and that tallies = with seeing more transactions per block: = https://blockchain.info/charts/n-transactions-per-block?timespan=3Dall&sho= wDataPoints=3Dfalse&daysAverageString=3D7&show_header=3Dtrue&scale=3D0&add= ress=3D = Unknown: Why are seeing more smaller transactions? Are we simply seeing = more efficient use of blockchain resources or have some large consumers = of block space going away? How much more block space compression might = be possible in, say, the next 12 months? Known: If we reach the point where all blocks are 1M bytes then there's = a major problem in terms of transaction confirmation. I published an = analysis of the impact of different mean block sizes against = confirmation times: = http://hashingit.com/analysis/34-bitcoin-traffic-bulletin = . The current = 35% to 45% mean block size doesn't have a huge impact on transaction = confirmations (assuming equal fees for all) but once we're up at 80% = then things start to get unpleasant. Instead of 50% of first = confirmations taking about 7 minutes they instead take nearer to 19 = minutes. Known: There are currently a reasonably large number of zero-fee = transactions getting relayed and mined. If things start to slow down = then there will be a huge incentive to delay them (or drop them = altogether). Unknown: If block space starts to get more scarce then how will this = affect the use of the blockchain? Do the zero-fee TXs move to some = batched transfer solution via third party? Do people start to get = smarter about how TXs are encoded? Do some TXs go away completely (there = are a lot of long-chain transactions that may simply be "noise" creating = an artificially inflated view of transaction volumes)? Known: There's a major problem looming for miners at the next block = reward halving. Many are already in a bad place and without meaningful = fees then sans a 2x increase in the USD:BTC ratio then many will simply = have to leave the network, increasing centralisation risks. There seems = to be a fairly pervasive assumption that the 300-ish MW of power that = they currently use is going to pay for itself (ignoring capital and = other operating costs). Unknown: If the block size is increased and yet more negligible fee = transactions are dumped onto the network then that might well motivate = some large fraction of miners to start to clamp block sizes or reject = transactions below a certain fee threshold; they can easily create their = own artificial scarcity if enough of them feel it is in their interest = (it's not the most tricky setting to change). One can well imagine VC = investors in mining groups asking why they're essentially subsidising = all of the other VC-funded Bitcoin startups. Known: the orphan rate is still pretty-high even with everyone's fast = connections. If we assume that 20M byte blocks become possible then = that's likely to increase. Unknown: What are the security implications for larger blocks (this one = (at least) can be simulated though)? For example, could large blocks = with huge numbers of trivial transactions be used to put other = validators at a disadvantage in a variant of a selfish mining attack? = I've seen objections that such bad actors could be blacklisted in the = future but it's not clear to me how. A private mining pool can trivially = be made to appear like 100 pools of 1% of the size without significantly = affecting the economics of running that private mine. Cheers, Dave --Apple-Mail=_D00D50D0-E09A-40EF-AC54-6B001E41379D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On 7 May 2015, at 11:52, Jorge Tim=C3=B3n <jtimon@jtimon.cc> = wrote:

On = Thu, May 7, 2015 at 11:25 AM, Mike Hearn <mike@plan99.net> = wrote:
I observed to = Wladimir and Gavin in private that this timeline meant a change to the = block size was unlikely to get into 0.11, leaving only 0.12, which would = give everyone only a few months to upgrade in order to fork the chain by = the end of the winter growth season. That seemed tight.

Can you please elaborate on what = terrible things will happen if we
don't increase the block = size by winter this year?
I assume that you are expecting = full blocks by then, have you used any
statistical = technique to come up with that date or is it just your
guess?
Because I love wild guesses and mine is = that full 1 MB blocks will not
happen until June 2017.

I've = been looking at this problem for quite a while (Gavin cited some of my = work a few days ago) so thought I'd chime in with a few thoughts (some = of which I've not published). I believe the major problem here is that = this isn't just an engineering decision; the reaction of the miners will = actually determine the success or failure of any course of action. In = fact any decision forced upon them may backfire if they collectively = take exception to it. It's worth bearing in mind that most of the hash = rate is now under the control of relatively large companies, many of = whom have investors who are expecting to see returns; it probably isn't = sufficient to just expect them to "do the right thing".

We're seeing plenty of = full 1M byte blocks already and have been for months. Typically whenever = we have one of the large inter-block gaps then these are often followed = by one (and sometimes several) completely full blocks (full by the = definition of whatever the miner wanted to use as a size = limit).

The = problem with this particular discussion is that there are quite a few = "knowns" but an equally large number of "unknowns". Let's look at = them:

Known: = There has been a steady trend towards the mean block size getting = larger. See https://blockchain.info/charts/avg-block-size?timespan=3Dall&am= p;showDataPoints=3Dfalse&daysAverageString=3D7&show_header=3Dtrue&= amp;scale=3D0&address=3D

Known: Now the trend was definitely = increasing quite quickly last year but for the last few months has been = slowing down, however we did see pretty much a 2x increase in mean block = sizes in 2014.

Known: For most of 2015 we've actually been seeing that rate = slow quite dramatically, but the total numbers of transactions are still = rising so we're seeing mean transaction sizes have been reducing, and = that tallies with seeing more transactions per block: https://blockchain.info/charts/n-transactions-per-block?timespa= n=3Dall&showDataPoints=3Dfalse&daysAverageString=3D7&show_head= er=3Dtrue&scale=3D0&address=3D

Unknown: Why are seeing more smaller = transactions? Are we simply seeing more efficient use of blockchain = resources or have some large consumers of block space going away? How = much more block space compression might be possible in, say, the next 12 = months?

Known: = If we reach the point where all blocks are 1M bytes then there's a major = problem in terms of transaction confirmation. I published an analysis of = the impact of different mean block sizes against confirmation = times: http://hashingit.com/analysis/34-bitcoin-traffic-bulletin. = The current 35% to 45% mean block size doesn't have a huge impact on = transaction confirmations (assuming equal fees for all) but once we're = up at 80% then things start to get unpleasant. Instead of 50% of first = confirmations taking about 7 minutes they instead take nearer to 19 = minutes.

Known: = There are currently a reasonably large number of zero-fee transactions = getting relayed and mined. If things start to slow down then there will = be a huge incentive to delay them (or drop them altogether).

Unknown: If block space = starts to get more scarce then how will this affect the use of the = blockchain? Do the zero-fee TXs move to some batched transfer solution = via third party? Do people start to get smarter about how TXs are = encoded? Do some TXs go away completely (there are a lot of long-chain = transactions that may simply be "noise" creating an artificially = inflated view of transaction volumes)?

Known: There's a major problem looming = for miners at the next block reward halving. Many are already in a bad = place and without meaningful fees then sans a 2x increase in the USD:BTC = ratio then many will simply have to leave the network, increasing = centralisation risks. There seems to be a fairly pervasive assumption = that the 300-ish MW of power that they currently use is going to pay for = itself (ignoring capital and other operating costs).

Unknown: If the block = size is increased and yet more negligible fee transactions are dumped = onto the network then that might well motivate some large fraction of = miners to start to clamp block sizes or reject transactions below a = certain fee threshold; they can easily create their own artificial = scarcity if enough of them feel it is in their interest (it's not the = most tricky setting to change). One can well imagine VC investors in = mining groups asking why they're essentially subsidising all of the = other VC-funded Bitcoin startups.

Known: the orphan rate is still = pretty-high even with everyone's fast connections. If we assume that 20M = byte blocks become possible then that's likely to increase.

Unknown: What are the = security implications for larger blocks (this one (at least) can be = simulated though)? For example, could large blocks with huge numbers of = trivial transactions be used to put other validators at a disadvantage = in a variant of a selfish mining attack? I've seen objections that such = bad actors could be blacklisted in the future but it's not clear to me = how. A private mining pool can trivially be made to appear like 100 = pools of 1% of the size without significantly affecting the economics of = running that private mine.


Cheers,
Dave

= --Apple-Mail=_D00D50D0-E09A-40EF-AC54-6B001E41379D--