Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 3B6D3107C for ; Sun, 30 Aug 2015 03:08:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9832A17D for ; Sun, 30 Aug 2015 03:08:39 +0000 (UTC) Received: from [192.168.50.29] ([69.50.179.106]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0LoJDJ-1Z3gpn0uuf-00gG4B; Sun, 30 Aug 2015 05:08:35 +0200 Content-Type: multipart/alternative; boundary="Apple-Mail=_98028B39-A9FC-4DC8-80A8-17F1FAA1DACA" Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) From: Peter R In-Reply-To: <55E26BDE.2080607@mattcorallo.com> Date: Sat, 29 Aug 2015 20:08:32 -0700 Message-Id: References: <55E21F2E.9000308@mattcorallo.com> <786493BB-2136-4587-A309-B8B1A34ED568@gmx.com> <55E26B82.2070805@mattcorallo.com> <55E26BDE.2080607@mattcorallo.com> To: Matt Corallo X-Mailer: Apple Mail (2.1510) X-Provags-ID: V03:K0:oA8R5tQ9PV2oLo4SdDWpu6CP+ZWrGVL/50URZbmbnALoz67x/e1 I0YH4sdV/gn5I3pxqNduLrpDipJhg6w2PT2yLHz2nUeQfb9PmnnXYWO9kHcJHqu/p5F/vmr cjiu0mwte+EOt9zE+0m89D7q3PcBmkmXV40beEHxV+n7F9pYyt9Z7Vcn7tKO3KZga1UN4tk DlAU5DxNwaWtYpwfxkoJg== X-UI-Out-Filterresults: notjunk:1;V01:K0:ngMycWM9luk=:u08oMLNvZjKhoUtwYDONtq o/uaoOTlJmQstX/bBHknmJ0sY73qTBAuXsi/m1PVM10mrq1Q1jEGbBdniF5ZvQm9V/Uja5q4f F82+AB3atn0+b6kmWVK+axxcq3l9DeoOqMlUdYfGw2ZRvCs/6380D7UOWtYc5U6ePSHAVEoTq Kc31MSL7D2umbZOfJTnqskc7l27SWNdWRF9A+FK/Ki0gAviNteJnapmWfxSaF3aR2caF5jpmM /WPefzT89aMf6gNUyDuF2Ku4CJoWkLlgODcqKwdRP3vEfwEW+ikh1282fS4zbXLmjPSiVMZxX 2+RfPduLnrH2dRCm4bJbIiiFeLVAAziyjvGlS1FGQVU3QthAHWGLJLRE5HX9oUq/uj76BjvK2 BOPyJUKJSoLQxDG/wFU2lndkBRzi510F4SEdoSyxkjKNfS2iWtk7UgZ8CzoGxFgJZfagxhDic R41eZDa9WwFTVaQ32H5qnY+r9rBLcHHO/Ugr7V8EPiUuSD7C30e6mqbAMw0dVN9ZJjsM2V1Lc T+tIAAJ14AwDKCDbOJKjCfOTqsbiiCIBicPUTyjLVTHg+TBY6FCrwIispRfPaBdPDwQgTGiGO 99KPHodP0Dy5an8rpTN/0RWNMNDp+ihYJEXw/zBKz+l2ueK09E8d0T7jwNOgRXkZuf7poNGrQ Pxx3W01y0d/uFGqd8IBzoShKxRyYAeRbQKVeejkK4h66qOs0yzgMXnlCynRlRqOQYQiCvWT+Z xg3SU363j+Yv8n6MrPO9Kbkgc4VaM9h+dVeHbA== X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: bitcoin-dev@lists.linuxfoundation.org, Daniele Pinna Subject: Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Aug 2015 03:08:41 -0000 --Apple-Mail=_98028B39-A9FC-4DC8-80A8-17F1FAA1DACA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > Of course this assumes the network does not change any as a result of > such a system. But such a system provides strong incentives for the > network to centralize in other ways (put all the mining nodes in one = DC > for all miners, etc). If all the mining nodes are in one data center, and if all the nodes are = programmed to build blocks in essentially the same way, then I would = agree that the orphan cost would be negligible! I will add this as an = example of a network configuration where the results of my paper would = be less relevant. =20 Peter =20 On 2015-08-29, at 7:35 PM, Matt Corallo = wrote: > Of course this assumes the network does not change any as a result of > such a system. But such a system provides strong incentives for the > network to centralize in other ways (put all the mining nodes in one = DC > for all miners, etc). >=20 > Matt >=20 > On 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote: >> It is not a purely academic scenario that blocks contain effectively = no >> information (that was not previously relayed). I'm not aware of any >> public code to do so, but I know several large miners who pre-relay = the >> block(s) they are working on to other nodes of theirs around the = globe. >> This means at announce-time you have only a few bytes to broadcast = (way >> less than a packet, and effects of using smaller packets to relay = things >> vs larger packets are very small, if anything). After you've = broadcast >> to all of your nodes, hops to other mining nodes are probably only a >> handful of ms away with very low packet loss, so relay time is no = longer >> connected to transaction inclusion at all (unless you're talking = about >> multi-GB blocks). Of course, this is relay time for large miners who = can >> invest time and money to build such systems. Small miners are = completely >> screwed in such a system. >>=20 >> Thus, the orphan risk for including a transaction is related to the >> validation time (which is only DB modify-utxo-set time, essentially, >> which maybe you can optimize much of that away, too, and only have to >> pass over mempool or so). Anyway, my point, really, is that though >> miners will have an incentive to not include transactions which will >> trigger validation by other nodes (ie things not already in their >> mempool), the incentive to not include transactions which have = already >> been relayed around sufficiently is, while not theoretically zero, as >> near to zero in practice as you can get. >>=20 >> Matt >>=20 >> On 08/29/15 23:17, Peter R wrote: >>> Hello Matt and Daniele, >>>=20 >>>> this seems to ignore the effects of transaction validation caches = and >>>> *block >>>> compression protocols. * >>>=20 >>> The effect of block compression protocols is included. This is what = I >>> call the "coding gain" and use the Greek letter "gamma" to = represent.=20 >>>=20 >>> As long as the block solution announcements contain information = (i.e., >>> Shannon Entropy) about the transactions included in a block, then = the >>> fee market will be "healthy" according to the definitions given in = the >>> linked paper (see below). This is the case right now, this is the = case >>> with your relay network, and this would be the case using any >>> implementation of IBLTs that I can imagine, so long as miners can = still >>> construct blocks according to their own volition. The "healthy fee >>> market" result follows from the Shannon-Hartley theorem; the = SH-theorem >>> describes the maximum rate at which information (Shannon Entropy) = can be >>> transmitted over a physical communication channel. =20 >>>=20 >>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf >>>=20 >>> I've exchanged emails with Greg Maxwell about (what IMO is) an = academic >>> scenario where the block solutions announcements contain *no = information >>> at all* about the transactions included in the blocks. Although the = fee >>> market would not be healthy in such a scenario, it is my feeling = that >>> this also requires miners to relinquish their ability to construct >>> blocks according to their own volition (i.e., the system would = already >>> be centralized). I look forward to a white paper demonstrating = otherwise! >>>=20 >>> Best regards, >>> Peter >>>=20 >>>=20 >>>=20 >>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev >>> >> > wrote: >>>=20 >>>> I believe it was pointed out previously in the discussion of the = Peter R >>>> paper, but I'll repeat it here so that its visible - this seems to >>>> ignore the effects of transaction validation caches and block >>>> compression protocols. Many large miners already have their own = network >>>> to relay blocks around the globe with only a few bytes on the wire = at >>>> block-time, and there is also the bitcoinrelaynetwork.org >>>> network, which >>>> does the same for smaller miners, albeit with slightly less = efficiency. >>>> Also, transaction validation time upon receiving a block can be = rather >>>> easily made negligible (ie the only validation time you should have = is >>>> the DB modify-utxo-set time). Thus, the increased orphan risk for >>>> including a transaction can be reduced to a very, very tiny amount, >>>> making the optimal blocksize, essentially, including everything = that >>>> you're confident is in the mempool of other reasonably large = miners. >>>>=20 >>>> Matt >>>>=20 >>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>>>> I'd like to submit this paper to the dev-list which analyzes how = miner >>>>> advantages scale with network and mempool properties in a scenario = of >>>>> uncapped block sizes. The work proceeds, in a sense, from where = Peter >>>>> R's work left off correcting a mistake and addressing the = critiques made >>>>> by the community to his work. >>>>>=20 >>>>> The main result of the work is a detailed analysis of mining = advantages >>>>> (defined as the added profit per unit of hash) as a function of = miner >>>>> hashrate. In it, I show how large block subsidies (or better, low >>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>>>> hashrates due to the steady increasing of marginal profits as = hashrates >>>>> grow. >>>>>=20 >>>>> The paper also shows that part of the large advantage the large = miners >>>>> have today is due to there being a barrier to entry into a >>>>> high-efficiency mining class which has access to expected profits = an >>>>> order of magnitude larger than everyone else. As block subsidies >>>>> decrease, this high-efficiency class is expected to vanish leading = to a >>>>> marginal profit structure which decreases as a function of = hashrate. >>>>>=20 >>>>> This work has vacuumed my entire life for the past two weeks = leading me >>>>> to lag behind on a lot of work. I apologize for typos which I may = not >>>>> have seen. I stand by for any comments the community may have and = look >>>>> forward to reigniting consideration of a block size scaling = proposal >>>>> (BIP101) which, due to the XT fork drama, I believe has been = placed >>>>> hastily and undeservedly on the chopping block. >>>>>=20 >>>>> = https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-= Uncapped-Block-Size-Fee-Markets >>>>>=20 >>>>>=20 >>>>> Regards, >>>>> Daniele >>>>>=20 >>>>>=20 >>>>> _______________________________________________ >>>>> bitcoin-dev mailing list >>>>> bitcoin-dev@lists.linuxfoundation.org >>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>>>=20 >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>=20 >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>=20 --Apple-Mail=_98028B39-A9FC-4DC8-80A8-17F1FAA1DACA Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Of course this assumes the network does = not change any as a result of
such a system. But such a system = provides strong incentives for the
network to centralize in other = ways (put all the mining nodes in one DC
for all miners, = etc).
If all the mining nodes are in one = data center, and if all the nodes are programmed to build blocks in = essentially the same way, then I would agree that the orphan cost would = be negligible!  I will add this as an example of a network = configuration where the results of my paper would be less relevant. =  

Peter =  


On 2015-08-29, at 7:35 PM, Matt = Corallo <lf-lists@mattcorallo.com> = wrote:

Of course this assumes the network does not change any as = a result of
such a system. But such a system provides strong = incentives for the
network to centralize in other ways (put all the = mining nodes in one DC
for all miners, etc).

Matt

On = 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote:
It is not a purely academic scenario that blocks contain = effectively no
information (that was not previously relayed). I'm not = aware of any
public code to do so, but I know several large miners = who pre-relay the
block(s) they are working on to other nodes of = theirs around the globe.
This means at announce-time you have only a = few bytes to broadcast (way
less than a packet, and effects of using = smaller packets to relay things
vs larger packets are very small, if = anything). After you've broadcast
to all of your nodes, hops to other = mining nodes are probably only a
handful of ms away with very low = packet loss, so relay time is no longer
connected to transaction = inclusion at all (unless you're talking about
multi-GB blocks). Of = course, this is relay time for large miners who can
invest time and = money to build such systems. Small miners are completely
screwed in = such a system.

Thus, the orphan risk for including a transaction = is related to the
validation time (which is only DB modify-utxo-set = time, essentially,
which maybe you can optimize much of that away, = too, and only have to
pass over mempool or so). Anyway, my point, = really, is that though
miners will have an incentive to not include = transactions which will
trigger validation by other nodes (ie things = not already in their
mempool), the incentive to not include = transactions which have already
been relayed around sufficiently is, = while not theoretically zero, as
near to zero in practice as you can = get.

Matt

On 08/29/15 23:17, Peter R wrote:
Hello Matt and Daniele,

= this seems to ignore the effects of transaction validation caches = and
*block
compression protocols. *

The effect = of block compression protocols is included.  This is what I
call = the "coding gain" and use the Greek letter "gamma" to represent. =

As long as the block solution announcements contain information = (i.e.,
Shannon Entropy) about the transactions included in a block, = then the
fee market will be "healthy" according to the definitions = given in the
linked paper (see below).  This is the case right = now, this is the case
with your relay network, and this would be the = case using any
implementation of IBLTs that I can imagine, so long as = miners can still
construct blocks according to their own volition. =  The "healthy fee
market" result follows from the = Shannon-Hartley theorem; the SH-theorem
describes the maximum rate at = which information (Shannon Entropy) can be
transmitted over a = physical communication channel.   

https:= //dl.dropboxusercontent.com/u/43331625/feemarket.pdf

I've = exchanged emails with Greg Maxwell about (what IMO is) an = academic
scenario where the block solutions announcements contain *no = information
at all* about the transactions included in the blocks. =  Although the fee
market would not be healthy in such a = scenario, it is my feeling that
this also requires miners to = relinquish their ability to construct
blocks according to their own = volition (i.e., the system would already
be centralized).  I = look forward to a white paper demonstrating otherwise!

Best = regards,
Peter



On 2015-08-29, at 2:07 PM, Matt Corallo = via bitcoin-dev
<bitcoin-dev@lists.li= nuxfoundation.org
<mailto:bitcoin-dev@l= ists.linuxfoundation.org>> wrote:

I believe it was pointed out previously in the discussion = of the Peter R
paper, but I'll repeat it here so that its visible - = this seems to
ignore the effects of transaction validation caches and = block
compression protocols. Many large miners already have their own = network
to relay blocks around the globe with only a few bytes on the = wire at
block-time, and there is also the bitcoinrelaynetwork.org
<= ;http://bitcoinrelaynetwork.org= > network, which
does the same for smaller miners, albeit with = slightly less efficiency.
Also, transaction validation time upon = receiving a block can be rather
easily made negligible (ie the only = validation time you should have is
the DB modify-utxo-set time). = Thus, the increased orphan risk for
including a transaction can be = reduced to a very, very tiny amount,
making the optimal blocksize, = essentially, including everything that
you're confident is in the = mempool of other reasonably large miners.

Matt

On 08/29/15 = 16:43, Daniele Pinna via bitcoin-dev wrote:
I'd like to submit this paper to the dev-list which = analyzes how miner
advantages scale with network and mempool = properties in a scenario of
uncapped block sizes. The work proceeds, = in a sense, from where Peter
R's work left off correcting a mistake = and addressing the critiques made
by the community to his = work.

The main result of the work is a detailed analysis of = mining advantages
(defined as the added profit per unit of hash) as a = function of miner
hashrate. In it, I show how large block subsidies = (or better, low
mempool fees-to-subsidy ratios) incentivize the = pooling of large
hashrates due to the steady increasing of marginal = profits as hashrates
grow.

The paper also shows that part of = the large advantage the large miners
have today is due to there being = a barrier to entry into a
high-efficiency mining class which has = access to expected profits an
order of magnitude larger than everyone = else. As block subsidies
decrease, this high-efficiency class is = expected to vanish leading to a
marginal profit structure which = decreases as a function of hashrate.

This work has vacuumed my = entire life for the past two weeks leading me
to lag behind on a lot = of work. I apologize for typos which I may not
have seen. I stand by = for any comments the community may have and look
forward to = reigniting consideration of a block size scaling proposal
(BIP101) = which, due to the XT fork drama, I believe has been placed
hastily = and undeservedly on the chopping block.

https://www.scribd.com/doc/276849= 939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets


Regards,
Daniele


_____________________________= __________________
bitcoin-dev mailing = list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfounda= tion.org/mailman/listinfo/bitcoin-dev

________________= _______________________________
bitcoin-dev mailing list
bitcoin-dev@lists.li= nuxfoundation.org
<mailto:bitcoin-dev@lists.linuxfoundation.org&= gt;
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
<= /blockquote>
_____________________________________________= __
bitcoin-dev mailing list
bitcoin-dev@lists.li= nuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinf= o/bitcoin-dev


= --Apple-Mail=_98028B39-A9FC-4DC8-80A8-17F1FAA1DACA--