Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 1E455ACC for ; Sat, 11 Jul 2015 13:17:56 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com [209.85.220.180]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id DFBBA17A for ; Sat, 11 Jul 2015 13:17:54 +0000 (UTC) Received: by qkbp125 with SMTP id p125so224498546qkb.2 for ; Sat, 11 Jul 2015 06:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0s0mmDq0Slx5McQ92L6FExafDDLv+2XCneWTTqk/Y1w=; b=AgFDmOSJ/fst/XJ6zAxcWfH+iywqeQPTCLLw7xb6KjiLAk4B7yWZVclBaljVE9YSP+ Tqu3j7Le+8i61+1OeOCliQluMvAVfbuBPQsMzkBJcxqb3V5ODRJBuRZqKfQ2+wTtWGrX RxWtP3Mfiuad+L+gASPp+5b2IZdJzbPS276leYYIrspV8vFpLmS4JWw6xJarXD7cXj37 1cwNIo8VaDULblax10aOLIshrVQ2ZakkzSOBjiUnA1b5Yj5tg4NG6zwaE/7l2BOTJ7DA sONwXKnpDjZwWXweZ+H5SnV4PqBPPVid73jHmCl/LJtAE333csFtxjPcrIeZvpgJb8hM hK5w== MIME-Version: 1.0 X-Received: by 10.141.23.136 with SMTP id z130mr43498136qhd.36.1436620674048; Sat, 11 Jul 2015 06:17:54 -0700 (PDT) Received: by 10.140.93.162 with HTTP; Sat, 11 Jul 2015 06:17:53 -0700 (PDT) In-Reply-To: References: Date: Sat, 11 Jul 2015 14:17:53 +0100 Message-ID: From: Tier Nolan To: bitcoin-dev@lists.linuxfoundation.org Content-Type: multipart/alternative; boundary=001a114232b6f429c0051a995037 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue. X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jul 2015 13:17:56 -0000 --001a114232b6f429c0051a995037 Content-Type: text/plain; charset=UTF-8 On Sat, Jul 11, 2015 at 1:09 PM, Nathan Wilcox wrote: > > If there's any cost to non-SPV mining, and the chance that an incoming > block contains only valid transactions is very high, isn't there still an > incentive to make this timeout longer and longer? > The benefit drops off pretty quickly as the timeout increases (and eventually goes negative). You could look at it that headers having 4 states. 1) Valid 2) Probably Valid 3) Probably Invalid 4) Invalid SPV mining puts newly received headers into the "probably valid" category. It builds empty (coinbase only) blocks on top of probably valid headers and build empty blocks on the header. Once it receives the full block, it can change the state to Valid. At that point, it can build full blocks on top of the header. As time passes without the full block being received/validated, it becomes less and less likely that the block is actually valid. The timeout is to recognize that fact. Making the timeout 24 hours is not likely to give the miner much benefit over making it 1-2 minutes. Setting the timeout to low means that the miner sometimes switches away from a header that turns out to be valid. Setting it to high means that the miner ends up staying to long on a header that turns out to be invalid. At some point, the second effect is stronger than the first effect. The timeout needs to be high enough so switching away from valid headers is rare but low enough so that it doesn't stay on invalid headers for ages. If 99% of full blocks are received (and validated) within 30 seconds of the header arriving, then it is reasonable for the miner to assume that if the full block hasn't arrived within 60 seconds of the header arriving, then the header is for an invalid block. Yes. If it's rare enough, then skipping transaction validation saves more > cost than the expected cost of mining atop an invalid block. ... but if > everyone follows that logic, the chance is no longer rare. > SPV miners don't actually produce invalid blocks though (other than building on invalid blocks). The full blocks they produce are still fully checked blocks. > SPV-mining is to prevent hashing hardware from having to waste power when >> it isn't needed. >> >> > It may be less of a problem if (when?) electricity costs dominate hardware >> capital costs. >> > > Oh. This is a different explanation than Peter Todd's I linked to above, > which claimed it was network latency in receiving transactions. > SPV mining is mainly to protect against latency. The reason that matters is that latency means that hashers end up building on blocks even though a new block has been found. You can look at it as wasting hashing power due to latency. In the world where minting fees are very low, there is no point in SPV mining. I assume at the point, the memory pool/queue is a few blocks deep. This means that the pool can create a full block without having to wait for new transactions to be sent in. It still needs to wait for the new full block before it knows which transactions to remove from its memory pool. Pools have to pay their hashers for hashing power. When minting fees are tiny, pools only get income only from tx fees. There is no point in creating empty blocks, since they don't pay anything. Between when the block is found and the pool has a new block ready to mine, there is no incentive for the pool to give out new work. The stratum protocol could be modified so pools can say. (It might already support this) -- block is found by some other pool -- -- pool builds new block -- The cancel command says that the pool will not accept any of the work that has been made obsolete. This gives a window of 20-30 seconds after each block where the pool has invalidated the old work, but does not send new work. Hashers' hardware would be stalled. On the other hand, the pool that found the block could create a new block straight away. There is an incentive for hashers to have a connection to multiple pools. Pools might go pure pay per share. The protocol would say how much they are willing to pay for a share and the local mining proxy would pick the most profitable pool. This eliminates pools having lots of ways of saying how they charge fees, you just connect to lots of pools and go with the one that pays the most. More interestingly, the average fee per byte for transactions in the memory pool is likely to increase as time passes since the last block. When two blocks are found very fast one after another, the second block is likely to have lower average fees. This is because the first block would have included most of the high value transactions and there wasn't enough time for new ones to arrive before the second block was found. Hashers would end up getting variable payouts based on how long since the last block was received. If some miners increase/decrease their output based on the fees the pools offer, then as time passes since the last block, the hashing rate would increase. This reduces the variation in block to block times. For example, if there was 1.5MB of transactions in the memory pool and they all paid the same fee per byte, then the block would be a full block at that rate. However, there would only be 0.5MB of transactions left. This means that the next block would be half full and only be able to pay out 50% of the fee, but the difficulty would be the same. The pay per hash would be 50% lower. Once 0.5MB of new transactions arrive, the fee would be back up to the same as the previous block. If there are major SHA256 altcoins (or side chains), then miners might end up switching between coins. The longer a coin went without a new block being found, the more tx fees available in the memory pool, so the more hashing power would decide to switch to that coin. There could be a situation where adding a few more transactions to the memory pool of a coin would cause a 100X increasing in hashing until the block was found and then it stalling again as the hashing power switches away. This is similar to alt coins getting blasted by coin switching pools and then dropping to almost no power. --001a114232b6f429c0051a995037 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On S= at, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <nathan@leastauthority.= com> wrote:

If there's a= ny cost to non-SPV mining, and the chance that an incoming block contains o= nly valid transactions is very high, isn't there still an incentive to = make this timeout longer and longer?

<= /div>
The benefit drops off pretty quickly as the timeout increases (an= d eventually goes negative).

You could look at it that he= aders having 4 states.

1) Valid
2) Probably= Valid
3) Probably Invalid
4) Invalid

SPV mining puts newly received headers into the "probably val= id" category.

It builds empty (coinbase only) blocks= on top of probably valid headers and build empty blocks on the header.
=
Once it receives the full block, it can change the state to = Valid.=C2=A0 At that point, it can build full blocks on top of the header.<= br>

As time passes without the full block being re= ceived/validated, it becomes less and less likely that the block is actuall= y valid.

The timeout is to recognize that fact.=C2=A0 Mak= ing the timeout 24 hours is not likely to give the miner much benefit over = making it 1-2 minutes.

Setting the timeout to low means t= hat the miner sometimes switches away from a header that turns out to be va= lid.=C2=A0

Setting it to high means that the miner ends = up staying to long on a header that turns out to be invalid.

<= div>At some point, the second effect is stronger than the first effect.=C2= =A0 The timeout needs to be high enough so switching away from valid header= s is rare but low enough so that it doesn't stay on invalid headers for= ages.

If 99% of full blocks are received (and= validated) within 30 seconds of the header arriving, then it is reasonable= for the miner to assume that if the full block hasn't arrived within 6= 0 seconds of the header arriving, then the header is for an invalid block.<= br>

Yes. If it's rare enough, then skipping transaction validatio= n saves more cost than the expected cost of mining atop an invalid block.= =C2=A0 ... but if everyone follows that logic, the chance is no longer rare= .

SPV miners don't actually p= roduce invalid blocks though (other than building on invalid blocks).=C2=A0= The full blocks they produce are still fully checked blocks.
=C2=A0
<= div>SPV-mining is to prevent hashing hardware from having to waste power wh= en it isn't needed.
=C2=A0
It may be less of a problem if (when?) electricity costs dominate ha= rdware capital costs.
=C2=A0
Oh.=C2=A0 This is a different explanation than Peter Todd's I= linked to above, which claimed it was network latency in receiving transac= tions.

SPV mining is mainly= to protect against latency.=C2=A0 The reason that matters is that latency = means that hashers end up building on blocks even though a new block has be= en found.

You can look at it as wasting hashing power due= to latency.

In the world where minting fees a= re very low, there is no point in SPV mining.

I assume at= the point, the memory pool/queue is a few blocks deep.=C2=A0 This means th= at the pool can create a full block without having to wait for new transact= ions to be sent in.

It still needs to wait for the new fu= ll block before it knows which transactions to remove from its memory pool.=

Pools have to pay their hashers for hashing p= ower.=C2=A0 When minting fees are tiny, pools only get income only from tx = fees.=C2=A0

There is no point in creating empty blocks, since they = don't pay anything.

Between when the block is found a= nd the pool has a new block ready to mine, there is no incentive for the po= ol to give out new work.=C2=A0 The stratum protocol could be modified so po= ols can say.=C2=A0 (It might already support this)

<Se= nd work to hashers>

-- block is found by some other po= ol --

<Cancel work for miners>

--= pool builds new block --

<Send new work to hashers>= ;

The cancel command says that the pool will not accept a= ny of the work that has been made obsolete.

Th= is gives a window of 20-30 seconds after each block where the pool has inva= lidated the old work, but does not send new work.=C2=A0 Hashers' hardwa= re would be stalled.

On the other hand, the pool that fou= nd the block could create a new block straight away.=C2=A0 There is an ince= ntive for hashers to have a connection to multiple pools.

Pools might go pure pay per share.=C2=A0 The protocol would say how much t= hey are willing to pay for a share and the local mining proxy would pick th= e most profitable pool.=C2=A0 This eliminates pools having lots of ways of = saying how they charge fees, you just connect to lots of pools and go with = the one that pays the most.

More interestingly, the average fee per byte for transactions in the m= emory pool is likely to increase as time passes since the last block.=C2=A0=

When two blocks are found very fast one after another, the second = block is likely to have lower average fees.=C2=A0 This is because the first= block would have included most of the high value transactions and there wa= sn't enough time for new ones to arrive before the second block was fou= nd.

Hashers would end up getting va= riable payouts based on how long since the last block was received.=C2=A0 I= f some miners increase/decrease their output based on the fees the pools of= fer, then as time passes since the last block, the hashing rate would incre= ase.=C2=A0 This reduces the variation in block to block times.

For example, if there was 1.5MB of transactions= in the memory pool and they all paid the same fee per byte, then the block= would be a full block at that rate.=C2=A0 However, there would only be 0.5= MB of transactions left.=C2=A0 This means that the next block would be half= full and only be able to pay out 50% of the fee, but the difficulty would = be the same.=C2=A0 The pay per hash would be 50% lower.=C2=A0 Once 0.5MB of= new transactions arrive, the fee would be back up to the same as the previ= ous block.

If there are major SHA25= 6 altcoins (or side chains), then miners might end up switching between coi= ns.=C2=A0 The longer a coin went without a new block being found, the more = tx fees available in the memory pool, so the more hashing power would decid= e to switch to that coin.

There cou= ld be a situation where adding a few more transactions to the memory pool o= f a coin would cause a 100X increasing in hashing until the block was found= and then it stalling again as the hashing power switches away.=C2=A0 This = is similar to alt coins getting blasted by coin switching pools and then dr= opping to almost no power.
--001a114232b6f429c0051a995037--