Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
	helo=mx.sourceforge.net)
	by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <jgarzik@bitpay.com>) id 1Uobze-0003i0-Sp
	for bitcoin-development@lists.sourceforge.net;
	Mon, 17 Jun 2013 16:09:22 +0000
Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of bitpay.com
	designates 74.125.82.180 as permitted sender)
	client-ip=74.125.82.180; envelope-from=jgarzik@bitpay.com;
	helo=mail-we0-f180.google.com; 
Received: from mail-we0-f180.google.com ([74.125.82.180])
	by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1Uobzb-0007uo-11
	for bitcoin-development@lists.sourceforge.net;
	Mon, 17 Jun 2013 16:09:22 +0000
Received: by mail-we0-f180.google.com with SMTP id w56so2465829wes.25
	for <bitcoin-development@lists.sourceforge.net>;
	Mon, 17 Jun 2013 09:09:12 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=google.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:x-gm-message-state;
	bh=vnKtlY6wRr5DmucoJZUZS1jssBG6epe451c7IuVpXcU=;
	b=FuPukY3Un+FxUtCP6Wz5h/zaghxyM8Br0WZXIQb8BZRoqYd5neBody2u4dmfCA+xJ2
	EEZiWIByeuF2OqsA4XCpZGWB6g+heDrZKkPXQ+WJsJSuuIHGmWo15KwXe1lPEcs3ch7S
	p0toc5qMs5sPWR9zA4++OZxpA0Q+K20bzKrH0hhmkFiZpCSMiGhAYiqf2dena2Acs2QF
	+QxJysMKf/g6ztR4/AmgVVXJCmUuxegOjxZAf+xXQJ1megHor1ABcTFWvrO+30Zop3q1
	VrLr9bC3cO5TlkhTBF4tSGBNUHV6rXez0QgbXAeFN16uvBG1d+zQn7IgiTckt2vMVRkK
	kGOQ==
MIME-Version: 1.0
X-Received: by 10.194.90.244 with SMTP id bz20mr8461463wjb.69.1371482161823;
	Mon, 17 Jun 2013 08:16:01 -0700 (PDT)
Received: by 10.194.178.69 with HTTP; Mon, 17 Jun 2013 08:16:01 -0700 (PDT)
In-Reply-To: <20130614200654.GB11509@petertodd.org>
References: <20130527111149.GB8955@tilt> <20130531165758.GA29135@petertodd.org>
	<20130610210913.GA17242@petertodd.org>
	<201306102123.15732.luke@dashjr.org>
	<20130614200654.GB11509@petertodd.org>
Date: Mon, 17 Jun 2013 11:16:01 -0400
Message-ID: <CAJHLa0P7gndRqubREeQqbVNy_nJgyu7CUdgK9no14RCahxx7Cg@mail.gmail.com>
From: Jeff Garzik <jgarzik@bitpay.com>
To: Peter Todd <pete@petertodd.org>
Content-Type: text/plain; charset=ISO-8859-1
X-Gm-Message-State: ALoCoQmBomjnYmH7u5XHyDMRuJ3TK10a9G5JcEtzCHwt9tCgUR4bRLXWviyjWyUPYKF9mJLvcU6X
X-Spam-Score: -1.6 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	-0.0 SPF_PASS               SPF: sender matches SPF record
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Uobzb-0007uo-11
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Decentralizing mining
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Mon, 17 Jun 2013 16:09:23 -0000

On Fri, Jun 14, 2013 at 4:06 PM, Peter Todd <pete@petertodd.org> wrote:
> It strikes me that this would work best if the pool has a mempool with
> child-pays-for-parent support where conflicts *are* allowed.
>
> IE you record whatever transactions you know about, conflicting or not,
> calculate which ones gives you the most fees/kb, and then figure out
> which set of non-conflicting ones are optimal. Of course, "optimal" is
> the knapsack problem...
>
> Now you can easilly tell the miners working on shares for you which tx's
> would be optimal if they wish to know, and at the same time whatever
> shares they send you are most likely to include transactions in your
> mempool inventory, and thus they can send just tx hashes to reduce
> bandwidth.
>
>
> Part of the broader issue that when we send peers INV advertisements we
> should be telling them what the fee/kb is so our peers can prioritize
> properly. That'd also help for the case where you want to broadcast two
> transactions in a row, but the pair is only profitable because the
> second is paying the fee for the first.

Interesting proposals, particularly this last.  The net result impact
is, however, that which was criticized in at least one forum thread:
replace-with-higher-fee.


> Speaking of, the way we tell peers about new blocks is really
> suboptimal: we tell every peer, in no particular order, about a new
> block via a block INV message, and then we give them the new block in
> parallel. I was looking through comp-sci papers on optimal
> flood-fill/gossip algorithms for random graph networks and it appears
> that optimal is to spend all your bandwidth to send the message to your
> fastest peer first, followed by your next fastest and so on. This works
> best because you get the exponential growth scaling faster by
> propagating the message as "deep" as possible in the network, and it
> then can flood outwards from there. Just sorting the peer list by
> #inv-recevied/time when doing INV pushes and when attending to incoming
> messages would probably be a big improvement.

In terms of packet size, I would like to look into the network-wide
costs of simply broadcasting block header + coinbase TX + TX list.  I
bet miners would love to opt into that.


>> > If the share does meet difficulty, hand it to both the pool and the
>> > local bitcoind. Should hand it to the pool first though, because the
>> > pool likely has the fastest block propagation, then hand it to local
>> > bitcoind. An optimized version may want to have some record of measured
>> > bandwidth - this applies Bitcoin in general too, although also has other
>> > issues.
>>
>> Currently, BFGMiner is doing submission to the pool, waiting for a response,
>> then submitting to a local bitcoind. This is because the pool might need to
>> receive/record the share before it processes the block on bitcoind, or you
>> could lose credit for it. The response from the pool is rather small (a single
>> TCP packet), so this shouldn't delay much longer.
>
> Right, I guess the pool wants to be sure you were actually the one who
> found the share, rather than just someone who was lucky enough to see it
> on the network and submitted it as your own.
>
>> > ## Reducing bandwidth
>> >
>> > How about for normal shares we just pass the block header, and have the
>> > pool randomly pick a subset of transactions to audit? Any fraud cancels
>> > the users shares. This will work best in conjunction with a UTXO proof
>> > tree to prove fees, or by just picking whole shares randomly to audit.
>>
>> Might as well just use higher difficulty shares (each one audited) for the
>> same effect. Block proposals allow the miner to tell the pool its transaction
>> set once (per txset change) for any number of shares.
>
> That's a good point - the current practice most pools seem to follow of
> about a share per second seems very excessive to me. On the other hand,
> it does have good user optics. The right solution might be something
> akin to P2Pool where the UI level is telling the user shares are being
> found so it's clear "stuff is happening", but under the hood only a
> small subset are ever sent to the pool.

With the onslaught of ASIC mining, most big pools are past a share per
second.  Variable difficulty or set-to-higher-difficulty quickly
became the norm, almost out of necessity.

Personally, I think most pools should target at _least_ 5-10 seconds
per share, no matter the strength of the miner.

-- 
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc.      https://bitpay.com/