To: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
From: Matt Corallo <lf-lists@mattcorallo.com>
Openpgp: preference=signencrypt
Autocrypt: addr=matt@bluematt.me; prefer-encrypt=mutual; keydata=
	xsFNBFPBqxoBEADnCJofKjaMG2qC0RkJAFgKNjyulG0YwrMpIMQJbD3ymYd8gty97RCvb+3E
	r5Lg1W6EwRXSDx88xmZSM+T2hSu5s9pUkpHWY+Ulcc8Iu4aMHiWvw7c6N4KPttMVjvinj9mk
	2aWiU1nr/nba7ZXFo4kjd9YPY5GBVEC1KvAY84AYg0lJvnzddq0CEjwOtaxFKRMIkRHeUhMK
	aBdQdKU4CjzsIt6qdmFoqgGdPWQ4w8NhGALxdeSX8LF5zwSeun/FZ6KIQqSo+3yBFrEQHMzM
	iIm3Xn8MZla/UwmaBDB+IfjSoKp5hM+l6NvCuOc9BmczK2+ZGEJNnvsooCpYoN3GK8G1dkWP
	FNUKTEG4QB0KsE6v877HfpG8mbhyNxKeZ36JdeoFjuTAwp4mP/c80focJjh8UNZsNCmunkI6
	erjxPtqRkVkU6BcLfFtGTrzM3+I8m2YJdw2ZvCzsEPU62X9QsCWrhHZsxhwS8/wtzGJpPTTO
	gx2sJQQW2E6E3m6wgKXWWj9oMo61wWwdeiDtJUnzT0DoB9CJLA1zT4V6lTrATPH8AMJ+73qh
	0nAEEXSL1kPwDLGlkZtylyqRi1UcAGg9r7bW/KLTEcIueSVSLm4JHtcZ0eyOYSgQ1rvutIc1
	xyoDKqkR015fJ7k4z6kuKhc/Zf9mXPcV19zLyAoZYsc0YcGLkQARAQABzS5NYXR0IENvcmFs
	bG8gKEJsdWVNYXR0KSA8bWF0dEBtYXR0Y29yYWxsby5jb20+wsF8BBMBCAAmAhsDBQkSzAMA
	Ah4BAheABQJWOsqGBAsHCQgFFQgKCQsFFgIDAQAACgkQibu4Zj4uZc5vSw/+JPZVYW3BSaCh
	rGqHTRvotdyD9y+IjcyTfaTbZruXU5xLZEP674PPoaYxPs47R1eykQqmC5dJvC4cKaYwnYXb
	q5I7j6PFpQuuzExmomfVzzyaJADhAJhxJdjpoDdwv2K9i9CBsIU9bdAA4fKWfP0+OKICSUD2
	xQfmkvt/2cDwm6+Mh1LHixSc0wA1TEXRF9+h9H7P68JJKxICfi4rZBl8HWPveiDHUMJXJiyg
	gVXwz4uU2WviDA5jqXklvH9oiyN+sKetk1GgCZ9jQqqB0JLtW7fGIIaUxPHMrOfq8VWulWQx
	HRBwBuZHsSKRd6jQAS6f3unjinpPfBPKDX1RGkTgSAZiPYcbIDtzQM95nCpR6fsib+Zii9Oe
	aABNkoXc+2YixNCnSolEzvIMi7rMjRncu81bcOWkU3O2VarITaPs0bAPReigtfbIPfcJN+SL
	5DCMcHThA593kxPfV5omIydJGniH9Z2OsJZQI75Vu5rvBo1lnnFvba2iuGRYFoUykXfMrrgt
	usm/2fWtQ7GHPaQ8bpRUN0BQuHX5G83MGSAScK1UzxSIeqazSwTBwGpDiJHsKqOErJ7tjoI7
	te9VVrplX7V7Y62GHX8JKJqM6OKGObFSiRqKD4l8sSndblPMU/WfIXE4mUoGIZW74GnFbz9O
	TQrx+l25KHnvZ5Ny4jIcxa/OwU0EVjrNnwEQANqyuRs5RspGyUZiuGvmFl+eD0emXZ08GoWh
	IGsKQA7vArckwCYvTabVOHUa3O9ChOdFrxzicqI/rO+V6cOBLqnuKtir+5JEYj61ShXnw6zh
	tvi/tfSdBoI7hE7WF2uzuiOXdmDzDYXnRnzPlYPtry/1BPFDD8rjG/9cTgXUa+y8w3E+0ah/
	KZ9OMMJsmXrAJ6QHcPnuTqQTgJIKVGQIroqmmwiRWjMNvToY85nnYKy7/WU1FCy5KLBrEGWZ
	2xHeHrJufuZ/SBmOc8Y745r5MDI7cGYgWUjQz3cUi/osF9BOwFA4B2BgnmLzoseoiQFuCCH5
	iVLjno8z80AIIrUAqaA2C/SjeiW9enFXKg13FpzKG98q6PJ7161oXidYr9dfr5aFKMOJzWbe
	FhpH0v7NJA4YN6ivUibVB/gBRLVQqWY3lhdHYQkfmNd//gQ/gzV7FTJmJh+Sq/hFvu6i78o4
	BqVN5iZYoOVbsD1IGzmUh4MjQXFty3ojSANKBAfvA2vl57VmQt9jD7zdEIYxVNmTBHt5oLtg
	0Sxi2ujGL24av7e5Qs2Eh3YJ9tXYiNAUHJdPGK1EWldr7rAKqGy2EU/Uu7DuQAo0C6WUI1ye
	nSQTC7a95j3Lxej/vTwRhwVesT7x5hsJ/WxZmiE+041q4GRN1F+xf45Ymi3ncIf9I1lUSGu/
	ABEBAAHCwWUEGAEIAA8FAlY6zZ8CGwwFCQWjmoAACgkQibu4Zj4uZc7uCxAAk5fvdX+ayVCL
	sTktFj6FkpYNcWwG/VAQ6F6TbFtNJpwXx+9zwgkzXZGuJf5Hu1xfVlflWaeyouADP08RfbeV
	hU4HAKoUYou+iGrRGsMkZmLrblay7pBQC7RnGLsQTMGbWAZgr6v15oJZllcKkmsQYETzzWlG
	/sPiS+p6oiVmgPOoh2HIqny8xs0sgfPvsvAElm0iPAK58V1BaTBrS5w3h5H2FsnjIhom9W/6
	KPxbTffeVh4jn/1z5Z+xVygdyZ2Ebdar22g6gPNznmbLEWSZO8HYtTSx8h7iZ1beb1XpgEiJ
	sFlbdZC/njslaOnfeF2JKB30LPQpoV+nZas89qqpQc84kySMgB7DXMN+lg55MnsvqhsPPA2/
	hv3oiljo0IWODTDNq3vkXkQ1aFJGAxKh3QNngkKpuj9TsBNtlXfAEFAmJhReukAmBZMB+5x6
	3LMBhsBLk4F8FO9lcHULAazliFIc8Eu2fgbbjjB6VsLQ352A7y2agtK3vaKjK013/hqoSYuG
	FU4Q3+Y6hJaiSB6P9syNcjJtsSiS8t9VmF7ZUAR3q958Q8XyGNXHtHfqxVj55mWSt9+eH0Wl
	h7ODk4L2GaXF1xWdBEyt6xypapmm2Z9/qcuX7X3eZcC4hMHgwHsfZh+LvIzeup3E8wYkr6gZ
	uWCHuEtIxpFG8chItvKHjHI=
Message-ID: <92215b88-75a4-6be7-dec6-89c567a74a9a@mattcorallo.com>
Date: Tue, 5 Jun 2018 14:44:57 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.8.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Subject: [bitcoin-dev] [BIP Proposal] BetterHash Mining Protocol Replacements
Precedence: list

Been working on this one for a while, so its already been through a few
rounds of feeback (thanks to all those who already have provided feedback)!

At a high level, this meets a few goals:

1) Replace getblocktemplate with something that is both more performant
(no JSON encoding, no full transactions sent over the wire to update a
job, hence we can keep the same CTransactionRef in Bitcoin Core making
lots of validation things way faster), more robust for consensus changes
(no need to add protocol changes to add commitments ala SegWit in the
future), and moves more block-switching logic inside of the work
provider (allowing Bitcoin Core to better optimize work switching as it
knows more than an outside pool server, specifically we can play more
games with how we do mempool eviction, empty block mining, and not
mining fresh transactions more easily by moving to a more "push" model
from the normal "pull" getblocktemplate implementation).

2) Replace Stratum with something more secure (sign messages when
applicable, without adding too much overhead to the pool), simpler to
implement (not JSON-wrapped-hex, no 32-byte-swapped-per-4-byte-byteorder
insanity), and better-defined (a clearly written spec, encompassing the
various things shoved backwards into stratum like suggested difficulty
in the password field and device identification by setting user to
"user.device") with VENDOR_MESSAGEs provided for extensibility instead
of conflicting specifications from various different vendors.

3) Provide the ability for a pool to accept work which the users of the
pool selected the transactions for, providing strong decentralization
pressure by removing the network-level centralization attacks pools can
do (or be compromised and used to perform) while still allowing them
full control of payout management and variance reduction.

While (1) and (2) stand on their own, making it all one set of protocols
to provide (3) provides at least the opportunity for drastically better
decentralization in Bitcoin mining in the future.

The latest version of the full BIP draft can be found at
https://github.com/TheBlueMatt/bips/blob/betterhash/bip-XXXX.mediawiki
and implementations of the work-generation part at
https://github.com/TheBlueMatt/bitcoin/commits/2018-02-miningserver and
pool/proxy parts at https://github.com/TheBlueMatt/mining-proxy (though
note that both implementations are currently on a slightly out-of-date
version of the protocol, I hope to get them brought up to date in the
coming day or two and make them much more full-featured. The whole stack
has managed to mine numerous testnet blocks on several different types
of hardware).

Matt