Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Sdiyk-0003T2-EU for bitcoin-development@lists.sourceforge.net; Sun, 10 Jun 2012 14:18:54 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 74.125.82.53 as permitted sender) client-ip=74.125.82.53; envelope-from=pieter.wuille@gmail.com; helo=mail-wg0-f53.google.com; Received: from mail-wg0-f53.google.com ([74.125.82.53]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Sdiyj-0005pT-EE for bitcoin-development@lists.sourceforge.net; Sun, 10 Jun 2012 14:18:54 +0000 Received: by wgbfm10 with SMTP id fm10so2038898wgb.10 for ; Sun, 10 Jun 2012 07:18:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.143.73 with SMTP id k51mr4734129wej.55.1339337927108; Sun, 10 Jun 2012 07:18:47 -0700 (PDT) Received: by 10.223.107.5 with HTTP; Sun, 10 Jun 2012 07:18:47 -0700 (PDT) Received: by 10.223.107.5 with HTTP; Sun, 10 Jun 2012 07:18:47 -0700 (PDT) In-Reply-To: <20120610090357.GA25567@vps7135.xlshosting.net> References: <20120610090357.GA25567@vps7135.xlshosting.net> Date: Sun, 10 Jun 2012 16:18:47 +0200 Message-ID: From: Pieter Wuille To: Bitcoin Dev Content-Type: multipart/alternative; boundary=0016e6db2e7061274e04c21ee9be X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (pieter.wuille[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Sdiyj-0005pT-EE Subject: [Bitcoin-development] BIP22/getmemorypool X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jun 2012 14:18:54 -0000 --0016e6db2e7061274e04c21ee9be Content-Type: text/plain; charset=ISO-8859-1 Hello everyone, Luke's getmemorypool/BIP22 pull request has been open for a long time, and didn't receive too much discussion. I think that having a stable and flexible API for negotiating block generation is important to be standardized. The fact that it allows moving block generation to specialized programs is a step in the right direction. However, it seems to me that too few people (myself included) understand all the details of BIP22 (or don't care enough) to judge their necessity. I gave up trying to follow all design decisions some time ago, and as it seems I'm not alone, nobody liked merging support for it in the Satoshi client. This is a pity, and I hope the situation can be improved soon. I'm sorry for being this late with these comments, but I think it's essential that the standard is not more complex than necessary (making it as easy as possible to write either servers or clients for it), and perhaps even more important, that its purpose and intended use cases are clear. From what I understand, the three subrequests are template, proposal and submit. The general idea is that 1) a miner requests a block template 2) builds/modifies a block based on this, and optionally uses propose to check whether the server is willing to accept it before doing work 3) submits when valid proof-of-work is found I'd like to see this process described in the BIP at least, it too me way too long to extract this. Regarding the block template: is there a particular reason for sending the full transactions (serialized in hex) both in templates and submissions? The server will always need to have access to the transaction database anyway, and clients will (afaics) rarely care about the actual transactions. My suggestion would be to just transfer txids - if the client is interested in the actual transactions, we have the gettransaction RPC call already. This seems to be captured by the several "submit/*" and "share/*" variations, but making it optional seems way more complex than just limiting the API to that way of working. That's another thing that bothers me in the standard: too many optional features. In particular, I understand the usefulness of having some flexibility in what miner clients are allowed to modify, but I'm unconvinced we need 10 individually selectable variations. In particular: * coinbase outputs: can we just add a list of required coinbase outputs (amount + scriptPubKey) to the template? If no generation+fee amount remains, nothing can be added. * coinbase input: put the required part in the template; miners can always add whatever they like. Is there any known use case where a server would not allow a client to add stuff to the coinbase? * noncerange limiting: if coinbase input variation is not limited, there is certainly no reason to limit nonceranges. This adds unnecessary complexity to clients, in my option. * time/*: put a minimum and maximum timestamp in the template (i believe those are already there anyway). Anything in between is valid. * transactions/add: what is the use case? * transactions/remove: i'd just standarize on having all transactions be removable (perhaps except those marked 'required'). * prevblock: one getmemorypool per new block shouldn't be a problem imho, so do a longpoll instead of having the client able to modify prevblock themselves. One more thing that I do not like is often several ways for specifying the same behaviour. For example, txrequires specifies that the first N transactions are mandatory, a 'required' field in the transaction list itself specifies that that transaction is mandatory, and the lack of transactions as variation means that they must not be touched at all. Pick one way that is flexible enough, and discard the others. In summary, I'd like to see the standard simplified. I have no problem merging code that makes getmemorypool compliant to a standard that is agreed upon, but like to understand it first. In my opinion - but I'm certainly open to discussion here - the standard could be simplified to: * getblocktemplate: create a new block template, and return it. The result contains: * bits, previousblockhash, version: as to be used in block * curtime, maxtimeoff, maxtimeoff: client chooses a timestamp between (curtime - local_time_at_receipt + local_time), decreased by mintimeoff and increased maxtimeoff * expires, sigoplimit, sizelimit: unchanged * subsidy: amount generated (5000000000 for now) * coinbaseaux: what generated coinbase's scriptSig must start with * coinbaseoutputs: list of objects, each specifying a required coinbase output. Each has fields: * amount: sent amount * scriptPubKey: hex serialized of the output script * transactions: list of object, each specifying a suggested transaction (except for the coinbase) in the generated block. Each has fields: * txid: transaction id * depends: list of dependencies (txids of earlier objects in this same transactions list). * fee: fee generated by this transaction, which increases the max output of the coinbase. * required: if present, transaction may not be dropped. * submitblocktemplate: submit an object containing a hex serialized block header, hex serialized coinbase transaction, and a list of txids. Returns true or string describing the problem. Proof of work is checked last, so that error is only returned if there is no other problem with the suggested block (this allows it to replace both propose and submit). Are there important use cases I'm missing? -- Pieter --0016e6db2e7061274e04c21ee9be Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable


Hello everyone,

Luke's getmemorypool/BIP22 pull request has been open for a long tim= e, and didn't receive too much discussion.

I think that having a stable and flexible API for negotiating block gene= ration is important to be standardized. The fact that it allows moving bloc= k generation to specialized programs is a step in the right direction. Howe= ver, it seems to me that too few people (myself included) understand all th= e details of BIP22 (or don't care enough) to judge their necessity. I g= ave up trying to follow all design decisions some time ago, and as it seems= I'm not alone, nobody liked merging support for it in the Satoshi clie= nt. This is a pity, and I hope the situation can be improved soon.

I'm sorry for being this late with these comments, but I think it= 9;s essential that the standard is not more complex than necessary (making = it as easy as possible to write either servers or clients for it), and perh= aps even more important, that its purpose and intended use cases are clear.=

From what I understand, the three subrequests are template, proposal and= submit. The general idea is that
=A01) a miner requests a block template
=A02) builds/modifies a block based on this, and optionally uses propose to= check whether the server is willing to accept it before doing work
=A03) submits when valid proof-of-work is found
I'd like to see this process described in the BIP at least, it too me w= ay too long to extract this.

Regarding the block template: is there a particular reason for sending t= he full transactions (serialized in hex) both in templates and submissions?= The server will always need to have access to the transaction database any= way, and clients will (afaics) rarely care about the actual transactions. M= y suggestion would be to just transfer txids - if the client is interested = in the actual transactions, we have the gettransaction RPC call already. Th= is seems to be captured by the several "submit/*" and "share= /*" variations, but making it optional seems way more complex than jus= t limiting the API to that way of working.

That's another thing that bothers me in the standard: too many optio= nal features. In particular, I understand the usefulness of having some fle= xibility in what miner clients are allowed to modify, but I'm unconvinc= ed we need 10 individually selectable variations. In particular:
* coinbase outputs: can we just add a list of required coinbase outputs (am= ount + scriptPubKey) to the template? If no generation+fee amount remains, = nothing can be added.
* coinbase input: put the required part in the template; miners can always = add whatever they like. Is there any known use case where a server would no= t allow a client to add stuff to the coinbase?
* noncerange limiting: if coinbase input variation is not limited, there is= certainly no reason to limit nonceranges. This adds unnecessary complexity= to clients, in my option.
* time/*: put a minimum and maximum timestamp in the template (i believe th= ose are already there anyway). Anything in between is valid.
* transactions/add: what is the use case?
* transactions/remove: i'd just standarize on having all transactions b= e removable (perhaps except those marked 'required').
* prevblock: one getmemorypool per new block shouldn't be a problem imh= o, so do a longpoll instead of having the client able to modify prevblock t= hemselves.

One more thing that I do not like is often several ways for specifying t= he same behaviour. For example, txrequires specifies that the first N trans= actions are mandatory, a 'required' field in the transaction list i= tself specifies that that transaction is mandatory, and the lack of transac= tions as variation means that they must not be touched at all. Pick one way= that is flexible enough, and discard the others.

In summary, I'd like to see the standard simplified. I have no probl= em merging code that makes getmemorypool compliant to a standard that is ag= reed upon, but like to understand it first.

In my opinion - but I'm certainly open to discussion here - the stan= dard could be simplified to:
* getblocktemplate: create a new block template, and return it. The result = contains:
=A0* bits, previousblockhash, version: as to be used in block
=A0* curtime, maxtimeoff, maxtimeoff: client chooses a timestamp between (c= urtime - local_time_at_receipt + local_time), decreased by mintimeoff and i= ncreased maxtimeoff
=A0* expires, sigoplimit, sizelimit: unchanged
=A0* subsidy: amount generated (5000000000 for now)
=A0* coinbaseaux: what generated coinbase's scriptSig must start with =A0* coinbaseoutputs: list of objects, each specifying a required coinbase = output. Each has fields:
=A0 =A0* amount: sent amount
=A0 =A0* scriptPubKey: hex serialized of the output script
=A0* transactions: list of object, each specifying a suggested transaction = (except for the coinbase) in the generated block. Each has fields:
=A0 =A0* txid: transaction id
=A0 =A0* depends: list of dependencies (txids of earlier objects in this sa= me transactions list).
=A0 =A0* fee: fee generated by this transaction, which increases the max ou= tput of the coinbase.
=A0 =A0* required: if present, transaction may not be dropped.
* submitblocktemplate: submit an object containing a hex serialized block h= eader, hex serialized coinbase transaction, and a list of txids. Returns tr= ue or string describing the problem. Proof of work is checked last, so that= error is only returned if there is no other problem with the suggested blo= ck (this allows it to replace both propose and submit).

Are there important use cases I'm missing?

--
Pieter

--0016e6db2e7061274e04c21ee9be--