Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Y0X5b-0000P0-0n for bitcoin-development@lists.sourceforge.net; Mon, 15 Dec 2014 14:57:35 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.212.176 as permitted sender) client-ip=209.85.212.176; envelope-from=btcdrak@gmail.com; helo=mail-wi0-f176.google.com; Received: from mail-wi0-f176.google.com ([209.85.212.176]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Y0X5Z-0001Za-Dz for bitcoin-development@lists.sourceforge.net; Mon, 15 Dec 2014 14:57:34 +0000 Received: by mail-wi0-f176.google.com with SMTP id ex7so9127337wid.9 for ; Mon, 15 Dec 2014 06:57:27 -0800 (PST) X-Received: by 10.180.88.33 with SMTP id bd1mr31900989wib.10.1418655447268; Mon, 15 Dec 2014 06:57:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.25.130 with HTTP; Mon, 15 Dec 2014 06:57:07 -0800 (PST) In-Reply-To: <20141215124730.GA8321@savin.petertodd.org> References: <20141215124730.GA8321@savin.petertodd.org> From: Btc Drak Date: Mon, 15 Dec 2014 14:57:07 +0000 Message-ID: To: Peter Todd Content-Type: multipart/alternative; boundary=f46d0444e8fdfe44dc050a427551 X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HK_RANDOM_FROM From username looks random -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.6 HK_RANDOM_ENVFROM Envelope sender username looks random 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (btcdrak[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Y0X5Z-0001Za-Dz Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Recent EvalScript() changes mean CHECKLOCKTIMEVERIFY can't be merged X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2014 14:57:35 -0000 --f46d0444e8fdfe44dc050a427551 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This is a pretty good example about refactoring discipline as well as premature/over optimisation. We all want to see more modular code, but the first steps should just be to relocate blocks of code so everything is more logically organised in smaller files (especially for consensus critical code). Refactoring should come in a second wave preferably after a stable release. Refactoring should be in the pure sense, optimising code with absolutely no change in behaviour. When it comes to actual API changes, I think we need to be a lot more careful and should be considered feature requests and get a lot more scrutiny as we are essentially breaking backwards compatibility. #4890 was pretty much merged with no discussion or thought yet other really simple and uncontroversial PRs remain unmerged for months. A key question in the case of EvalScript() would have been, "why are we passing txTo and nIn here, and are there any future use cases that might require them? Why should this be removed from the API and the entire method signature changed?". BC breaks always need strong justification. So I've expressed my concern a few times about the speed and frequency of refactoring and also the way it's being done. I am not alone, as others not directly connected with the Bitcoin Core project have also expressed concerns about the number of refactorings "for the sake of refactoring", especially of consensus critical code. Careful as we may be, we know from history that small edge case bugs can creep in very easily and cause a lot of unforeseen problems. BtcDrak On Mon, Dec 15, 2014 at 12:47 PM, Peter Todd wrote: > > BtcDrak was working on rebasing my CHECKLOCKTIMEVERIFY=C2=B9 patch to mas= ter a > few > days ago and found a fairly large design change that makes merging it > currently > impossible. Pull-req #4890=C2=B2, specifically commit c7829ea7, changed t= he > EvalScript() function to take an abstract SignatureChecker object, > removing the > txTo and nIn arguments that used to contain the transaction the script wa= s > in > and the txin # respectively. CHECKLOCKTIMEVERIFY needs txTo to obtain the > nLockTime field of the transaction, and it needs nIn to obtain the > nSequence of > the txin. > > We need to fix this if CHECKLOCKTIMEVERIFY is to be merged. > > Secondly, that this change was made, and the manner in which is was made, > is I > think indicative of a development process that has been taking significan= t > risks with regard to refactoring the consensus critical codebase. I know = I > personally have had a hard time keeping up with the very large volume of > code > being moved and changed for the v0.10 release, and I know BtcDrak - who i= s > keeping Viacoin up to date with v0.10 - has also had a hard time giving t= he > changes reasonable review. The #4890 pull-req in question had no ACKs at > all, > and only two untested utACKS, which I find worrying for something that ma= de > significant consensus critical code changes. > > While it would be nice to have a library encapsulating the consensus code= , > this > shouldn't come at the cost of safety, especially when the actual users of > that > library or their needs is still uncertain. This is after all a > multi-billion > project where a simple fork will cost miners alone tens of thousands of > dollars > an hour; easily much more if it results in users being defrauded. That's > also > not taking into account the significant negative PR impact and loss of > trust. I > personally would recommend *not* upgrading to v0.10 due to these issues. > > A much safer approach would be to keep the code changes required for a > consensus library to only simple movements of code for this release, acce= pt > that the interface to that library won't be ideal, and wait until we have > feedback from multiple opensource projects with publicly evaluatable code > on > where to go next with the API. > > 1) https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki > 2) https://github.com/bitcoin/bitcoin/pull/4890 > > -- > 'peter'[:-1]@petertodd.org > 00000000000000001b18a596ecadd07c0e49620fb71b16f9e41131df9fc52fa6 > > > -------------------------------------------------------------------------= ----- > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=3D164703151&iu=3D/4140/ostg= .clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --f46d0444e8fdfe44dc050a427551 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
This is a pretty good example about refactoring discipline= as well as premature/over optimisation.

We all want to = see more modular code, but the first steps should just be to relocate block= s of code so everything is more logically organised in smaller files (espec= ially for consensus critical code). Refactoring should come in a second wav= e preferably after a stable release. Refactoring should be in the pure sens= e, optimising code with absolutely no change in behaviour.

When it comes to actual API changes, I think we need to be a lot m= ore careful and should be considered feature requests and get a lot more sc= rutiny as we are essentially breaking backwards compatibility. #4890 was pr= etty much merged with no discussion or thought yet other really simple and = uncontroversial PRs remain unmerged for months. A key question in the case = of EvalScript() would have been, "why are we passing txTo and nIn here= , and are there any future use cases that might require them? Why should th= is be removed from the API and the entire method signature changed?". = BC breaks always need strong justification.

So I've = expressed my concern a few times about the speed and frequency of refactori= ng and also the way it's being done. I am not alone, as others not dire= ctly connected with the Bitcoin Core project have also expressed concerns a= bout the number of refactorings "for the sake of refactoring", es= pecially of consensus critical code. Careful as we may be, we know from his= tory that small edge case bugs can creep in very easily and cause a lot of = unforeseen problems.

BtcDrak


On Mon, Dec 15, 2014 at 12:47 PM, Peter Todd <pete@petertodd= .org> wrote:
BtcDrak was workin= g on rebasing my CHECKLOCKTIMEVERIFY=C2=B9 patch to master a few
days ago and found a fairly large design change that makes merging it curre= ntly
impossible. Pull-req #4890=C2=B2, specifically commit c7829ea7, changed the=
EvalScript() function to take an abstract SignatureChecker object, removing= the
txTo and nIn arguments that used to contain the transaction the script was = in
and the txin # respectively. CHECKLOCKTIMEVERIFY needs txTo to obtain the nLockTime field of the transaction, and it needs nIn to obtain the nSequenc= e of
the txin.

We need to fix this if CHECKLOCKTIMEVERIFY is to be merged.

Secondly, that this change was made, and the manner in which is was made, i= s I
think indicative of a development process that has been taking significant<= br> risks with regard to refactoring the consensus critical codebase. I know I<= br> personally have had a hard time keeping up with the very large volume of co= de
being moved and changed for the v0.10 release, and I know BtcDrak - who is<= br> keeping Viacoin up to date with v0.10 - has also had a hard time giving the=
changes reasonable review. The #4890 pull-req in question had no ACKs at al= l,
and only two untested utACKS, which I find worrying for something that made=
significant consensus critical code changes.

While it would be nice to have a library encapsulating the consensus code, = this
shouldn't come at the cost of safety, especially when the actual users = of that
library or their needs is still uncertain. This is after all a multi-billio= n
project where a simple fork will cost miners alone tens of thousands of dol= lars
an hour; easily much more if it results in users being defrauded. That'= s also
not taking into account the significant negative PR impact and loss of trus= t. I
personally would recommend *not* upgrading to v0.10 due to these issues.
A much safer approach would be to keep the code changes required for a
consensus library to only simple movements of code for this release, accept=
that the interface to that library won't be ideal, and wait until we ha= ve
feedback from multiple opensource projects with publicly evaluatable code o= n
where to go next with the API.

1) https://github.com/bitcoin/bips/blob/master/bip-0065.m= ediawiki
2) https://github.com/bitcoin/bitcoin/pull/4890

--
'peter'[:-1]@pet= ertodd.org
00000000000000001b18a596ecadd07c0e49620fb71b16f9e41131df9fc52fa6

---------------------------------------------------------= ---------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & mo= re
Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gam= pad/clk?id=3D164703151&iu=3D/4140/ostg.clktrk
__________________= _____________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--f46d0444e8fdfe44dc050a427551--