Return-Path: <jlrubin@mit.edu> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id C2EA0AA5 for <bitcoin-dev@lists.linuxfoundation.org>; Sun, 2 Jun 2019 21:32:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D8965844 for <bitcoin-dev@lists.linuxfoundation.org>; Sun, 2 Jun 2019 21:32:36 +0000 (UTC) Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (authenticated bits=0) (User authenticated as jlrubin@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id x52LWYve003684 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT) for <bitcoin-dev@lists.linuxfoundation.org>; Sun, 2 Jun 2019 17:32:35 -0400 Received: by mail-ed1-f44.google.com with SMTP id w37so23926043edw.4 for <bitcoin-dev@lists.linuxfoundation.org>; Sun, 02 Jun 2019 14:32:35 -0700 (PDT) X-Gm-Message-State: APjAAAW6P85Ll1jtTd3uiXtd1Ya10AzIPW6BYCgnOILkMCqK4YBAnFz4 w0jjAVEOuXX9o4rnp9nmFyh/YtmkRMZNzacBqp8= X-Google-Smtp-Source: APXvYqw9IxWIVeVmuekFpAJEjKVMTMyU13BLJIuDbuGxN8CP1gORVWr3Ci+Shoukk2/tQhBkfC7JAKnUfB3VRtp20UQ= X-Received: by 2002:a17:906:30c4:: with SMTP id b4mr3062002ejb.276.1559511153764; Sun, 02 Jun 2019 14:32:33 -0700 (PDT) MIME-Version: 1.0 References: <CAD5xwhjSj82YYuQHHbwgSLvUNV2RDY0b=yMYeLj-p6j7PpS9-Q@mail.gmail.com> <CAMZUoKm9aZMCnJzP3YvLZ5oycDG-pss8cYZwan2N71_gc95GDg@mail.gmail.com> In-Reply-To: <CAMZUoKm9aZMCnJzP3YvLZ5oycDG-pss8cYZwan2N71_gc95GDg@mail.gmail.com> From: Jeremy <jlrubin@mit.edu> Date: Sun, 2 Jun 2019 14:32:20 -0700 X-Gmail-Original-Message-ID: <CAD5xwhieGt3n+PrnZaqpSGM-fXUnEP_BWMtXH77KuPzSLGF79A@mail.gmail.com> Message-ID: <CAD5xwhieGt3n+PrnZaqpSGM-fXUnEP_BWMtXH77KuPzSLGF79A@mail.gmail.com> To: "Russell O'Connor" <roconnor@blockstream.io> Content-Type: multipart/alternative; boundary="00000000000057f178058a5dfcde" X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Mon, 03 Jun 2019 15:10:25 +0000 Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> Subject: Re: [bitcoin-dev] OP_SECURETHEBAG (supersedes OP_CHECKOUTPUTSVERIFY) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Sun, 02 Jun 2019 21:32:37 -0000 --00000000000057f178058a5dfcde Content-Type: text/plain; charset="UTF-8" Hi Russell, Thanks for the response. I double checked my work in drafting my response and realized I didn't address all the malleability concerns, I believe I have now (fingers crossed) addressed all points of malleability. *The malleability concerns are as follows:* A TXID is computed as: def txid(self): r = b"" r += struct.pack("<i", self.nVersion) r += ser_vector(self.vin) r += ser_vector(self.vout) r += struct.pack("<I", self.nLockTime) return sha256(r) if the bag hash is just: def get_bag_hash(self): r = b"" r += ser_vector(self.vout) return TaggedHash("BagHash", r) We allow changing a few things: nVersion, nLockTime, scriptSig (per input), number of inputs, nSequence (per input) which can change the TXID/what the transaction does. changing nVersion: can disable BIP68, change TXID changing nLockTime: can change TXID changing nSequence: can change TXID changing number of inputs: half spend problem, change TXID changing scriptsigs: change TXID if co-spent with legacy input Instead, we can use the following digest: def get_bag_hash(self): r = b"" r += struct.pack("<i", self.nVersion) r += struct.pack("<I", self.nLockTime) r += sha256(b"".join(out.serialize() for out in self.vout)) r += sha256(b"".join(struct.pack("<I", inp.nSequence) for inp in self.vin)) r += struct.pack("<Q", len(self.vin)) for inp in self.vin: r += ser_string(inp.scriptSig) return TaggedHash("BagHash", r) which should lock in all the relevant bits. The only part left out is the COutpoint, which can't be known ahead of time (because it depends on the creating txn). Technically, len(vin) is redundant with sha256(b"".join(struct.pack("<I", inp.nSequence) for inp in self.vin)), because the length padding on the hash implied the number of inputs, but I figured it's best to err on explicit. A further benefit (in a CISC sense) of committing to all these values is that we enforce CLTV and CSV semantics for free on OP_SECURETHEBAG scripts, which helps with channels. *Treating OP_SECURETHEBAG as a PUSHDATA:* I agree in theory it's nicer, and am 100% open to implementing it that way. The only concern I have with doing it this way is that it means that a flags must be added to GetOp (or GetOp must be modularized to be per-script version) because it affects script parsing, as opposed to using a multibyte opcode which contains a pushdata, which remain compatible with prior script parsing. I'd like to get rough consensus on the best approach for compatibility with downstream software, hence choosing this option for the draft. Personally, my preference is to *not* do flags and just have a separate parser version which cleans up some of our past sins. We can experiment with a fancier parser (as you've shown in Haskell/Rust/Coq), perhaps even bitwise huffman encoding opcodes to save space on scripts (i.e. the 7 most common opcodes could fit within 3 bits) or whatever else we like. I just didn't want to have the scope creep too far on this particular BIP, but I'm with you that lookahead is a hack compared to an actual parametrized argument. I think you'd also appreciate the template script expansion approach mentioned in the BIP -- it gets around some of these concerns, but requires changes to Taproot. --00000000000057f178058a5dfcde Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:arial,helvetica,sans-serif;font-size:small;color:#000000">Hi Russe= ll,</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,= sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_def= ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color= :#000000">Thanks for the response. I double checked my work in drafting my = response and realized I didn't address all the malleability concerns, I= believe I have now (fingers crossed) addressed all points of malleability.= </div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,san= s-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_defaul= t" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#0= 00000"><b>The malleability concerns are as follows:</b></div><div class=3D"= gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sm= all;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-fam= ily:arial,helvetica,sans-serif;font-size:small;color:#000000">A TXID is com= puted as:</div><div class=3D"gmail_default" style=3D"font-family:arial,helv= etica,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gma= il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small= ;color:#000000">def txid(self):<br></div><div class=3D"gmail_default" style= =3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">= =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 r =3D b""<br>=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0r +=3D struct.pack("<i", self.nVersion)<br>= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_vector(self.vin)<br>=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_vector(self.vout)<br>=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0r +=3D struct.pack("<I", self.nLockTime)<br>= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return sha256(r)<br></div><div class=3D"g= mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma= ll;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-fami= ly:arial,helvetica,sans-serif;font-size:small;color:#000000">if the bag has= h is just:</div><div class=3D"gmail_default" style=3D"font-family:arial,hel= vetica,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gm= ail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:smal= l;color:#000000"><div class=3D"gmail_default" style=3D"font-family:arial,he= lvetica,sans-serif;font-size:small;color:rgb(0,0,0)">def get_bag_hash(self)= :<br></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetic= a,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 r =3D b""<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_= vector(self.vout)</div><div class=3D"gmail_default" style=3D"font-family:ar= ial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0return TaggedHash("BagHash", r)</div><div cla= ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s= ize:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D= "font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">W= e allow changing a few things: nVersion, nLockTime, scriptSig (per input), = number of inputs, nSequence (per input) which can change the TXID/what the = transaction does.</div><div class=3D"gmail_default" style=3D"font-family:ar= ial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div c= lass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font= -size:small;color:rgb(0,0,0)">changing nVersion: can disable BIP68, change = TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica= ,sans-serif;font-size:small;color:rgb(0,0,0)">changing nLockTime: can chang= e TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helveti= ca,sans-serif;font-size:small;color:rgb(0,0,0)">changing nSequence: can cha= nge TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helve= tica,sans-serif;font-size:small;color:rgb(0,0,0)">changing number of inputs= : half spend problem, change TXID</div><div class=3D"gmail_default" style= =3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)= ">changing scriptsigs: change TXID if co-spent with legacy input<br></div><= div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif= ;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" s= tyle=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,= 0,0)">Instead, we can use the following digest:<br></div><div class=3D"gmai= l_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;= color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-fami= ly:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2= =A0 def get_bag_hash(self):<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r =3D b&qu= ot;"<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct.pack("<= ;i", self.nVersion)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct= .pack("<I", self.nLockTime)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0r +=3D sha256(b"".join(out.serialize() for out in self.vout))<= br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D sha256(b"".join(struc= t.pack("<I", inp.nSequence) for inp in self.vin))<br>=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct.pack("<Q", len(self.v= in))<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for inp in self.vin:<br>=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_string(inp.scriptSig)<b= r>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return TaggedHash("BagHash", = r)<br></div><div class=3D"gmail_default" style=3D"font-family:arial,helveti= ca,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gma= il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small= ;color:rgb(0,0,0)">which should lock in all the relevant bits. The only par= t left out is the COutpoint, which can't be known ahead of time (becaus= e it depends on the creating txn). Technically, len(vin) is redundant with = sha256(b"".join(struct.pack("<I", inp.nSequence) for= inp in self.vin)), because the length padding on the hash implied the numb= er of inputs, but I figured it's best to err on explicit.</div><div cla= ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s= ize:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D= "font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">A= further benefit (in a CISC sense) of committing to all these values is tha= t we enforce CLTV and CSV semantics for free on OP_SECURETHEBAG scripts, wh= ich helps with channels.<br></div><div class=3D"gmail_default" style=3D"fon= t-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br><= /div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans= -serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_defa= ult" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:= rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:ari= al,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b>Treating OP_SE= CURETHEBAG as a PUSHDATA:</b></div><div class=3D"gmail_default" style=3D"fo= nt-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b><= br></b></div><div class=3D"gmail_default" style=3D"font-family:arial,helvet= ica,sans-serif;font-size:small;color:rgb(0,0,0)">I agree in theory it's= nicer, and am 100% open to implementing it that way. The only concern I ha= ve with doing it this way is that it means that a flags must be added to Ge= tOp (or GetOp must be modularized to be per-script version) because it affe= cts script parsing, as opposed to using a multibyte opcode which contains a= pushdata, which remain compatible with prior script parsing.<br></div><div= class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo= nt-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" styl= e=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0= )">I'd like to get rough consensus on the best approach for compatibili= ty with downstream software, hence choosing this option for the draft.</div= ><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-ser= if;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default"= style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(= 0,0,0)">Personally, my preference is to *not* do flags and just have a sepa= rate parser version which cleans up some of our past sins. We can experimen= t with a fancier parser (as you've shown in Haskell/Rust/Coq), perhaps = even bitwise huffman encoding opcodes to save space on scripts (i.e. the 7 = most common opcodes could fit within 3 bits) or whatever else we like. I ju= st didn't want to have the scope creep too far on this particular BIP, = but I'm with you that lookahead is a hack compared to an actual paramet= rized argument.<br></div><div class=3D"gmail_default" style=3D"font-family:= arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div= class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo= nt-size:small;color:rgb(0,0,0)">I think you'd also appreciate the templ= ate script expansion approach mentioned in the BIP -- it gets around some o= f these concerns, but requires changes to Taproot.<br></div><div class=3D"g= mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma= ll;color:rgb(0,0,0)"><br></div></div><div class=3D"gmail_default" style=3D"= font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br><= /div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans= -serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_default= " style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#00= 0000"><br></div></div></div> --00000000000057f178058a5dfcde--