MIME-Version: 1.0
References: <CAD5xwhjSj82YYuQHHbwgSLvUNV2RDY0b=yMYeLj-p6j7PpS9-Q@mail.gmail.com>
	<CAMZUoKm9aZMCnJzP3YvLZ5oycDG-pss8cYZwan2N71_gc95GDg@mail.gmail.com>
In-Reply-To: <CAMZUoKm9aZMCnJzP3YvLZ5oycDG-pss8cYZwan2N71_gc95GDg@mail.gmail.com>
From: Jeremy <jlrubin@mit.edu>
Date: Sun, 2 Jun 2019 14:32:20 -0700
Message-ID: <CAD5xwhieGt3n+PrnZaqpSGM-fXUnEP_BWMtXH77KuPzSLGF79A@mail.gmail.com>
To: "Russell O'Connor" <roconnor@blockstream.io>
Content-Type: multipart/alternative; boundary="00000000000057f178058a5dfcde"
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] OP_SECURETHEBAG (supersedes OP_CHECKOUTPUTSVERIFY)
Precedence: list

--00000000000057f178058a5dfcde
Content-Type: text/plain; charset="UTF-8"

Hi Russell,

Thanks for the response. I double checked my work in drafting my response
and realized I didn't address all the malleability concerns, I believe I
have now (fingers crossed) addressed all points of malleability.

*The malleability concerns are as follows:*

A TXID is computed as:

def txid(self):
         r = b""
         r += struct.pack("<i", self.nVersion)
         r += ser_vector(self.vin)
         r += ser_vector(self.vout)
         r += struct.pack("<I", self.nLockTime)
         return sha256(r)

if the bag hash is just:

def get_bag_hash(self):
         r = b""
         r += ser_vector(self.vout)
         return TaggedHash("BagHash", r)

We allow changing a few things: nVersion, nLockTime, scriptSig (per input),
number of inputs, nSequence (per input) which can change the TXID/what the
transaction does.

changing nVersion: can disable BIP68, change TXID
changing nLockTime: can change TXID
changing nSequence: can change TXID
changing number of inputs: half spend problem, change TXID
changing scriptsigs: change TXID if co-spent with legacy input

Instead, we can use the following digest:

    def get_bag_hash(self):
         r = b""
         r += struct.pack("<i", self.nVersion)
         r += struct.pack("<I", self.nLockTime)
         r += sha256(b"".join(out.serialize() for out in self.vout))
         r += sha256(b"".join(struct.pack("<I", inp.nSequence) for inp in
self.vin))
         r += struct.pack("<Q", len(self.vin))
         for inp in self.vin:
             r += ser_string(inp.scriptSig)
         return TaggedHash("BagHash", r)

which should lock in all the relevant bits. The only part left out is the
COutpoint, which can't be known ahead of time (because it depends on the
creating txn). Technically, len(vin) is redundant with
sha256(b"".join(struct.pack("<I", inp.nSequence) for inp in self.vin)),
because the length padding on the hash implied the number of inputs, but I
figured it's best to err on explicit.

A further benefit (in a CISC sense) of committing to all these values is
that we enforce CLTV and CSV semantics for free on OP_SECURETHEBAG scripts,
which helps with channels.



*Treating OP_SECURETHEBAG as a PUSHDATA:*

I agree in theory it's nicer, and am 100% open to implementing it that way.
The only concern I have with doing it this way is that it means that a
flags must be added to GetOp (or GetOp must be modularized to be per-script
version) because it affects script parsing, as opposed to using a multibyte
opcode which contains a pushdata, which remain compatible with prior script
parsing.

I'd like to get rough consensus on the best approach for compatibility with
downstream software, hence choosing this option for the draft.

Personally, my preference is to *not* do flags and just have a separate
parser version which cleans up some of our past sins. We can experiment
with a fancier parser (as you've shown in Haskell/Rust/Coq), perhaps even
bitwise huffman encoding opcodes to save space on scripts (i.e. the 7 most
common opcodes could fit within 3 bits) or whatever else we like. I just
didn't want to have the scope creep too far on this particular BIP, but I'm
with you that lookahead is a hack compared to an actual parametrized
argument.

I think you'd also appreciate the template script expansion approach
mentioned in the BIP -- it gets around some of these concerns, but requires
changes to Taproot.

--00000000000057f178058a5dfcde
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-family:arial,helvetica,sans-serif;font-size:small;color:#000000">Hi Russe=
ll,</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:#000000">Thanks for the response. I double checked my work in drafting my =
response and realized I didn&#39;t address all the malleability concerns, I=
 believe I have now (fingers crossed) addressed all points of malleability.=
</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,san=
s-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_defaul=
t" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#0=
00000"><b>The malleability concerns are as follows:</b></div><div class=3D"=
gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sm=
all;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-fam=
ily:arial,helvetica,sans-serif;font-size:small;color:#000000">A TXID is com=
puted as:</div><div class=3D"gmail_default" style=3D"font-family:arial,helv=
etica,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:#000000">def txid(self):<br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">=
=C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 r =3D b&quot;&quot;<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0r +=3D struct.pack(&quot;&lt;i&quot;, self.nVersion)<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_vector(self.vin)<br>=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_vector(self.vout)<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0r +=3D struct.pack(&quot;&lt;I&quot;, self.nLockTime)<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return sha256(r)<br></div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-fami=
ly:arial,helvetica,sans-serif;font-size:small;color:#000000">if the bag has=
h is just:</div><div class=3D"gmail_default" style=3D"font-family:arial,hel=
vetica,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gm=
ail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:smal=
l;color:#000000"><div class=3D"gmail_default" style=3D"font-family:arial,he=
lvetica,sans-serif;font-size:small;color:rgb(0,0,0)">def get_bag_hash(self)=
:<br></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetic=
a,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2=A0 =C2=A0 =C2=A0=
=C2=A0 r =3D b&quot;&quot;<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_=
vector(self.vout)</div><div class=3D"gmail_default" style=3D"font-family:ar=
ial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0return TaggedHash(&quot;BagHash&quot;, r)</div><div cla=
ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s=
ize:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D=
"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">W=
e allow changing a few things: nVersion, nLockTime, scriptSig (per input), =
number of inputs, nSequence (per input) which can change the TXID/what the =
transaction does.</div><div class=3D"gmail_default" style=3D"font-family:ar=
ial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div c=
lass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font=
-size:small;color:rgb(0,0,0)">changing nVersion: can disable BIP68, change =
TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica=
,sans-serif;font-size:small;color:rgb(0,0,0)">changing nLockTime: can chang=
e TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helveti=
ca,sans-serif;font-size:small;color:rgb(0,0,0)">changing nSequence: can cha=
nge TXID</div><div class=3D"gmail_default" style=3D"font-family:arial,helve=
tica,sans-serif;font-size:small;color:rgb(0,0,0)">changing number of inputs=
: half spend problem, change TXID</div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)=
">changing scriptsigs: change TXID if co-spent with legacy input<br></div><=
div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif=
;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" s=
tyle=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,=
0,0)">Instead, we can use the following digest:<br></div><div class=3D"gmai=
l_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;=
color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-fami=
ly:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">=C2=A0 =C2=
=A0 def get_bag_hash(self):<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r =3D b&qu=
ot;&quot;<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct.pack(&quot;&lt=
;i&quot;, self.nVersion)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct=
.pack(&quot;&lt;I&quot;, self.nLockTime)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0r +=3D sha256(b&quot;&quot;.join(out.serialize() for out in self.vout))<=
br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D sha256(b&quot;&quot;.join(struc=
t.pack(&quot;&lt;I&quot;, inp.nSequence) for inp in self.vin))<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D struct.pack(&quot;&lt;Q&quot;, len(self.v=
in))<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for inp in self.vin:<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r +=3D ser_string(inp.scriptSig)<b=
r>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return TaggedHash(&quot;BagHash&quot;, =
r)<br></div><div class=3D"gmail_default" style=3D"font-family:arial,helveti=
ca,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:rgb(0,0,0)">which should lock in all the relevant bits. The only par=
t left out is the COutpoint, which can&#39;t be known ahead of time (becaus=
e it depends on the creating txn). Technically, len(vin) is redundant with =
sha256(b&quot;&quot;.join(struct.pack(&quot;&lt;I&quot;, inp.nSequence) for=
 inp in self.vin)), because the length padding on the hash implied the numb=
er of inputs, but I figured it&#39;s best to err on explicit.</div><div cla=
ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s=
ize:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D=
"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">A=
 further benefit (in a CISC sense) of committing to all these values is tha=
t we enforce CLTV and CSV semantics for free on OP_SECURETHEBAG scripts, wh=
ich helps with channels.<br></div><div class=3D"gmail_default" style=3D"fon=
t-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br><=
/div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans=
-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_defa=
ult" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:=
rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:ari=
al,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b>Treating OP_SE=
CURETHEBAG as a PUSHDATA:</b></div><div class=3D"gmail_default" style=3D"fo=
nt-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b><=
br></b></div><div class=3D"gmail_default" style=3D"font-family:arial,helvet=
ica,sans-serif;font-size:small;color:rgb(0,0,0)">I agree in theory it&#39;s=
 nicer, and am 100% open to implementing it that way. The only concern I ha=
ve with doing it this way is that it means that a flags must be added to Ge=
tOp (or GetOp must be modularized to be per-script version) because it affe=
cts script parsing, as opposed to using a multibyte opcode which contains a=
 pushdata, which remain compatible with prior script parsing.<br></div><div=
 class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo=
nt-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" styl=
e=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0=
)">I&#39;d like to get rough consensus on the best approach for compatibili=
ty with downstream software, hence choosing this option for the draft.</div=
><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-ser=
if;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default"=
 style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(=
0,0,0)">Personally, my preference is to *not* do flags and just have a sepa=
rate parser version which cleans up some of our past sins. We can experimen=
t with a fancier parser (as you&#39;ve shown in Haskell/Rust/Coq), perhaps =
even bitwise huffman encoding opcodes to save space on scripts (i.e. the 7 =
most common opcodes could fit within 3 bits) or whatever else we like. I ju=
st didn&#39;t want to have the scope creep too far on this particular BIP, =
but I&#39;m with you that lookahead is a hack compared to an actual paramet=
rized argument.<br></div><div class=3D"gmail_default" style=3D"font-family:=
arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div=
 class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo=
nt-size:small;color:rgb(0,0,0)">I think you&#39;d also appreciate the templ=
ate script expansion approach mentioned in the BIP -- it gets around some o=
f these concerns, but requires changes to Taproot.<br></div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:rgb(0,0,0)"><br></div></div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br><=
/div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans=
-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_default=
" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#00=
0000"><br></div></div></div>

--00000000000057f178058a5dfcde--