path: root/97/e103ffada6e8835f08d14eb9fd6d63a4be0b9e

Return-Path: <jeremy.l.rubin@gmail.com>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
 by lists.linuxfoundation.org (Postfix) with ESMTP id E67D9C000B
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat,  5 Mar 2022 12:20:18 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp2.osuosl.org (Postfix) with ESMTP id D36DA40195
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat,  5 Mar 2022 12:20:18 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level: 
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp2.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp2.osuosl.org ([127.0.0.1])
 by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id uyHQVXdjTuOw
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat,  5 Mar 2022 12:20:16 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [IPv6:2a00:1450:4864:20::12b])
 by smtp2.osuosl.org (Postfix) with ESMTPS id 0716740022
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat,  5 Mar 2022 12:20:15 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id b9so18550223lfv.7
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 05 Mar 2022 04:20:15 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=XiYqqU49sAhsSixQDhn1d1t6v8VfFfz+5EBgUGXZkQY=;
 b=p0xfk7YgnVsNmXJ+6F6ee64iq5nVxHOIEGTUFwvlOkxaUO1KJ40tNKH19Q1A/ga7NY
 jCK4i8uYPw9eB2iQfdTM+7srakiy0DtZxSYD06siVHvk5NiWtlH8zjGbRdwMpSAiQHt5
 WSQv1UcEqFfBDKQG4QHHq671RWw9C6OKIyZHErupl1SpF9RPK59cLwCRyfsOlgB64cyJ
 5eLEgbiJ+Z/AaeX+dakB9waUBK/qDcrACnS9jUXA7YBqZ4KPSG4cXuLZOTdnudbYzm8F
 uWwtJp3hmnrjiBfg2J1D7Q6C1zcq+a9ss7LvZKaWz3mqxMjIC+9q7LzlZNB7UN3+3Jy+
 GtGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=XiYqqU49sAhsSixQDhn1d1t6v8VfFfz+5EBgUGXZkQY=;
 b=H2VvWwCUhGX6IpUdLMDw2UPg7kJlJ4gn2cqbqUPeRVvPV0eRBuEWtF/KdwbkRu8LBY
 yv8KrOdYbhqtH6sCh1PWb9VFip780Ebn+mqasDVvtj7ZMQdowhXc33KNIBcUbuiKcse8
 +bBPmRSUtwfhGwiES7SVS4EZE+qcpXZGo1zKQebNBjDyqaMUlEscJwoqHoTVylBEAxL2
 h9CAJwhdCC2Y5+kn2TArXCKraOF5qTh2y21wed24Qp9cIlWnhxifllFypdrdlBQqJucF
 PMVg7VNLVQ0FaWnH/Kjjmag3hSThRX+Zg6yLU8CWbDxYX3Wfno+UrgulDRwCbgN0ekLi
 hcMw==
X-Gm-Message-State: AOAM531VxyAbDXvOTRI3gW37BhE8aQck/TWuYXiMW+JVXtpi37ApPo0i
 3eogJlwYXYeGz3d3kpHwPnJU55ZE59B7bBF8JTxonY9N/cjM6A==
X-Google-Smtp-Source: ABdhPJyJtVCEnqMq4fRyZ7q7nhhSLxDBHlP7qzcWyqxWqFMHrh103S477DuNGQhz67OeL8owwHGKSu9HbhP9cM+psVw=
X-Received: by 2002:a05:6512:114c:b0:443:4d18:86c0 with SMTP id
 m12-20020a056512114c00b004434d1886c0mr2217506lfg.226.1646482813582; Sat, 05
 Mar 2022 04:20:13 -0800 (PST)
MIME-Version: 1.0
References: <CAD5xwhgXE9sB-hdzz_Bgz6iEA-M5-Yu2VOn1qRzkaq+DdVsgmw@mail.gmail.com>
 <20220305055924.GB5308@erisian.com.au>
In-Reply-To: <20220305055924.GB5308@erisian.com.au>
From: Jeremy Rubin <jeremy.l.rubin@gmail.com>
Date: Sat, 5 Mar 2022 12:20:02 +0000
Message-ID: <CAD5xwhjG7kN=LatZRpQxqmaqtoRP31BcyeN2zHtOUsGt=6oJ3w@mail.gmail.com>
To: Anthony Towns <aj@erisian.com.au>
Content-Type: multipart/alternative; boundary="0000000000003b82f205d977a744"
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Annex Purpose Discussion: OP_ANNEX,
 Turing Completeness, and other considerations
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 05 Mar 2022 12:20:19 -0000

--0000000000003b82f205d977a744
Content-Type: text/plain; charset="UTF-8"

On Sat, Mar 5, 2022 at 5:59 AM Anthony Towns <aj@erisian.com.au> wrote:

> On Fri, Mar 04, 2022 at 11:21:41PM +0000, Jeremy Rubin via bitcoin-dev
> wrote:
> > I've seen some discussion of what the Annex can be used for in Bitcoin.
>
>
> https://www.erisian.com.au/meetbot/taproot-bip-review/2019/taproot-bip-review.2019-11-12-19.00.log.html
>
> includes some discussion on that topic from the taproot review meetings.
>
> The difference between information in the annex and information in
> either a script (or the input data for the script that is the rest of
> the witness) is (in theory) that the annex can be analysed immediately
> and unconditionally, without necessarily even knowing anything about
> the utxo being spent.
>

I agree that should happen, but there are cases where this would not work.
E.g., imagine OP_LISP_EVAL + OP_ANNEX... and then you do delegation via the
thing in the annex.

Now the annex can be executed as a script.



>
> The idea is that we would define some simple way of encoding (multiple)
> entries into the annex -- perhaps a tag/length/value scheme like
> lightning uses; maybe if we add a lisp scripting language to consensus,
> we just reuse the list encoding from that? -- at which point we might
> use one tag to specify that a transaction uses advanced computation, and
> needs to be treated as having a heavier weight than its serialized size
> implies; but we could use another tag for per-input absolute locktimes;
> or another tag to commit to a past block height having a particular hash.
>

Yes, this seems tough to do without redefining checksig to allow partial
annexes. Hence thinking we should make our current checksig behavior
require it be 0, future operations should be engineered with specific
structured annex in mind.



>
> It seems like a good place for optimising SIGHASH_GROUP (allowing a group
> of inputs to claim a group of outputs for signing, but not allowing inputs
> from different groups to ever claim the same output; so that each output
> is hashed at most once for this purpose) -- since each input's validity
> depends on the other inputs' state, it's better to be able to get at
> that state as easily as possible rather than having to actually execute
> other scripts before your can tell if your script is going to be valid.
>

I think SIGHASH_GROUP could be some sort of mutable stack value, not ANNEX.
you want to be able to compute what range you should sign, and then the
signature should cover the actual range not the argument itself.

Why sign the annex literally?

Why require that all signatures in one output sign the exact same digest?
What if one wants to sign for value and another for value + change?



>
> > The BIP is tight lipped about it's purpose
>
> BIP341 only reserves an area to put the annex; it doesn't define how
> it's used or why it should be used.
>
>
It does define how it's used, Checksig must commit to it. Were there no
opcodes dependent on it I would agree, and that would be preferable.




> > Essentially, I read this as saying: The annex is the ability to pad a
> > transaction with an additional string of 0's
>
> If you wanted to pad it directly, you can do that in script already
> with a PUSH/DROP combo.
>

You cannot, because the push/drop would not be signed and would be
malleable.

The annex is not malleable, so it can be used to this as authenticated
padding.



>
> The point of doing it in the annex is you could have a short byte
> string, perhaps something like "0x010201a4" saying "tag 1, data length 2
> bytes, value 420" and have the consensus intepretation of that be "this
> transaction should be treated as if it's 420 weight units more expensive
> than its serialized size", while only increasing its witness size by
> 6 bytes (annex length, annex flag, and the four bytes above). Adding 6
> bytes for a 426 weight unit increase seems much better than adding 426
> witness bytes.
>
>
Yes, that's what I say in the next sentence,

*> Or, we might somehow make the witness a small language (e.g., run length
encoded zeros) such that we can very quickly compute an equivalent number
of zeros to 'charge' without actually consuming the space but still
consuming a linearizable resource... or something like that.*

so I think we concur on that.



> > Introducing OP_ANNEX: Suppose there were some sort of annex pushing
> opcode,
> > OP_ANNEX which puts the annex on the stack
>
> I think you'd want to have a way of accessing individual entries from
> the annex, rather than the annex as a single unit.
>

Or OP_ANNEX + OP_SUBSTR + OP_POVARINTSTR? Then you can just do 2 pops for
the length and the tag and then get the data.


>
> > Now suppose that I have a computation that I am running in a script as
> > follows:
> >
> > OP_ANNEX
> > OP_IF
> >     `some operation that requires annex to be <1>`
> > OP_ELSE
> >     OP_SIZE
> >     `some operation that requires annex to be len(annex) + 1 or does a
> > checksig`
> > OP_ENDIF
> >
> > Now every time you run this,
>
> You only run a script from a transaction once at which point its
> annex is known (a different annex gives a different wtxid and breaks
> any signatures), and can't reference previous or future transactions'
> annexes...
>
>
In a transaction validator, yes. But in a satisfier, no.

And it doesn't break the signatures if we add the ability to only sign over
a part of an annex either/multiple annexes, since the annex could be
mutable partially.


Not true about accessing previous TXNs annexes. All coins spend from
Coinbase transactions. If you can get the COutpoint you're spending, you
can get the parent of the COutpoint... and iterate backwards so on and so
forth. Then you have the CB txn, which commits to the tree of wtxids. So
you get previous transactions annexes comitted there.


For future transactions, you can, as a miner with decent hashrate you could
promise what your Coinbase transaction would be for a future block and what
the Outputs would be, and then you can pop open that as well... but you
can't show valid PoW for that one so I'm not sure that's different than
authenticated data. But where it does have a use is that you could, if you
had OP_COUTPOINTVERIFY, say that this coin is only spendable if a miner
mines the specific block that you want at a certain height (e.g., with only
your txn in it?) and then they can claim the outpoint in the future... so
maybe there is something there bizzare that can happen with that
capability....



> > Because the Annex is signed, and must be the same, this can also be
> > inconvenient:
>
> The annex is committed to by signatures in the same way nVersion,
> nLockTime and nSequence are committed to by signatures; I think it helps
> to think about it in a similar way.
>

nSequence, yes, nLockTime is per-tx.

BTW i think we now consider nSeq/nLock to be misdesigned given desire to
vary these per-input/per-tx....\

so if the annex is like these perhaps it's also misdesigned.

>
> > Suppose that you have a Miniscript that is something like: and(or(PK(A),
> > PK(A')), X, or(PK(B), PK(B'))).
> >
> > A or A' should sign with B or B'. X is some sort of fragment that might
> > require a value that is unknown (and maybe recursively defined?) so
> > therefore if we send the PSBT to A first, which commits to the annex, and
> > then X reads the annex and say it must be something else, A must sign
> > again. So you might say, run X first, and then sign with A and C or B.
> > However, what if the script somehow detects the bitstring WHICH_A WHICH_B
> > and has a different Annex per selection (e.g., interpret the bitstring
> as a
> > int and annex must == that int). Now, given and(or(K1, K1'),... or(Kn,
> > Kn')) we end up with needing to pre-sign 2**n annex values somehow...
> this
> > seems problematic theoretically.
>
> Note that you need to know what the annex will contain before you sign,
> since the annex is committed to via the signature. If "X" will need
> entries in the annex that aren't able to be calculated by the other
> parties, then they need to be the first to contribute to the PSBT, not A.
>
> I think the analogy to locktimes would be "I need the locktime to be at
> least block 900k, should I just sign that now, or check that nobody else
> is going to want it to be block 950k or something? Or should I just sign
> with nLockTime at 900k, 910k, 920k, 930k, etc and let someone else pick
> the right one?" The obvious solution is just to work out what the
> nLockTime should be first, then run signing rounds. Likewise, work out
> what the annex should be first, then run the signing rounds.
>


Yes, my point is this is computationally hard to do sometimes.

>
> CLTV also has the problem that if you have one script fragment with
> CLTV by time, and another with CLTV by height, you can't come up with
> an nLockTime that will ever satisfy both. If you somehow have script
> fragments that require incompatible interpretations of the annex, you're
> likewise going to be out of luck.
>
>
Yes, see above. If we don't know how the annex will be structured or used,
this is the point of this thread....

We need to drill down how to not introduce these problems.



> Having a way of specifying locktimes in the annex can solve that
> particular problem with CLTV (different inputs can sign different
> locktimes, and you could have different tags for by-time/by-height so
> that even the same input can have different clauses requiring both),
> but the general problem still exists.
>
> (eg, you might have per-input by-height absolute locktimes as annex
> entry 3, and per-input by-time absolute locktimes as annex entry 4,
> so you might convert:
>
>  "900e3 CLTV DROP" -> "900e3 3 PUSH_ANNEX_ENTRY GREATERTHANOREQUAL VERIFY"
>
>  "500e6 CLTV DROP" -> "500e6 4 PUSH_ANNEX_ENTRY GREATERTHANOREQUAL VERIFY"
>
> for height/time locktime checks respectively)
>
> > Of course this wouldn't be miniscript then. Because miniscript is just
> for
> > the well behaved subset of script, and this seems ill behaved. So maybe
> > we're OK?
>
> The CLTV issue hit miniscript:
>
> https://medium.com/blockstream/dont-mix-your-timelocks-d9939b665094


Maybe the humour didn't hit -- we can only define well behaved as best we
know, and the solution was to re-define miniscript to only be the well
defined subset of miniscript once the bug in the spec was found.



>
> > It seems like one good option is if we just go on and banish the
> OP_ANNEX.
> > Maybe that solves some of this? I sort of think so. It definitely seems
> > like we're not supposed to access it via script, given the quote from
> above:
>
> How the annex works isn't defined, so it doesn't make any sense to
> access it from script. When how it works is defined, I expect it might
> well make sense to access it from script -- in a similar way that the
> CLTV and CSV opcodes allow accessing nLockTime and nSequence from script.
>

That's false: CLTV and CSV expressly do not allow accessing it from script,
only lower bounding it (and transitively proving that it was not of the
other flavour).

So you can't actually get the exact nLockTime / Sequence on the stack
(exception: if you use the maximum allowable value, then there are no other
values...)


Given that it's not defined at all, that's why I'm skeptical about signing
it at all presently.

If theres a future upgrade, it would be compatible as we can add new
sighash flags to cover that.


> > One solution would be to... just soft-fork it out. Always must be 0. When
> > we come up with a use case for something like an annex, we can find a way
> > to add it back.
>
> The point of reserving the annex the way it has been is exactly this --
> it should not be used now, but when we agree on how it should be used,
> we have an area that's immediately ready to be used.


> (For the cases where you don't need script to enforce reasonable values,
> reserving it now means those new consensus rules can be used immediately
> with utxos that predate the new consensus rules -- so you could update
> offchain contracts from per-tx to per-input locktimes immediately without
> having to update the utxo on-chain first)
>

I highly doubt that we will not need new sighash flags once it is ready to
allow partial covers of the annex, e.g. like the structured ones described
above.

We're already doing a soft fork for the new annex rules, so this isn't a
big deal...

Legacy outputs can use these new sighash flags as well, in theory (maybe
I'll do a post on why we shouldn't...)



Cheers,

Jeremy

--0000000000003b82f205d977a744
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></di=
v></div><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On=
 Sat, Mar 5, 2022 at 5:59 AM Anthony Towns &lt;<a href=3D"mailto:aj@erisian=
.com.au">aj@erisian.com.au</a>&gt; wrote:<br></div><blockquote class=3D"gma=
il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-le=
ft-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">On Fri,=
 Mar 04, 2022 at 11:21:41PM +0000, Jeremy Rubin via bitcoin-dev wrote:<br>
&gt; I&#39;ve seen some discussion of what the Annex can be used for in Bit=
coin. <br>
<br>
<a href=3D"https://www.erisian.com.au/meetbot/taproot-bip-review/2019/tapro=
ot-bip-review.2019-11-12-19.00.log.html" rel=3D"noreferrer" target=3D"_blan=
k">https://www.erisian.com.au/meetbot/taproot-bip-review/2019/taproot-bip-r=
eview.2019-11-12-19.00.log.html</a><br>
<br>
includes some discussion on that topic from the taproot review meetings.<br=
>
<br>
The difference between information in the annex and information in<br>
either a script (or the input data for the script that is the rest of<br>
the witness) is (in theory) that the annex can be analysed immediately<br>
and unconditionally, without necessarily even knowing anything about<br>
the utxo being spent.<br></blockquote><div><br></div><div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:rgb(0,0,0)">I agree that should happen, but there are cases where th=
is would not work. E.g., imagine OP_LISP_EVAL + OP_ANNEX... and then you do=
 delegation via the thing in the annex.</div><div class=3D"gmail_default" s=
tyle=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,=
0,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,hel=
vetica,sans-serif;font-size:small;color:rgb(0,0,0)">Now the annex can be ex=
ecuted as a script.</div><br></div><div>=C2=A0</div><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-l=
eft-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
The idea is that we would define some simple way of encoding (multiple)<br>
entries into the annex -- perhaps a tag/length/value scheme like<br>
lightning uses; maybe if we add a lisp scripting language to consensus,<br>
we just reuse the list encoding from that? -- at which point we might<br>
use one tag to specify that a transaction uses advanced computation, and<br=
>
needs to be treated as having a heavier weight than its serialized size<br>
implies; but we could use another tag for per-input absolute locktimes;<br>
or another tag to commit to a past block height having a particular hash.<b=
r></blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"fo=
nt-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Yes,=
 this seems tough to do without redefining checksig to allow partial annexe=
s. Hence thinking we should make our current checksig behavior require it b=
e 0, future operations should be engineered with specific structured annex =
in mind.</div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:s=
olid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
It seems like a good place for optimising SIGHASH_GROUP (allowing a group<b=
r>
of inputs to claim a group of outputs for signing, but not allowing inputs<=
br>
from different groups to ever claim the same output; so that each output<br=
>
is hashed at most once for this purpose) -- since each input&#39;s validity=
<br>
depends on the other inputs&#39; state, it&#39;s better to be able to get a=
t<br>
that state as easily as possible rather than having to actually execute<br>
other scripts before your can tell if your script is going to be valid.<br>=
</blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"font=
-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">I thin=
k SIGHASH_GROUP could be some sort of mutable stack value, not ANNEX. you w=
ant to be able to compute what range you should sign, and then the signatur=
e should cover the actual range not the argument itself.</div><div class=3D=
"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:s=
mall;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font=
-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Why si=
gn the annex literally?</div><div class=3D"gmail_default" style=3D"font-fam=
ily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div>=
<div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-seri=
f;font-size:small;color:rgb(0,0,0)">Why require that all signatures in one =
output sign the exact same digest? What if one wants to sign for value and =
another for value=C2=A0+ change?</div><br></div><div>=C2=A0</div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width=
:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-lef=
t:1ex">
<br>
&gt; The BIP is tight lipped about it&#39;s purpose<br>
<br>
BIP341 only reserves an area to put the annex; it doesn&#39;t define how<br=
>
it&#39;s used or why it should be used.<br>
<br></blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">It=
 does define how it&#39;s used, Checksig must commit to it. Were there no o=
pcodes dependent on it I would agree, and that would be preferable.</div><b=
r></div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:so=
lid;border-left-color:rgb(204,204,204);padding-left:1ex">
&gt; Essentially, I read this as saying: The annex is the ability to pad a<=
br>
&gt; transaction with an additional string of 0&#39;s <br>
<br>
If you wanted to pad it directly, you can do that in script already<br>
with a PUSH/DROP combo.<br></blockquote><div><br></div><div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:rgb(0,0,0)">You cannot, because the push/drop would not be signed =
and would be malleable.</div><div class=3D"gmail_default" style=3D"font-fam=
ily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div>=
<div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-seri=
f;font-size:small;color:rgb(0,0,0)">The annex is not malleable, so it can b=
e used to this as authenticated padding.</div><br></div><div>=C2=A0</div><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);pad=
ding-left:1ex">
<br>
The point of doing it in the annex is you could have a short byte<br>
string, perhaps something like &quot;0x010201a4&quot; saying &quot;tag 1, d=
ata length 2<br>
bytes, value 420&quot; and have the consensus intepretation of that be &quo=
t;this<br>
transaction should be treated as if it&#39;s 420 weight units more expensiv=
e<br>
than its serialized size&quot;, while only increasing its witness size by<b=
r>
6 bytes (annex length, annex flag, and the four bytes above). Adding 6<br>
bytes for a 426 weight unit increase seems much better than adding 426<br>
witness bytes.<br>
<br></blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Ye=
s, that&#39;s what I say in the next sentence,</div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:rgb(0,0,0)"><i><br></i></div><div class=3D"gmail_default" style=3D"font-fa=
mily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><i>&gt; O=
r, we might somehow make the witness a small language (e.g., run length enc=
oded zeros) such that we can very quickly compute an equivalent number of z=
eros to &#39;charge&#39; without actually consuming the space but still con=
suming a linearizable resource... or something like that.</i></div><br></di=
v><div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sa=
ns-serif;font-size:small;color:rgb(0,0,0)">so I think we concur on that.</d=
iv><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border=
-left-color:rgb(204,204,204);padding-left:1ex">
&gt; Introducing OP_ANNEX: Suppose there were some sort of annex pushing op=
code,<br>
&gt; OP_ANNEX which puts the annex on the stack<br>
<br>
I think you&#39;d want to have a way of accessing individual entries from<b=
r>
the annex, rather than the annex as a single unit.<br></blockquote><div><br=
></div><div><div class=3D"gmail_default" style=3D"font-family:arial,helveti=
ca,sans-serif;font-size:small;color:rgb(0,0,0)">Or OP_ANNEX + OP_SUBSTR + O=
P_POVARINTSTR? Then you can just do 2 pops for the length and the tag and t=
hen get the data.</div></div><div>=C2=A0</div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-st=
yle:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
&gt; Now suppose that I have a computation that I am running in a script as=
<br>
&gt; follows:<br>
&gt; <br>
&gt; OP_ANNEX<br>
&gt; OP_IF<br>
&gt;=C2=A0 =C2=A0 =C2=A0`some operation that requires annex to be &lt;1&gt;=
`<br>
&gt; OP_ELSE<br>
&gt;=C2=A0 =C2=A0 =C2=A0OP_SIZE<br>
&gt;=C2=A0 =C2=A0 =C2=A0`some operation that requires annex to be len(annex=
) + 1 or does a<br>
&gt; checksig`<br>
&gt; OP_ENDIF<br>
&gt; <br>
&gt; Now every time you run this,<br>
<br>
You only run a script from a transaction once at which point its<br>
annex is known (a different annex gives a different wtxid and breaks<br>
any signatures), and can&#39;t reference previous or future transactions&#3=
9;<br>
annexes...<br>
<br></blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">In=
 a transaction validator, yes. But in a satisfier, no.</div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-f=
amily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">And it d=
oesn&#39;t break the signatures if we add the ability to only sign over a p=
art of an annex either/multiple annexes, since the annex could be mutable p=
artially.</div><div class=3D"gmail_default" style=3D"font-family:arial,helv=
etica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"=
gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sm=
all;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-=
family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Not tru=
e about accessing previous TXNs annexes. All coins spend from Coinbase tran=
sactions. If you can get the COutpoint you&#39;re spending, you can get the=
 parent of the COutpoint... and iterate backwards so on and so forth. Then =
you have the CB txn, which commits to the tree of wtxids. So you get previo=
us transactions annexes comitted=C2=A0there.</div><div class=3D"gmail_defau=
lt" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:r=
gb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:aria=
l,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div cla=
ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s=
ize:small;color:rgb(0,0,0)">For future transactions, you can, as a miner wi=
th decent hashrate you could promise what your Coinbase transaction would b=
e for a future block and what the Outputs would be, and then you can pop op=
en that as well... but you can&#39;t show valid PoW for that one so I&#39;m=
 not sure that&#39;s different than authenticated data. But where it does h=
ave a use is that you could, if you had OP_COUTPOINTVERIFY, say that this c=
oin is only spendable if a miner mines the specific block that you want at =
a certain height (e.g., with only your txn in it?) and then they can claim =
the outpoint in the future...=C2=A0so maybe there is something there bizzar=
e that can happen with that capability....</div><br></div><div>=C2=A0</div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);p=
adding-left:1ex">
&gt; Because the Annex is signed, and must be the same, this can also be<br=
>
&gt; inconvenient:<br>
<br>
The annex is committed to by signatures in the same way nVersion,<br>
nLockTime and nSequence are committed to by signatures; I think it helps<br=
>
to think about it in a similar way.<br></blockquote><div><br></div><div cla=
ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s=
ize:small;color:rgb(0,0,0)">nSequence, yes, nLockTime is per-tx.</div><div =
class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fon=
t-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)=
">BTW i think we now consider nSeq/nLock to be misdesigned=C2=A0given desir=
e to vary these per-input/per-tx....\</div><div class=3D"gmail_default" sty=
le=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,=
0)"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,helve=
tica,sans-serif;font-size:small;color:rgb(0,0,0)">so if the annex is like t=
hese perhaps it&#39;s also misdesigned.</div><blockquote class=3D"gmail_quo=
te" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-sty=
le:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
&gt; Suppose that you have a Miniscript that is something like: and(or(PK(A=
),<br>
&gt; PK(A&#39;)), X, or(PK(B), PK(B&#39;))).<br>
&gt; <br>
&gt; A or A&#39; should sign with B or B&#39;. X is some sort of fragment t=
hat might<br>
&gt; require a value that is unknown (and maybe recursively defined?) so<br=
>
&gt; therefore if we send the PSBT to A first, which commits to the annex, =
and<br>
&gt; then X reads the annex and say it must be something else, A must sign<=
br>
&gt; again. So you might say, run X first, and then sign with A and C or B.=
<br>
&gt; However, what if the script somehow detects the bitstring WHICH_A WHIC=
H_B<br>
&gt; and has a different Annex per selection (e.g., interpret the bitstring=
 as a<br>
&gt; int and annex must =3D=3D that int). Now, given and(or(K1, K1&#39;),..=
. or(Kn,<br>
&gt; Kn&#39;)) we end up with needing to pre-sign 2**n annex values somehow=
... this<br>
&gt; seems problematic theoretically.<br>
<br>
Note that you need to know what the annex will contain before you sign,<br>
since the annex is committed to via the signature. If &quot;X&quot; will ne=
ed<br>
entries in the annex that aren&#39;t able to be calculated by the other<br>
parties, then they need to be the first to contribute to the PSBT, not A.<b=
r>
<br>
I think the analogy to locktimes would be &quot;I need the locktime to be a=
t<br>
least block 900k, should I just sign that now, or check that nobody else<br=
>
is going to want it to be block 950k or something? Or should I just sign<br=
>
with nLockTime at 900k, 910k, 920k, 930k, etc and let someone else pick<br>
the right one?&quot; The obvious solution is just to work out what the<br>
nLockTime should be first, then run signing rounds. Likewise, work out<br>
what the annex should be first, then run the signing rounds.<br></blockquot=
e><div><br></div><div><br></div><div class=3D"gmail_default" style=3D"font-=
family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Yes, my=
 point is this is computationally hard to do sometimes.</div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px=
;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1e=
x">
<br>
CLTV also has the problem that if you have one script fragment with<br>
CLTV by time, and another with CLTV by height, you can&#39;t come up with<b=
r>
an nLockTime that will ever satisfy both. If you somehow have script<br>
fragments that require incompatible interpretations of the annex, you&#39;r=
e<br>
likewise going to be out of luck.<br>
<br></blockquote><div><br></div><div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Ye=
s, see above. If we don&#39;t know how the annex will be structured or used=
, this is the point of this thread....</div><div class=3D"gmail_default" st=
yle=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0=
,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,helv=
etica,sans-serif;font-size:small;color:rgb(0,0,0)">We need to drill down ho=
w to not introduce these problems.</div><br></div><div>=C2=A0</div><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-wid=
th:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-l=
eft:1ex">
Having a way of specifying locktimes in the annex can solve that<br>
particular problem with CLTV (different inputs can sign different<br>
locktimes, and you could have different tags for by-time/by-height so<br>
that even the same input can have different clauses requiring both),<br>
but the general problem still exists.<br>
<br>
(eg, you might have per-input by-height absolute locktimes as annex<br>
entry 3, and per-input by-time absolute locktimes as annex entry 4,<br>
so you might convert:<br>
<br>
=C2=A0&quot;900e3 CLTV DROP&quot; -&gt; &quot;900e3 3 PUSH_ANNEX_ENTRY GREA=
TERTHANOREQUAL VERIFY&quot;<br>
<br>
=C2=A0&quot;500e6 CLTV DROP&quot; -&gt; &quot;500e6 4 PUSH_ANNEX_ENTRY GREA=
TERTHANOREQUAL VERIFY&quot;<br>
<br>
for height/time locktime checks respectively)<br>
<br>
&gt; Of course this wouldn&#39;t be miniscript then. Because miniscript is =
just for<br>
&gt; the well behaved subset of script, and this seems ill behaved. So mayb=
e<br>
&gt; we&#39;re OK?<br>
<br>
The CLTV issue hit miniscript:<br>
<br>
<a href=3D"https://medium.com/blockstream/dont-mix-your-timelocks-d9939b665=
094" rel=3D"noreferrer" target=3D"_blank">https://medium.com/blockstream/do=
nt-mix-your-timelocks-d9939b665094</a></blockquote><div><br></div><div><div=
 class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo=
nt-size:small;color:rgb(0,0,0)">Maybe the humour didn&#39;t hit -- we can o=
nly define well behaved as best we know, and the solution was to re-define =
miniscript to only be the well defined subset of miniscript once the bug in=
 the spec was found.</div><br></div><div>=C2=A0</div><blockquote class=3D"g=
mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-=
left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
&gt; It seems like one good option is if we just go on and banish the OP_AN=
NEX.<br>
&gt; Maybe that solves some of this? I sort of think so. It definitely seem=
s<br>
&gt; like we&#39;re not supposed to access it via script, given the quote f=
rom above:<br>
<br>
How the annex works isn&#39;t defined, so it doesn&#39;t make any sense to<=
br>
access it from script. When how it works is defined, I expect it might<br>
well make sense to access it from script -- in a similar way that the<br>
CLTV and CSV opcodes allow accessing nLockTime and nSequence from script.<b=
r></blockquote><div><br></div><div class=3D"gmail_default" style=3D"font-fa=
mily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">That&#39;=
s false: CLTV and CSV expressly do not allow accessing it from script, only=
 lower bounding it (and transitively proving that it was not of the other f=
lavour).</div><div class=3D"gmail_default" style=3D"font-family:arial,helve=
tica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:rgb(0,0,0)">So you can&#39;t actually get the exact nLockTime / Se=
quence on the stack (exception: if you use the maximum allowable value, the=
n there are no other values...)</div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b=
r></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,s=
ans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class=3D"gmail_d=
efault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;col=
or:rgb(0,0,0)">Given that it&#39;s not defined at all, that&#39;s why I&#39=
;m skeptical about signing it at all presently.</div><div class=3D"gmail_de=
fault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;colo=
r:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D"font-family:a=
rial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">If theres=C2=A0=
a future upgrade, it would be compatible as we can add new sighash flags to=
 cover that.</div><div class=3D"gmail_default" style=3D"font-family:arial,h=
elvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><blockquote=
 class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:=
1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left=
:1ex"><br>
&gt; One solution would be to... just soft-fork it out. Always must be 0. W=
hen<br>
&gt; we come up with a use case for something like an annex, we can find a =
way<br>
&gt; to add it back.<br>
<br>
The point of reserving the annex the way it has been is exactly this --<br>
it should not be used now, but when we agree on how it should be used,<br>
we have an area that&#39;s immediately ready to be used.</blockquote><block=
quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-w=
idth:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding=
-left:1ex">
<br>
(For the cases where you don&#39;t need script to enforce reasonable values=
,<br>
reserving it now means those new consensus rules can be used immediately<br=
>
with utxos that predate the new consensus rules -- so you could update<br>
offchain contracts from per-tx to per-input locktimes immediately without<b=
r>
having to update the utxo on-chain first)<br></blockquote><div><br></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
color:rgb(0,0,0)">I highly doubt that we will not need new sighash flags on=
ce it is ready to allow partial covers of the annex, e.g. like the structur=
ed ones described above.</div><div class=3D"gmail_default" style=3D"font-fa=
mily:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0=
,0)">We&#39;re already doing a soft fork for the new annex rules, so this i=
sn&#39;t a big deal...</div><div class=3D"gmail_default" style=3D"font-fami=
ly:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0=
)">Legacy outputs can use these new sighash flags as well, in theory (maybe=
 I&#39;ll do a post on why we shouldn&#39;t...)</div><div class=3D"gmail_de=
fault" style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><b=
r></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,s=
ans-serif;color:rgb(0,0,0)"><br></div><div class=3D"gmail_default" style=3D=
"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div cl=
ass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;color=
:rgb(0,0,0)">Cheers,</div><div class=3D"gmail_default" style=3D"font-family=
:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div class=3D"gmail=
_default" style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"=
>Jeremy</div></div></div>

--0000000000003b82f205d977a744--