path: root/12/ea68c3c1d0650d1be7e383273d118093aff7ec

Return-Path: <fresheneesz@gmail.com>
Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 46671C000B
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Mar 2022 05:05:56 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp3.osuosl.org (Postfix) with ESMTP id 2E5EE60AB0
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Mar 2022 05:05:56 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level: 
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: smtp3.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp3.osuosl.org ([127.0.0.1])
 by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 7MnKOWjLZr9G
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Mar 2022 05:05:54 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [IPv6:2a00:1450:4864:20::62c])
 by smtp3.osuosl.org (Postfix) with ESMTPS id B6D3B60A77
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 10 Mar 2022 05:05:53 +0000 (UTC)
Received: by mail-ej1-x62c.google.com with SMTP id r13so9608020ejd.5
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Wed, 09 Mar 2022 21:05:53 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=D+DuiM4DtVGlNxfJiE5rqnIJK/3zdJOyAnzWbH+tTBM=;
 b=RIHUX5iuWMRMkO1OkxvTSnXCBw4bdhvTH05cvginQPjcHUeeOZ0oyGwtkrispbLFi4
 lrK4HbAJ4rUcUUd1dhGcUq/YaOy/TdQsv3dOdgKoz5nfjxsOHMcCCpe4w5jFo9V9f6jx
 s/aWRRPj11CnHkM7vC7IxCtQ8mFjj+fEFRVdfLgiV3vRTJeKOv6viNHzjxeW6bmQ+Zdq
 pVEVGAPgmaVenst1PO0aDUc+ki7F18srSa1tTF8P6s1+kQGrP1sFNzvgtrKrxBTb28Tk
 ADaabqa1MYUrwpCGmbQW7ENYj7LNBPsxdt3PCdCI2KjhkXvUanpN+H5wflItfWHZKs5A
 teag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=D+DuiM4DtVGlNxfJiE5rqnIJK/3zdJOyAnzWbH+tTBM=;
 b=w5rG1rfVGhDS5H7rdgVfTHtlKnLxwNtMvl5SU4MbtQvJW431Qy6gqe3aMjwTYJW1TY
 UO/pzitTEe8ICVM85a54Bov0iaanhmZhN0hklf8XJjKMU5lGXZQJ8EZlchAB8vQmOiXk
 D1jyUs/IcEfgVcJOoXeV3lQ7CP0a44qgvEw5whtadH0+lQlcWaiP2jmgsFmS0r7pxGmG
 gCntfgBP1VVA5Xb+gFMfflTj+j6hBuyexMn+nGIY//brsCRmO0IUMt+IZqpp/v6IICM7
 j/7e7aGxkv6nbe+WZXP6p0QmRaLx3JrxucZA4l2q38UOD60nAanDqgHr4A8D7i40CoSx
 flZA==
X-Gm-Message-State: AOAM530CE8QpRTGKy6uYo4SwG3HDDiHXByke+qbkuCtZYPZ/jZGhKx45
 SSR2LfAuGCcAtUgC3qbbgUTZQxHZHReh3QC1vxJuFZXzpnQ=
X-Google-Smtp-Source: ABdhPJyJ7jmBDY8hW7xHIc0GYKVRIXv1SMlrifYUN9++phqGymu5UH3zGvPfwrXOcRWAIqiMd/Wlf5v+nlJkG3u0eNY=
X-Received: by 2002:a17:907:206a:b0:6da:b2d9:5e32 with SMTP id
 qp10-20020a170907206a00b006dab2d95e32mr2724371ejb.176.1646888751677; Wed, 09
 Mar 2022 21:05:51 -0800 (PST)
MIME-Version: 1.0
References: <EIwjydT0d68Z7Jv8_JlrCbQW6NHSSnIU5sWwE8eX2rm9K3djfzU3nQqUrmt44U8-L9sObegelHCV6Sk7h2nwq_HS1d26FophzjNU7xC_6SE=@protonmail.com>
In-Reply-To: <EIwjydT0d68Z7Jv8_JlrCbQW6NHSSnIU5sWwE8eX2rm9K3djfzU3nQqUrmt44U8-L9sObegelHCV6Sk7h2nwq_HS1d26FophzjNU7xC_6SE=@protonmail.com>
From: Billy Tetrud <billy.tetrud@gmail.com>
Date: Wed, 9 Mar 2022 23:05:35 -0600
Message-ID: <CAGpPWDafWGcZJOUs4wSEt0DzFP8OXB4nrbx+9sUtTe5JfdwE_w@mail.gmail.com>
To: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Content-Type: multipart/alternative; boundary="0000000000000744ad05d9d62be8"
X-Mailman-Approved-At: Thu, 10 Mar 2022 08:53:38 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Jets (Was: `OP_FOLD`: A Looping Construct For
	Bitcoin SCRIPT)
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 10 Mar 2022 05:05:56 -0000

--0000000000000744ad05d9d62be8
Content-Type: text/plain; charset="UTF-8"

Hi ZmnSCPxj,

>  Just ask a bunch of fullnodes to add this 1Mb of extra ignored data in
this tiny 1-input-1-output transaction so I pay only a small fee

I'm not suggesting that you wouldn't have to pay a fee for it. You'd pay a
fee for it as normal, so there's no DOS vector. Doesn't adding
extra witness data do what would be needed here? Eg simply adding extra
data onto the witness script that will remain unconsumed after successful
execution of the script?

> how do new jets get introduced?

In scenario A, new jets get introduced by being added to bitcoin software
as basically relay rules.

> If a new jet requires coordinated deployment over the network, then you
might as well just softfork and be done with it.

It would not need a coordinated deployment. However, the more nodes that
supported that jet, the more efficient using it would be for the network.

> If a new jet can just be entered into some configuration file, how do you
coordinate those between multiple users so that there *is* some benefit for
relay?

When a new version of bitcoin comes out, people generally upgrade to it
eventually. No coordination is needed. 100% of the network need not support
a jet. Just some critical mass to get some benefit.

> Having a static lookup table is better since you can pattern-match on
strings of specific, static length

Sorry, better than what exactly?

> How does the unupgraded-to-upgraded boundary work?

This is what I'm thinking. Imagine a simple script:

OP_DUP
OP_ADD

with witness

1

This would execute as 1+1 = 2 -> success. Let's say the script is
jettified so we can instead write it as:

OP_JET
1b5f03cf # adler32 hash of the replaced script

with a witness:

OP_JET   # Some number that represents OP_JET
1b5f03cf
0
1

A jet-aware node transmitting to another jet-aware node can transmit it as
is (tho it would need to do a swap out to validate). For a jet-aware node
to transmit this to a non-jet aware node, it would need to swap the OP_JET
call with the script it represents. So the transaction sent to the non-jet
aware node would have:

Script:

OP_DUP
OP_ADD

Witness:

OP_JET
1b5f03cf
0
1

And you can see that this would execute and succeed by adding 1+1 and
ending up with the stack:

2
0
1b5f03cf
OP_JET

Which would succeed because of the non-zero top of stack.

When the non-jet aware node sends this to a jet-aware node, that node would
see the extra items on the stack after script execution, and would
interpret them as an OP_JET call specifying that OP_JET should replace the
witness items starting at index 0 with `1b5f03cf  OP_JET`. It does this and
then sends that along to the next hop.

In order to support this without a soft fork, this extra otherwise
unnecessary data would be needed, but for jets that represent long scripts,
the extra witness data could be well worth it (for the network).

However, this extra data would be a disincentive to do transactions this
way, even when its better for the network. So it might not be worth doing
it this way without a soft fork. But with a soft fork to upgrade nodes to
support an OP_JET opcode, the extra witness data can be removed (replaced
with out-of-band script fragment transmission for nodes that don't support
a particular jet).

One interesting additional thing that could be done with this mechanism is
to add higher-order function ability to jets, which could allow nodes to
add OP_FOLD or similar functions as a jet without requiring additional soft
forks.  Hypothetically, you could imagine a jet script that uses an OP_LOOP
jet be written as follows:

5             # Loop 5 times
1             # Loop the next 1 operation
3c1g14ad
OP_JET
OP_ADD  # The 1 operation to loop

The above would sum up 5 numbers from the stack. And while this summation
jet can't be represented in bitcoin script on its own (since bitcoin script
can't manipulate opcode calls), the jet *call* can still be represented as:

OP_ADD
OP_ADD
OP_ADD
OP_ADD
OP_ADD

which means all of the above replacement functionality would work just as
well.

So my point here is that jets implemented in a way similar to this would
give a much wider range of "code as compression" possibilities than
implementing a single opcode like op_fold.

> To make jets more useful, we should redesign the language so that
`OP_PUSH` is not in the opcode stream, but instead, we have a separate
table of constants that is attached / concatenated to the actual SCRIPT.

This can already be done, right? You just have to redesign the script to
consume and swap/rot around the data in the right way to separate them out
from the main script body.


On Mon, Mar 7, 2022 at 5:35 PM ZmnSCPxj <ZmnSCPxj@protonmail.com> wrote:

> Good morning Billy,
>
> Changed subject since this is only tangentially related to `OP_FOLD`.
>
> > Let me organize my thoughts on this a little more clearly. There's a
> couple possibilities I can think of for a jet-like system:
> >
> > A. We could implement jets now without a consensus change, and
> without requiring all nodes to upgrade to new relay rules. Probably. This
> would give upgraded nodes improved validation performance and many upgraded
> nodes relay savings (transmitting/receiving fewer bytes). Transactions
> would be weighted the same as without the use of jets tho.
> > B. We could implement the above + lighter weighting by using a soft fork
> to put the jets in a part of the blockchain hidden from unupgraded nodes,
> as you mentioned.
> > C. We could implement the above + the jet registration idea in a soft
> fork.
> >
> > For A:
> >
> > * Upgraded nodes query each connection for support of jets in general,
> and which specific jets they support.
> > * For a connection to another upgraded node that supports the jet(s)
> that a transaction contains, the transaction is sent verbatim with the jet
> included in the script (eg as some fake opcode line like 23 OP_JET,
> indicating to insert standard jet 23 in its place). When validation
> happens, or when a miner includes it in a block, the jet opcode call is
> replaced with the script it represents so hashing happens in a way that is
> recognizable to unupgraded nodes.
> > * For a connection to a non-upgraded node that doesn't support jets, or
> an upgraded node that doesn't support the particular jet included in the
> script, the jet opcode call is replaced as above before sending to that
> node. In addition, some data is added to the transaction that unupgraded
> nodes propagate along but otherwise ignore. Maybe this is extra witness
> data, maybe this is some kind of "annex", or something else. But that data
> would contain the original jet opcode (in this example "23 OP_JET") so that
> when that transaction data reaches an upgraded node that recognizes that
> jet again, it can swap that back in, in place of the script fragment it
> represents.
> >
> > I'm not 100% sure the required mechanism I mentioned of "extra ignored
> data" exists, and if it doesn't, then all nodes would at least need to be
> upgraded to support that before this mechanism could fully work.
>
> I am not sure that can even be *made* to exist.
> It seems to me a trivial way to launch a DDoS: Just ask a bunch of
> fullnodes to add this 1Mb of extra ignored data in this tiny
> 1-input-1-output transaction so I pay only a small fee if it confirms but
> the bandwidth of all fullnodes is wasted transmitting and then ignoring
> this block of data.
>
> > But even if such a mechanism doesn't exist, a jet script could still be
> used, but it would be clobbered by the first nonupgraded node it is relayed
> to, and can't then be converted back (without using a potentially expensive
> lookup table as you mentioned).
>
> Yes, and people still run Bitcoin Core 0.8.x.....
>
> > > If the script does not weigh less if it uses a jet, then there is no
> incentive for end-users to use a jet
> >
> > That's a good point. However, I'd point out that nodes do lots of things
> that there's no individual incentive for, and this might be one where
> people either altruistically use jets to be lighter on the network, or use
> them in the hopes that the jet is accepted as a standard, reducing the cost
> of their scripts. But certainly a direct incentive to use them is better.
> Honest nodes can favor connecting to those that support jets.
>
> Since you do not want a dynamic lookup table (because of the cost of
> lookup), how do new jets get introduced?
> If a new jet requires coordinated deployment over the network, then you
> might as well just softfork and be done with it.
> If a new jet can just be entered into some configuration file, how do you
> coordinate those between multiple users so that there *is* some benefit for
> relay?
>
> > >if a jet would allow SCRIPT weights to decrease, upgraded nodes need to
> hide them from unupgraded nodes
> > > we have to do that by telling unupgraded nodes "this script will
> always succeed and has weight 0"
> >
> > Right. It doesn't have to be weight zero, but that would work fine
> enough.
> >
> > > if everybody else has not upgraded, a user of a new jet has no
> security.
> >
> > For case A, no security is lost. For case B you're right. For case C,
> once nodes upgrade to the initial soft fork, new registered jets can take
> advantage of relay-cost weight savings (defined by the soft fork) without
> requiring any nodes to do any upgrading, and nodes could be further
> upgraded to optimize the validation of various of those registered jets,
> but those processing savings couldn't change the weighting of transactions
> without an additional soft fork.
> >
> > > Consider an attack where I feed you a SCRIPT that validates trivially
> but is filled with almost-but-not-quite-jettable code
> >
> > I agree a pattern-matching lookup table is probably not a great design.
> But a lookup table like that is not needed for the jet registration idea.
> After the necessary soft fork, there would be standard rules for which
> registered jets nodes are required to keep an index of, and so the lookup
> table would be a straightforward jet hash lookup rather than a
> pattern-matching lookup, which wouldn't have the same DOS problems. A node
> would simply find a jet opcode call like "ab38cd39e OP_JET" and just lookup
> ab38cd39e in its index.
>
> How does the unupgraded-to-upgraded boundary work?
> Having a static lookup table is better since you can pattern-match on
> strings of specific, static length, and we can take a page from `rsync` and
> use its "rolling checksum" idea which works with identifying strings of a
> certain specific length at arbitrary offsets.
>
> Say you have jetted sequences where the original code is 42 bytes, and
> another jetted sequence where the original code is 54 bytes, you would keep
> a 42-byte rolling checksum and a separate 54-byte rolling checksum, and
> then when it matches, you check if the last 42 or 54 bytes matched the
> jetted sequences.
>
> It does imply having a bunch of rolling checksums around, though.
> Sigh.
>
> ---
>
> To make jets more useful, we should redesign the language so that
> `OP_PUSH` is not in the opcode stream, but instead, we have a separate
> table of constants that is attached / concatenated to the actual SCRIPT.
>
> So for example instead of an HTLC having embedded `OP_PUSH`es like this:
>
>    OP_IF
>        OP_HASH160 <hash> OP_EQUALVERIFY OP_DUP OP_HASH160 <acceptor pkh>
>    OP_ELSE
>        <timeout> OP_CHECKLOCKTIMEVERIFY OP_DROP OP_DUP OP_HASH160
> <offerrer pkh>
>    OP_ENDIF
>    OP_EQUALVERIFY
>    OP_CHECKSIG
>
> We would have:
>
>    constants:
>        h = <hash>
>        a = <acceptor pkh>
>        t = <timeout>
>        o = <offerer pkh>
>    script:
>        OP_IF
>            OP_HASH160 h OP_EQUALVERIFY OP_DUP OP_HASH160 a
>        OP_ELSE
>            t OP_CHECKLOCKTIMEVERIFY OP_DROP OP_DUP OP_HASH160 o
>        OP_ENDIF
>        OP_EQUALVERIFY
>        OP_CHECKSIG
>
> The above allows for more compressibility, as the entire `script` portion
> can be recognized as a jet outright.
> Move the incompressible hashes out of the main SCRIPT body.
>
> We should note as well that this makes it *easier* to create recursive
> covenants (for good or ill) out of `OP_CAT` and whatever opcode you want
> that allows recursive covenants in combination with `OP_CAT`.
> Generally, recursive covenants are *much* more interesting if they can
> change some variables at each iteration, and having a separate
> table-of-constants greatly facilitates that.
>
> Indeed, the exercise of `OP_TLUV` in
> [drivechains-over-recursive-convenants][] puts the loop variables into the
> front of the SCRIPT to make it easier to work with the SCRIPT manipulation.
>
> [drivechains-over-recursive-covenants]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-February/019976.html
>
> ---
>
> Perhaps we can consider the general vs specific tension in
> information-theoretic terms.
>
> A language which supports more computational power --- i.e. more general
> --- must, by necessity, have longer symbols, as a basic law of information
> theory.
> After all, a general language can express more things.
>
> However, we do recognize that certain sequences of things-to-say are much
> more likely than others.
> That is, we expect that certain sequences "make sense" to do.
> That is why "jets" are even proposed, they are shortcuts towards those.
>
> Assuming a general language is already deployed for Bitcoin, then a new
> opcode is a jet as it simply makes the SCRIPT shorter.
>
> Instead of starting with a verbose (by necessity) general language, we
> could instead start with a terse but restricted language, and slowly loosen
> up its restrictions by adding new capabilities in softforks.
>
> Regards,
> ZmnSCPxj
>
>

--0000000000000744ad05d9d62be8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi=C2=A0<span style=3D"color:rgb(32,33,36);font-size:=
0.875rem;letter-spacing:0.2px;font-family:Roboto,RobotoDraft,Helvetica,Aria=
l,sans-serif;white-space:nowrap">ZmnSCPxj,</span></div><div><br></div>&gt;=
=C2=A0

Just ask a bunch of fullnodes to add this 1Mb of extra ignored data in this=
 tiny 1-input-1-output transaction so I pay only a small fee<div><br></div>=
<div>I&#39;m not suggesting that you wouldn&#39;t have to pay a fee for it.=
 You&#39;d pay a fee for it as normal, so there&#39;s no DOS vector. Doesn&=
#39;t adding extra=C2=A0witness data do what would be needed here? Eg simpl=
y adding extra data onto the witness script that will remain unconsumed aft=
er successful execution of the script?</div><div><br></div><div>&gt; how do=
 new jets get introduced?</div><div><br></div><div>In scenario A, new jets =
get introduced by being added to bitcoin software as basically relay rules.=
=C2=A0</div><div><br></div><div>&gt; If a new jet requires coordinated depl=
oyment over the network, then you might as well just softfork and be done w=
ith it.</div><div><br></div><div>It would not need a coordinated deployment=
. However, the more nodes that supported that jet, the more efficient using=
 it would be for the network.=C2=A0</div><div><br></div><div>&gt; If a new =
jet can just be entered into some configuration file, how do you coordinate=
 those between multiple users so that there *is* some benefit for relay?</d=
iv><div><br></div><div>When a new version of bitcoin comes out, people gene=
rally upgrade to it eventually. No coordination is needed. 100% of the netw=
ork need not support a jet. Just some critical mass to=C2=A0get some benefi=
t.=C2=A0</div><div><br></div>&gt; Having a static lookup table is better si=
nce you can pattern-match on strings of specific, static length<div><br></d=
iv><div>Sorry, better than what exactly?=C2=A0</div><div><div><br></div><di=
v>&gt; How does the unupgraded-to-upgraded boundary work?</div><div></div><=
/div><div><div><br></div><div>This is what I&#39;m thinking. Imagine a simp=
le script:=C2=A0</div><div><br></div><div>OP_DUP</div><div>OP_ADD</div><div=
><br></div><div>with witness</div><div><br></div><div>1</div><div><br></div=
><div>This would execute as 1+1 =3D 2 -&gt; success. Let&#39;s say the scri=
pt is jettified=C2=A0so we can instead write it as:</div><div><br></div><di=
v><div><div>OP_JET</div></div><div>1b5f03cf # adler32 hash of the replaced =
script</div></div><div><br></div><div>with a witness:</div><div><br></div><=
div><div>OP_JET=C2=A0 =C2=A0# Some number that represents OP_JET</div></div=
><div>1b5f03cf=C2=A0<br></div><div>0</div><div>1</div><div><br></div><div>A=
 jet-aware node transmitting to another jet-aware node can transmit it as i=
s (tho it would need to do a swap out to validate). For a jet-aware node to=
 transmit this to a non-jet aware node, it would need to swap the OP_JET ca=
ll with the script it represents. So the transaction sent to the non-jet aw=
are node would have:</div><div><br></div><div>Script:</div><div><br></div><=
div><div>OP_DUP</div><div>OP_ADD</div></div><div><br></div><div>Witness:</d=
iv><div><br></div><div><div><div>OP_JET</div></div><div>1b5f03cf=C2=A0<br><=
/div><div>0</div><div>1</div></div><div><br></div><div>And you can see that=
 this would execute and succeed by adding 1+1 and ending up with the stack:=
<br><br>2</div><div>0</div><div>1b5f03cf=C2=A0<br></div><div>OP_JET</div><d=
iv><br></div><div>Which would succeed=C2=A0because of the non-zero top of s=
tack.=C2=A0</div><div><br></div><div>When the non-jet aware node sends this=
 to a jet-aware node, that node would see the extra items on the stack afte=
r script execution, and would interpret them as an OP_JET call specifying t=
hat OP_JET should replace the witness items starting at index 0 with `1b5f0=
3cf=C2=A0=C2=A0OP_JET`. It does this and then sends that along to the next =
hop.</div><div><br></div><div>In order to support this without a soft fork,=
 this extra otherwise unnecessary=C2=A0data would be needed, but for jets t=
hat represent long scripts, the extra witness data could be well worth it (=
for the network).=C2=A0</div><div><br></div><div>However, this extra data w=
ould be a disincentive to do transactions this way, even when its=C2=A0bett=
er for the network. So it might not be worth doing it this way without a so=
ft fork. But with a soft fork to upgrade nodes to support an OP_JET opcode,=
 the extra witness data can be removed (replaced with out-of-band script fr=
agment transmission for nodes that don&#39;t support a particular jet).=C2=
=A0</div><div><br></div><div>One interesting additional thing that could be=
 done with this mechanism is to add higher-order function ability to jets, =
which could allow nodes to add OP_FOLD or similar functions as a jet withou=
t requiring additional soft forks.=C2=A0 Hypothetically, you could imagine =
a jet script that uses an OP_LOOP jet be written as follows:</div><div><br>=
</div><div>5=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Loop 5 times<=
/div><div>1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Loop the next =
1 operation</div><div>3c1g14ad=C2=A0<br></div><div>OP_JET</div><div>OP_ADD=
=C2=A0 # The 1 operation to loop</div><div><br></div></div><div>The above w=
ould sum up 5 numbers from the stack. And while this summation jet can&#39;=
t be represented in bitcoin script on its own (since bitcoin script can&#39=
;t manipulate opcode calls), the jet *call* can still be represented as:</d=
iv><div><br></div><div>OP_ADD=C2=A0=C2=A0<br></div><div>OP_ADD=C2=A0=C2=A0<=
br></div><div>OP_ADD=C2=A0=C2=A0<br></div><div>OP_ADD=C2=A0=C2=A0<br></div>=
<div>OP_ADD=C2=A0=C2=A0<br></div><div><br></div><div>which means all of the=
 above replacement functionality would work just as well.=C2=A0</div><div><=
br></div><div>So my point here is that jets implemented in a way similar to=
 this would give a much wider range of &quot;code as compression&quot; poss=
ibilities than implementing a single opcode like op_fold.=C2=A0</div><div><=
br></div><div>&gt; To make jets more useful, we should redesign the languag=
e so that `OP_PUSH` is not in the opcode stream, but instead, we have a sep=
arate table of constants that is attached / concatenated to the actual SCRI=
PT.</div><div><br></div><div>This can already be done, right? You just have=
 to redesign the script to consume and swap/rot around the data in the righ=
t way to separate them out from the main script body.=C2=A0</div><div><br><=
/div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a=
ttr">On Mon, Mar 7, 2022 at 5:35 PM ZmnSCPxj &lt;<a href=3D"mailto:ZmnSCPxj=
@protonmail.com" target=3D"_blank">ZmnSCPxj@protonmail.com</a>&gt; wrote:<b=
r></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
;border-left:1px solid rgb(204,204,204);padding-left:1ex">Good morning Bill=
y,<br>
<br>
Changed subject since this is only tangentially related to `OP_FOLD`.<br>
<br>
&gt; Let me organize my thoughts on this a little more clearly. There&#39;s=
 a couple possibilities I can think of for a jet-like system:<br>
&gt;<br>
&gt; A. We could implement jets now without a consensus change, and without=
=C2=A0requiring all nodes to upgrade to new relay rules. Probably. This wou=
ld give upgraded nodes improved=C2=A0validation performance and many upgrad=
ed nodes relay savings (transmitting/receiving fewer bytes). Transactions w=
ould be weighted the same as without the use of jets tho.<br>
&gt; B. We could implement the above=C2=A0+ lighter weighting by using a so=
ft fork to put the jets in a part of the blockchain hidden from unupgraded =
nodes, as you mentioned.=C2=A0<br>
&gt; C. We could implement the above=C2=A0+ the jet registration idea in a =
soft fork.=C2=A0<br>
&gt;<br>
&gt; For A:<br>
&gt;<br>
&gt; * Upgraded nodes query each connection for support of jets in general,=
 and which specific jets they support.<br>
&gt; * For a connection to another upgraded node that supports the jet(s) t=
hat a transaction contains, the transaction is sent verbatim=C2=A0with the =
jet included in the script (eg as some fake opcode line like 23 OP_JET, ind=
icating to insert standard jet 23 in its place). When validation happens, o=
r when a miner includes it in a block, the jet opcode call is replaced with=
 the script it represents so hashing happens in a way that is recognizable =
to unupgraded nodes.<br>
&gt; * For a connection to a non-upgraded node that doesn&#39;t support jet=
s, or an upgraded node that doesn&#39;t support the particular jet included=
 in the script, the jet opcode call is replaced as above before sending to =
that node. In addition, some data is added to the transaction that unupgrad=
ed nodes propagate along but otherwise ignore. Maybe this is extra witness =
data, maybe this is some kind of &quot;annex&quot;, or something else. But =
that data would contain the original jet opcode (in this example &quot;23 O=
P_JET&quot;) so that when that transaction data reaches an upgraded node th=
at recognizes that jet again, it can swap that back in, in place of the scr=
ipt fragment it represents.=C2=A0<br>
&gt;<br>
&gt; I&#39;m not 100% sure the required mechanism I mentioned of &quot;extr=
a ignored data&quot; exists, and if it doesn&#39;t, then all nodes would at=
 least need to be upgraded to support that before this mechanism could full=
y work.<br>
<br>
I am not sure that can even be *made* to exist.<br>
It seems to me a trivial way to launch a DDoS: Just ask a bunch of fullnode=
s to add this 1Mb of extra ignored data in this tiny 1-input-1-output trans=
action so I pay only a small fee if it confirms but the bandwidth of all fu=
llnodes is wasted transmitting and then ignoring this block of data.<br>
<br>
&gt; But even if such a mechanism doesn&#39;t exist, a jet script could sti=
ll be used, but it would be clobbered by the first nonupgraded node it is r=
elayed to, and can&#39;t then be converted back (without using a potentiall=
y expensive lookup table as you mentioned).=C2=A0<br>
<br>
Yes, and people still run Bitcoin Core 0.8.x.....<br>
<br>
&gt; &gt; If the script does not weigh less if it uses a jet, then there is=
 no incentive for end-users to use a jet<br>
&gt;<br>
&gt; That&#39;s a good point. However, I&#39;d point out that nodes do lots=
 of things that there&#39;s no individual incentive for, and this might be =
one where people either altruistically use jets to be lighter on the networ=
k, or use them in the hopes that the jet is accepted as a standard, reducin=
g the cost of their scripts. But certainly a direct incentive to use them i=
s better. Honest nodes can favor connecting to those that support jets.<br>
<br>
Since you do not want a dynamic lookup table (because of the cost of lookup=
), how do new jets get introduced?<br>
If a new jet requires coordinated deployment over the network, then you mig=
ht as well just softfork and be done with it.<br>
If a new jet can just be entered into some configuration file, how do you c=
oordinate those between multiple users so that there *is* some benefit for =
relay?<br>
<br>
&gt; &gt;if a jet would allow SCRIPT weights to decrease, upgraded nodes ne=
ed to hide them from unupgraded nodes<br>
&gt; &gt; we have to do that by telling unupgraded nodes &quot;this script =
will always succeed and has weight 0&quot;<br>
&gt;<br>
&gt; Right. It doesn&#39;t have to be weight zero, but that would work fine=
 enough.=C2=A0<br>
&gt;<br>
&gt; &gt; if everybody else has not upgraded, a user of a new jet has no se=
curity.<br>
&gt;<br>
&gt; For case A, no security is lost. For case B you&#39;re right. For case=
 C, once nodes upgrade to the initial soft fork, new registered jets can ta=
ke advantage of relay-cost weight savings (defined by the soft fork) withou=
t requiring any nodes to do any upgrading, and nodes could be further upgra=
ded to optimize the validation of various of those registered jets, but tho=
se processing savings couldn&#39;t change the weighting of transactions wit=
hout an additional soft fork.<br>
&gt;<br>
&gt; &gt; Consider an attack where I feed you a SCRIPT that validates trivi=
ally but is filled with almost-but-not-quite-jettable code<br>
&gt;<br>
&gt; I agree a pattern-matching lookup table is probably not a great design=
. But a lookup table like that is not needed for the jet registration idea.=
 After the necessary soft fork, there would be standard rules for which reg=
istered jets nodes are required to keep an index of, and so the lookup tabl=
e would be a straightforward jet hash lookup rather than a pattern-matching=
 lookup, which wouldn&#39;t have the same DOS problems. A node would simply=
 find a jet opcode call like &quot;ab38cd39e OP_JET&quot; and just lookup a=
b38cd39e in its index.=C2=A0<br>
<br>
How does the unupgraded-to-upgraded boundary work?<br>
Having a static lookup table is better since you can pattern-match on strin=
gs of specific, static length, and we can take a page from `rsync` and use =
its &quot;rolling checksum&quot; idea which works with identifying strings =
of a certain specific length at arbitrary offsets.<br>
<br>
Say you have jetted sequences where the original code is 42 bytes, and anot=
her jetted sequence where the original code is 54 bytes, you would keep a 4=
2-byte rolling checksum and a separate 54-byte rolling checksum, and then w=
hen it matches, you check if the last 42 or 54 bytes matched the jetted seq=
uences.<br>
<br>
It does imply having a bunch of rolling checksums around, though.<br>
Sigh.<br>
<br>
---<br>
<br>
To make jets more useful, we should redesign the language so that `OP_PUSH`=
 is not in the opcode stream, but instead, we have a separate table of cons=
tants that is attached / concatenated to the actual SCRIPT.<br>
<br>
So for example instead of an HTLC having embedded `OP_PUSH`es like this:<br=
>
<br>
=C2=A0 =C2=A0OP_IF<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_HASH160 &lt;hash&gt; OP_EQUALVERIFY OP_DUP OP=
_HASH160 &lt;acceptor pkh&gt;<br>
=C2=A0 =C2=A0OP_ELSE<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;timeout&gt; OP_CHECKLOCKTIMEVERIFY OP_DROP O=
P_DUP OP_HASH160 &lt;offerrer pkh&gt;<br>
=C2=A0 =C2=A0OP_ENDIF<br>
=C2=A0 =C2=A0OP_EQUALVERIFY<br>
=C2=A0 =C2=A0OP_CHECKSIG<br>
<br>
We would have:<br>
<br>
=C2=A0 =C2=A0constants:<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0h =3D &lt;hash&gt;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0a =3D &lt;acceptor pkh&gt;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0t =3D &lt;timeout&gt;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0o =3D &lt;offerer pkh&gt;<br>
=C2=A0 =C2=A0script:<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_IF<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0OP_HASH160 h OP_EQUALVERIFY OP_DUP=
 OP_HASH160 a<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_ELSE<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0t OP_CHECKLOCKTIMEVERIFY OP_DROP O=
P_DUP OP_HASH160 o<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_ENDIF<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_EQUALVERIFY<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0OP_CHECKSIG<br>
<br>
The above allows for more compressibility, as the entire `script` portion c=
an be recognized as a jet outright.<br>
Move the incompressible hashes out of the main SCRIPT body.<br>
<br>
We should note as well that this makes it *easier* to create recursive cove=
nants (for good or ill) out of `OP_CAT` and whatever opcode you want that a=
llows recursive covenants in combination with `OP_CAT`.<br>
Generally, recursive covenants are *much* more interesting if they can chan=
ge some variables at each iteration, and having a separate table-of-constan=
ts greatly facilitates that.<br>
<br>
Indeed, the exercise of `OP_TLUV` in [drivechains-over-recursive-convenants=
][] puts the loop variables into the front of the SCRIPT to make it easier =
to work with the SCRIPT manipulation.<br>
<br>
[drivechains-over-recursive-covenants]: <a href=3D"https://lists.linuxfound=
ation.org/pipermail/bitcoin-dev/2022-February/019976.html" rel=3D"noreferre=
r" target=3D"_blank">https://lists.linuxfoundation.org/pipermail/bitcoin-de=
v/2022-February/019976.html</a><br>
<br>
---<br>
<br>
Perhaps we can consider the general vs specific tension in information-theo=
retic terms.<br>
<br>
A language which supports more computational power --- i.e. more general --=
- must, by necessity, have longer symbols, as a basic law of information th=
eory.<br>
After all, a general language can express more things.<br>
<br>
However, we do recognize that certain sequences of things-to-say are much m=
ore likely than others.<br>
That is, we expect that certain sequences &quot;make sense&quot; to do.<br>
That is why &quot;jets&quot; are even proposed, they are shortcuts towards =
those.<br>
<br>
Assuming a general language is already deployed for Bitcoin, then a new opc=
ode is a jet as it simply makes the SCRIPT shorter.<br>
<br>
Instead of starting with a verbose (by necessity) general language, we coul=
d instead start with a terse but restricted language, and slowly loosen up =
its restrictions by adding new capabilities in softforks.<br>
<br>
Regards,<br>
ZmnSCPxj<br>
<br>
</blockquote></div>

--0000000000000744ad05d9d62be8--