path: root/01/14305a0f40f59fa70236b7bebe4b917ea8adf2

Return-Path: <laolu32@gmail.com>
Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 455DFC000E
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu,  4 Nov 2021 22:08:06 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp1.osuosl.org (Postfix) with ESMTP id 3595E81A50
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu,  4 Nov 2021 22:08:06 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: 0.149
X-Spam-Level: 
X-Spam-Status: No, score=0.149 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, PDS_OTHER_BAD_TLD=1.997,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001]
 autolearn=no autolearn_force=no
Authentication-Results: smtp1.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp1.osuosl.org ([127.0.0.1])
 by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id OtKIe2_eW2sv
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu,  4 Nov 2021 22:08:05 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com
 [IPv6:2607:f8b0:4864:20::b29])
 by smtp1.osuosl.org (Postfix) with ESMTPS id DAD1D81A3E
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu,  4 Nov 2021 22:08:04 +0000 (UTC)
Received: by mail-yb1-xb29.google.com with SMTP id d10so17994295ybe.3
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Thu, 04 Nov 2021 15:08:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
 bh=2LKZJqbRCfivMlgrJ7cimVgGhfZpMCbfshzZ9vQIwgU=;
 b=Mn6tyqxCmjMEhi4+1loL6mRdihtykvuTgFQ+GFc0Je7YPP3KFtcXDn2vMqtYjsmEPi
 lHYl3lX6LsxQwlTOdwOUWf9pycUxer6WJyqIVpDU3wPAGhdTZjIzgPX7tIHFscN+1Fmc
 TtvMN2EiORVFZ0jbNhx+ELput8e3OwEOkptVt/ZXALAwZtkZ2AIfpBCtfY7XKWpnHJ9t
 O3I6s7SDQUQHU/NCMs5VyJhNbTix8CcaF/bWnzjBPtUDi+mSRm2nwdAFKb7Yc2yJu79m
 IwgBUIFlZek7/cnk1AH4tcT8XAv1qfCu5OSxY0Tf70VcCBBvICOezLOxB2mIJk3h0pMl
 uOpg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to;
 bh=2LKZJqbRCfivMlgrJ7cimVgGhfZpMCbfshzZ9vQIwgU=;
 b=KM+1SS80EadzFlspuT0PErcHLNlWq0nyptoO0F0CwJ9TzX8C2w3f9hrGWsCbLQRoi+
 dFlqrDM4ubAQ4TF/5Ch5rK2yjCU+0ZeBSn8wJK71zi/fkby94HJozmjSaDw6xBlIxb3x
 O+B9eTBGZFbVLMhV0eegcxlywm2vNWeu3jhMIENcetJWfNUjVZ85T9XlqQ6jn9wtyvvk
 jQCEWHj5YOCZRoDBHFEwmNK0D98W2Zydjoc0XOgtm5SHi4dEeGjq0Q2+EJweUaiettuv
 O0965Fi/UDiiln233KiLENBCsNloAWmNgkAC6MF6tYRoo7my+ogS3tuBzmhSCpn7DIrc
 roeg==
X-Gm-Message-State: AOAM5300ri9hq6zkMpiFn/Nq5NNYbBLgaebggdt5izONXMI7f517CFdE
 Qcb5o9pRZ58V6gSBlNFvedKpSoBmvf4oCc3TyseE7uTZno4=
X-Google-Smtp-Source: ABdhPJxEyr4t3fVw3QZNcLTegz6lpvQrHWX9VEoGewuK7OPJ4bIp06u38+okHofGyUe31laSC7ty5vssI8122M1y2b4=
X-Received: by 2002:a25:aa67:: with SMTP id s94mr47649860ybi.343.1636063683646; 
 Thu, 04 Nov 2021 15:08:03 -0700 (PDT)
MIME-Version: 1.0
References: <CAO3Pvs-Jazo27Vmi3Rforke++T5rkboS=2PK9CSSTtCk4enF2w@mail.gmail.com>
In-Reply-To: <CAO3Pvs-Jazo27Vmi3Rforke++T5rkboS=2PK9CSSTtCk4enF2w@mail.gmail.com>
From: Olaoluwa Osuntokun <laolu32@gmail.com>
Date: Thu, 4 Nov 2021 15:07:52 -0700
Message-ID: <CAO3Pvs_hLDGtBRP9J4Zokk6BpaBTUQAy01O3BjC6xbBzsudKjg@mail.gmail.com>
To: lnd@lightning.engineering, 
 Arnoud Kouwenhoven - Pukaki Corp via bitcoin-dev
 <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000b199ca05cffdc228"
Subject: Re: [bitcoin-dev] Neutrino, Taproot,
	and The Evolution of BiPs 157/158
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Nov 2021 22:08:06 -0000

--000000000000b199ca05cffdc228
Content-Type: text/plain; charset="UTF-8"

> In terms of adding more taproot specific functionality, I've had a number
of
> items in my laundry list such as:

Forgot to add this other item (also the list wasn't meant to be only tapoot
stuff):
  * reviving old projects to include a micropayment-for-data layer to
    incentivize nodes to serve the filters and other data

On Thu, Nov 4, 2021 at 3:01 PM Olaoluwa Osuntokun <laolu32@gmail.com> wrote:

> Hi y'all,
>
> If you're an active user of neutrino [8], then you probably heard about
> what
> went down over the past week or so on testnet as related to Taproot. First,
> i just wanted to reassure everyone that nothing is fundamentally broken
> with
> BIP 157/158 as it relates to taproot, and we already have a mitigation
> patch
> in place for the issue we encountered.
>
> The rest of this mail is structured in a FAQ style to make it easy to skim
> and extract the information that may be relevant to the reader.
>
> ## What happened on testnet?
>
> Neutrino nodes on testnet rejected a filter (thinking it was invalid) due
> to this transaction spending a taproot input [1]. This was due to a faulty
> heuristics in the neutrino _client code_ (not part of the protocol) that
> attempted to verify the contents of a filter more completely.
>
> In retrospect, the heuristic in question wasn't full proof, as it attempted
> to derive the _pk script_ of a transaction based on its input
> witness/sigScript. This worked pretty well in the context of segwit v0, but
> it isn't possible to exhaustively do as we don't know what future spends
> will look like.
>
> ## Is neutrino broken?
>
> No, the client side is fine, and the protocol is fine.
>
> The problematic heuristic has been removed in this PR [2], which will be
> included in lnd 0.14, and has been tagged with neutrino 0.13 [3].
>
> To dig into _why_ we attempted to use such a heuristic, we'll need to
> revisit how BIP 158 works briefly. For each block, we insert the
> `pkScript`s
> of all the outputs, and also the prev out's pkScript (the script being
> spent) as well. This lets the filter compress script re-use in both inputs
> and outputs, and also makes it possible to implement some protocols in a
> more light-client friendly manner (for example Loop uses this to has the
> client watch HTLC _scripts_ being spent, as it doesn't necessarily know the
> txid/outpoint).
>
> The one issue with this, is that while clients can ensure that all the
> `pkScripts` of outputs have been inserted, they can't do the same for the
> inputs (which is why we added that heuristic in the client code). Luckily
> we
> know how to properly fix this at the protocol level, more on that below.
>
> ## How can I make sure my neutrino clients handle the Taproot upgrade on
> mainnet smoothly?
>
> Upgrade to 0.14 (assuming it's out in time), or apply this small patch [4].
> The patch just demotes an error case to a warning message, so anyone
> running
> a fork should be able to easily apply the fix.
>
> Alongside, optionally extend these filter header guides [7].
>
> We're looking into some intermediate ground where we can verify the scripts
> that we know are relevant to the node.
>
> ## How will BIP 158/157 evolve post taproot?
>
> In terms of adding more taproot specific functionality, I've had a number
> of
> items in my laundry list such as:
>
>   * creating new segwit-only filters with re-parameterized fp rates (also
>     examine other filter types such as pure outpoints, etc)
>
>   * creating filters that include witness data to allow matching on
>     internal/external keys, the control block, merkle root, annex, etc
>
>   * add a new protocol extension to btcd (with a corresponding BIP) to
>     allow notes to fetch block undo data (as described here [5]) to fully
>     verify fetched filters or a node needs to reconcile conflicting filters
>
>   * new filters that span across multiple blocks as previously worked on by
>     Kalle Alm (couldn't find a link to his PR when typing this...)
>
> Make further progress towards a proposal that allows filters to be
> committed
> either as a soft-fork, or a "velvet fork" [6] where miners optionally
> commit to
> the past filter header chain.
>
>
> -- Laolu
>
> [1]:
> https://mempool.space/testnet/tx/4b425a1f5c0fcf4794c48b810c53078773fb768acd2be1398e3f561cc3f19fb8
> [2]: https://github.com/lightninglabs/neutrino/pull/234
> [3]: https://github.com/lightninglabs/neutrino/releases/tag/v0.13.0
> [4]: https://github.com/lightninglabs/neutrino/pull/234/files
> [5]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-February/016649.html
> [6]: https://eprint.iacr.org/2018/087
> [7]:
> https://github.com/lightninglabs/neutrino/blob/5e09bd9b5d65e90c6ff07aa11b3b9d80d42afb86/chainsync/filtercontrol.go#L15
> [8]: https://github.com/lightninglabs/neutrino
>

--000000000000b199ca05cffdc228
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">&gt; In terms of adding more taproot specific functionalit=
y, I&#39;ve had a number of<br>&gt; items in my laundry list such as:<div>=
=C2=A0=C2=A0</div><div>Forgot to add this other item (also the list wasn&#3=
9;t meant to be only tapoot stuff):<br>=C2=A0 * reviving old projects to in=
clude a micropayment-for-data layer to<br>=C2=A0 =C2=A0 incentivize nodes t=
o serve the filters and other data<br></div></div><br><div class=3D"gmail_q=
uote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Nov 4, 2021 at 3:01 PM =
Olaoluwa Osuntokun &lt;<a href=3D"mailto:laolu32@gmail.com">laolu32@gmail.c=
om</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex=
"><div dir=3D"ltr">Hi y&#39;all,<br><br>If you&#39;re an active user of neu=
trino [8], then you probably heard about what<br>went down over the past we=
ek or so on testnet as related to Taproot. First,<br>i just wanted to reass=
ure everyone that nothing is fundamentally broken with<br>BIP 157/158 as it=
 relates to taproot, and we already have a mitigation patch<br>in place for=
 the issue we encountered.<br><br>The rest of this mail is structured in a =
FAQ style to make it easy to skim<br>and extract the information that may b=
e relevant to the reader.<br><br>## What happened on testnet?<br><br>Neutri=
no nodes on testnet rejected a filter (thinking it was invalid) due<br>to t=
his transaction spending a taproot input [1]. This was due to a faulty<br>h=
euristics in the neutrino _client code_ (not part of the protocol) that<br>=
attempted to verify the contents of a filter more completely. <br><br>In re=
trospect, the heuristic in question wasn&#39;t full proof, as it attempted<=
br>to derive the _pk script_ of a transaction based on its input<br>witness=
/sigScript. This worked pretty well in the context of segwit v0, but<br>it =
isn&#39;t possible to exhaustively do as we don&#39;t know what future spen=
ds<br>will look like.<br><br>## Is neutrino broken?<br><br>No, the client s=
ide is fine, and the protocol is fine.<br><br>The problematic heuristic has=
 been removed in this PR [2], which will be<br>included in lnd 0.14, and ha=
s been tagged with neutrino 0.13 [3].<br><br>To dig into _why_ we attempted=
 to use such a heuristic, we&#39;ll need to<br>revisit how BIP 158 works br=
iefly. For each block, we insert the `pkScript`s<br>of all the outputs, and=
 also the prev out&#39;s pkScript (the script being<br>spent) as well. This=
 lets the filter compress script re-use in both inputs<br>and outputs, and =
also makes it possible to implement some protocols in a<br>more light-clien=
t friendly manner (for example Loop uses this to has the<br>client watch HT=
LC _scripts_ being spent, as it doesn&#39;t necessarily know the<br>txid/ou=
tpoint).<br><br>The one issue with this, is that while clients can ensure t=
hat all the<br>`pkScripts` of outputs have been inserted, they can&#39;t do=
 the same for the<br>inputs (which is why we added that heuristic in the cl=
ient code). Luckily we<br>know how to properly fix this at the protocol lev=
el, more on that below.<br><br>## How can I make sure my neutrino clients h=
andle the Taproot upgrade on mainnet smoothly?<br><br>Upgrade to 0.14 (assu=
ming it&#39;s out in time), or apply this small patch [4].<br>The patch jus=
t demotes an error case to a warning message, so anyone running<br>a fork s=
hould be able to easily apply the fix.<br><br>Alongside, optionally extend =
these filter header guides [7].<br><br>We&#39;re looking into some intermed=
iate ground where we can verify the scripts<br>that we know are relevant to=
 the node.<br><br>## How will BIP 158/157 evolve post taproot?<br><br>In te=
rms of adding more taproot specific functionality, I&#39;ve had a number of=
<br>items in my laundry list such as:<br><br>=C2=A0 * creating new segwit-o=
nly filters with re-parameterized fp rates (also<br>=C2=A0 =C2=A0 examine o=
ther filter types such as pure outpoints, etc)<br><br>=C2=A0 * creating fil=
ters that include witness data to allow matching on<br>=C2=A0 =C2=A0 intern=
al/external keys, the control block, merkle root, annex, etc<br><br>=C2=A0 =
* add a new protocol extension to btcd (with a corresponding BIP) to<br>=C2=
=A0 =C2=A0 allow notes to fetch block undo data (as described here [5]) to =
fully<br>=C2=A0 =C2=A0 verify fetched filters or a node needs to reconcile =
conflicting filters<br><br>=C2=A0 * new filters that span across multiple b=
locks as previously worked on by<br>=C2=A0 =C2=A0 Kalle Alm (couldn&#39;t f=
ind a link to his PR when typing this...)<br><br>Make further progress towa=
rds a proposal that allows filters to be committed<br>either as a soft-fork=
, or a &quot;velvet fork&quot; [6] where miners optionally commit to<br>the=
 past filter header chain.<br><br><br>-- Laolu<br><br>[1]: <a href=3D"https=
://mempool.space/testnet/tx/4b425a1f5c0fcf4794c48b810c53078773fb768acd2be13=
98e3f561cc3f19fb8" target=3D"_blank">https://mempool.space/testnet/tx/4b425=
a1f5c0fcf4794c48b810c53078773fb768acd2be1398e3f561cc3f19fb8</a><br>[2]: <a =
href=3D"https://github.com/lightninglabs/neutrino/pull/234" target=3D"_blan=
k">https://github.com/lightninglabs/neutrino/pull/234</a><br>[3]: <a href=
=3D"https://github.com/lightninglabs/neutrino/releases/tag/v0.13.0" target=
=3D"_blank">https://github.com/lightninglabs/neutrino/releases/tag/v0.13.0<=
/a><br>[4]: <a href=3D"https://github.com/lightninglabs/neutrino/pull/234/f=
iles" target=3D"_blank">https://github.com/lightninglabs/neutrino/pull/234/=
files</a><br>[5]: <a href=3D"https://lists.linuxfoundation.org/pipermail/bi=
tcoin-dev/2019-February/016649.html" target=3D"_blank">https://lists.linuxf=
oundation.org/pipermail/bitcoin-dev/2019-February/016649.html</a><br>[6]: <=
a href=3D"https://eprint.iacr.org/2018/087" target=3D"_blank">https://eprin=
t.iacr.org/2018/087</a><br>[7]: <a href=3D"https://github.com/lightninglabs=
/neutrino/blob/5e09bd9b5d65e90c6ff07aa11b3b9d80d42afb86/chainsync/filtercon=
trol.go#L15" target=3D"_blank">https://github.com/lightninglabs/neutrino/bl=
ob/5e09bd9b5d65e90c6ff07aa11b3b9d80d42afb86/chainsync/filtercontrol.go#L15<=
/a><br>[8]: <a href=3D"https://github.com/lightninglabs/neutrino" target=3D=
"_blank">https://github.com/lightninglabs/neutrino</a><br></div>
</blockquote></div>

--000000000000b199ca05cffdc228--