Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id CFB47ABC for ; Mon, 18 Dec 2017 21:59:03 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ot0-f179.google.com (mail-ot0-f179.google.com [74.125.82.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 87B7F171 for ; Mon, 18 Dec 2017 21:59:02 +0000 (UTC) Received: by mail-ot0-f179.google.com with SMTP id h9so14691178oti.0 for ; Mon, 18 Dec 2017 13:59:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sGyL32OowT264Gkgi8ulSnaoFH5mOSiVxytGGfdPAvg=; b=KTTSrRa1P7NsC89YPtwjs8bytZMfu3hmimO5ZLt3vbgARQq/3rMWHc21RVvZugIR67 TBlSl8cEPs2DpzqHcAMdCSif9O+VdUgwWNdJPBHn0GGpu/0U7VIZIwK22ZVgSQWNgYC4 zQrbVXp+VfzmE8vXqe62HAMgXloHL92EuJJ5URgt3Onfce29mn4UPQl58QNdCQLrGycL wR0M9h3leGiuBobor1z3yn9XTf+dB9wDoWtCVmgihJPJHwnCr2ziLPX05wU8I5udfWSm a6M6dott/L2fUlzsp5kPr7JVKXfNeZk8eL/U6na/K5Y2oLtn/VQJZkCA88q056a3GM+i kqVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sGyL32OowT264Gkgi8ulSnaoFH5mOSiVxytGGfdPAvg=; b=Q/tLwwSGX/Z5mzDAxpUT1u3d8Hhnp8jayTu99GJUTspCiCFldtYP8Lb/IQGWX0bCts bySCoUTEPnVImHUUf8vntqlIOWx0JpGzl4bEsaYWWe9bpvXI1pZ/kRDsh1AYs2UfoLu7 Rtdkr1SLUPQ3hvYJlhz8ilsSEELAh8zyn3pMl5upfeXM4vwwsW8zXU16yWbnaakX4PDB Y+ozGD957iL8ROQcwUPv1DDndoWxUJ9jEqvA+EJhjg4xivhJn11tR9rNg8NEhd+IWsXa fbRGHgnLdV+2a1xqEB9Gzu6djFcIrguIX9AqyVLQY6OfZk7tmMwgvogQaYPbySHCo569 BXUw== X-Gm-Message-State: AKGB3mIlXjZnAuZpsAqZ1eOlsmJjHACHNnU27VLUTDBP+cCrsAUo0asN xCcuyw2dM+WRH8/BjEDWD/OHj4tTNTc= X-Google-Smtp-Source: ACJfBotVIvYahXMubl4N6qXMPlhhoLQIGwSoa8Hc+4unJ7UBF+xdAsmw1o/MO7/1ggx+Ux+7uCrS3g== X-Received: by 10.157.53.98 with SMTP id l31mr930718ote.314.1513634341707; Mon, 18 Dec 2017 13:59:01 -0800 (PST) Received: from ?IPv6:2600:380:606b:27b4:1841:2452:2040:4e3? ([2600:380:606b:27b4:1841:2452:2040:4e3]) by smtp.gmail.com with ESMTPSA id g143sm5404051oic.6.2017.12.18.13.59.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Dec 2017 13:59:00 -0800 (PST) Content-Type: multipart/alternative; boundary=Apple-Mail-29E69DCE-23FB-4F26-B284-C27B22449429 Mime-Version: 1.0 (1.0) From: Eric Voskuil X-Mailer: iPhone Mail (14G60) In-Reply-To: Date: Mon, 18 Dec 2017 16:58:58 -0500 Content-Transfer-Encoding: 7bit Message-Id: References: <61B0AEC9-3B1D-416F-8883-A030E5109538@friedenbach.org> To: Kalle Rosenbaum , Bitcoin Protocol Discussion X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HTML_MESSAGE, MIME_QP_LONG_LINE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Mon, 18 Dec 2017 22:14:12 +0000 Subject: Re: [bitcoin-dev] Why not witnessless nodes? X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2017 21:59:03 -0000 --Apple-Mail-29E69DCE-23FB-4F26-B284-C27B22449429 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable How does one know what consensus has formed (around a UTXO set)? e > On Dec 18, 2017, at 16:27, Kalle Rosenbaum via bitcoin-dev wrote: >=20 > Hi Mark >=20 > Yes, it seems like sign-to-contract protocols, which I just now briefly re= ad about [1][2], may need to use historic witnesses. That raises the questio= n, what are Bitcoin witnesses for? >=20 > To me it seems witnesses should be regarded as temporary. But it seems bot= h respondents to this thread, Eric and Mark, mean that witnesses are forever= . I regard witnesses as a way to authenticate updates to the UTXO set, and o= nce buried deep enough in the blockchain, the witness is no longer needed, b= ecause consensus has formed around the UTXO set update. >=20 > Suppose a transaction with an invalid witness happens to enter the blockch= ain and gets buried 100000 blocks down with the witness still available. Is t= he blockchain above it valid? I'd say the blockchain is valid and that it wa= s a bug that the transaction made it into the blockchain. We will have to li= ve with such bugs. >=20 > Another way to put it: Suppose that all witnesses from 2017 dissappears fr= om all nodes in 2020. Is the blockchain still valid? I think so. I would con= tinue using it without looking back. >=20 > With that approach, I think sign-to-contract protocols has to find ways to= work in a witnessless environment. For example, users of such protocols can= setup their own archival nodes. >=20 > I'd love to hear alternative views on this. >=20 > Thanks, > /Kalle >=20 > [1] https://download.wpsoftware.net/bitcoin/wizardry/mw-slides/2017-03-mit= -bitcoin-expo/slides.pdf > [2] https://bitcointalk.org/index.php?topic=3D893898.msg9861102#msg9861102= >=20 > 2017-12-18 18:30 GMT+01:00 Mark Friedenbach via bitcoin-dev : >> Sign-to-contract enables some interesting protocols, none of which are in= wide use as far as I=E2=80=99m aware. But if they were (and arguably this i= s an area that should be more developed), then SPV nodes validating these pr= otocols will need access to witness data. If a node is performing IBD with a= ssumevalid set to true, and is also intending to prune history, then there=E2= =80=99s no reason to fetch those witnesses as far as I=E2=80=99m aware. But i= t would be a great disservice to the network for nodes intending to serve SP= V clients to prune this portion of the block history.=20 >>=20 >>=20 >>> On Dec 18, 2017, at 8:19 AM, Eric Voskuil via bitcoin-dev wrote: >>>=20 >>> You can't know (assume) a block is valid unless you have previously vali= dated the block yourself. But in the case where you have, and then intend to= rely on it in a future sync, there is no need for witness data for blocks y= ou are not going to validate. So you can just not request it.=20 >>>=20 >>> However you will not be able to provide those blocks to nodes that *are*= validating; the client is pruned and therefore not a peer (cannot reciproca= te). (An SPV client is similarly not a peer; it is a more deeply pruned clie= nt than the witnessless client.) >>>=20 >>> There is no other reason that a node requires witness data. SPV clients d= on't need it as it is neither require it to verify header commitment to tran= sactions nor to extract payment addresses from them. >>>=20 >>> The harm to the network by pruning is that eventually it can become hard= er and even impossible for anyone to validate the chain. But because you are= fully validating you individually remain secure, so there is no individual i= ncentive working against this system harm. >>>=20 >>> e >>>=20 >>>> On Dec 18, 2017, at 08:35, Kalle Rosenbaum wrote: >>>>=20 >>>> 2017-12-18 13:43 GMT+01:00 Eric Voskuil : >>>>>=20 >>>>> > On Dec 18, 2017, at 03:32, Kalle Rosenbaum via bitcoin-dev wrote: >>>>> > >>>>> > Dear list, >>>>> > >>>>> > I find it hard to understand why a full node that does initial block= >>>>> > download also must download witnesses if they are going to skip veri= fication anyway. >>>>>=20 >>>>> Why run a full node if you are not going to verify the chain? >>>>=20 >>>> I meant to say "I find it hard to understand why a full node that does i= nitial block >>>> download also must download witnesses when it is going to skip verifica= tion of the witnesses anyway." >>>>=20 >>>> I'm referring to the "assumevalid" feature of Bitcoin Core that skips s= ignature verification up to block X. Or have I misunderstood assumevalid? >>>>=20 >>>> /Kalle >>>> =20 >>>>>=20 >>>>> > If my full node skips signature verification for >>>>> > blocks earlier than X, it seems the reasons for downloading the >>>>> > witnesses for those blocks are: >>>>> > >>>>> > * to be able to send witnesses to other nodes. >>>>> > >>>>> > * to verify the witness root hash of the blocks >>>>> > >>>>> > I suppose that it's important to verify the witness root hash becaus= e >>>>> > a bad peer may send me invalid witnesses during initial block >>>>> > download, and if I don't verify that the witness root hash actually >>>>> > commits to them, I will get banned by peers requesting the blocks fr= om >>>>> > me because I send them garbage. >>>>> > So both the reasons above (there may be more that I don't know about= ) >>>>> > are actually the same reason: To be able to send witnesses to others= >>>>> > without getting banned. >>>>> > >>>>> > What if a node could chose not to download witnesses and thus chose t= o >>>>> > send only witnessless blocks to peers. Let's call these nodes >>>>> > witnessless nodes. Note that witnessless nodes are only witnessless >>>>> > for blocks up to X. Everything after X is fully verified. >>>>> > >>>>> > Witnessless nodes would be able to sync faster because it needs to >>>>> > download less data to calculate their UTXO set. They would therefore= >>>>> > more quickly be able to provide full service to SPV wallets and its >>>>> > local wallets as well as serving blocks to other witnessless nodes >>>>> > with same or higher assumevalid block. For witnessless nodes with >>>>> > lower assumevalid they can serve at least some blocks. It could also= >>>>> > serve blocks to non-segwit nodes. >>>>> > >>>>> > Do witnessless nodes risk dividing the network in two parts, one >>>>> > witnessless and one with full nodes, with few connections between th= e >>>>> > parts? >>>>> > >>>>> > So basically, what are the reasons not to implement witnessless >>>>> > nodes? >>>>> > >>>>> > Thank you, >>>>> > /Kalle >>>>> > _______________________________________________ >>>>> > bitcoin-dev mailing list >>>>> > bitcoin-dev@lists.linuxfoundation.org >>>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>>=20 >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>=20 >>=20 >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>=20 >=20 > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --Apple-Mail-29E69DCE-23FB-4F26-B284-C27B22449429 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
How does one know what cons= ensus has formed (around a UTXO set)?

e
<= br>On Dec 18, 2017, at 16:27, Kalle Rosenbaum via bitcoin-dev <bitcoin-dev@lists.linuxfounda= tion.org> wrote:

Hi Mark

Yes, it seems like sign-to-contrac= t protocols, which I just now briefly read about [1][2], may need to use his= toric witnesses. That raises the question, what are Bitcoin witnesses for?
To me it seems witnesses should be regarded as temporary. B= ut it seems both respondents to this thread, Eric and Mark, mean that w= itnesses are forever. I regard witnesses as a way to authenticate updates to= the UTXO set, and once buried deep enough in the blockchain, the witness is= no longer needed, because consensus has formed around the UTXO set update.<= /div>

Suppose a transaction with an invalid witness happe= ns to enter the blockchain and gets buried 100000 blocks down with the witne= ss still available. Is the blockchain above it valid? I'd say the blockchain= is valid and that it was a bug that the transaction made it into the blockc= hain. We will have to live with such bugs.

Another w= ay to put it: Suppose that all witnesses from 2017 dissappears from all node= s in 2020. Is the blockchain still valid? I think so. I would continue using= it without looking back.

With that approach, I thi= nk sign-to-contract protocols has to find ways to work in a witnessless envi= ronment. For example, users of such protocols can setup their own archival n= odes.


2017-12-18 18:30 GMT+01:00 Mark Friedenbach via bitc= oin-dev <bitcoin-dev@lists.linuxfoundation.org>:
Sign-to-contract enables some interesting proto= cols, none of which are in wide use as far as I=E2=80=99m aware. But if they= were (and arguably this is an area that should be more developed), then SPV= nodes validating these protocols will need access to witness data. If a nod= e is performing IBD with assumevalid set to true, and is also intending to p= rune history, then there=E2=80=99s no reason to fetch those witnesses as far= as I=E2=80=99m aware. But it would be a great disservice to the network for= nodes intending to serve SPV clients to prune this portion of the block his= tory. 


On Dec 18, 2017, at 8:19 AM, Eric Voskuil via bitcoin-dev <bitcoin-dev@l= ists.linuxfoundation.org> wrote:

Y= ou can't know (assume) a block is valid unless you have previously validated= the block yourself. But in the case where you have, and then intend to rely= on it in a future sync, there is no need for witness data for blocks you ar= e not going to validate. So you can just not request it. 
However you will not be able to provide those blocks to nodes th= at *are* validating; the client is pruned and therefore not a peer (cannot r= eciprocate). (An SPV client is similarly not a peer; it is a more deeply pru= ned client than the witnessless client.)

There is n= o other reason that a node requires witness data. SPV clients don't need it a= s it is neither require it to verify header commitment to transactions nor t= o extract payment addresses from them.

The harm to t= he network by pruning is that eventually it can become harder and even impos= sible for anyone to validate the chain. But because you are fully validating= you individually remain secure, so there is no individual incentive working= against this system harm.

e

On Dec 1= 8, 2017, at 08:35, Kalle Rosenbaum <kalle@rosenbaum.se> wrote:

2017-12-18 13:43 GMT+01:00 Eric Voskuil <= eric@voskuil.org&g= t;:

> On Dec 18, 2017, at 03:32, Kalle Rosenbaum via bitcoin-dev <bitcoin-dev= @lists.linuxfoundation.org> wrote:
>
> Dear list,
>
> I find it hard to understand why a full node that does initial block > download also must download witnesses if they are going to skip verific= ation anyway.

Why run a full node if you are not going to verify the chain?

I meant to say "I find it hard to understand why a full node that does initi= al block
download also must download witness= es when it is going to skip verification of the witnesses anyway."

I'm referring to the "assumevalid" feature of Bitc= oin Core that skips signature verification up to block X. Or have I misunder= stood assumevalid?

/Kalle
 

> If my full node skips signature verification for
> blocks earlier than X, it seems the reasons for downloading the
> witnesses for those blocks are:
>
> * to be able to send witnesses to other nodes.
>
> * to verify the witness root hash of the blocks
>
> I suppose that it's important to verify the witness root hash because > a bad peer may send me invalid witnesses during initial block
> download, and if I don't verify that the witness root hash actually
= > commits to them, I will get banned by peers requesting the blocks from<= br> > me because I send them garbage.
> So both the reasons above (there may be more that I don't know about) > are actually the same reason: To be able to send witnesses to others > without getting banned.
>
> What if a node could chose not to download witnesses and thus chose to<= br> > send only witnessless blocks to peers. Let's call these nodes
> witnessless nodes. Note that witnessless nodes are only witnessless
= > for blocks up to X. Everything after X is fully verified.
>
> Witnessless nodes would be able to sync faster because it needs to
> download less data to calculate their UTXO set. They would therefore > more quickly be able to provide full service to SPV wallets and its
= > local wallets as well as serving blocks to other witnessless nodes
> with same or higher assumevalid block. For witnessless nodes with
> lower assumevalid they can serve at least some blocks. It could also > serve blocks to non-segwit nodes.
>
> Do witnessless nodes risk dividing the network in two parts, one
> witnessless and one with full nodes, with few connections between the > parts?
>
> So basically, what are the reasons not to implement witnessless
> nodes?
>
> Thank you,
> /Kalle
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.= org/mailman/listinfo/bitcoin-dev

______________________________________________= _
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listin= fo/bitcoin-dev

_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.<= wbr>linuxfoundation.org
https://lists.linuxfoundation.org/m= ailman/listinfo/bitcoin-dev


____________________= ___________________________
bitcoin-dev mailing list<= br>bitcoin-de= v@lists.linuxfoundation.org
https://lists.linuxfoundation= .org/mailman/listinfo/bitcoin-dev
= --Apple-Mail-29E69DCE-23FB-4F26-B284-C27B22449429--