Return-Path: <shymaa.arafat@gmail.com> Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3E286C000B for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 7 Feb 2022 16:52:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 25EF38136B for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 7 Feb 2022 16:52:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JAH5ET7eqQct for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 7 Feb 2022 16:52:07 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by smtp1.osuosl.org (Postfix) with ESMTPS id A276D81353 for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 7 Feb 2022 16:52:07 +0000 (UTC) Received: by mail-ed1-x52e.google.com with SMTP id eg42so15660484edb.7 for <bitcoin-dev@lists.linuxfoundation.org>; Mon, 07 Feb 2022 08:52:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=RDdDtonQlhLr+DKBCFbn/pbN40kRX5Rz/xz4TtsPvvs=; b=FDzemIh+eDYN8l52bSmwFbwkBzYp+H1tc6IMwVLeBm3jrCxGeBVVNpedcEREqZYPzu wxX8lIkSrPMdZGYDOXlia/7cErEh1ED7jVtxzv7CPXbZs7K6QluOxh9hS1QRkfoHv12W ume3tGgTfHASgMNSpmflwf3OPzNgDWfOYiIKDNpNPm4GIbFyeHsroZ0KvNfkEF+8oKlx HzCChMIv0otiGCstxxUg0hqAGscS6bkSj1HVcgCNqicEBqhzfMu9iGmq9Rdx1a+Q7R1F Xx5Xaf93DcAhtE/ZhuX71CjOfKWneqrFGHysQ8aJa2HHa4QhhhAKMNj3a6wJAPX/AEQM lQ3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=RDdDtonQlhLr+DKBCFbn/pbN40kRX5Rz/xz4TtsPvvs=; b=zmeMfNP6S5es3UbF2uo7Aif1yF5gyieWgj85HsNwluXS581tA9d0o2ZIDtSzcB3vH7 krsOtK7rmEwah6qQE2oSj4kFNIBCFe0xtgtR2e3OuTx5MrssfLzLhwNezu1ZwZH/PRsm GWDPqOBOApk4rjQBcR78QSC73YTavilTC6BZeyPgWMPYLmRiEBjfwFMW0XMiR8/DoRDy 0YYQS1ne7KlkdumKaKy62EA+cAxDKFW1sai3+XYztPR7/4jmWLKprdJ5/BKvDVDbzNja crpwQ90+7GTAbqvONbHD2vXgzOYFxkc0bRGbl6ywYhn8QEcEHdD18yk1MuayWWxxhnKg /T5Q== X-Gm-Message-State: AOAM533konQcdpfHUzAPZ1a2tKWXwvCuN8e5pK20HqLAHt6jdB2952X8 kjnCXiIGMC0tEsv7nJAIUq5+6rE848PG8Zivx0vjsTlU X-Google-Smtp-Source: ABdhPJxZ6P4SzfVkX4lTVcexEiKlM5btYO7y++AHQMMZUxRFEysGT+PF0G51388dB+5ss92oc0aEFUDB0XPiLjbz7Vg= X-Received: by 2002:aa7:c793:: with SMTP id n19mr396751eds.74.1644252725828; Mon, 07 Feb 2022 08:52:05 -0800 (PST) MIME-Version: 1.0 References: <c7bdbBVd0KmLFPUeYk0QUdni7tbDwJSj4HGLlEOkdPzIYzOyaX147HWJPKE-isTL267nQeJds8-rsKNyzRrBhucsZvwZcg5dZjQxDnbwxAA=@wuille.net> <86BAFB7B-5ECB-4790-A19B-6E296A063C59@voskuil.org> <CAGpPWDbR5ctxf=HjLjqy0ADcQZMy9HQv-ZJfyFKmSBTntvkE9A@mail.gmail.com> In-Reply-To: <CAGpPWDbR5ctxf=HjLjqy0ADcQZMy9HQv-ZJfyFKmSBTntvkE9A@mail.gmail.com> From: shymaa arafat <shymaa.arafat@gmail.com> Date: Mon, 7 Feb 2022 18:51:54 +0200 Message-ID: <CAM98U8mT8SFfd4dPfFBbofrJQsXv+GX6Q5-Xb2y0hgqR5XMGSA@mail.gmail.com> To: Billy Tetrud <billy.tetrud@gmail.com>, Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> Content-Type: multipart/alternative; boundary="000000000000a4deb905d7706bdf" X-Mailman-Approved-At: Mon, 07 Feb 2022 16:59:07 +0000 Subject: Re: [bitcoin-dev] A suggestion to periodically destroy (or remove to secondary storage for Archiving reasons) dust, Non-standard UTXOs, and also detected burn X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Mon, 07 Feb 2022 16:52:09 -0000 --000000000000a4deb905d7706bdf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Feb 7, 2022, 16:44 Billy Tetrud via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > every lightning network transaction adds one dust UTXO > > Could you clarify what you mean here? What dust do lightning transactions > create? > I mean this msg https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-December/01963= 6.html Even though, the writer clarified after my enquiry I still think it is the same meaning most of the time only one will be spent. His words: .............. *My statement was technically incorrect, it should have been "most of the time only one of them is spent".* *But nothing prevents them to be both spent, or none of them to be spent.* *They are strictly equivalent, the only difference is the public key that can sign for them: one of these outputs belongs to you, the other belongs to your peer.* *You really cannot distinguish anything when inserting them into the utxo set, they are perfectly symmetrical and you cannot know beforehand for sure which one will be spent.* *You can guess which one will be spent most of the time, but your heuristic will never be 100% correct, so I don't think it's worth pursuing.* *.........*........ > > I do think that UTXO set size is something that will need to be addressed > at some point. I liked the idea of utreexo or some other accumulator as t= he > ultimate solution to this problem. In the mean time, I kind of agree with > Eric that outputs unlikely to be spent can easily be stored off ram and s= o > I wouldn't expect them to really be much of an issue to keep around. 3 > million utxos is only like 100MB. If software could be improved to move > dust off ram, that sounds like a good win tho. > > On Sun, Feb 6, 2022, 13:14 Eric Voskuil via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> >> >> > On Feb 6, 2022, at 10:52, Pieter Wuille via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> > >> > =EF=BB=BF >> >> Dear Bitcoin Developers, >> > >> >> -When I contacted bitInfoCharts to divide the first interval of >> addresses, they kindly did divided to 3 intervals. From here: >> >> https://bitinfocharts.com/top-100-richest-bitcoin-addresses.html >> >> -You can see that there are more than 3.1m addresses holding =E2=89= =A4 >> 0.000001 BTC (1000 Sat) with total value of 14.9BTC; an average of 473 S= at >> per address. >> > >> >> -Therefore, a simple solution would be to follow the difficulty >> adjustment idea and just delete all those >> > >> > That would be a soft-fork, and arguably could be considered theft. >> While commonly (but non universally) implemented standardness rules may >> prevent spending them currently, there is no requirement that such a rul= e >> remain in place. Depending on how feerate economics work out in the futu= re, >> such outputs may not even remain uneconomical to spend. Therefore, dropp= ing >> them entirely from the UTXO set is potentially destroying potentially >> useful funds people own. >> > >> >> or at least remove them to secondary storage >> > >> > Commonly adopted Bitcoin full nodes already have two levels of storage >> effectively (disk and in-RAM cache). It may be useful to investigate usi= ng >> amount as a heuristic about what to keep and how long. IIRC, not even ev= ery >> full node implementation even uses a UTXO model. >> >> You recall correctly. Libbitcoin has never used a UTXO store. A full nod= e >> has no logical need for an additional store of outputs, as transactions >> already contain them, and a full node requires all of them, spent or >> otherwise. >> >> The hand-wringing over UTXO set size does not apply to full nodes, it is >> relevant only to pruning. Given linear worst case growth, even that is >> ultimately a non-issue. >> >> >> for Archiving with extra cost to get them back, along with >> non-standard UTXOs and Burned ones (at least for publicly known, publish= ed, >> burn addresses). >> > >> > Do you mean this as a standardness rule, or a consensus rule? >> > >> > * As a standardness rule it's feasible, but it makes policy (further) >> deviate from economically rational behavior. There is no reason for mine= rs >> to require a higher price for spending such outputs. >> > * As a consensus rule, I expect something like this to be very >> controversial. There are currently no rules that demand any minimal fee = for >> anything, and given uncertainly over how fee levels could evolve in the >> future, it's unclear what those rules, if any, should be. >> > >> > Cheers, >> > >> > -- >> > Pieter >> > >> > _______________________________________________ >> > bitcoin-dev mailing list >> > bitcoin-dev@lists.linuxfoundation.org >> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --000000000000a4deb905d7706bdf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Mon, Feb 7, 2022, 16:44 Billy Tetrud via bitcoin-de= v <<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@= lists.linuxfoundation.org</a>> wrote:<br></div><blockquote class=3D"gmai= l_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left= :1ex"><div dir=3D"auto">>=C2=A0<span style=3D"font-size:12.8px">every li= ghtning network transaction adds one dust UTXO</span><div dir=3D"auto"><spa= n style=3D"font-size:12.8px"><br></span></div><div dir=3D"auto"><span style= =3D"font-size:12.8px">Could you clarify what you mean here? What dust do li= ghtning transactions create?</span></div></div></blockquote></div></div><di= v dir=3D"auto">I mean this msg</div><div dir=3D"auto"><a href=3D"https://li= sts.linuxfoundation.org/pipermail/bitcoin-dev/2021-December/019636.html">ht= tps://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-December/019636.= html</a></div><div dir=3D"auto">Even though, the writer clarified after my = enquiry I still think it is the same meaning most of the time only one will= be spent. His words:</div><div dir=3D"auto">..............</div><div dir= =3D"auto"><i>My statement was technically incorrect, it should have been &q= uot;most of the time only one of them is spent".</i></div><div dir=3D"= auto"><i>But nothing prevents them to be both spent, or none of them to be = spent.</i></div><div dir=3D"auto"><i>They are strictly equivalent, the only= difference is the public key that can sign for them: one of these outputs = belongs to you, the other belongs to your peer.</i></div><div dir=3D"auto">= <i><br></i></div><div dir=3D"auto"><i>You really cannot distinguish anythin= g when inserting them into the utxo set, they are perfectly symmetrical and= you cannot know beforehand for sure which one will be spent.</i></div><div= dir=3D"auto"><i>You can guess which one will be spent most of the time, bu= t your heuristic will never be 100% correct, so I don't think it's = worth pursuing.</i></div><div dir=3D"auto"><i>.........</i>........</div><d= iv dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote= " style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><= div dir=3D"auto"><div dir=3D"auto"><span style=3D"font-size:12.8px"><br></s= pan></div><div dir=3D"auto"><span style=3D"font-size:12.8px">I do think tha= t UTXO set size is something that will need to be addressed at some point. = I liked the idea of utreexo or some other accumulator as the ultimate solut= ion to this problem. In the mean time, I kind of agree with Eric that outpu= ts unlikely to be spent can easily be stored off ram and so I wouldn't = expect them to really be much of an issue to keep around. 3 million utxos i= s only like 100MB. If software could be improved to move dust off ram, that= sounds like a good win tho.</span></div></div><br><div class=3D"gmail_quot= e"><div dir=3D"ltr" class=3D"gmail_attr">On Sun, Feb 6, 2022, 13:14 Eric Vo= skuil via bitcoin-dev <<a href=3D"mailto:bitcoin-dev@lists.linuxfoundati= on.org" target=3D"_blank" rel=3D"noreferrer">bitcoin-dev@lists.linuxfoundat= ion.org</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"= margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br> <br> > On Feb 6, 2022, at 10:52, Pieter Wuille via bitcoin-dev <<a href=3D= "mailto:bitcoin-dev@lists.linuxfoundation.org" rel=3D"noreferrer noreferrer= " target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a>> wrote:<br= > > <br> > =EF=BB=BF<br> >> Dear Bitcoin Developers,<br> > <br> >> -When I contacted bitInfoCharts to divide the first interval of ad= dresses, they kindly did divided to 3 intervals. From here:<br> >> <a href=3D"https://bitinfocharts.com/top-100-richest-bitcoin-addre= sses.html" rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">https= ://bitinfocharts.com/top-100-richest-bitcoin-addresses.html</a><br> >> -You can see that there are more than 3.1m addresses holding =E2= =89=A4 0.000001 BTC (1000 Sat) with total value of 14.9BTC; an average of 4= 73 Sat per address.<br> > <br> >> -Therefore, a simple solution would be to follow the difficulty ad= justment idea and just delete all those<br> > <br> > That would be a soft-fork, and arguably could be considered theft. Whi= le commonly (but non universally) implemented standardness rules may preven= t spending them currently, there is no requirement that such a rule remain = in place. Depending on how feerate economics work out in the future, such o= utputs may not even remain uneconomical to spend. Therefore, dropping them = entirely from the UTXO set is potentially destroying potentially useful fun= ds people own.<br> > <br> >> or at least remove them to secondary storage<br> > <br> > Commonly adopted Bitcoin full nodes already have two levels of storage= effectively (disk and in-RAM cache). It may be useful to investigate using= amount as a heuristic about what to keep and how long. IIRC, not even ever= y full node implementation even uses a UTXO model.<br> <br> You recall correctly. Libbitcoin has never used a UTXO store. A full node h= as no logical need for an additional store of outputs, as transactions alre= ady contain them, and a full node requires all of them, spent or otherwise.= <br> <br> The hand-wringing over UTXO set size does not apply to full nodes, it is re= levant only to pruning. Given linear worst case growth, even that is ultima= tely a non-issue.<br> <br> >> for Archiving with extra cost to get them back, along with non-sta= ndard UTXOs and Burned ones (at least for publicly known, published, burn a= ddresses).<br> > <br> > Do you mean this as a standardness rule, or a consensus rule?<br> > <br> > * As a standardness rule it's feasible, but it makes policy (furth= er) deviate from economically rational behavior. There is no reason for min= ers to require a higher price for spending such outputs.<br> > * As a consensus rule, I expect something like this to be very controv= ersial. There are currently no rules that demand any minimal fee for anythi= ng, and given uncertainly over how fee levels could evolve in the future, i= t's unclear what those rules, if any, should be.<br> > <br> > Cheers,<br> > <br> > --<br> > Pieter<br> > <br> > _______________________________________________<br> > bitcoin-dev mailing list<br> > <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" rel=3D"norefe= rrer noreferrer" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a= ><br> > <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-= dev" rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">https://lis= ts.linuxfoundation.org/mailman/listinfo/bitcoin-dev</a><br> _______________________________________________<br> bitcoin-dev mailing list<br> <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" rel=3D"noreferrer = noreferrer" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><br> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">https://lists.li= nuxfoundation.org/mailman/listinfo/bitcoin-dev</a><br> </blockquote></div> _______________________________________________<br> bitcoin-dev mailing list<br> <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank" = rel=3D"noreferrer">bitcoin-dev@lists.linuxfoundation.org</a><br> <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = rel=3D"noreferrer noreferrer" target=3D"_blank">https://lists.linuxfoundati= on.org/mailman/listinfo/bitcoin-dev</a><br> </blockquote></div></div></div> --000000000000a4deb905d7706bdf--