Return-Path: Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B16CC002D for ; Sun, 2 Oct 2022 04:07:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4FA0F40106 for ; Sun, 2 Oct 2022 04:07:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4FA0F40106 X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YKUtG6Fl7lAD for ; Sun, 2 Oct 2022 04:07:06 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C17B4400D2 Received: from azure.erisian.com.au (azure.erisian.com.au [172.104.61.193]) by smtp2.osuosl.org (Postfix) with ESMTPS id C17B4400D2 for ; Sun, 2 Oct 2022 04:07:05 +0000 (UTC) Received: from aj@azure.erisian.com.au (helo=sapphire.erisian.com.au) by azure.erisian.com.au with esmtpsa (Exim 4.92 #3 (Debian)) id 1oeqFn-0001tX-Fr; Sun, 02 Oct 2022 14:07:01 +1000 Received: by sapphire.erisian.com.au (sSMTP sendmail emulation); Sun, 02 Oct 2022 14:06:54 +1000 Date: Sun, 2 Oct 2022 14:06:54 +1000 From: Anthony Towns To: Michael Folkson , Bitcoin Protocol Discussion Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score-int: -18 X-Spam-Bar: - Subject: Re: [bitcoin-dev] bitcoin-inquistion: evaluating soft forks on signet X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2022 04:07:07 -0000 On Wed, Sep 28, 2022 at 11:48:32AM +0000, Michael Folkson via bitcoin-dev wrote: > SegWit was added > to a new testnet (Segnet) for testing rather than the pre-existing testnet > and I think future soft fork proposals should follow a similar approach. I think past history falls into a few groups: * p2sh was briefly tested on testnet (and an alternative was tested on mainnet) https://bitcointalk.org/index.php?topic=58579.msg786939#msg786939 * cltv and csv were mostly tested on elements alpha (liquid precursor testnet); though they were activated on testnet 6 and 11 weeks prior to mainnet http://diyhpl.us/wiki/transcripts/gmaxwell-sidechains-elements/ * segwit was also tested via elements alpha, though in a different form to what was deployed for bitcoin (ie, the elements approach would have been a hard fork). because of the p2p changes (you need additional data to validate blocks post segwit), segwit had dedicated test networks, up to segnet4, from 1st Jan 2016 to 30th Mar 2016. segwit was activated on testnet on 13th May 2016, merged into core on 25th June 2016, and included in the 0.13.1 released on 27th October 2016. I couldn't find very good public references about segnet, and don't think it saw much use outside of people implementing segwit consensus features themselves. * taproot was merged 15th October 2020 (#18267), and activated on signet as of genesis around 21st October 2020 (#20157). It was locked in on mainnet on 12th June 2021, activated on testnet on 8th July 2021, and activated on mainnet on 14th November 2021. * CTV had ctv-signet created around 17th December 2020, but it wasn't really announced or encouraged until 17th Feb 2022. The core PR (#21702) was opened 16th April 2021. https://www.erisian.com.au/bitcoin-core-dev/log-2020-12-17.html#l-845 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-February/019925.html * I think Drivechains has a DriveNet testnet (since 2018 or earlier?), though I don't see an explorer, and it looks like the bitcoin code it's based on predates taproot. * Other than CTV, Drivechains and ideas being explored on Liquid, most other ideas for bitcoin consensus changes haven't really progressed past a gist, mailing list post, bip or draft patch to somewhere that you can actually experiment (great consensus cleanup, anyprevout, OP_TX/TXHASH, TLUV, SIGHASH_GROUP, PUSH_ANNEX, checkblockatheight, g'root/graftroot, etc...) I thing segnet was mostly used for development of segwit itself, rather than testing or application development -- it was reset about once a month as changes to the segwit design occurred, and after the design was finalised, was active on testnet, either using -addnode to connect directly to know segwit-enabled peers, or, eventually, with seed nodes updated and filtering via the WITNESS feature. The 23rd June 2016 meeting logs have some relevant discussion: https://www.erisian.com.au/bitcoin-core-dev/log-2016-06-23.html#l-178 > Even if there is community consensus on what soft fork proposals should be added to the default signet today (which may or may not be case) I find it highly unlikely this will always be the case. The point of doing it via signet and outside of core is there doesn't need to be any community consensus on soft forks. Unlike mainnet, signet sBTC isn't money, and it isn't permissionless; and unlike merging it into core, there isn't a risk of a mege having unintended side effects impacting mainnet operation. > We then get into the situation where the block signers (currently AJ and Kalle) are the gatekeepers on what soft fork proposals are added. Because signet mining is a closed set (determined by the first block after genesis when the signetchallenge is locked in), signet soft forks always have gatekeepers. If signet miners don't opt-in to new soft forks (by upgrading their node to allow mempool acceptance according to new soft fork rules, and thus allow inclusion in block templates; or, if they're running with -acceptnonstdtxn, to reject txs that don't comply with the rules, so that funds using the new rules aren't actually anyone can spend) then you can't test the new soft fork rules on signet. > I don't think it is fair on the signet block signers to put them in that position and I don't think it is wise to put other Bitcoin Core contributors/maintainers in the position of having to defend why some proposed soft forks are accessible on the default signet while others aren't. So, I think it's more accurate to say signet miners are fundamentally *already* in that position. They can delegate that power of course, saying "we'll just implement the default rules for the software we run", but that just moves the responsibility around. > The default signet was a long term project to address the unreliability and weaknesses of testnet. That's certainly one goal. For me, that's one facet of the broader goal, which is to make it easier to develop and test new and improved bitcoin applications. So applications like vaults and eltoo that rely on the creation of new consensus (or relay) features are important to me too. Having taproot enabled on signet relatively early seemed like a win to me, and, personally, I think it's something we should work out how to repeat. > Many default signet users won't be interested in testing soft fork proposals and it is not reasonable for them to be subject to a stalling or forked blockchain Sure, that's absolutely a concern. I even raised it in the original post: >>> before actually mining blocks I want to make the signet miner able >>> to automatically detect/recover if the bitcoin-inquisition node either >>> crashes or starts producing incompatible blocks. I think I've got a specific approach there that I'm happy with now; it's PRed as https://github.com/bitcoin-inquisition/bitcoin/pull/7 The idea is that if you want to mine "inquisition" blocks to test out a soft fork, you run both a core and an inquistion bitcoind, but only use the inquisition bitcoind for generating blocks -- you *always* submit blocks to the core bitcoind (and if core rejects and inquisition block, you generate a new template using core and mine and submit that). That way blocks are never submitted to the network unless they satisfy core's consensus rules, and from core's perspective the chain always progresses. > because changes to a soft fork proposal or a buggy soft fork proposal pushed to the default signet makes previous valid/invalid transactions invalid/valid. The above addresses the invalid to valid case for consensus rules (ie, the part that would make a change not be a "soft fork"). The normal cases for a soft fork are: * making a consensus valid transaction, invalid * making a relay invalid transaction, valid That leaves the final category for buggy behaviour as making a relay valid transaction invalid for relay. It's not clear to me that dealing with that automatically is a good idea -- that's a pretty severe class of bug that could result in people being unable to spend their funds; so maybe the best thing is for txs like that to be ignored by the miners until the build up in the mempool and people start complaining and insisting the bug get fixed? That's also a reason to have it happen on the global signet: you don't want people developing new soft fork rules to introduce such a bug and not notice it because the people making such transactions don't care about the soft fork, and then have it suddenly appear once it's already locked in on mainnet. Better to have the bug impact signet transactions first, before there's any risk of bugs affecting real money. > If they want to test proposed soft forks on a custom signet they are opting in to possible disruption rather than it being forced upon them. If you don't want to risk any disruption, then regtest or a private signet is a better option. A global p2p network *always* has risk of disruption at some level or another. But again, opting-in to potential disruption from future soft forks that are anywhere near the deployment phase is a *good* idea: far better for the potential disruption to become real and observable when it only affects fake money, and to fix it then, rather than having the risk of losing real money due to bugs in features you didn't think you cared about. Cheers, aj