Date: Fri, 16 Sep 2022 17:15:45 +1000
From: Anthony Towns <aj@erisian.com.au>
To: bitcoin-dev@lists.linuxfoundation.org
Message-ID: <YyQioS3F942wu1HW@erisian.com.au>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Subject: [bitcoin-dev] bitcoin-inquistion: evaluating soft forks on signet
Precedence: list

Subhead: "Nobody expects a Bitcoin Inquistion? C'mon man, *everyone*
expects a Bitcoin Inquisition."

As we've seen from the attempt at a CHECKTEMPLATEVERIFY activation earlier
in the year [0], the question of "how to successfully get soft fork
ideas from concept to deployment" doesn't really have a good answer today.

Obviously, a centralised solution to this problem exists: we could
establish a trusted group, perhaps one containing devs, industry
representatives, investors, etc, have them review proposals and their
implementations, and follow their lead when they decide that a proposal
has met their standards and should be widely deployed. Some might even
say "sipa is precisely that group". The problem with having a group of
that nature isn't one of effectiveness, but rather that they are then
vulnerable to pressure and corruption, which isn't desirable if we want
everyone using Bitcoin to truly be peers, and often isn't desirable for
the prospective members of the group either. So that's not something we
should want people to volunteer for, nor is it a duty we should thrust
on people. Or at least, that's my opinion, anyway.

I think any alternative approach to doing consensus changes (while
avoiding a chain split) has to look something like this:

 * propose an idea (research phase)
 * implement the idea (development phase)
 * demonstrate the idea is worthwhile (evaluation phase)
 * once everyone is convinced, activate (deployment phase)

Without an evaluation phase that is thorough enough to convince (almost)
everyone, I think deployment becomes controversial and perhaps effectively
impossible (at least without some trusted leadership group). But with an
evaluation phase that demonstrates to everyone who's interested that the
proposal has actual value, minimal cost and no risk, I think activation
could be fairly easy and straightforward.

I contend that the most significant problem we have is in the "evaluation
phase". How do you convince enough people that a change is sufficiently
beneficial to justify the risk of messing with their money? If you're
only trying to convince a few experts, then perhaps you can do that with
papers and talks; but limiting the evaluation to only a few experts is
effectively just falling back to the centralised approach.

So I think that means that part of the "evaluation phase" should involve
implementing real systems on top of the proposed change, so that you
can demonstrate real value from the change. It's easy to say that
"CTV can enable vaults" or "CTV can make opening a lightning channel
non-interactive" -- but it's harder to go from saying something
is possible to actually making it happen, so, at least to me, it
seems reasonable to be skeptical of people claiming benefits without
demonstrating they're achievable in practice.

I contend the easiest way we could make it easy to demonstrate a soft
fork working as designed is to deploy it on the default global signet,
essentially as soon as it has a fully specified proposal and a reasonably
high-quality implementation.

The problem with that idea is that creates a conundrum: you can't activate
a soft fork on the default signet without first merging the code into
bitcoin core, you can't merge the code into bitcoin core until it's been
fully evaluated, and the way you evaluate it is by activating it on the
default signet?

I think the weakest link in that loop is the first one: what if we did
activate soft forks on the default signet prior to the code being merged
into core? To that end, I'm proposing a fork of core that I'm calling
"bitcoin-inquisition", with the idea that it branches from stable
releases of core and adds support for proposed changes to consensus
(CTV, ANYPREVOUT, TLUV, OP_CAT, etc...) and potentially also relay
policy (relay changes are often implied by consensus changes, but also
potentially things like package relay).

  https://github.com/bitcoin-inquisition/bitcoin/wiki
  https://github.com/bitcoin-inquisition/bitcoin/pulls

The idea being that if you're trying to work on "upgrading lightning
to support eltoo", you can iterate through changes needed to consensus
(via bitcoin-inquisition) and client apps (cln, lnd, eclair etc), while
testing them in public (on signet) and having any/all the pre-existing
signet infrastructure available (faucets, explorers etc) without having
to redeploy it yourself. Having multiple consensus changes deployed in
one place also seems like it might make it easier to compare alternative
approaches (eg CTV vs ANYPREVOUT vs OP_TXHASH vs OP_TX, etc).

So that's the concept. For practical purposes, I haven't yet merged
either CTV or APO support into the bitcoin-inquisition 23.0 branch yet,
and before actually mining blocks I want to make the signet miner able
to automatically detect/recover if the bitcoin-inquisition node either
crashes or starts producing incompatible blocks.

Anyway, I wanted to post the idea publicly, both to give folks an
opportunity to poke holes in the idea, or to suggest any further
improvements or otherwise do any review before the CTV and APO patches
get merged.

Some other details that may be of interest.

The biggest challenge with soft forks and the idea of "iterating
through changes" is that making improvements can create a hard fork,
which then forces everyone running old software to update, which can be
pretty inconvenient, especially if you don't actually care about that
change. Since signet (and regtest) mining is effectively permissioned,
we can avoid that problem by having all these proposed soft forks
come with a pre-baked ability to abandon the soft fork (much as David
Harding described in [1]). Once a soft fork is abandoned, it can either
be ignored forever (and later versions of the software can not include
the code to enforce it at all), or it can be replaced by a new version
of the soft fork.

Another benefit that comes from signet chains being permissioned is
that miners can be expected to coordinate upgrading out of band, so
there is no need for a 90% signalling threshold. Instead, activation
(and abandonment) of a soft fork can be triggered by a single block
signalling. That further means there is no need for any individual
block to signal for multiple forks, and instead of having 29 different
signals, we can instead easily have up to 2**29. I've chosen to make
the standard signal have 16 bits for specifying a bip number (0-65535)
and 8 bits for specifying a version of that bip, which seems like it
should be more than enough at least for a while. More details at [2].

I'm basing bitcoin-inquisition solely off stable releases. This is partly
because it can be annoying to constantly rebase consensus changes aginst
bitcoin core's master branch, but also I think it might help consensus
changes be easily backported once they pass the "evaluation phase"
and move into the "deployment phase".

I'm not sure what level of code quality PRs should have before being
merged into bitcoin-inquisition. I think CTV is plenty good enough,
but I'm not sure about APO, particularly its test coverage. If you want
to influence what becomes the tradition here, contributing a review,
or posting patches against the upsteam branch might be a good start?

Does this make the global default signet miners, or perhaps the
bitcoin-inquisition maintainers the "trusted group" that we want to
avoid? Hopefully not -- anyone can run their own fork or do their own
fork of bitcoin core, so if the miners/maintainers start trying to
arbitrarily block proposals they can be worked around without too much
hassle. And since they're clearly separate from any of the actions that
need to be taken for actual deployment once activation is complete,
they shouldn't have any ability to unduly promote fork proposals that
people aren't fully satisfied are ready for deployment.

Cheers,
aj

[0] https://bitcoinops.org/en/newsletters/2022/04/27/#discussion-about-activating-ctv

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-April/020242.html

[2] https://github.com/bitcoin-inquisition/bitcoin/wiki/Heretical-Deployments