Date: Fri, 18 Dec 2020 15:27:20 +0000
From: Andrew Poelstra <apoelstra@wpsoftware.net>
To: bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Message-ID: <20201218152720.GW106279@boulet>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature"; boundary="SSZWk8LhJ5PcIQAS"
Content-Disposition: inline
Subject: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
Precedence: list


--SSZWk8LhJ5PcIQAS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

I have gone over BIP-0322 and substantially rewritten the text.
Everything I did is (I think) simply clarifying the existing
protocol, which felt like it was written by committee and wasn't
easy to follow, EXCEPT:

1. I rewrote the motivation section, which I believe originally
   was a paraphrase of Luke-jr's general objections to having any
   signmessage functionality. I hope Luke in particular can take
   a look at what I wrote under "Motivation" and see if it
   captures his concerns.

2. I merged the "consensus" and "upgradeable" rules to simply be
   one set of rules, consisting of consensus checks plus additional
   restrictions, all of which must be included. The new "Extensions"
   section allows validators to output the state "consensus-valid"
   if they really don't want to check the additional restrictions.

3. The "inconclusive" state, which was originally used for what I've
   called "consensus-valid", now indicates that a validator does not
   understand the script that it is checking (also described in the
   new "Extensions" section). The goal is that implementors are able
   to be meaningfully BIP-0322 while only supporting a subset of
   Script, e.g. the templates that their own software supports, or
   Miniscript, or the non-raw non-address set of output descriptors,
   or whatever.

   We have seen opposition to supporting BIP-322, e.g. [1] because
   of the requirement that you either have a full script interpreter
   (plus an open-ended list of Core's standardness flags, which is
   not even available through libbitcoinconsensus) or nothing. On
   the other hand, the vast majority of outputs are single-key p2pkh,
   p2pkwh or p2sh-wpkh.

The new text is here (and for posterity I will also include it
inline below, though unless Github deletes it it will be easier
to read in rendered form):

https://github.com/apoelstra/bips/blob/2020-12--bip322-overhaul/bip-0322.me=
diawiki

I'll also PR this to the BIPs repo in the next day or two, and
comments on Github are then welcome.


[1] https://bitcointalk.org/index.php?topic=3D5261605.0


* * * * * Full text of the above link * * * * *

<pre>
  BIP: 322
  Layer: Applications
  Title: Generic Signed Message Format
  Author: Karl-Johan Alm <karljohan-alm@garage.co.jp>
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
  Status: Draft
  Type: Standards Track
  Created: 2018-09-10
  License: CC0-1.0
</pre>

=3D=3D Abstract =3D=3D

A standard for interoperable signed messages based on the Bitcoin Script fo=
rmat, either for proving fund availability, or committing to a message as t=
he intended recipient of funds sent to the invoice address.

=3D=3D Motivation =3D=3D

The current message signing standard only works for P2PKH (1...) invoice ad=
dresses. We propose to extend and generalize the standard by using a Bitcoi=
n Script based approach. This ensures that any coins, no matter what script=
 they are controlled by, can in-principle be signed for. For easy interoper=
ability with existing signing hardware, we also define a signature message =
format which resembles a Bitcoin transaction (except that it contains an in=
valid input, so it cannot be spent on any real network).

Additionally, the current message signature format uses ECDSA signatures wh=
ich do not commit to the public key, meaning that they do not actually prov=
e knowledge of any secret keys. (Indeed, valid signatures can be tweaked by=
 3rd parties to become valid signatures on certain related keys.)

Ultimately no message signing protocol can actually prove control of funds,=
 both because a signature is obsolete as soon as it is created, and because=
 the possessor of a secret key may be willing to sign messages on others' b=
ehalf even if it would not sign actual transactions. No signmessage protoco=
l can fix these limitations.

=3D=3D Types of Signatures =3D=3D

This BIP specifies three formats for signing messages: ''legacy'', ''simple=
'' and ''full''. Additionally, a variant of the ''full'' format can be used=
 to demonstrate control over a set of UTXOs.

=3D=3D=3D Legacy =3D=3D=3D

New proofs should use the new format for all invoice address formats, inclu=
ding P2PKH.

The legacy format MAY be used, but must be restricted to the legacy P2PKH i=
nvoice address format.

=3D=3D=3D Simple =3D=3D=3D

A ''simple'' signature consists of a witness stack, consensus encoded as a =
vector of vectors of bytes, and base64-encoded. Validators should construct=
 <code>to_spend</code> and <code>to_sign</code> as defined below, with defa=
ult values for all fields except that

* <code>message_hash</code> is a BIP340-tagged hash of the message, as spec=
ified below
* <code>message_challenge</code> in <code>to_spend</code> is set to the scr=
iptPubKey being signed with
* <code>message_signature</code> in <code>to_sign</code> is set to the prov=
ided simple signature.

and then proceed as they would for a full signature.

=3D=3D=3D Full =3D=3D=3D

Full signatures follow an analogous specification to the BIP-325 challenges=
 and solutions used by Signet.

Let there be two virtual transactions to_spend and to_sign.

The "to_spend" transaction is:

    nVersion =3D 0
    nLockTime =3D 0
    vin[0].prevout.hash =3D 0000...000
    vin[0].prevout.n =3D 0xFFFFFFFF
    vin[0].nSequence =3D 0
    vin[0].scriptSig =3D OP_0 PUSH32[ message_hash ]
    vin[0].scriptWitness =3D []
    vout[0].nValue =3D 0
    vout[0].scriptPubKey =3D message_challenge

where <code>message_hash</code> is a BIP340-tagged hash of the message, i.e=
=2E sha256_tag(m), where tag =3D <code>BIP0322-signed-message</code>, and <=
code>message_challenge</code> is the to be proven (public) key script.

The "to_sign" transaction is:

    nVersion =3D 0 or as appropriate (e.g. 2, for time locks)
    nLockTime =3D 0 or as appropriate (for time locks)
    vin[0].prevout.hash =3D to_spend.txid
    vin[0].prevout.n =3D 0
    vin[0].nSequence =3D 0 or as appropriate (for time locks)
    vin[0].scriptWitness =3D message_signature
    vout[0].nValue =3D 0
    vout[0].scriptPubKey =3D OP_RETURN

A full signature consists of the base64-encoding of the to_spend and to_sig=
n transactions concatenated in standard network serialisation.

=3D=3D=3D Full (Proof of Funds) =3D=3D=3D

A signer may construct a proof of funds, demonstrating control of a set of =
UTXOs, by constructing a full signature as above, with the following modifi=
cations.

* <code>message_challenge</code> is unused and shall be set to <code>OP_TRU=
E</code>
* Similarly, <code>message_signature</code> is then empty.
* All outputs that the signer wishes to demonstrate control of are included=
 as additional outputs to <code>to_sign</code>, and their witness and scrip=
tSig data should be set as though these outputs were actually being spent.

Unlike an ordinary signature, validators of a proof of funds need access to=
 the current UTXO set, to learn that the claimed inputs exist on the blockc=
hain, and to learn their scriptPubKeys.

=3D=3D Detailed Specification =3D=3D

For all signature types, except legacy, the <code>to_spend</code> and <code=
>to_sign</code> transactions must be valid transactions which pass all cons=
ensus checks, except of course that the output with prevout <code>000...000=
:FFFFFFFF</code> does not exist.

We additionally require the following restrictions be met.

* All signatures must use the SIGHASH_ALL flag.
* The use of <code>CODESEPARATOR</code> or <code>FindAndDelete</code> is fo=
rbidden.
* The use of NOPs reserved for upgrades is forbidden.
* The use of segwit versions greater than 1 are forbidden.
* <code>LOW_S</code>, <code>STRICTENC</code> and <code>NULLFAIL</code>: val=
id ECDSA signatures must be strictly DER-encoded and have a low-S value; in=
valid ECDSA signature must be the empty push
* <code>MINIMALDATA</code>: all pushes must be minimally encoded
* <code>CLEANSTACK</code>: require that only a single stack element remains=
 after evaluation
* <code>MINIMALIF</code>: the argument of <code>IF</code>/<code>NOTIF</code=
> must be exactly 0x01 or empty push

Future versions of this BIP may relax these rules, in particular those arou=
nd NOPs and future Segwit versions, as they are deployed on Bitcoin.

=3D=3D=3D Verification =3D=3D=3D

Validation consists of the following steps. A validator is given as input a=
n address ''A'' (which may be omitted in a proof-of-funds), signature ''s''=
 and message ''m'', and outputs one of four states (although validators are=
 only required to be able to output the first and last):
* ''valid'' indicates that the signature passed all checks described below
* ''valid at time t and age s'' indicates that the signature has set timelo=
cks but is otherwise valid (see "Extensions" below)
* ''consensus-valid'' indicates that the signature passed validation except=
 for the additonal restrictions in the above section (see "Extensions" belo=
w)
* ''inconclusive'' means the validator was unable to check the scripts (see=
 "Extensions" below)
* ''invalid'' means none of the other states

# Decode ''s'' as the transactions <code>to_sign</code> and <code>to_spend<=
/code>
# Confirm that <code>message_hash</code> is the correct hash of ''m''
# Confirm that <code>message_challenge</code> is the scriptPubKey correspon=
ding to ''A'' if ''A'' is present, and otherwise must be <code>OP_TRUE</cod=
e>
# Confirm that all other fields are set as specified above; in particular t=
hat
** <code>to_spend</code> has exactly one input and one output
** <code>to_sign</code> has at least one input and its first input spends t=
he output of </code>to_spend</code>
** <code>to_sign</code> has exactly one output, as specified above
# Confirm that the two transactions together satisfy all consensus rules, e=
xcept for <code>to_spend</code>'s missing input, and except that ''nSequenc=
e'' of <code>to_sign</code>'s first input and ''nLockTime'' of <code>to_sig=
n</code> are not checked.
# Confirm that all of the above restrictions are met.

If the above conditions are met, the signature is considered ''valid''. Oth=
erwise the signature is ''invalid''.

=3D=3D=3D Signing =3D=3D=3D

Signers who control an address ''A'' who wish to sign a message ''m'' act a=
s follows:

# They construct <code>to_spend</code> and <code>to_sign</code> as specifie=
d above, using the scriptPubKey of ''A'' for <code>message_challenge</code>=
 and tagged hash of ''m'' as <code>message_hash</code>.
# Optionally, they may set nLockTime of <code>to_sign</code> or nSequence o=
f its first input.
# Optionally, they may add any additional outputs to <code>to_sign</code> t=
hat they wish to prove control of.
# They satisfy <code>to_sign</code> as they would any other transaction.

They then encode their signature, choosing either ''simple'' or ''full'' as=
 follows:

* If they added no inputs to <code>to_sign</code>, left nSequence and nLock=
Time at 0, and ''A'' is a Segwit address (either pure or P2SH-wrapped), the=
n they may base64-encode <code>message_signature</code>
* Otherwise they must base64-encode the concatenation of <code>to_spend</co=
de> followed by <code>to_sign</code>.

=3D=3D Extensions =3D=3D

To ease implementation, we allow some additional states to be output rather=
 than ''valid'' or ''invalid''. Users who do not understand or who do not w=
ish to deal with these states may treat them as ''invalid''.

=3D=3D=3D Timelocks =3D=3D=3D=20

If the nLockTime of <code>to_sign</code> is set to ''t'', and the nSequence=
 of the first input of <code>to_sign</code> is set to ''s'', the validator =
may output the state ''valid at time t and age s''.

If both ''t'' and ''s'' are 0, the validator must instead output ''valid''.

Users may then wish to interpret this state as ''valid'' or ''invalid'' rel=
ative to the state of the current blockchain, but the rules for doing so ar=
e out of scope of this BIP.

=3D=3D=3D Incomplete Validation =3D=3D=3D

Some validators may not wish to implement a full script interpreter, choosi=
ng instead to support only specific script templates, or only Miniscript, f=
or example. In this case, if they are unable to execute the scripts used by=
 <code>to_sign</code>, they should output the state ''inconclusive''.

Users should interpret this state as the same thing as ''invalid'', althoug=
h take it as a sign that they should find more capable software.

=3D=3D=3D Consensus-Only Validation =3D=3D=3D

Validators which are only able to check consensus-correctness of witnesses,=
 but not the additional restrictions imposed by this BIP, may output the st=
ate ''consensus-valid'' to indicate that a signature has passed all consens=
us and structural checks.

Users should interpret this state as the same thing as ''valid'' but be awa=
re that other software may fail to validate the same signature.

=3D=3D Compatibility =3D=3D

This specification is backwards compatible with the legacy signmessage/veri=
fymessage specification through the special case as described above.

=3D=3D Reference implementation =3D=3D

TODO

=3D=3D Acknowledgements =3D=3D

Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, Andrew =
Poelstra, and many others for their feedback on the specification.

=3D=3D References =3D=3D

# Original mailing list thread: https://lists.linuxfoundation.org/pipermail=
/bitcoin-dev/2018-March/015818.html

=3D=3D Copyright =3D=3D

This document is licensed under the Creative Commons CC0 1.0 Universal lice=
nse.

=3D=3D Test vectors =3D=3D

TODO

* * * * * End full text * * * * *

--=20
Andrew Poelstra
Director of Research, Blockstream
Email: apoelstra at wpsoftware.net
Web:   https://www.wpsoftware.net/andrew

The sun is always shining in space
    -Justin Lewis-Webster


--SSZWk8LhJ5PcIQAS
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEkPnKPD7Je+ki35VexYjWPOQbl8EFAl/cylUACgkQxYjWPOQb
l8H5lwf/X6H5eLJR13y4wU0KG9PnTCYjbQjPoIXhxNEMSZMaQm7VPiFpUNxqHsHI
8lIYGCYeQ8z62aVTzGfvMzt0VEcdj/E7GtfoD3pqDza6fSOyVwquC9+TR4a0zvkT
9Ri/PxJnoSKD2nccGBcYiD2jsfxTtnEvgfo9JPIECpAQ4iOzIwsY19nVn1Fl5b4I
P+QJbg9RdmYDLrlreut+dZJK8Bpmqta8u+CF1NOZz/cTTs9ZwOJXIwRN15Rb+hH7
XPmLlmKqKtVsC49qxVChNeYecjMkBbIfXYrRbF6mL1EhqZC9YX43+gcd6AKjEp1v
s5arYtziGjLIK+I26kDBLqjsVbwCUA==
=hba1
-----END PGP SIGNATURE-----

--SSZWk8LhJ5PcIQAS--