Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0481B9D2 for ; Tue, 29 May 2018 09:13:46 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from bitcoin.jonasschnelli.ch (bitcoinsrv.jonasschnelli.ch [138.201.55.219]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 5DEE16C5 for ; Tue, 29 May 2018 09:13:44 +0000 (UTC) Received: by bitcoin.jonasschnelli.ch (Postfix, from userid 1002) id 35F3E15E4BAE; Tue, 29 May 2018 11:13:43 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE autolearn=ham version=3.3.1 Received: from [192.168.0.13] (cable-static-238-67.teleport.ch [213.188.238.67]) by bitcoin.jonasschnelli.ch (Postfix) with ESMTPSA id A15D615E19E2 for ; Tue, 29 May 2018 11:13:42 +0200 (CEST) From: Jonas Schnelli Content-Type: multipart/signed; boundary="Apple-Mail=_3A741525-D3A7-4193-A3B4-AC55A5C8B0CC"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Message-Id: <03557E21-8CFC-4822-8494-F4A78E23860B@jonasschnelli.ch> Date: Tue, 29 May 2018 11:13:37 +0200 To: Bitcoin Protocol Discussion X-Mailer: Apple Mail (2.3445.6.18) X-Virus-Scanned: clamav-milter 0.99.4 at bitcoinsrv.jonasschnelli.ch X-Virus-Status: Clean Subject: [bitcoin-dev] New serialization/encoding format for key material X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2018 09:13:46 -0000 --Apple-Mail=_3A741525-D3A7-4193-A3B4-AC55A5C8B0CC Content-Type: multipart/alternative; boundary="Apple-Mail=_5B46F54D-49BF-4A7F-A169-4E437A1E01F7" --Apple-Mail=_5B46F54D-49BF-4A7F-A169-4E437A1E01F7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Extended public and private keys are defined in BIP32 [1]. Encoded extended private keys should not be confused with a wallet = =E2=80=9Eseed=E2=80=9C (proposals like BIP39) while they can also partially serve the purpose = to =E2=80=9Eseed=E2=80=9C a wallet (there may be an overlap in the = use-case). Recovering a wallet by its extended private master key (xpriv; may or = may not be at depth 0) is a complex task with risks of failing to recover all = available funds. It may be reasonable to consider that recovering a wallet purely based = on the existence of an extended private master key is a forensic funds recovery process and should probably be the last resort in case of a = backup-recovery situation. A simple example here is, that it was/is possible to have = used an xpriv (referring to extended private master key) in production that = is/was used to derive BIP45 based P2SH multisig addresses (1of1, used by Bitpays BWS = for while), later used for bare BIP45ish multisig 1of1 as well as for P2PKH = after BIP44 & vanilla BIP32 P2WPKH (m/0=E2=80=99/k=E2=80=99). I=E2=80=99m not aware of any wallet that would recover 100% of those = funds, leading to the risk that forwarding the unspents and destroying the extended master = key may result in coins forever lost. The case above may be an edge case, but I=E2=80=99m generally under the = assumption that recovering funds based on the sole existence of an xpriv (or seed) = without further metadata is a fragile concept. Second, the missing birthday-metadata tend to lead to non-optimal = blockchain scans (eventually increased p2p traffic). Recovering funds can take = hours. Additionally, the BIP44 gap limit seems to be a weak construct. The = current gap limit in BIP44 is set to 20 [2] which basically means, handing out more = then 20 incoming payment requests (addresses) results in taking the risks that = funds may be destroyed (or at least not detected) during a recovery. The Gap limit value may also depend on the use case, but the current = proposals do not allow to set an arbitrary value. High load merchants very likely = need a different gap limit value then individuals create a transaction once a = year. During creation time of an xpriv/xpub, it is impossible to know if the = created xpriv will be used for an unforeseen derivation scheme. Future proposals = may want to limit an extended key to a single derivation scheme. This is an early draft in order to allow discussion that may lead to a = possible proposal. This proposals could also make BIP 178 obsolete since it can be replace = the WIF[3] standard. Thanks for feedback /jonas ------------------------------------ Titel ###### Bech32 encoded key material including metadata Abstract ######## An error tolerant encoding format for key material up to 520bits with a = minimal amount of metadata. Motivation ########## (See above; intro text) Specification ############# ## Serialization format 1 bit version bit 15 bits (bit 1 to 16) key-birthday (0-32767) (12 bit gap limit) 3 or 5 bits script type 256 or 512 or 520 bits key material =3D Total 275, 545, 553 bits The initial version bit allows extending the serialization-format in = future. The encoding format must hint the total length and thus allow to = calculate the length of the key material. The total length for 256 or 512 bit key material is optimised for Bech32 = (power of 5). ### Key material If the key material length is 520 bits, it must contain an extended = public key If the key material length is 512 bits, it must contain an extended = private key Key material length other then 256, 512, 520 bits and invalid. If 520 bits are present, first 256 bits are the BIP32 chain code, to = second 264 bits (33 bytes) define the public key (according to BIP32) If 512 bits are present, first 256 bits are the BIP32 chain code, to = second 256 bits define the private key If 256 bits are present, those bits represent a pure private key (or = seed) ### Key birthday A 15 bit timestamp expressed in days since genesis (valid up to ~2098). = The birthday must be set to the first possible derivation of the according = extended key, if unknown, the used seed birthday must be used. If both unknown, 0 (16x0bit) must be used. ### Gap limit delta 12 bits, results in a possible range from 0 to 4095. If the total decoded serialization length is 275 bits (decode) or if the = key material is 256 bits (encode), the gap limit must not be present. The base gap limit value is 20 (to disallow insane gap limits). The = final gap limit is the base value + the gap limit delta stored in those 12 bits. Key derivation gap limit must not be exceeded when deriving child keys = and must be respected during transaction rescans. Child key derivation must not be possible if gap limit is hit. ### Script type restriction 3 or 5 bits (range 0-7 / 0-31) 0 no restriction 1 P2PKH compressed 2 P2PKH | P2SH 3 P2WPKH P2WSH nested in P2SH 4 P2WPKH | P2WSH If the total decoded serialization length is 275 bits (decode) or if the = key material is 256 bits (encode), 3 bits are used for the script type. 5 = bits are used for key material with the size of 512, 520 bits. If the script type restriction is set, the according extended key must = only be used to derive addresses with the selected script type. This does not stands in contradiction to derivation path proposals = ([4]). It does allow to derive and encode an extended key at a keypath where users = assume restricted script types in derivation due to other supported proposals. Encoding ######## Bech32 must be used as encoding format (see the Bech32 rational [5]). = Encoding 545 or 553 bits (results in 109 resp. 111 x 5 bits) will exceed the = Bech32 property of a guaranteed detection of 4 errors (only 3 are). It is possible that there are more efficient BCH codes, especially for = encoding extended private keys. Since a Bech32 implementation needs to be present = in modern Bitcoin software, re-using Bech32 will allow to migrate to this = proposal with a minimal implementation effort. Forensic, cpu-intense key-recovery (including brute-force techniques) = may allow to recover keys beyond the guaranteed error detection limits. Bech32 HRPs Mainnet Private Extended: xp Mainnet Public Extended: xpu Testnet Private Extended: tp Testnet Public Extended: tpu Mainnet Key: pk- Testnet Key: tk- Compatibility ########### Only new software will be able to use these serialization and encoding = format. References ########## [1] https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki = [2] https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki [3] https://github.com/bitcoin/bips/blob/master/bip-0178.mediawiki [4] https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki [5] = https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#rationale --Apple-Mail=_5B46F54D-49BF-4A7F-A169-4E437A1E01F7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi

Extended public and private keys are defined in BIP32 = [1].

Encoded = extended private keys should not be confused with a wallet = =E2=80=9Eseed=E2=80=9C 
(proposals like BIP39) = while they can also partially serve the purpose to 
=E2=80=9Eseed=E2=80=9C a wallet (there may be an overlap in = the use-case).

Recovering a wallet by its extended private master key = (xpriv; may or may not 
be at depth 0) is a = complex task with risks of failing to recover all = available 
funds.

It may be reasonable to consider that = recovering a wallet purely based on the 
existence of an extended private master key is a forensic = funds recovery 
process and should probably be = the last resort in case of a backup-recovery 
situation. A simple example here is, that it was/is possible = to have used an 
xpriv (referring to extended = private master key) in production that is/was used 
to derive BIP45 based P2SH multisig addresses (1of1, used by = Bitpays BWS for 
while), later used for bare = BIP45ish multisig 1of1 as well as for P2PKH after 
BIP44 & vanilla BIP32 P2WPKH (m/0=E2=80=99/k=E2=80=99).
I=E2=80=99m not aware of any wallet that would recover = 100% of those funds, leading to 
the risk that = forwarding the unspents and destroying the extended master = key 
may result in coins forever = lost.

The case = above may be an edge case, but I=E2=80=99m generally under the = assumption that 
recovering funds based on the = sole existence of an xpriv (or seed) without further 
metadata is a fragile concept.

Second, the missing birthday-metadata = tend to lead to non-optimal blockchain 
scans = (eventually increased p2p traffic). Recovering funds can take = hours.

Additionally, the BIP44 gap limit seems to be a weak = construct. The current gap 
limit in BIP44 is = set to 20 [2] which basically means, handing out more then = 20 
incoming payment requests (addresses) = results in taking the risks that funds 
may be = destroyed (or at least not detected) during a recovery.
The Gap limit value may also depend on the use case, but the = current proposals 
do not allow to set an = arbitrary value. High load merchants very likely need a 
different gap limit value then individuals create a = transaction once a year.

During creation time of an xpriv/xpub, it is impossible to = know if the created 
xpriv will be used for an = unforeseen derivation scheme. Future proposals may 
want to limit an extended key to a single derivation = scheme.


This is an early draft in order to = allow discussion that may lead to a possible
proposal.
This proposals could also make = BIP 178 obsolete since it can be replace the 
WIF[3] standard.


Thanks for = feedback
/jonas


------------------------------------


Titel
######
Bech32 = encoded key material including metadata

Abstract
########
An error tolerant encoding = format for key material up to 520bits with a minimal 
amount of metadata.

Motivation
##########
(See above; intro = text)


Specification
#############

## Serialization format

1 bit version bit
15= bits (bit 1 to 16) key-birthday (0-32767)
(12 bit = gap limit)
3 or 5 bits script type
256 or 512 or 520 bits key material
=3D = Total 275, 545, 553 bits

The initial version bit allows extending the = serialization-format in future.
The encoding format = must hint the total length and thus allow to calculate = the 
length of the key material.

The total length for 256 = or 512 bit key material is optimised for Bech32 (power 
of 5).

### Key material
If the key material = length is 520 bits, it must contain an extended public key
If the key material length is 512 bits, it must contain an = extended private key
Key material length other then = 256, 512, 520 bits and invalid.

If 520 bits are present, first 256 bits = are the BIP32 chain code, to second 264 
bits = (33 bytes) define the public key (according to BIP32)

If 512 bits are present, = first 256 bits are the BIP32 chain code, to second 256 
bits define the private key

If 256 bits are present, those bits = represent a pure private key (or seed)

### Key birthday 
A 15 bit timestamp expressed in days since genesis (valid up = to ~2098). The 
birthday must be set to the = first possible derivation of the according extended 
key, if unknown, the used seed birthday must be used. If both = unknown, 0 
(16x0bit) must be used.

### Gap limit = delta
12 bits, results in a possible range from 0 = to 4095.

If = the total decoded serialization length is 275 bits (decode) or if the = key 
material is 256 bits (encode), the gap = limit must not be present.

The base gap limit value is 20 (to disallow insane gap = limits). The final gap 
limit is the base = value + the gap limit delta stored in those 12 bits.
Key derivation gap limit must not be exceeded when deriving = child keys and must 
be respected during = transaction rescans.
Child key derivation must not = be possible if gap limit is hit.

### Script type restriction
3 or 5 bits (range 0-7 / 0-31)
0 no = restriction
1 P2PKH compressed
2= P2PKH | P2SH
3 P2WPKH P2WSH nested in = P2SH
4 P2WPKH | P2WSH

If the total decoded serialization = length is 275 bits (decode) or if the key 
material is 256 bits (encode), 3 bits are used for the script = type. 5 bits are 
used for key material with = the size of 512, 520 bits.

If the script type restriction is set, the according extended = key must only be 
used to derive addresses = with the selected script type.
This does not stands = in contradiction to derivation path proposals ([4]). It 
does allow to derive and encode an extended key at a keypath = where users assume 
restricted script types in = derivation due to other supported proposals.


Encoding
########
Bech32 must be used as encoding format = (see the Bech32 rational [5]). Encoding 
545 = or 553 bits (results in 109 resp. 111 x 5 bits) will exceed the Bech32 = property of a 
guaranteed detection of 4 = errors (only 3 are).
It is possible that there are = more efficient BCH codes, especially for encoding 
extended private keys. Since a Bech32 implementation needs to = be present in 
modern Bitcoin software, = re-using Bech32 will allow to migrate to this proposal 
with a minimal implementation effort.
Forensic, cpu-intense key-recovery (including brute-force = techniques) may allow 
to recover keys beyond = the guaranteed error detection limits.

Bech32 HRPs
Mainnet = Private Extended: xp
Mainnet Public Extended: = xpu
Testnet Private Extended: tp
Testnet Public Extended: tpu
Mainnet = Key: pk-
Testnet Key: tk-

Compatibility
###########
Only new software will be = able to use these serialization and encoding format.

References
##########


[1] https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki<= /a>
[2] https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki<= /a>
[3] https://github.com/bitcoin/bips/blob/master/bip-0178.mediawiki<= /a>
[4] https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki<= /a>
[5] https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#= rationale

= --Apple-Mail=_5B46F54D-49BF-4A7F-A169-4E437A1E01F7-- --Apple-Mail=_3A741525-D3A7-4193-A3B4-AC55A5C8B0CC Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEyhopCNzi8TB0xizeHrd2uwPHki0FAlsNGcEACgkQHrd2uwPH ki1ICRAAnzyZxlQizVPFNlLlrDN8N1YbRPZHCQavMNuagsSaUh/2f32e03TvI9tM RTPsAyhQ52VtyoT0IxJdlRqFGw60BB9QKEHlNngT3gsisZ9GneK+ouqMPUQh14hJ yi0+lJdDRhMtXDLp+Wkdf907zXQQFA/iwVIQeBUjllN72tK/Apx+0lyHdK+Br1FF +5q98Yl79o/7XNI1X+clzlqe0I43qDJ+MxcKzV8vFEq41HdLtLstj8qPk7mMLgx/ iQ80peUGlclmeTfDORDUgaa6oBD1qTA1+GHfDxTnd3eQpxcE42w/nXnUijM3UYqn OR2bdY81lkJZqb59Wnu40Qapbnr3LKGJvDw1SzFVm1fuYBbIWeVQi7rTb+QStfh6 luEFVCQG557vtTdTCFZEM2Y2TwTLwXB0t4yOqod8FbOo5075kW+pYGVeGcUj2wWO 78I6JN3rOIiAKe+z90Y0bxblQncwoL43m288hIjO5D0/ojSxCsaKt8EE2ZY6VmGG +tE0ox7Y46mHO1zOK215Qz0X0Wt+k1M9BvMR2+51gsXidYHTvWrV1cSlbQfVz/eU nWA1OLIofbSImmV9GFPNggHEUYe3n0quVqlfQFB1r7S26U9WF6ezeq9IblEKaQ0j 14MM6EIglmp08yRSPuv/Krs62KGljEfHsKwb1r+e2U5bRvUbiD0= =3XYm -----END PGP SIGNATURE----- --Apple-Mail=_3A741525-D3A7-4193-A3B4-AC55A5C8B0CC--