1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
|
Return-Path: <ZmnSCPxj@protonmail.com>
Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133])
by lists.linuxfoundation.org (Postfix) with ESMTP id 1547BC0881
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Dec 2019 01:50:47 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by hemlock.osuosl.org (Postfix) with ESMTP id 046F987CC9
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Dec 2019 01:50:47 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from hemlock.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id ahWpyQcPXSgL
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Dec 2019 01:50:44 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch
[185.70.40.134])
by hemlock.osuosl.org (Postfix) with ESMTPS id A7433879DA
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Dec 2019 01:50:44 +0000 (UTC)
Date: Tue, 10 Dec 2019 01:50:38 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
s=default; t=1575942641;
bh=dUnndObwS0G8bwQacRcWpN2grBuKb9yVNyO6nNqQfBI=;
h=Date:To:From:Reply-To:Subject:In-Reply-To:References:Feedback-ID:
From;
b=P/LhOZ2s6HNMmYPEAIwXi4P9nZZq6hGeWZLjqpdCAVxP71RVJCoFxppigswLBUryN
De5vbPKUNh0MmN31+P04uYYynVBYvU3+RXpLVRyVmNM+qyIK49nf3MTdsyepuv9Sjo
lfIka4xn9nNzOzsk7ifejMyZNs+b3sXUElodCnA8=
To: Pieter Wuille <pieter.wuille@gmail.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
From: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Reply-To: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Message-ID: <2YyEOYXhcEvfGJLUX4zswzSpBA0vWOC_Jwl_MOcphySLZqjfBDqqDkBvZB6kX7nvVsGNPqwuh63lgBGO5BcURaig2r5tqxFoaEZTLDMTG7U=@protonmail.com>
In-Reply-To: <CAPg+sBgdspmfK1qpG=9N9fAtVNkMDd+xym7yHdjq=zYnqeh5dA@mail.gmail.com>
References: <CAPg+sBgdspmfK1qpG=9N9fAtVNkMDd+xym7yHdjq=zYnqeh5dA@mail.gmail.com>
Feedback-ID: el4j0RWPRERue64lIQeq9Y2FP-mdB86tFqjmrJyEPR9VAtMovPEo9tvgA0CrTsSHJeeyPXqnoAu6DN-R04uJUg==:Ext:ProtonMail
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: [bitcoin-dev] Analysis of Bech32 swap/insert/delete detection
and next steps
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Dec 2019 01:50:47 -0000
Good morning Pieter,
> Hi all,
>
> I've made a writeup on Bech32's detection abilities, analysing how it
> behaves in the presence of not just substitution errors, but also
> swapping of characters, and insertions and deletions:
> https://gist.github.com/sipa/a9845b37c1b298a7301c33a04090b2eb
>
> It shows that the "insert or delete a 'q' right before a final 'p'" is
> in fact the only deviation from the expected at-most-1-in-a-billion
> failure to detect chance, at least when restricted to the classes of
> errors analyzed with various uniformity assumptions. There is some
> future work left, such as analyzing combinations of insertions and
> substitutions, but I would be surprising if additional weaknesses
> exist there.
>
> It also shows that changing one constant in Bech32 would resolve this
> issue, while not affecting the error detection properties for other
> classes of errors.
>
> So my suggestion for the next steps are:
>
> - Update BIP173 to include the insertion weakness as an erratum, and
> the results of this analysis.
>
To clarify, this step does not modify anything about the implementation of =
BIP173, only adds this as an additional erratum section?
> - Amend segwit addresses (either by amending BIP173, or by writing a
> short updated BIP to modify it) to be restricted to only length 20 or
> 32 (as fixed-length strings are unaffected by the insertion issue, an=
d
> I don't think inserting 20 characters is an interesting error class).
To clarify, this refers to all SegWit address versions from 1 to 15, as thi=
s restriction exists for SegWit address v0?
>
> - Define a variant of Bech32 with the modified constant, so that
> non-BIP173 uses of Bech32 can choose a non-impacted version if they
> worry about this class of errors.
>
Okay, this probably needs to be raised in lightning-dev as well, for invoic=
e formats, as well as planned offers feature.
By my understanding, best practice for readers of Bech32-based formats woul=
d be something like the below:
1. Define two variants of checksum, the current Bech32 checksum and the mo=
dified Bech32 checksum.
2. Support both variants (software tries one first, then tries the other i=
f it fails).
3. Flag or signal some deprecation warning if current Bech32 checksum was =
detected.
4. At some undefined point in the future, drop support for the current Bec=
h32 checksum.
> - Later, if and when we expect a need for non-32-byte witness programs
> in the medium term, define an updated segwit address scheme that uses
> the modified Bech32 variant.
Okay, so we will only use the modified Bech32 if and only if we expect to n=
eed a non-32-byte witness program for a particular non-0 SegWit version.
Regards,
ZmnSCPxj
|
