1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
|
Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
helo=mx.sourceforge.net)
by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mh.in.england@gmail.com>) id 1X71SU-0000AG-H1
for bitcoin-development@lists.sourceforge.net;
Tue, 15 Jul 2014 12:03:46 +0000
Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.219.52 as permitted sender)
client-ip=209.85.219.52; envelope-from=mh.in.england@gmail.com;
helo=mail-oa0-f52.google.com;
Received: from mail-oa0-f52.google.com ([209.85.219.52])
by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1X71SS-0007kl-GX
for bitcoin-development@lists.sourceforge.net;
Tue, 15 Jul 2014 12:03:46 +0000
Received: by mail-oa0-f52.google.com with SMTP id o6so3774200oag.39
for <bitcoin-development@lists.sourceforge.net>;
Tue, 15 Jul 2014 05:03:37 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.60.52.5 with SMTP id p5mr10859126oeo.55.1405425817027; Tue,
15 Jul 2014 05:03:37 -0700 (PDT)
Sender: mh.in.england@gmail.com
Received: by 10.76.35.234 with HTTP; Tue, 15 Jul 2014 05:03:36 -0700 (PDT)
Date: Tue, 15 Jul 2014 14:03:36 +0200
X-Google-Sender-Auth: RCm5k5l2rMqvmKbZ1RwhxH4zskc
Message-ID: <CANEZrP3ZzCBohXWZmZxE=ofP74Df4Hd-hCLH6jYn=JKbiqNQXA@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>,
Aaron Voisine <voisine@gmail.com>
Content-Type: multipart/alternative; boundary=001a11332c8e9508bd04fe3a328d
X-Spam-Score: -0.5 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(mh.in.england[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1X71SS-0007kl-GX
Cc: Andreas Schildbach <andreas@schildbach.de>
Subject: [Bitcoin-development] BIP 38 NFC normalisation issue
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 15 Jul 2014 12:03:46 -0000
--001a11332c8e9508bd04fe3a328d
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
[+cc aaron]
We recently added an implementation of BIP 38 (password protected private
keys) to bitcoinj. It came to my attention that the third test vector may
be broken. It gives a hex version of what the NFC normalised version of the
input string should be, but this does not match the results of the Java
unicode normaliser, and in fact I can't even get Python to print the names
of the characters past the embedded null. I'm curious where this normalised
version came from.
Given that "pile of poo" is not a character I think any sane user would put
into a passphrase, I question the value of this test vector. NFC form is
intended to collapse things like umlaut control characters onto their prior
code point, but here we're feeding the algorithm what is basically garbage
so I'm not totally surprised that different implementations appear to
disagree on the outcome.
Proposed action: we remove this test vector as it does not represent any
real world usage of the spec, or if we desperately need to verify NFC
normalisation I suggest using a different, more realistic test string, like
Z=C3=BCrich, or something written in Thai.
Test 3:
- Passphrase =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 (\u03D2\u0301=
\u0000\U00010400\U0001F4A9; GREEK
UPSILON WITH HOOK <http://codepoints.net/U+03D2>, COMBINING ACUTE ACCENT
<http://codepoints.net/U+0301>, NULL <http://codepoints.net/U+0000>, DES=
ERET
CAPITAL LETTER LONG I <http://codepoints.net/U+10400>, PILE OF POO
<http://codepoints.net/U+1F4A9>)
- Encrypted key:
6PRW5o9FLp4gJDDVqJQKJFTpMvdsSGJxMYHtHaQBF3ooa8mwD69bapcDQn
- Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
- Unencrypted private key (WIF):
5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
- *Note:* The non-standard UTF-8 characters in this passphrase should be
NFC normalized to result in a passphrase of0xcf9300f0909080f09f92a9 befo=
re
further processing
--001a11332c8e9508bd04fe3a328d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">[+cc aaron]<div><br></div><div>We recently added an implem=
entation of BIP 38 (password protected private keys) to bitcoinj. It came t=
o my attention that the third test vector may be broken. It gives a hex ver=
sion of what the NFC normalised version of the input string should be, but =
this does not match the results of the Java unicode normaliser, and in fact=
I can't even get Python to print the names of the characters past the =
embedded null. I'm curious where this normalised version came from.<div=
>
<br></div><div>Given that "pile of poo" is not a character I thin=
k any sane user would put into a passphrase, I question the value of this t=
est vector. NFC form is intended to collapse things like umlaut control cha=
racters onto their prior code point, but here we're feeding the algorit=
hm what is basically garbage so I'm not totally surprised that differen=
t implementations appear to disagree on the outcome.</div>
<div><br></div><div>Proposed action: we remove this test vector as it does =
not represent any real world usage of the spec, or if we desperately need t=
o verify NFC normalisation I suggest using a different, more realistic test=
string, like Z=C3=BCrich, or something written in Thai.<br>
<div><br></div><div><br><div><br></div><div><span style=3D"color:rgb(51,51,=
51);font-family:Helvetica,arial,freesans,clean,sans-serif,'Segoe UI Emo=
ji','Segoe UI Symbol';font-size:15.454545021057129px;line-heigh=
t:23.18181800842285px">Test 3:</span><ul style=3D"padding:0px 0px 0px 30px;=
margin:15px 0px;color:rgb(51,51,51);font-family:Helvetica,arial,freesans,cl=
ean,sans-serif,'Segoe UI Emoji','Segoe UI Symbol';font-size=
:15.454545021057129px;line-height:23.18181800842285px">
<li>Passphrase =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 (<tt style=3D"=
font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font=
-size:12px;margin:0px;border:1px solid rgb(221,221,221);border-top-left-rad=
ius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-b=
ottom-left-radius:3px;padding:0px;background-color:rgb(248,248,248)">\u03D2=
\u0301\u0000\U00010400\U0001F4A9</tt>;=C2=A0<a href=3D"http://codepoints.ne=
t/U+03D2" target=3D"_blank" style=3D"color:rgb(65,131,196);text-decoration:=
none;background:transparent">GREEK UPSILON WITH HOOK</a>,=C2=A0<a href=3D"h=
ttp://codepoints.net/U+0301" target=3D"_blank" style=3D"color:rgb(65,131,19=
6);text-decoration:none;background:transparent">COMBINING ACUTE ACCENT</a>,=
=C2=A0<a href=3D"http://codepoints.net/U+0000" target=3D"_blank" style=3D"c=
olor:rgb(65,131,196);text-decoration:none;background:transparent">NULL</a>,=
=C2=A0<a href=3D"http://codepoints.net/U+10400" target=3D"_blank" style=3D"=
color:rgb(65,131,196);text-decoration:none;background:transparent">DESERET =
CAPITAL LETTER LONG I</a>,=C2=A0<a href=3D"http://codepoints.net/U+1F4A9" t=
arget=3D"_blank" style=3D"color:rgb(65,131,196);text-decoration:none;backgr=
ound:transparent">PILE OF POO</a>)</li>
<li>Encrypted key: 6PRW5o9FLp4gJDDVqJQKJFTpMvdsSGJxMYHtHaQBF3ooa8mwD69bapcD=
Qn</li><li>Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF</li><li>Unen=
crypted private key (WIF): 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTM=
SZ4</li>
<li><i>Note:</i>=C2=A0The non-standard UTF-8 characters in this passphrase =
should be NFC normalized to result in a passphrase of<tt style=3D"font-fami=
ly:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12p=
x;margin:0px;border:1px solid rgb(221,221,221);border-top-left-radius:3px;b=
order-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-lef=
t-radius:3px;padding:0px;background-color:rgb(248,248,248)">0xcf9300f090908=
0f09f92a9</tt>=C2=A0before further processing</li>
</ul><div><font color=3D"#333333" face=3D"Helvetica, arial, freesans, clean=
, sans-serif, Segoe UI Emoji, Segoe UI Symbol"><span style=3D"font-size:15p=
x;line-height:23.18181800842285px"><br></span></font></div></div><div><br><=
/div>
</div></div></div></div>
--001a11332c8e9508bd04fe3a328d--
|
