summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMike Hearn <mike@plan99.net>2014-07-17 12:59:26 +0200
committerbitcoindev <bitcoindev@gnusha.org>2014-07-17 10:59:34 +0000
commit73037dea0f852445c5020682f1269f127eb40c74 (patch)
tree205cecf1b376b787c8663b79905a456bd70b8c16
parent2c83830c4c03b77ba1e3f5f21d87b93ca82fbfbc (diff)
downloadpi-bitcoindev-73037dea0f852445c5020682f1269f127eb40c74.tar.gz
pi-bitcoindev-73037dea0f852445c5020682f1269f127eb40c74.zip
Re: [Bitcoin-development] BIP 38 NFC normalisation issue
-rw-r--r--d1/a1bc23babb9be987002357f9986f3a067ca6df752
1 files changed, 752 insertions, 0 deletions
diff --git a/d1/a1bc23babb9be987002357f9986f3a067ca6df b/d1/a1bc23babb9be987002357f9986f3a067ca6df
new file mode 100644
index 000000000..9072fa45c
--- /dev/null
+++ b/d1/a1bc23babb9be987002357f9986f3a067ca6df
@@ -0,0 +1,752 @@
+Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
+ helo=mx.sourceforge.net)
+ by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
+ (envelope-from <mh.in.england@gmail.com>) id 1X7jPS-0003u6-Ah
+ for bitcoin-development@lists.sourceforge.net;
+ Thu, 17 Jul 2014 10:59:34 +0000
+Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
+ designates 209.85.213.172 as permitted sender)
+ client-ip=209.85.213.172; envelope-from=mh.in.england@gmail.com;
+ helo=mail-ig0-f172.google.com;
+Received: from mail-ig0-f172.google.com ([209.85.213.172])
+ by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
+ (Exim 4.76) id 1X7jPQ-0003n4-33
+ for bitcoin-development@lists.sourceforge.net;
+ Thu, 17 Jul 2014 10:59:34 +0000
+Received: by mail-ig0-f172.google.com with SMTP id h15so5837274igd.17
+ for <bitcoin-development@lists.sourceforge.net>;
+ Thu, 17 Jul 2014 03:59:26 -0700 (PDT)
+MIME-Version: 1.0
+X-Received: by 10.60.132.203 with SMTP id ow11mr44174756oeb.47.1405594766703;
+ Thu, 17 Jul 2014 03:59:26 -0700 (PDT)
+Sender: mh.in.england@gmail.com
+Received: by 10.76.35.234 with HTTP; Thu, 17 Jul 2014 03:59:26 -0700 (PDT)
+Received: by 10.76.35.234 with HTTP; Thu, 17 Jul 2014 03:59:26 -0700 (PDT)
+In-Reply-To: <lq6sq5$71h$1@ger.gmane.org>
+References: <CANEZrP3ZzCBohXWZmZxE=ofP74Df4Hd-hCLH6jYn=JKbiqNQXA@mail.gmail.com>
+ <CAObn+gfbH61kyv_ttT4vsQuNFRWLB5H3xaux7GQ0co82ucO_eA@mail.gmail.com>
+ <CANg-TZAe2PO9nwQktmDSJFtaLsg6hogOw6mj0SaROdJJr33vog@mail.gmail.com>
+ <CANEZrP3E2mdvOUHiW9W_hM3Z_kn9um8E6aX5vf-S9tA7KgnpUQ@mail.gmail.com>
+ <CACq0ZD5qTc-2f+puPaXMDFZNWUx8kvOZQMxqkM_e4YafhTW7cA@mail.gmail.com>
+ <lq5fvo$4s6$1@ger.gmane.org>
+ <CANEZrP0x2Ypb063VkcoE+h_OHfRVOusmXB2X1VQx77sZhAuTFw@mail.gmail.com>
+ <lq5l5a$4fl$1@ger.gmane.org> <lq5m78$j23$1@ger.gmane.org>
+ <CACq0ZD6Nib4kU8kAo6jdXamw7NTTd5_JwO4x1fsVOMP2-F1=cA@mail.gmail.com>
+ <lq6sq5$71h$1@ger.gmane.org>
+Date: Thu, 17 Jul 2014 12:59:26 +0200
+X-Google-Sender-Auth: W1V9nEXY4trg4l1r1KdPK5QQwwU
+Message-ID: <CANEZrP2=e-JSRjuRgyeGNd2-fvXxEi5t4PAS3BrT-Y7SieywdQ@mail.gmail.com>
+From: Mike Hearn <mike@plan99.net>
+To: Andreas Schildbach <andreas@schildbach.de>
+Content-Type: multipart/alternative; boundary=047d7b41cc74c4831904fe618858
+X-Spam-Score: -0.5 (/)
+X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
+ See http://spamassassin.org/tag/ for more details.
+ -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
+ sender-domain
+ 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
+ (mh.in.england[at]gmail.com)
+ -0.0 SPF_PASS SPF: sender matches SPF record
+ 1.0 HTML_MESSAGE BODY: HTML included in message
+ 0.1 DKIM_SIGNED Message has a DKIM or DK signature,
+ not necessarily valid
+ -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
+X-Headers-End: 1X7jPQ-0003n4-33
+Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
+Subject: Re: [Bitcoin-development] BIP 38 NFC normalisation issue
+X-BeenThere: bitcoin-development@lists.sourceforge.net
+X-Mailman-Version: 2.1.9
+Precedence: list
+List-Id: <bitcoin-development.lists.sourceforge.net>
+List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
+ <mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
+List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
+List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
+List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
+List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
+ <mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
+X-List-Received-Date: Thu, 17 Jul 2014 10:59:34 -0000
+
+--047d7b41cc74c4831904fe618858
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: quoted-printable
+
+Glad we got to the bottom of that. That's quite a nasty compiler/language
+bug I must say. Not even a warning. Still, python crashes when trying to
+print the name of a null character. It wouldn't surprise me if there are
+other weird issues lurking. Would definitely sleep better with a more
+restricted character set.
+On 17 Jul 2014 00:04, "Andreas Schildbach" <andreas@schildbach.de> wrote:
+
+> Please excuse me. I had a more thorough look at the original problem and
+> found that the only problem with the original test case was that you
+> cannot specify codepoints from the SMP using \u in Java. I always tried
+> \u010400 but that doesn't work.
+>
+> Here is a fix for bitcoinj. The test now passes.
+>
+> https://github.com/bitcoinj/bitcoinj/pull/143
+>
+> We can (and probably should) still need to filter control chars, I'll
+> have a look at that now again.
+>
+>
+> On 07/16/2014 11:06 PM, Aaron Voisine wrote:
+> > If I first remove \u0000, so the non-normalized passphrase is
+> > "\u03D2\u0301\U00010400\U0001F4A9", and then NFC normalize it, it
+> > becomes "\u03D3\U00010400\U0001F4A9"
+> >
+> > UTF-8 encoded this is: 0xcf93f0909080f09f92a9 (not the same as what
+> > you got, Andreas!)
+> >
+> > Encoding private key: 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMS=
+Z4
+> > with this passphrase, I get a BIP38 key of:
+> > 6PRW5o9FMb4hAYRQPmgcvVDTyDtr6R17VMXGLmvKjKVpGkYhBJ4uYuR9wZ
+> >
+> > I recommend rather than simply removing control characters from the
+> > password that instead the spec require that passwords containing
+> > control characters are invalid. We don't want people trying to be
+> > clever and putting them in thinking they are adding to the password
+> > entropy.
+> >
+> > Also for UI compatibility across many platforms, I'm also in favor
+> > disallowing any character below U+0020 (space)
+> >
+> > I can submit a PR once we figure out why Andreas's passphrase was
+> > different than what I got.
+> >
+> > Aaron Voisine
+> > breadwallet.com
+> >
+> >
+> > On Wed, Jul 16, 2014 at 4:04 AM, Andreas Schildbach
+> > <andreas@schildbach.de> wrote:
+> >> Damn, I just realized that I implement only the decoding side of BIP38=
+.
+> >> So I cannot propose a complete test vector. Here is what I have:
+> >>
+> >>
+> >> Passphrase: =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 (\u03D2\u030=
+1\u0000\U00010400\U0001F4A9; GREEK
+> >> UPSILON WITH HOOK, COMBINING ACUTE ACCENT, NULL, DESERET CAPITAL LETTE=
+R
+> >> LONG I, PILE OF POO)
+> >>
+> >> Passphrase bytes after removing ISO control characters and NFC
+> >> normalization: 0xcf933034303066346139
+> >>
+> >> Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
+> >>
+> >> Unencrypted private key (WIF):
+> >> 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
+> >>
+> >>
+> >> Can someone calculate the encrypted key from it (using whatever
+> >> implementation) and I will verify it decodes properly in bitcoinj?
+> >>
+> >>
+> >>
+> >> On 07/16/2014 12:46 PM, Andreas Schildbach wrote:
+> >>> I will change the bitcoinj implementation and propose a new test
+> vector.
+> >>>
+> >>>
+> >>>
+> >>> On 07/16/2014 11:29 AM, Mike Hearn wrote:
+> >>>> Yes sorry, you're right, the issue starts with the null code point.
+> >>>> Python seems to have problems starting there too. It might work if w=
+e
+> >>>> took that out.
+> >>>>
+> >>>>
+> >>>> On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach
+> >>>> <andreas@schildbach.de <mailto:andreas@schildbach.de>> wrote:
+> >>>>
+> >>>> Guys, you are always talking about the Unicode astral plane, but
+> in fact
+> >>>> its a plain old (ASCII) control character where this problem
+> starts and
+> >>>> likely ends: \u0000.
+> >>>>
+> >>>> Let's ban/filter ISO control characters and be done with it. Mos=
+t
+> >>>> control characters will never be enterable by any keyboard into =
+a
+> >>>> password field. Of course I assume that Character.isISOControl()
+> works
+> >>>> consistently across platforms.
+> >>>>
+> >>>>
+> http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#isISOCo=
+ntrol%28char%29
+> >>>>
+> >>>>
+> >>>> On 07/16/2014 12:23 AM, Aaron Voisine wrote:
+> >>>> > If the user creates a password on an iOS device with an astral
+> >>>> > character and then can't enter that password on a JVM wallet,
+> that
+> >>>> > sucks. If JVMs really can't support unicode NFC then that's a
+> strong
+> >>>> > case to limit the spec to the subset of unicode that all popul=
+ar
+> >>>> > platforms can support, but it sounds like it might just be a J=
+VM
+> >>>> > string library bug that could hopefully be reported and fixed.
+> I get
+> >>>> > the same result as in the test case using apple's
+> >>>> > CFStringNormalize(passphrase, kCFStringNormalizationFormC);
+> >>>> >
+> >>>> > Aaron Voisine
+> >>>> > breadwallet.com <http://breadwallet.com>
+> >>>> >
+> >>>> >
+> >>>> > On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn <mike@plan99.net
+> >>>> <mailto:mike@plan99.net>> wrote:
+> >>>> >> Yes, we know, Andreas' code is indeed doing normalisation.
+> >>>> >>
+> >>>> >> However it appears the output bytes end up being different.
+> What
+> >>>> I get back
+> >>>> >> is:
+> >>>> >>
+> >>>> >> cf930001303430300166346139
+> >>>> >>
+> >>>> >> vs
+> >>>> >>
+> >>>> >> cf9300f0909080f09f92a9
+> >>>> >>
+> >>>> >> from the spec.
+> >>>> >>
+> >>>> >> I'm not sure why. It appears this is due to the character fro=
+m
+> >>>> the astral
+> >>>> >> planes. Java is old and uses 16 bit characters internally - i=
+t
+> >>>> wouldn't
+> >>>> >> surprise me if there's some weirdness that means it
+> doesn't/won't
+> >>>> support
+> >>>> >> this kind of thing.
+> >>>> >>
+> >>>> >> I recommend instead that any implementation that wishes to be
+> >>>> compatible
+> >>>> >> with JVM based wallets (I suspect Android is the same) just
+> >>>> refuse any
+> >>>> >> passphrase that includes characters outside the BMP. At least
+> >>>> unless someone
+> >>>> >> can find a fix. I somehow doubt this will really hurt anyone.
+> >>>> >>
+> >>>> >>
+> >>>>
+> -------------------------------------------------------------------------=
+-----
+> >>>> >> Want fast and easy access to all the code in your enterprise?
+> >>>> Index and
+> >>>> >> search up to 200,000 lines of code with a free copy of Black
+> Duck
+> >>>> >> Code Sight - the same software that powers the world's larges=
+t
+> code
+> >>>> >> search on Ohloh, the Black Duck Open Hub! Try it now.
+> >>>> >> http://p.sf.net/sfu/bds
+> >>>> >> _______________________________________________
+> >>>> >> Bitcoin-development mailing list
+> >>>> >> Bitcoin-development@lists.sourceforge.net
+> >>>> <mailto:Bitcoin-development@lists.sourceforge.net>
+> >>>> >>
+> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+> >>>> >>
+> >>>> >
+> >>>> >
+> >>>>
+> -------------------------------------------------------------------------=
+-----
+> >>>> > Want fast and easy access to all the code in your enterprise?
+> >>>> Index and
+> >>>> > search up to 200,000 lines of code with a free copy of Black
+> Duck
+> >>>> > Code Sight - the same software that powers the world's largest
+> code
+> >>>> > search on Ohloh, the Black Duck Open Hub! Try it now.
+> >>>> > http://p.sf.net/sfu/bds
+> >>>> >
+> >>>>
+> >>>>
+> >>>>
+> >>>>
+> -------------------------------------------------------------------------=
+-----
+> >>>> Want fast and easy access to all the code in your enterprise?
+> Index and
+> >>>> search up to 200,000 lines of code with a free copy of Black Duc=
+k
+> >>>> Code Sight - the same software that powers the world's largest
+> code
+> >>>> search on Ohloh, the Black Duck Open Hub! Try it now.
+> >>>> http://p.sf.net/sfu/bds
+> >>>> _______________________________________________
+> >>>> Bitcoin-development mailing list
+> >>>> Bitcoin-development@lists.sourceforge.net
+> >>>> <mailto:Bitcoin-development@lists.sourceforge.net>
+> >>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+> >>>>
+> >>>>
+> >>>>
+> >>>>
+> >>>>
+> -------------------------------------------------------------------------=
+-----
+> >>>> Want fast and easy access to all the code in your enterprise? Index
+> and
+> >>>> search up to 200,000 lines of code with a free copy of Black Duck
+> >>>> Code Sight - the same software that powers the world's largest code
+> >>>> search on Ohloh, the Black Duck Open Hub! Try it now.
+> >>>> http://p.sf.net/sfu/bds
+> >>>>
+> >>>>
+> >>>>
+> >>>> _______________________________________________
+> >>>> Bitcoin-development mailing list
+> >>>> Bitcoin-development@lists.sourceforge.net
+> >>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+> >>>>
+> >>>
+> >>>
+> >>>
+> >>>
+> -------------------------------------------------------------------------=
+-----
+> >>> Want fast and easy access to all the code in your enterprise? Index a=
+nd
+> >>> search up to 200,000 lines of code with a free copy of Black Duck
+> >>> Code Sight - the same software that powers the world's largest code
+> >>> search on Ohloh, the Black Duck Open Hub! Try it now.
+> >>> http://p.sf.net/sfu/bds
+> >>>
+> >>
+> >>
+> >>
+> >>
+> -------------------------------------------------------------------------=
+-----
+> >> Want fast and easy access to all the code in your enterprise? Index an=
+d
+> >> search up to 200,000 lines of code with a free copy of Black Duck
+> >> Code Sight - the same software that powers the world's largest code
+> >> search on Ohloh, the Black Duck Open Hub! Try it now.
+> >> http://p.sf.net/sfu/bds
+> >> _______________________________________________
+> >> Bitcoin-development mailing list
+> >> Bitcoin-development@lists.sourceforge.net
+> >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+> >
+> >
+> -------------------------------------------------------------------------=
+-----
+> > Want fast and easy access to all the code in your enterprise? Index and
+> > search up to 200,000 lines of code with a free copy of Black Duck
+> > Code Sight - the same software that powers the world's largest code
+> > search on Ohloh, the Black Duck Open Hub! Try it now.
+> > http://p.sf.net/sfu/bds
+> > _______________________________________________
+> > Bitcoin-development mailing list
+> > Bitcoin-development@lists.sourceforge.net
+> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+> >
+>
+>
+>
+>
+> -------------------------------------------------------------------------=
+-----
+> Want fast and easy access to all the code in your enterprise? Index and
+> search up to 200,000 lines of code with a free copy of Black Duck
+> Code Sight - the same software that powers the world's largest code
+> search on Ohloh, the Black Duck Open Hub! Try it now.
+> http://p.sf.net/sfu/bds
+> _______________________________________________
+> Bitcoin-development mailing list
+> Bitcoin-development@lists.sourceforge.net
+> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
+>
+
+--047d7b41cc74c4831904fe618858
+Content-Type: text/html; charset=UTF-8
+Content-Transfer-Encoding: quoted-printable
+
+<p dir=3D"ltr">Glad we got to the bottom of that. That&#39;s quite a nasty =
+compiler/language bug I must say. Not even a warning. Still, python crashes=
+ when trying to print the name of a null character. It wouldn&#39;t surpris=
+e me if there are other weird issues lurking. Would definitely sleep better=
+ with a more restricted character set.</p>
+
+<div class=3D"gmail_quote">On 17 Jul 2014 00:04, &quot;Andreas Schildbach&q=
+uot; &lt;<a href=3D"mailto:andreas@schildbach.de">andreas@schildbach.de</a>=
+&gt; wrote:<br type=3D"attribution"><blockquote class=3D"gmail_quote" style=
+=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+Please excuse me. I had a more thorough look at the original problem and<br=
+>
+found that the only problem with the original test case was that you<br>
+cannot specify codepoints from the SMP using \u in Java. I always tried<br>
+\u010400 but that doesn&#39;t work.<br>
+<br>
+Here is a fix for bitcoinj. The test now passes.<br>
+<br>
+<a href=3D"https://github.com/bitcoinj/bitcoinj/pull/143" target=3D"_blank"=
+>https://github.com/bitcoinj/bitcoinj/pull/143</a><br>
+<br>
+We can (and probably should) still need to filter control chars, I&#39;ll<b=
+r>
+have a look at that now again.<br>
+<br>
+<br>
+On 07/16/2014 11:06 PM, Aaron Voisine wrote:<br>
+&gt; If I first remove \u0000, so the non-normalized passphrase is<br>
+&gt; &quot;\u03D2\u0301\U00010400\U0001F4A9&quot;, and then NFC normalize i=
+t, it<br>
+&gt; becomes &quot;\u03D3\U00010400\U0001F4A9&quot;<br>
+&gt;<br>
+&gt; UTF-8 encoded this is: 0xcf93f0909080f09f92a9 (not the same as what<br=
+>
+&gt; you got, Andreas!)<br>
+&gt;<br>
+&gt; Encoding private key: 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTM=
+SZ4<br>
+&gt; with this passphrase, I get a BIP38 key of:<br>
+&gt; 6PRW5o9FMb4hAYRQPmgcvVDTyDtr6R17VMXGLmvKjKVpGkYhBJ4uYuR9wZ<br>
+&gt;<br>
+&gt; I recommend rather than simply removing control characters from the<br=
+>
+&gt; password that instead the spec require that passwords containing<br>
+&gt; control characters are invalid. We don&#39;t want people trying to be<=
+br>
+&gt; clever and putting them in thinking they are adding to the password<br=
+>
+&gt; entropy.<br>
+&gt;<br>
+&gt; Also for UI compatibility across many platforms, I&#39;m also in favor=
+<br>
+&gt; disallowing any character below U+0020 (space)<br>
+&gt;<br>
+&gt; I can submit a PR once we figure out why Andreas&#39;s passphrase was<=
+br>
+&gt; different than what I got.<br>
+&gt;<br>
+&gt; Aaron Voisine<br>
+&gt; <a href=3D"http://breadwallet.com" target=3D"_blank">breadwallet.com</=
+a><br>
+&gt;<br>
+&gt;<br>
+&gt; On Wed, Jul 16, 2014 at 4:04 AM, Andreas Schildbach<br>
+&gt; &lt;<a href=3D"mailto:andreas@schildbach.de">andreas@schildbach.de</a>=
+&gt; wrote:<br>
+&gt;&gt; Damn, I just realized that I implement only the decoding side of B=
+IP38.<br>
+&gt;&gt; So I cannot propose a complete test vector. Here is what I have:<b=
+r>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt; Passphrase: =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 (\u03D2\=
+u0301\u0000\U00010400\U0001F4A9; GREEK<br>
+&gt;&gt; UPSILON WITH HOOK, COMBINING ACUTE ACCENT, NULL, DESERET CAPITAL L=
+ETTER<br>
+&gt;&gt; LONG I, PILE OF POO)<br>
+&gt;&gt;<br>
+&gt;&gt; Passphrase bytes after removing ISO control characters and NFC<br>
+&gt;&gt; normalization: 0xcf933034303066346139<br>
+&gt;&gt;<br>
+&gt;&gt; Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF<br>
+&gt;&gt;<br>
+&gt;&gt; Unencrypted private key (WIF):<br>
+&gt;&gt; 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4<br>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt; Can someone calculate the encrypted key from it (using whatever<br=
+>
+&gt;&gt; implementation) and I will verify it decodes properly in bitcoinj?=
+<br>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt; On 07/16/2014 12:46 PM, Andreas Schildbach wrote:<br>
+&gt;&gt;&gt; I will change the bitcoinj implementation and propose a new te=
+st vector.<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt; On 07/16/2014 11:29 AM, Mike Hearn wrote:<br>
+&gt;&gt;&gt;&gt; Yes sorry, you&#39;re right, the issue starts with the nul=
+l code point.<br>
+&gt;&gt;&gt;&gt; Python seems to have problems starting there too. It might=
+ work if we<br>
+&gt;&gt;&gt;&gt; took that out.<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach<br>
+&gt;&gt;&gt;&gt; &lt;<a href=3D"mailto:andreas@schildbach.de">andreas@schil=
+dbach.de</a> &lt;mailto:<a href=3D"mailto:andreas@schildbach.de">andreas@sc=
+hildbach.de</a>&gt;&gt; wrote:<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Guys, you are always talking about the Unico=
+de astral plane, but in fact<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 its a plain old (ASCII) control character wh=
+ere this problem starts and<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 likely ends: \u0000.<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Let&#39;s ban/filter ISO control characters =
+and be done with it. Most<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 control characters will never be enterable b=
+y any keyboard into a<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 password field. Of course I assume that Char=
+acter.isISOControl() works<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 consistently across platforms.<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 <a href=3D"http://docs.oracle.com/javase/7/d=
+ocs/api/java/lang/Character.html#isISOControl%28char%29" target=3D"_blank">=
+http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#isISOCont=
+rol%28char%29</a><br>
+
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 On 07/16/2014 12:23 AM, Aaron Voisine wrote:=
+<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; If the user creates a password on an iO=
+S device with an astral<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; character and then can&#39;t enter that=
+ password on a JVM wallet, that<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; sucks. If JVMs really can&#39;t support=
+ unicode NFC then that&#39;s a strong<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; case to limit the spec to the subset of=
+ unicode that all popular<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; platforms can support, but it sounds li=
+ke it might just be a JVM<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; string library bug that could hopefully=
+ be reported and fixed. I get<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; the same result as in the test case usi=
+ng apple&#39;s<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; CFStringNormalize(passphrase, kCFString=
+NormalizationFormC);<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; Aaron Voisine<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; <a href=3D"http://breadwallet.com" targ=
+et=3D"_blank">breadwallet.com</a> &lt;<a href=3D"http://breadwallet.com" ta=
+rget=3D"_blank">http://breadwallet.com</a>&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; On Tue, Jul 15, 2014 at 11:20 AM, Mike =
+Hearn &lt;<a href=3D"mailto:mike@plan99.net">mike@plan99.net</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &lt;mailto:<a href=3D"mailto:mike@plan99.net=
+">mike@plan99.net</a>&gt;&gt; wrote:<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; Yes, we know, Andreas&#39; code is =
+indeed doing normalisation.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; However it appears the output bytes=
+ end up being different. What<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 I get back<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; is:<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; cf930001303430300166346139<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; vs<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; cf9300f0909080f09f92a9<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; from the spec.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; I&#39;m not sure why. It appears th=
+is is due to the character from<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 the astral<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; planes. Java is old and uses 16 bit=
+ characters internally - it<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 wouldn&#39;t<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; surprise me if there&#39;s some wei=
+rdness that means it doesn&#39;t/won&#39;t<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 support<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; this kind of thing.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; I recommend instead that any implem=
+entation that wishes to be<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 compatible<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; with JVM based wallets (I suspect A=
+ndroid is the same) just<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 refuse any<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; passphrase that includes characters=
+ outside the BMP. At least<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 unless someone<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; can find a fix. I somehow doubt thi=
+s will really hurt anyone.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 --------------------------------------------=
+----------------------------------<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; Want fast and easy access to all th=
+e code in your enterprise?<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Index and<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; search up to 200,000 lines of code =
+with a free copy of Black Duck<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; Code Sight - the same software that=
+ powers the world&#39;s largest code<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; search on Ohloh, the Black Duck Ope=
+n Hub! Try it now.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; <a href=3D"http://p.sf.net/sfu/bds"=
+ target=3D"_blank">http://p.sf.net/sfu/bds</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; ___________________________________=
+____________<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; Bitcoin-development mailing list<br=
+>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; <a href=3D"mailto:Bitcoin-developme=
+nt@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &lt;mailto:<a href=3D"mailto:Bitcoin-develop=
+ment@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a>&g=
+t;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt; <a href=3D"https://lists.sourceforg=
+e.net/lists/listinfo/bitcoin-development" target=3D"_blank">https://lists.s=
+ourceforge.net/lists/listinfo/bitcoin-development</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 --------------------------------------------=
+----------------------------------<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; Want fast and easy access to all the co=
+de in your enterprise?<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Index and<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; search up to 200,000 lines of code with=
+ a free copy of Black Duck<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; Code Sight - the same software that pow=
+ers the world&#39;s largest code<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; search on Ohloh, the Black Duck Open Hu=
+b! Try it now.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt; <a href=3D"http://p.sf.net/sfu/bds" tar=
+get=3D"_blank">http://p.sf.net/sfu/bds</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 --------------------------------------------=
+----------------------------------<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Want fast and easy access to all the code in=
+ your enterprise? Index and<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 search up to 200,000 lines of code with a fr=
+ee copy of Black Duck<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Code Sight - the same software that powers t=
+he world&#39;s largest code<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 search on Ohloh, the Black Duck Open Hub! Tr=
+y it now.<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 <a href=3D"http://p.sf.net/sfu/bds" target=
+=3D"_blank">http://p.sf.net/sfu/bds</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 ____________________________________________=
+___<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 Bitcoin-development mailing list<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 <a href=3D"mailto:Bitcoin-development@lists.=
+sourceforge.net">Bitcoin-development@lists.sourceforge.net</a><br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 &lt;mailto:<a href=3D"mailto:Bitcoin-develop=
+ment@lists.sourceforge.net">Bitcoin-development@lists.sourceforge.net</a>&g=
+t;<br>
+&gt;&gt;&gt;&gt; =C2=A0 =C2=A0 <a href=3D"https://lists.sourceforge.net/lis=
+ts/listinfo/bitcoin-development" target=3D"_blank">https://lists.sourceforg=
+e.net/lists/listinfo/bitcoin-development</a><br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; ----------------------------------------------------------=
+--------------------<br>
+&gt;&gt;&gt;&gt; Want fast and easy access to all the code in your enterpri=
+se? Index and<br>
+&gt;&gt;&gt;&gt; search up to 200,000 lines of code with a free copy of Bla=
+ck Duck<br>
+&gt;&gt;&gt;&gt; Code Sight - the same software that powers the world&#39;s=
+ largest code<br>
+&gt;&gt;&gt;&gt; search on Ohloh, the Black Duck Open Hub! Try it now.<br>
+&gt;&gt;&gt;&gt; <a href=3D"http://p.sf.net/sfu/bds" target=3D"_blank">http=
+://p.sf.net/sfu/bds</a><br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;&gt; _______________________________________________<br>
+&gt;&gt;&gt;&gt; Bitcoin-development mailing list<br>
+&gt;&gt;&gt;&gt; <a href=3D"mailto:Bitcoin-development@lists.sourceforge.ne=
+t">Bitcoin-development@lists.sourceforge.net</a><br>
+&gt;&gt;&gt;&gt; <a href=3D"https://lists.sourceforge.net/lists/listinfo/bi=
+tcoin-development" target=3D"_blank">https://lists.sourceforge.net/lists/li=
+stinfo/bitcoin-development</a><br>
+&gt;&gt;&gt;&gt;<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt;<br>
+&gt;&gt;&gt; --------------------------------------------------------------=
+----------------<br>
+&gt;&gt;&gt; Want fast and easy access to all the code in your enterprise? =
+Index and<br>
+&gt;&gt;&gt; search up to 200,000 lines of code with a free copy of Black D=
+uck<br>
+&gt;&gt;&gt; Code Sight - the same software that powers the world&#39;s lar=
+gest code<br>
+&gt;&gt;&gt; search on Ohloh, the Black Duck Open Hub! Try it now.<br>
+&gt;&gt;&gt; <a href=3D"http://p.sf.net/sfu/bds" target=3D"_blank">http://p=
+.sf.net/sfu/bds</a><br>
+&gt;&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt;<br>
+&gt;&gt; ------------------------------------------------------------------=
+------------<br>
+&gt;&gt; Want fast and easy access to all the code in your enterprise? Inde=
+x and<br>
+&gt;&gt; search up to 200,000 lines of code with a free copy of Black Duck<=
+br>
+&gt;&gt; Code Sight - the same software that powers the world&#39;s largest=
+ code<br>
+&gt;&gt; search on Ohloh, the Black Duck Open Hub! Try it now.<br>
+&gt;&gt; <a href=3D"http://p.sf.net/sfu/bds" target=3D"_blank">http://p.sf.=
+net/sfu/bds</a><br>
+&gt;&gt; _______________________________________________<br>
+&gt;&gt; Bitcoin-development mailing list<br>
+&gt;&gt; <a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitco=
+in-development@lists.sourceforge.net</a><br>
+&gt;&gt; <a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
+velopment" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/b=
+itcoin-development</a><br>
+&gt;<br>
+&gt; ----------------------------------------------------------------------=
+--------<br>
+&gt; Want fast and easy access to all the code in your enterprise? Index an=
+d<br>
+&gt; search up to 200,000 lines of code with a free copy of Black Duck<br>
+&gt; Code Sight - the same software that powers the world&#39;s largest cod=
+e<br>
+&gt; search on Ohloh, the Black Duck Open Hub! Try it now.<br>
+&gt; <a href=3D"http://p.sf.net/sfu/bds" target=3D"_blank">http://p.sf.net/=
+sfu/bds</a><br>
+&gt; _______________________________________________<br>
+&gt; Bitcoin-development mailing list<br>
+&gt; <a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-d=
+evelopment@lists.sourceforge.net</a><br>
+&gt; <a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-develo=
+pment" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitco=
+in-development</a><br>
+&gt;<br>
+<br>
+<br>
+<br>
+---------------------------------------------------------------------------=
+---<br>
+Want fast and easy access to all the code in your enterprise? Index and<br>
+search up to 200,000 lines of code with a free copy of Black Duck<br>
+Code Sight - the same software that powers the world&#39;s largest code<br>
+search on Ohloh, the Black Duck Open Hub! Try it now.<br>
+<a href=3D"http://p.sf.net/sfu/bds" target=3D"_blank">http://p.sf.net/sfu/b=
+ds</a><br>
+_______________________________________________<br>
+Bitcoin-development mailing list<br>
+<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
+pment@lists.sourceforge.net</a><br>
+<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
+" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
+velopment</a><br>
+</blockquote></div>
+
+--047d7b41cc74c4831904fe618858--
+
+