MIME-Version: 1.0
Date: Sun, 6 Mar 2016 15:04:32 -0500
Message-ID: <CADL_X_cFOknYejoQPRarwRxpFz5D8ffqfohh799NR7e2_Jgx4w@mail.gmail.com>
From: Jameson Lopp <jameson.lopp@gmail.com>
To: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=089e0112c0224a0185052d66db27
Subject: [bitcoin-dev] BIP44 & BIP32 chain address look-ahead limits
Precedence: list

--089e0112c0224a0185052d66db27
Content-Type: text/plain; charset=UTF-8

I recently ran into an issue while importing a Mycelium HD wallet where it
was not finding all of my funds - upon further investigation with Mycelium
devs we realized that the wallet was following the BIP44 spec correctly,
but BIP44 may have a flaw.

The problem was a result of my creating 16 transactions in Mycelium in a
fairly short timeframe, but the first 15 transactions ended up never
confirming while the 16th was confirmed. As a result, when I later
reimported the account from the master seed, the chain derivation stopped
upon hitting this large gap of unused addresses on the internal / change
chain.

BIP44 recommends that there need not be a lookahead on internal chains
"because internal chains receive only coins that come from the associated
external chains."
https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#Address_gap_limit

BIP32 also notes that "the look-ahead for internal chains can be very
small, as no gaps are to be expected here."
https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#Full_wallet_sharing_m

It seems to me that there /is/ an edge case that can result in significant
gaps in internal chain address usage and as such, the recommendation should
be to look ahead on both external and internal chains when performing
account discovery. On a related note, the recommended look-ahead of 20 may
not be safe enough - perhaps it should be raised to 100 if not higher.

In addition to recommending a larger look-ahead, it may also be advisable
for BIP44 to recommend that wallets "fill in" gaps of unused chain
addresses by "looking back" from the current tip of the internal chain's
index when the wallet decides to create a new change address. This could
help mitigate the size of gaps caused by failed transactions.

- Jameson

--089e0112c0224a0185052d66db27
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div>I recently ran into an issue while imp=
orting a Mycelium HD wallet where it was not finding all of my funds - upon=
 further investigation with Mycelium devs we realized that the wallet was f=
ollowing the BIP44 spec correctly, but BIP44 may have a flaw.<br><br></div>=
The problem was a result of my creating 16 transactions in Mycelium in a fa=
irly short timeframe, but the first 15 transactions ended up never confirmi=
ng while the 16th was confirmed. As a result, when I later reimported the a=
ccount from the master seed, the chain derivation stopped upon hitting this=
 large gap of unused addresses on the internal / change chain.<br><br></div=
>BIP44 recommends that there need not be a lookahead on internal chains &qu=
ot;because
internal chains receive only coins that come from the associated external c=
hains.&quot; <a href=3D"https://github.com/bitcoin/bips/blob/master/bip-004=
4.mediawiki#Address_gap_limit">https://github.com/bitcoin/bips/blob/master/=
bip-0044.mediawiki#Address_gap_limit</a><br><br></div>BIP32 also notes that=
 &quot;the look-ahead for internal chains can be very small, as no gaps are=
 to be expected here.&quot; <a href=3D"https://github.com/bitcoin/bips/blob=
/master/bip-0032.mediawiki#Full_wallet_sharing_m">https://github.com/bitcoi=
n/bips/blob/master/bip-0032.mediawiki#Full_wallet_sharing_m</a><br><br></di=
v><div>It seems to me that there /is/ an edge case that can result in signi=
ficant gaps in internal chain address usage and as such, the recommendation=
 should be to look ahead on both external and internal chains when performi=
ng account discovery. On a related note, the recommended look-ahead of 20 m=
ay not be safe enough - perhaps it should be raised to 100 if not higher.<b=
r><br></div><div>In addition to recommending a larger look-ahead, it may al=
so be advisable for BIP44 to recommend that wallets &quot;fill in&quot; gap=
s of unused chain addresses by &quot;looking back&quot; from the current ti=
p of the internal chain&#39;s index when the wallet decides to create a new=
 change address. This could help mitigate the size of gaps caused by failed=
 transactions.<br></div><div><br></div>- Jameson<br></div>

--089e0112c0224a0185052d66db27--