Date: Fri, 21 Aug 2015 15:06:16 -0700
From: Peter Todd <pete@petertodd.org>
To: Matt Corallo <lf-lists@mattcorallo.com>
Message-ID: <20150821220603.GC7450@muck>
References: <55D6AD19.10305@mattcorallo.com> <20150821053819.GA18176@muck>
	<20150821054219.GB18176@muck> <55D7662E.4090104@mattcorallo.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="EgVrEAR5UttbsTXg"
Content-Disposition: inline
In-Reply-To: <55D7662E.4090104@mattcorallo.com>
Cc: bitcoin-dev@lists.linuxfoundation.org, Mike Hearn <mike@plan99.net>
Subject: Re: [bitcoin-dev] Revisiting NODE_BLOOM: Proposed BIP
Precedence: list


--EgVrEAR5UttbsTXg
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Aug 21, 2015 at 05:55:58PM +0000, Matt Corallo wrote:
> Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
> sure we want to encourage more people aside from bitcoinj to use
> that...I thought about adding a DNS seed section to this bip, but
> decided against it...still, I think we should add the option to select
> service bits to DNS seeds ASAP.

Well, in general relying on seeds every time you start your node is a
really bad idea; doing so much be carefully weighed against the
downsides and should be used only as a last resort. Nodes should be
doing caching and proper gossip protocol participation whenever
possible. (note how bitcoinj nodes *do* rely on centralized servers,
implemented with an unauthenticated, unencrypted, protocol - the worst
of all possible solutions with many possible MITM vectors and privacy
security holes)

To that end, I'd be inclined to leave the DNS seed protocol as it is and
let others solve the centralized server use-case, for which Cartographer
isn't all that bad of a load balancing mechanism. Also as gmaxwell noted
on IRC, adding flag bits does have privacy implications.

> Re: need to "shard" the blockchain: not sure what you're referring to
> here. The bloom filter stuff requires you to download the chain
> in-order, sure, but you have to do that for headers anyway, and
> hopefully your total data isnt too much more than headers alone.

Any protocol change that would split blocks themselves into multiples.
Not an easy problem to solve, but given the inherent O(n^2) scaling of
global consensus blockchains, it's the only kind of solution that could
in the future make the blockchain itself have reasonable scalability.

> Anyone have the best reference for the DoS issues?

Well actually, we can reference the DoS attacks that Bitcoin XT nodes
are undergoing right now - part of the attack is repeated Bloom filter
requests to soak up disk IO bandwidth. I've CC'd Gavin and Mike - as far
as I know they haven't published details of those attacks - a write-up
would be very helpful.

While so far those are being directed only at XT nodes, obviously this
is a potential issue for Core nodes as well. Like I mentioned last time
around, it's critical that miners aren't affected by these attacks -
nodes simply serving SPV wallet clients are much less latency sensitive,
so a good DoS attack mitigation strategy would be to have the two
classes of nodes out there "in the wild"

> BIP: ?
> Title: NODE_BLOOM service bit
> Author: Matt Corallo <bip@bluematt.me>, Peter Todd <pete@petertodd.org>
> Type: Standards Track (draft)
> Created: 20-08-2015
>=20
> Abstract
> =3D=3D=3D=3D=3D=3D=3D=3D
>=20
> This BIP extends BIP 37, Connection Bloom filtering, by defining a
> service bit to allow peers to advertise that they support bloom filters
> explicitly. It also bumps the protocol version to allow peers to
> identify old nodes which allow bloom filtering of the connection despite
> lacking the new service bit.
>=20
>=20
> Motivation
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> BIP 37 did not specify a service bit for the bloom filter service, thus
> implicitly assuming that all nodes that serve peers data support it.
> However, the connection filtering algorithm proposed in BIP 37, and
> implemented in several clients today, has been shown to provide little
> to no privacy[1], as well as being a large DoS risk on some nodes[2].
> Thus, allowing node operators to disable connection bloom filtering is a
> much-needed feature.
>=20
>=20
> Specification
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> The following protocol bit is added:
>=20
>     NODE_BLOOM =3D (1 << 2)
>=20
> Nodes which support bloom filters should set that protocol bit.
> Otherwise it should remain unset. In addition the protocol version is
> increased from 70002 to 70011 in the reference implementation. It is
> often the case that nodes which have a protocol version smaller than
> 70011, but larger than 70000 support bloom filtered connections without
> the NODE_BLOOM bit set, however clients which require bloom filtered
> connections should avoid making this assumption.
>=20
> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
> which, nonetheless, provide filtered access to the data which they do hav=
e).
>=20
> If a node does not support bloom filters but receives a "filterload",
> "filteradd", or "filterclear" message from a peer the node should
> disconnect that peer immediately. For backwards compatibility, in
> initial implementations, nodes may choose to only disconnect nodes which
> have the new protocol version set and attempt to send a filter command.
>=20
> While outside the scope of this BIP it is suggested that DNS seeds and
> other peer discovery mechanisms support the ability to specify the
> services required; current implementations simply check only that
> NODE_NETWORK is set.
>=20
>=20
> Design rational
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> A service bit was chosen as applying a bloom filter is a service.
>=20
> The increase in protocol version is for backwards compatibility. In
> initial implementations, old nodes which are not yet aware of NODE_BLOOM
> and use a protocol version < 70011 may still send filter* messages to a
> node without NODE_BLOOM. This feature may be removed after there are
> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
> allowing node operators to fully close the bloom-related DoS vectors.
>=20
>=20
> Reference Implementation
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> https://github.com/bitcoin/bitcoin/pull/6579
>=20
>=20
> Copyright
> =3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> This document is placed in the public domain.
>=20
>=20
> References
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> [1] http://eprint.iacr.org/2014/763
> [2] ???? is one example where the issues were found, though others
> independently discovered issues as well. Sample DoS exploit code
> available at https://github.com/petertodd/bloom-io-attack.

--=20
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

--EgVrEAR5UttbsTXg
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----

iQGrBAEBCACVBQJV16DVXhSAAAAAABUAQGJsb2NraGFzaEBiaXRjb2luLm9yZzAw
MDAwMDAwMDAwMDAwMDAwNDAyZmU2ZmI5YWQ2MTNjOTNlMTJiZGRmYzZlYzAyYTJi
ZDkyZjAwMjA1MDU5NGQvFIAAAAAAFQARcGthLWFkZHJlc3NAZ251cGcub3JncGV0
ZUBwZXRlcnRvZC5vcmcACgkQwIXyHOf0udz8yAgAjBLUkgwM6t7vh4c/mGoEk7Fa
aOotVdpm8WcUkw/6DN/Tlt1A917zdXn/1H45UkEeY7eGVXhZowYBEDeN3ZMtov+V
zBrmuyA2enPj0auOoZMj5fwF7IQKWut1jwXrhPbN/47oc1d7hjqlkQvBfEqHc2Jc
8Xed4+bjtcClZGOfhQW6M8cdA2OnolUi+hKu7cZj9rX+zSnvf9dO8KdP3mP63CBl
OwSJ7sdBNy/2s7vAP4LumR/IVXu2NIaha5DdKsE1+VEGq3fDq4tEHhZX1N2DPT5D
p0cNQ0l+MLiGpetqb5k4GV3MfG10Ut+GQWzstiW9BupCnMu/ICLuxJo0dxQJrg==
=q7Zq
-----END PGP SIGNATURE-----

--EgVrEAR5UttbsTXg--