MIME-Version: 1.0
From: Bram Cohen <bram@bittorrent.com>
Date: Tue, 21 Feb 2017 14:00:23 -0800
Message-ID: <CA+KqGkpVLfGjQUJEYdRppqvN263rHrY-rGL2k22eMryQbb6Q8g@mail.gmail.com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a11440182c1b425054911818d
Subject: [bitcoin-dev] Proposal for utxo commitment format
Precedence: list

--001a11440182c1b425054911818d
Content-Type: text/plain; charset=UTF-8

Here is a Merkle set data structure, whose format may be useful for utxo
commitments in Bitcoin blocks. It may also be useful for any other
distributed computation which wants an audit trail:

https://github.com/bramcohen/MerkleSet

This is a fairly straightforward Patricia Trie, with a simple format and a
simple reference implementation plus a performance optimized non-reference
implementation which is much more cache coherent. It will need to be ported
to C and be properly turned before the potential performance gains can be
realized though.

The clever things which affect the format spec are:

It uses blake2s as the internal hash function. This is the fastest hash
function to use on 512 bit inputs because blake2b uses a 1024 bit block
size. It might make sense to use a hypothetical variant of blake which is
optimized for 64 bits with a 512 bit block size, but that hasn't been
specified. Sha256 would take advantage of hardware acceleration, but that
isn't available everywhere.

Two bits of security are sacrificed to include metadata inline which halves
the CPU cost of hashing.

When one side of a node is empty and the other contains exactly two things
the secure hash of the child is adopted verbatim rather than rehashing it.
This roughly halves the amount of hashing done, and makes it more resistant
to malicious data, and cleans up some implementation details, at the cost
of some extra complexity.

--001a11440182c1b425054911818d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Here is a Merkle set data structure, whose format may be u=
seful for utxo commitments in Bitcoin blocks. It may also be useful for any=
 other distributed computation which wants an audit trail:<div><br></div><d=
iv><a href=3D"https://github.com/bramcohen/MerkleSet">https://github.com/br=
amcohen/MerkleSet</a><br></div><div><br></div><div>This is a fairly straigh=
tforward Patricia Trie, with a simple format and a simple reference impleme=
ntation plus a performance optimized non-reference implementation which is =
much more cache coherent. It will need to be ported to C and be properly tu=
rned before the potential performance gains can be realized though.=C2=A0</=
div><div><br></div><div>The clever things which affect the format spec are:=
</div><div><br></div><div>It uses blake2s as the internal hash function. Th=
is is the fastest hash function to use on 512 bit inputs because blake2b us=
es a 1024 bit block size. It might make sense to use a hypothetical variant=
 of blake which is optimized for 64 bits with a 512 bit block size, but tha=
t hasn&#39;t been specified. Sha256 would take advantage of hardware accele=
ration, but that isn&#39;t available everywhere.</div><div><br></div><div>T=
wo bits of security are sacrificed to include metadata inline which halves =
the CPU cost of hashing.</div><div><br></div><div>When one side of a node i=
s empty and the other contains exactly two things the secure hash of the ch=
ild is adopted verbatim rather than rehashing it. This roughly halves the a=
mount of hashing done, and makes it more resistant to malicious data, and c=
leans up some implementation details, at the cost of some extra complexity.=
</div></div>

--001a11440182c1b425054911818d--