MIME-version: 1.0 (Mac OS X Mail 8.2 \(2098\))
Content-type: multipart/signed;
	boundary="Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9";
	protocol="application/pgp-signature"; micalg=pgp-sha512
From: Jean-Paul Kogelman <jeanpaulkogelman@me.com>
In-reply-to: <20150702071814.GA1904@banane.informatik.uni-ulm.de>
Date: Thu, 02 Jul 2015 01:33:08 -0700
Message-id: <7A4D6052-CAA8-4E22-92E4-83828E589570@me.com>
References: <F6C7E867-1CCA-4DFB-8A88-361316A76FC3@me.com>
	<20150702071814.GA1904@banane.informatik.uni-ulm.de>
To: Henning Kopp <henning.kopp@uni-ulm.de>
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] Defining a min spec
Precedence: list


--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

In the case of Bitcoin Core, for a starting point, you basically have to =
work backwards from what we have right now. We know some of the bounds =
already. Block size already tells you a lot about your bandwidth =
requirements, and Pieter=E2=80=99s simulations gives you even more =
information when you take orphan rates into account. There=E2=80=99s =
also a hard cap on the number of SigOps if I recall correctly, so =
that=E2=80=99s probably a good starting point for a MIPS metric, etc.

Memory is probably the hardest to pin down since some memory structures =
like (from what I understand, correct me if I=E2=80=99m wrong) the UTXO =
database live fully in memory and are basically unbounded. Perhaps this =
can somehow be capped at a certain size and move all the really old =
UTXO=E2=80=99s that are unlikely to move to disk and just take the CPU / =
disk hit in the unlikely event that they are referenced by a new block. =
Has the address database been capped yet? Mempool? I realize that it=E2=80=
=99s probably debatable whether or not this behaviour should be =
independent of available memory since any bugs here could affect =
consensus (especially the UTXO db).

Ultimately, what comes out of it is a list of numbers. A Mbit network =
I/O, B MIPS, C MB memory, D Disk space, etc. At that point you can =
debate whether or not such a machine can be considered an entrypoint =
bitcoin full node. You round up the numbers that are not really =
available anymore in off the shelf hardware (like disk space) and you =
round down the numbers that seem too high. For all we know we=E2=80=99re =
already over budget on some of the metrics that we decide to track as =
min spec. Network I/O for example. At that point you can start focussed =
research into bringing Bitcoin Core back into budget on those metrics. =
Then the discussion moves from =E2=80=9Cit=E2=80=99s probably too =
high=E2=80=9D to =E2=80=9Cwe=E2=80=99re X% over budget=E2=80=9D.

The most valuable thing that could come out of this is to get some kind =
of formulation how all the different levers in Bitcoin Core affect the =
min spec and ideally have a benchmark tool. For example, we could settle =
on a min spec that would exclude the Raspberry Pi 1 on MIPS, but when =
secp256k1 is enabled for validation, the MIPS requirement could drop =
significantly, allowing us to adjust the min spec downward to include =
the Raspberry Pi 1 again (again, just an example).

Ideally some people would have the actual min spec machine built and =
running. The cost of that shouldn=E2=80=99t be too high (it=E2=80=99s =
the min spec after all) and I=E2=80=99m sure people would be happy to =
chip in a couple bits for this.

Remember, the min spec should be able to handle Bitcoin Core running =
under full load; that=E2=80=99s maxed out blocks with maxed out SigOps, =
etc.


jp


> On Jul 2, 2015, at 12:18 AM, Henning Kopp <henning.kopp@uni-ulm.de> =
wrote:
>=20
> Hi Jean-Paul,
>=20
> that's a very interesting point of view and I have never thought about
> it this way, since I have a totally different background.
>=20
> How would you go on about defining a min spec? Is this done by testing
> the software on different hardware configurations or are you looking
> at the requirements a priori?
>=20
> Best regards
> Henning
>=20
>=20
> On Wed, Jul 01, 2015 at 09:04:19PM -0700, Jean-Paul Kogelman wrote:
>> Hi folks,
>>=20
>> I=E2=80=99m a game developer. I write time critical code for a living =
and have to deal with memory, CPU, GPU and I/O budgets on a daily basis. =
These budgets are based on what we call a minimum specification (of =
hardware); min spec for short. In most cases the min spec is based on =
entry model machines that are available during launch, and will give the =
user an enjoyable experience when playing our games. Obviously, we can =
turn on a number of bells and whistles for people with faster machines, =
but that=E2=80=99s not the point of this mail.
>>=20
>> The point is, can we define a min spec for Bitcoin Core? The number =
one reason for this is: if you know how your changes affect your =
available budgets, then the risk of breaking something due to capacity =
problems is reduced to practically zero.
>>=20
>> One way of doing so is to work backwards from what we have right now: =
Block size (network / disk I/O), SigOps/block (CPU), UTXO size (memory), =
etc. Then there=E2=80=99s Pieter=E2=80=99s analysis of network =
bottlenecks and how it affects orphan rates that could be used to set =
some form of cap on what transfer time + verification time should be to =
keep the orphan rate at an acceptable level.
>>=20
>> So taking all of the above (and more) into account, what =
configuration would be the bare minimum to comfortably run Bitcoin Core =
at maximum load and can it be reasonably expected to still be out there =
in the field running Bitcoin Core? Also, can the parameters that were =
used to determine this min spec be codified in some way so that they can =
later be used if Bitcoin Core is optimized (or extended with new =
functionality) and see how it affects the min spec? Basically, with any =
reasonably big change, one of the first questions could be: =E2=80=9CHow =
does this change affect min spec?"
>>=20
>> For example, currently OpenSSL is used to verify the signatures in =
the transactions. The new secp256k1 implementation is several times =
faster than (depending on CPU architecture, I=E2=80=99m sure) =
OpenSSL=E2=80=99s implementation. So it would result in faster =
verification time. This can then result in the following things; either =
network I/O and CPU requirements are adjusted downward in the min spec =
(you can run the new Bitcoin Core on a cheaper configuration), or other =
parameters can be adjusted upwards (number of SigOps / transaction, =
block size?), through proper rollout obviously. Since we know how min =
spec is affected by these changes, they should be non-controversial by =
default. Nobody running min spec is going to be affected by it, etc.
>>=20
>> Every change that has a positive effect on min spec (do more on the =
same hardware) basically pushes the need to start following any of the =
curve laws (Nielsen, Moore) forward. No need for miners / node operators =
to upgrade.
>>=20
>> Once we hit what we call SOL (Speed Of Light, the fastest something =
can go on a specific platform) it=E2=80=99s time to start looking at =
periodically adjusting min spec upwards, or by that time maybe it=E2=80=99=
s possible to use conservative plots of the curve laws as a basis.
>>=20
>> Lastly, a benchmark test could be developed that can tell everyone =
running Bitcoin Core how their setup compares to the min spec and how =
long they can expect to run on this setup.
>>=20
>> What do you guys think?
>>=20
>>=20
>> jp
>=20
>=20
>=20
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>=20
>=20
> --
> Henning Kopp
> Institute of Distributed Systems
> Ulm University, Germany
>=20
> Office: O27 - 3402
> Phone: +49 731 50-24138
> Web: http://www.uni-ulm.de/in/vs/~kopp


--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCgAGBQJVlPdEAAoJEG93Vo4Z7tpFmYgQAMYaSUrped38/ASQnjME/jG7
ojcykAFIAArlHKh4qLhka7JnsZ47i3dYrzEZN0IOUGyOANnaa7zddsfDo29uT89O
kk3t/LSE04Bdk/e2J18cwnqsp8yfdhQyJ+XyOLzwVtaSUuwm5SVZd0RwkScB+Crq
bm8muzut/2IDD+RTCmuXnBBLSp1Hss1tnDuhv1sQh9w1UAfh278Z6XHtLu/Qgm6y
45DTWi8btvST7I68CGY5ftYOQOxNGGLz7ZLRSVwrxBDwGtU0NS4p4OUu/L5Vt/lZ
5f2lDFUOCsyuXVBfmm8eozMECTCm1aCneEwOKt9k0B4q6hv8YYj154qFzPn5mVYp
d5SEHO7riSKNFYcqC/pWYu7rFInTxw2BRWrs1gjDA7AlwFHjd+GTTkX5H/fLMAe1
atyI+4xFVbCAE4HI9K0yAXi9oJ9uo/tC8qaXSSfpOTWhpXG+iACKpA6BHJJg7Q7s
eeu5gNKNSF/AWBYykcuHwBlI9dVOiDtDifah8XpkF7jeqwMN0sFnZdBqQxuSGrFy
tihcJ2yC3e8kciLJ6ls9Y1rZbobi/7RBTATKK7VwatIcR+Wv2FxULJumKwtENYcu
kdZj5njc8gdMpcditWT0h/cpX6kHCbA+E4uqu+LVDCV2QLaIbalQo5ZHjOqC9EP5
D6Uzafx1EHzHdTxOW/9E
=9JCZ
-----END PGP SIGNATURE-----

--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9--