Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Eric Voskuil <eric@voskuil.org>
In-Reply-To: <CAFVRnyr-Z9YWtT3r+7-fGejzgxKhH3-kQuo8JQFqKDpyZNBBdg@mail.gmail.com>
Date: Fri, 31 Mar 2017 11:58:44 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <F7248427-367E-4666-9171-5FD6898F4144@voskuil.org>
References: <CAEgR2PEG1UMqY0hzUH4YE_an=qOvQUgfXreSRsoMWfFWxG3Vqg@mail.gmail.com>
	<CAFVRnyq9Qgw88RZqenjQTDZHEWeuNCdh12Dq7wCGZdu9ZuEN9w@mail.gmail.com>
	<CAD1TkXvd4yLHZDAdMi78WwJ_siO1Vt7=DgYiBmP45ffVveuHBg@mail.gmail.com>
	<SINPR04MB1949AB581C6870184445E0C4C2340@SINPR04MB1949.apcprd04.prod.outlook.com>
	<CAD1TkXsj53JRYhqot2aHSQR+HEDKm7+6S5kGtaLYBCoc24PuWg@mail.gmail.com>
	<SINPR04MB1949A0AF3AD33B4664417068C2370@SINPR04MB1949.apcprd04.prod.outlook.com>
	<CAD1TkXtPZ7w+qYqr_hvyeq95aJ2ge1YYkoC1taDkzv1vEMKpog@mail.gmail.com>
	<SINPR04MB1949BE883C69CFF1477AFAEFC2370@SINPR04MB1949.apcprd04.prod.outlook.com>
	<CAD1TkXvXYX0f+jMMc41vhANuKfw-rNg9tUOG0bCS=T-YGYYjPw@mail.gmail.com>
	<CAFVRnyqSMVj2Ttc4_5vuk73Z5yRJdxeSodvkdjqsrHbgghcmUQ@mail.gmail.com>
	<CAD1TkXvmo8ygbFdPJxdiL5-QiN6+ujeSgOJ7_4eit43aZ2bzyg@mail.gmail.com>
	<CAFVRnyr-Z9YWtT3r+7-fGejzgxKhH3-kQuo8JQFqKDpyZNBBdg@mail.gmail.com>
To: David Vorick <david.vorick@gmail.com>,
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Hard fork proposal from last week's meeting
Precedence: list

As an independently verifiable, decentralized store of public information, t=
he Bitcoin block tree and transaction DAG do have an advantage over systems s=
uch as Visa. The store is just a cache. There is no need to implement reliab=
ility in storage or in communications. It is sufficient to be able to detect=
 invalidity. And even if a subset of nodes fail to do so, the system overall=
 compensates.

As such the architecture of a Bitcoin node and its supporting hardware requi=
rements are very different from an unverifiable, centralized store of privat=
e information. So in that sense the comparison below is not entirely fair. M=
any, if not most, of the high costs of a Visa datacenter do not apply becaus=
e of Bitcoin's information architecture.

However, if the system cannot remain decentralized these architectural advan=
tages will not hold. At that point your considerations below are entirely va=
lid. Once the information is centralized it necessarily becomes private and f=
ragile. Conversely, once it becomes private it necessarily becomes centraliz=
ed and fragile. This fragility requires significant investment by the centra=
l authority to maintain.

So as has been said, we can have decentralization and its benefit of trustle=
ssness or we can have Visa. We already have Visa. Making another is entirely=
 uninteresting.

e=20

> On Mar 31, 2017, at 11:23 AM, David Vorick via bitcoin-dev <bitcoin-dev@li=
sts.linuxfoundation.org> wrote:
>=20
> Sure, your math is pretty much entirely irrelevant because scaling systems=
 to massive sizes doesn't work that way.
>=20
> At 400B transactions per year we're looking at block sizes of 4.5 GB, and a=
 database size of petabytes. How much RAM do you need to process blocks like=
 that? Can you fit that much RAM into a single machine? Okay, you can't fit t=
hat much RAM into a single machine. So you have to rework the code to operat=
e on a computer cluster.
>=20
> Already we've hit a significant problem. You aren't going to rewrite Bitco=
in to do block validation on a computer cluster overnight. Further, are stor=
age costs consistent when we're talking about setting up clusters? Are bandw=
idth costs consistent when we're talking about setting up clusters? Are RAM a=
nd CPU costs consistent when we're talking about setting up clusters? No, th=
ey aren't. Clusters are a lot more expensive to set up per-resource because t=
hey need to talk to eachother and synchronize with eachother and you have a L=
OT more parts, so you have to build in redundancies that aren't necessary in=
 non-clusters.
>=20
> Also worth pointing out that peak transaction volumes are typically 20-50x=
 the size of typical transaction volumes. So your cluster isn't going to nee=
d to plan to handle 15k transactions per second, you're really looking at mo=
re like 200k or even 500k transactions per second to handle peak-volumes. An=
d if it can't, you're still going to see full blocks.
>=20
> You'd need a handful of experts just to maintain such a thing. Disks are g=
oing to be failing every day when you are storing multiple PB, so you can't j=
ust count a flat cost of $20/TB and expect that to work. You're going to nee=
d redundancy and tolerance so that you don't lose the system when a few of y=
our hard drives all fail within minutes of eachother. And you need a way to r=
ebuild everything without taking the system offline.
>=20
> This isn't even my area of expertise. I'm sure there are a dozen other sig=
nificant issues that one of the Visa architects could tell you about when de=
aling with mission-critical data at this scale.
>=20
> --------
>=20
> Massive systems operate very differently and are much more costly per-unit=
 than tiny systems. Once we grow the blocksize large enough that a single co=
mputer can't do all the processing all by itself we get into a world of much=
 harder, much more expensive scaling problems. Especially because we're talk=
ing about a distributed system where the nodes don't even trust each other. A=
nd transaction processing is largely non-parallel. You have to check each tr=
ansaction against each other transaction to make sure that they aren't doubl=
e spending eachother. This takes synchronization and prevents 500 CPUs from a=
ll crunching the data concurrently. You have to be a lot more clever than th=
at to get things working and consistent.
>=20
> When talking about scalability problems, you should ask yourself what othe=
r systems in the world operate at the scales you are talking about. None of t=
hem have cost structures in the 6 digit range, and I'd bet (without actually=
 knowing) that none of them have cost structures in the 7 digit range either=
. In fact I know from working in a related industry that the cost structures=
 for the datacenters (plus the support engineers, plus the software manageme=
nt, etc.) that do airline ticket processing are above $5 million per year fo=
r the larger airlines. Visa is probably even more expensive than that (thoug=
h I can only speculate).
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev