1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
|
Return-Path: <gmaxwell@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 14E53258
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 29 Oct 2015 08:17:28 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f174.google.com (mail-io0-f174.google.com
[209.85.223.174])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9FD35DF
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 29 Oct 2015 08:17:27 +0000 (UTC)
Received: by iofz202 with SMTP id z202so37513295iof.2
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 29 Oct 2015 01:17:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc:content-type;
bh=vBj9vLBozmBy0zz4eVHJ3cVSnj1AfJe+O0G+yHbCGaM=;
b=qQMhUNpL6ancR52ihOB55vH5NpQNRw63bhLxLpnjo0lmLkdYAmrV113Zc0VGWjdgat
Y5fdNUJiIS6/viQvwpDQRXYMEEvEOdLalFWX+NupkMbujKTZ9rcKO7vlUa1my/MSDMWk
i3bW7s5Cq5GdQ4BhaHZy1+VEgBppsfkh+7IX4Q4OvLFfVqFmFrIaBKfKIrar1hr3mIFX
Hw1eY8LZnuz4NL6DiIfMYsaLWEspowrs74d3PHUQaGlNGRNT6V3ImVg2jF6zhFKIhRKe
6qCqpefXE62CQmUaDaUtfiMG5ZAsEbN58vr0MW5rlY9hNrn29S7DngGiifTdmpyHTBtR
l2Yg==
MIME-Version: 1.0
X-Received: by 10.107.25.199 with SMTP id 190mr1891580ioz.134.1446106647103;
Thu, 29 Oct 2015 01:17:27 -0700 (PDT)
Received: by 10.107.192.199 with HTTP; Thu, 29 Oct 2015 01:17:27 -0700 (PDT)
In-Reply-To: <5631C363.5060705@neomailbox.net>
References: <5631C363.5060705@neomailbox.net>
Date: Thu, 29 Oct 2015 08:17:27 +0000
Message-ID: <CAAS2fgQ=+5b4tht9SeGVJPXDbqKan=Y8x7Q5F=uT5iiq+BnHJA@mail.gmail.com>
From: Gregory Maxwell <gmaxwell@gmail.com>
To: telemaco <telemaco@neomailbox.net>
Content-Type: text/plain; charset=UTF-8
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM,
RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Oct 2015 08:17:28 -0000
On Thu, Oct 29, 2015 at 6:57 AM, telemaco via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Why not "outsource" totally that data management part to the already
> existing with decades of experience database world. People would be able to
> create incredibly easy bitcoin statistics/graphs/analisys with existing
> software packages (hey even excel or libreoffice like) or connect bitcoin
> data to their own sources and if so they chose analyze bitcoin data on a
> datawarehouse or any imaginable approach. Of course every transaction would
> be have to do through the bitcoin node and only the data management would be
> on rdbms side.
The word "database" is likely confusing people here. This is not a
database in an ordinary sense.
The bitcoin core consensus engine requires a highly optimized ultra
compact data structure to perform the lookups for coin existence. The
data stored is highly compressed and very specialized, it would not be
useful to other applications. Right now, on boring laptop hardware,
during network synchronization updates to this database run at over
10,000 records per second, while the system is also busy doing the
other validation chores of a node. This is backended by a high
performance transactional key value store. The need for performance
here is essential to even keeping up with the network, it's not about
enabling any kind of fancy querying (bitcoin core does not offer fancy
querying), it's about the base load that every node must handle to
usably sync up and keep up with the Bitcoin network.
The backend can be swapped out for something else that provides the
same properties, but doing so does not give you any of the
inspection/analytics that you're looking for. Systems that do that
exist, and they require databases taking hundreds of gigabytes of
storage and take days to weeks to import the network data. They're
great for what they're for, but they're not suitable for consensus use
in the system for space efficiency, performance, and consensus
consistency reasons.
|
