1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mh.in.england@gmail.com>) id 1Uz1vR-00086U-19
for bitcoin-development@lists.sourceforge.net;
Tue, 16 Jul 2013 09:52:05 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.219.44 as permitted sender)
client-ip=209.85.219.44; envelope-from=mh.in.england@gmail.com;
helo=mail-oa0-f44.google.com;
Received: from mail-oa0-f44.google.com ([209.85.219.44])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1Uz1vP-0008Qg-9e
for bitcoin-development@lists.sourceforge.net;
Tue, 16 Jul 2013 09:52:04 +0000
Received: by mail-oa0-f44.google.com with SMTP id l10so558249oag.3
for <bitcoin-development@lists.sourceforge.net>;
Tue, 16 Jul 2013 02:51:58 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.60.79.131 with SMTP id j3mr797568oex.96.1373968317898; Tue,
16 Jul 2013 02:51:57 -0700 (PDT)
Sender: mh.in.england@gmail.com
Received: by 10.76.23.36 with HTTP; Tue, 16 Jul 2013 02:51:57 -0700 (PDT)
In-Reply-To: <2F20A509-13A9-4C84-86D7-A15C21BACD53@include7.ch>
References: <B87F1213-5BD8-43F5-9744-F69947561ED5@grabhive.com>
<CANEZrP2xh=m8yWLt-o2UrrUVfU+cUYBuMVxkVFF5mVMtWdRwOQ@mail.gmail.com>
<C1727B41-AF61-44FA-BE17-5FE4425FDEA8@grabhive.com>
<CANEZrP0_H9+prDSF92q8a4QzP=fzDM6cTDv0+KcfV9NF9thkmw@mail.gmail.com>
<3E7894A0-06F3-453D-87F8-975A244EBACF@include7.ch>
<CANEZrP2jmWkDbpJEm0vd2CKF-prFNbz_ZeNJfDWtSCKb8k5ZXA@mail.gmail.com>
<2BDA0943-22BB-4405-9AF0-86FB41FD04A6@include7.ch>
<CANEZrP0McSrVzwv=-qimPyX41EEDmyQdYW5QjPr_i+KWyJZSZw@mail.gmail.com>
<2F20A509-13A9-4C84-86D7-A15C21BACD53@include7.ch>
Date: Tue, 16 Jul 2013 11:51:57 +0200
X-Google-Sender-Auth: BI6VhVigBv-yq5UtCbHeOlFVcds
Message-ID: <CANEZrP2yQvmvwP_ZULdS2i+X6L9MeZ+DfidiuZPD2EHwLsN2MA@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Jonas Schnelli <jonas.schnelli@include7.ch>,
Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Content-Type: multipart/alternative; boundary=047d7b67812685670d04e19ded42
X-Spam-Score: -0.5 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(mh.in.england[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1Uz1vP-0008Qg-9e
Subject: Re: [Bitcoin-development] Introducing BitcoinKit.framework
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 16 Jul 2013 09:52:05 -0000
--047d7b67812685670d04e19ded42
Content-Type: text/plain; charset=UTF-8
Let's re-add the list as this is a topic of general interest.
On Tue, Jul 16, 2013 at 11:36 AM, Jonas Schnelli <jonas.schnelli@include7.ch
> wrote:
> What do you think about extending the BitcoinKit.framework (based on
> bitcoind) so that the kit could handle the very basic SPV concept
> (getHeaders aka fast catchup, wallet-birthday, bloom filters)?
> Maybe it would be possible to only port some of the bitcoinj work to c++
> (and use it for extending BitcoinKit or bitcoind)?
> Maybe then it could also be a starting point for someone who likes to
> create a SPV mode for bitcoind?
>
Making bitcoind/Bitcoin-Qt support SPV mode was the original plan some
years ago, Satoshi even sent me some code he wrote that did the first
parts, but it was incomplete.
At the time, I decided to do a separate implementation for a few different
reasons. One is that my understanding of his code wasn't so good back then
and I lacked confidence to change it. Especially as there were no unit
tests back then (and still aren't any for most of it), making invasive
changes to the core validation code was and is highly risky. A separate
code base seemed to reduce the risk a lot.
Another reason is that Satoshi encouraged me to write a simple
re-implementation that people could learn from. And I wanted a documented,
object oriented API that people could use to build a variety of apps.
Yet another reason was bitcoind is security critical code that scrapes
complex data structures from untrusted sources on the internet, and it's
written in an unmanaged language. Ordinarily this would be a recipe for
disaster as a single overflow or memory management error could lead to
hacking and theft on a massive scale. It's like taking a chainsaw and using
it to carve an ice sculpture. Satoshi, incredibly, pulled it off, mostly by
using advanced C++ features that made his code hard to read for many people
and by being very, very careful. I was not convinced I could do such a good
job and was worried about accidentally introducing vulnerabilities.
A final reason is that it was clear that the bitcoind codebase would need
serious changes for mobiles, beyond that required for ordinary SPV support.
For example, Satoshi's code assumes it has access to block headers via a
std::map and that assumption is made in a lot of places. On Android phones,
you can't fit all the block headers in RAM. bitcoinj uses a circular ring
buffer of the last N thousand headers for this reason. It's quite different
to how bitcoind works.
All that said, it was a ton of work and it's still unclear that it was the
right call.
Anyway, your situation is a little different. Firstly you don't care about
mobiles, your app is intended for desktops. So the changes required are
less invasive. Also, there are more unit tests and more people with a good
understanding of the code these days, so perhaps the risk of introducing
bugs is lower. And these days we have some nice APIs for building apps so
that need is already met.
If you wanted to implement SPV mode in bitcoind, Gavin or I could send you
Satoshi's old patch although of course it is no longer usable. It would
indicate the basic cut lines though.
--047d7b67812685670d04e19ded42
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Let's re-add the list as this is a topic of general in=
terest.<div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, Ju=
l 16, 2013 at 11:36 AM, Jonas Schnelli <span dir=3D"ltr"><<a href=3D"mai=
lto:jonas.schnelli@include7.ch" target=3D"_blank">jonas.schnelli@include7.c=
h</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word"><div><di=
v class=3D"im"></div></div><div>What do you think about extending the Bitco=
inKit.framework (based on bitcoind) so that the kit could handle the very b=
asic SPV concept (getHeaders aka fast catchup, wallet-birthday, bloom filte=
rs)?<br>
</div><div>Maybe it would be possible to only port some of the bitcoinj wor=
k to c++ (and use it for extending BitcoinKit or bitcoind)?</div><div>Maybe=
then it could also be a starting point for someone who likes to create a S=
PV mode for bitcoind?</div>
</div></blockquote><div><br></div><div>Making bitcoind/Bitcoin-Qt support S=
PV mode was the original plan some years ago, Satoshi even sent me some cod=
e he wrote that did the first parts, but it was incomplete.</div><div><br>
</div><div>At the time, I decided to do a separate implementation for a few=
different reasons. One is that my understanding of his code wasn't so =
good back then and I lacked confidence to change it. Especially as there we=
re no unit tests back then (and still aren't any for most of it), makin=
g invasive changes to the core validation code was and is highly risky. A s=
eparate code base seemed to reduce the risk a lot.</div>
<div><br></div><div>Another reason is that Satoshi encouraged me to write a=
simple re-implementation that people could learn from. And I wanted a docu=
mented, object oriented API that people could use to build a variety of app=
s.</div>
<div><br></div><div>Yet another reason was bitcoind is security critical co=
de that scrapes complex data structures from untrusted sources on the inter=
net, and it's written in an unmanaged language. Ordinarily this would b=
e a recipe for disaster as a single overflow or memory management error cou=
ld lead to hacking and theft on a massive scale. It's like taking a cha=
insaw and using it to carve an ice sculpture. Satoshi, incredibly, pulled i=
t off, mostly by using advanced C++ features that made his code hard to rea=
d for many people and by being very, very careful. I was not convinced I co=
uld do such a good job and was worried about accidentally introducing vulne=
rabilities.</div>
<div><br></div><div>A final reason is that it was clear that the bitcoind c=
odebase would need serious changes for mobiles, beyond that required for or=
dinary SPV support. For example, Satoshi's code assumes it has access t=
o block headers via a std::map and that assumption is made in a lot of plac=
es. On Android phones, you can't fit all the block headers in RAM. bitc=
oinj uses a circular ring buffer of the last N thousand headers for this re=
ason. It's quite different to how bitcoind works.</div>
<div><br></div><div>All that said, it was a ton of work and it's still =
unclear that it was the right call.</div><div><br></div><div>Anyway, your s=
ituation is a little different. Firstly you don't care about mobiles, y=
our app is intended for desktops. So the changes required are less invasive=
. Also, there are more unit tests and more people with a good understanding=
of the code these days, so perhaps the risk of introducing bugs is lower. =
And these days we have some nice APIs for building apps so that need is alr=
eady met.</div>
<div><br></div><div>If you wanted to implement SPV mode in bitcoind, Gavin =
or I could send you Satoshi's old patch although of course it is no lon=
ger usable. It would indicate the basic cut lines though.=C2=A0</div></div>=
</div>
</div>
--047d7b67812685670d04e19ded42--
|
