1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191]
helo=mx.sourceforge.net)
by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <sipa@ulyssis.org>) id 1QrDyc-0000xb-JH
for bitcoin-development@lists.sourceforge.net;
Wed, 10 Aug 2011 18:58:02 +0000
X-ACL-Warn:
Received: from rhcavuit01.kulnet.kuleuven.be ([134.58.240.129]
helo=cavuit01.kulnet.kuleuven.be)
by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76)
id 1QrDyb-00021T-1s for bitcoin-development@lists.sourceforge.net;
Wed, 10 Aug 2011 18:58:02 +0000
X-KULeuven-Envelope-From: sipa@ulyssis.org
X-Spam-Status: not spam, SpamAssassin (not cached, score=-48.788, required 5,
autolearn=disabled, DKIM_ADSP_CUSTOM_MED 0.00,
FREEMAIL_FROM 0.00, KUL_SMTPS -50.00, NML_ADSP_CUSTOM_MED 1.20,
T_TO_NO_BRKTS_FREEMAIL 0.01)
X-KULeuven-Scanned: Found to be clean
X-KULeuven-ID: BB64D1382E5.A7323
X-KULeuven-Information: Katholieke Universiteit Leuven
Received: from smtps01.kuleuven.be (smtpshost01.kulnet.kuleuven.be
[134.58.240.74])
by cavuit01.kulnet.kuleuven.be (Postfix) with ESMTP id BB64D1382E5
for <bitcoin-development@lists.sourceforge.net>;
Wed, 10 Aug 2011 20:57:53 +0200 (CEST)
Received: from smtp.ulyssis.org (mail.ulyssis.student.kuleuven.be
[193.190.253.235])
by smtps01.kuleuven.be (Postfix) with ESMTP id 91DBA31E702
for <bitcoin-development@lists.sourceforge.net>;
Wed, 10 Aug 2011 20:57:53 +0200 (CEST)
Received: from wop.ulyssis.org (wop.intern.ulyssis.org [192.168.0.182])
by smtp.ulyssis.org (Postfix) with ESMTP id F3222F8004
for <bitcoin-development@lists.sourceforge.net>;
Wed, 10 Aug 2011 21:00:48 +0200 (CEST)
Received: by wop.ulyssis.org (Postfix, from userid 615)
id 9992587C1B0; Wed, 10 Aug 2011 20:57:53 +0200 (CEST)
Date: Wed, 10 Aug 2011 20:57:53 +0200
X-Kuleuven: This mail passed the K.U.Leuven mailcluster
From: Pieter Wuille <pieter.wuille@gmail.com>
To: bitcoin-development@lists.sourceforge.net
Message-ID: <20110810185752.GA18562@ulyssis.org>
References: <CABsx9T2pTg8YG_Q09cnAvsrxquLO-6cWr1tb=fdWtLPBEyJzng@mail.gmail.com>
<1312995554.17416.22.camel@BMThinkPad.lan.bluematt.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1312995554.17416.22.camel@BMThinkPad.lan.bluematt.me>
X-PGP-Key: http://sipa.ulyssis.org/pubkey.asc
User-Agent: Mutt/1.5.20 (2009-06-14)
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(pieter.wuille[at]gmail.com)
0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is
CUSTOM_MED 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit,
and not from a mailing list
0.0 T_TO_NO_BRKTS_FREEMAIL To: misformatted and free email service
X-Headers-End: 1QrDyb-00021T-1s
Subject: Re: [Bitcoin-development] Roadmap/schedules
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 10 Aug 2011 18:58:02 -0000
On Wed, Aug 10, 2011 at 06:59:14PM +0200, Matt Corallo wrote:
> On Wed, 2011-08-10 at 12:29 -0400, Gavin Andresen wrote:
> > I've been wading through the pull requests and bug lists to figure out
> > a roadmap for the next few months.
> >
> > Here are the things on my priority list:
>
> > 2. We've got a chronic problem with new code causing CRITICAL_SECTION
> > deadlocks (see issue #453 for the latest). Detecting potential
> > deadlocks early should be done; longer term I think re-architecting to
> > be single-threaded/asio is probably the right thing to do.
> Sipa had begin looking at doing some redoing of the locking system (to
> support more broad stuff like read-only locks, etc) to solve that exact
> bug, but I never heard anything about if he actually started writing
> code or how far he got.
No I didn't start writing anything - I've been quite busy the past few weeks,
and will be more so the coming weeks. Anyway, some ideas:
Either we try to make everything single threaded, and aim towards a bitcoin
library which you pass events (which can be network, rpc, UI, ...) and
it always processes in finite time, without any separate threads. That
would be a serious rewrite, and maybe a limitation on potential growth
(there *will* be a time where a full node doesn't run on anything but
a 16-core machine...).
The alternative is doing a very careful checking/rework of the locking
system. I think you want some per-object locking instead of per single
data structure. Making it so fine-grained forces careful checking of
the order in which things are locked. That is hard to keep track of,
and probably doesn't gain you very much (just a guess, experiments could
prove me wrong, obviously)
I would propose a system with one lock for the node-handling code
(mapTransactions, mapBlockIndex, mapOrphanBlocks, ...), one lock for
the wallet-handling code (mapWallet, CKeyStore), and one lock for
network-handling code. No access to any inner data structures of
these components is exposed, and everything goes through accessor
functions. All exposed functions of each component take the respective
lock upon entering the component. This includes functions that only
need read-only access (which currently often don't take a lock at
all, iirc).
However, I think we can move to reader-writer locks (boost's shared_mutex).
A lot of code does not need an exclusive lock on the data, as multiple
threads reading the internal data structures simultaneously is not a
problem. This would mean that all inspector functions are wrapped in a
lock_shared/unlock_shared blocks, all mutator functions are wrapped in
a lock_upgrade/unlock_upgrade block, and code that actually modifies
data structures is wrapped in a unlock_upgrade_and_lock/
unlock_and_lock_upgrade block.
This is clearly part of a larger code-cleanup effort, as it would mean
moving all code in GUI and RPC that take locks on various things, to
the component they are taking locks on. That's immediately a nice step
towards "librarification" of the code...
> > Sipa's wallet and key export/import
> I was under the impression the plan was for this to go in 0.4 aka next
> release, but personally, I don't care too much either way.
I think it should be more or less finished by now in terms of
functionality, at least for dumpprivkey, importprivkey, removeprivkey.
I'm somewhat less sure about dumpwallet/importwallet, as some changes
to the json dump format might be useful still. It does require testing
though...
> > Move from wxWidgets to qt for the GUI
I'd really like to see that - with or without autotools, if some degree
of consistent config/build architecture can be maintained for the
different platforms.
> > Un-hardcode fee handling (anybody already working on this?)
> Sipa did some good thinking through for algorithms that could be really
> useful here, but I don't think any code was ever written, nor did he
> finish (is he off doing the studying thing?)
I was working on a draft for a reworked fee system. I didn't get to
write things out nicely, but the main idea was: assign a score to each
transaction group, in a way that scores always keep increasing over time.
Keep the memory pool sorted according to those scores, and drop the lowest
scoring ones when a configurable memory limit is reached (no limit on the
score itself). Finally, for mining, select the top N transaction groups
from the pool in such a way that an average configurable fee per byte
is maintained.
As each mining node chooses a (hopefully more or less fixed, or at least
only slowly changing) cutoff score above which transactions are included,
the network should converge to a more or less fixed probability distribution
for the score at which transactions are included.
Nodes can measure and estimate this distribution, and calculate expected time
to inclusion for a given fee.
The devil is in the details, as it is kinda hard to define a scoring system
for transactions that is independent from the current exchange value of
bitcoins, from which kind of transactions are common on the network, but still
tries to mimic the cost for the network to handle that transaction.
Anyway, as said, I currently don't have the time to implement these ideas
right now. I do read the mailing list, though :)
--
Pieter
|
