summaryrefslogtreecommitdiff
path: root/67/def7ef724fd027c025e774130ab43462b21755
blob: 0d2683d79d1fbdf8a581e2e902bde19bb3c158a3 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
Return-Path: <micaroni3@gmail.com>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 8A398C002D
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 20 Aug 2022 14:17:36 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp2.osuosl.org (Postfix) with ESMTP id 58D7040993
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 20 Aug 2022 14:17:36 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 58D7040993
Authentication-Results: smtp2.osuosl.org;
 dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com
 header.a=rsa-sha256 header.s=20210112 header.b=UQIp95Ot
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -1.848
X-Spam-Level: 
X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from smtp2.osuosl.org ([127.0.0.1])
 by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id k2ApQbx07023
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 20 Aug 2022 14:17:35 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 90085401ED
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [IPv6:2a00:1450:4864:20::22c])
 by smtp2.osuosl.org (Postfix) with ESMTPS id 90085401ED
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 20 Aug 2022 14:17:34 +0000 (UTC)
Received: by mail-lj1-x22c.google.com with SMTP id e2so1992750ljj.1
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Sat, 20 Aug 2022 07:17:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=cc:to:subject:message-id:date:from:mime-version:from:to:cc;
 bh=YpjZAnY9kaaAPKmuXrURbnr6GstmHIEwRCjcecxlE1w=;
 b=UQIp95OtdZzGpdDQSg8MO1BS4YkTTlPM+RddTnDYTjWlRKceDYMGQ7fmWIX6vQU4ET
 ZkKaVFX4g27wNGRBUaG48K1ltWx+/jZ6EeDBR5tAz8DStwmkgRxas3HquUMmIJyvyONk
 M94RAf6a0xREPepzxIRWz/qOq7Uxew5y00GuBcQsPetpw+E68HYKpDb3czAzoB5Lv6gl
 SSNDEC9tB7q6t4AdyfJbs+flBnsDAkoXmeIP644fH77iquvdnUMQUOnU2G8wpm3hLcZi
 7iZIQ0ewCJPschaOVZMjBivbhkzfH2FQlwxsm8R9fSjUrh9QhCaipSG9Ghv6sQWFus0U
 zQqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
 :from:to:cc;
 bh=YpjZAnY9kaaAPKmuXrURbnr6GstmHIEwRCjcecxlE1w=;
 b=zE4G+4joymuSsTOwCrAe0c3feun0G6T8b/8voHU5J3Sl2Q0cCWMQawWr/5bj8nDoBX
 7C+g0RhEfX5J3GqbOrkWo25cmtgD1NKcBea0iwxnr5fA3N77iiyv29KJLx3Mg0GHGOpG
 BmjSIvGcqaSXhfpyoRQHSEQ9qhtDc0faBBJyAaW0H0nSlBYZl2Pz4b2J9L+X1ZSV76WW
 t+4OMUsuyg5om1XCuSHobNLjQA+03IHE/pKTq6zSsA0BSKz01WUuEj+tu0wB/LQe2+9O
 e8djy1zec7ecXPSvTl/pxCprBEtKLZlS1+7BY3DJBlCH1DLwcEc/+3WCd7cwhY3M7y7U
 aLmg==
X-Gm-Message-State: ACgBeo1R0qBAgZo1vMsrYvN7AHtm07vkFITIAtfuIIVteAWoPMDt0pIk
 B8Zc13iTd3Ot5UfZIaFp1LcZx25ws3Vx8WUsibMBHP0nYphhXhnQ1P8=
X-Google-Smtp-Source: AA6agR4HVeUk7PPpyCop1OjeMJyOYK9kEh3Htr6kJFLS8Kr+SXunCK/oEcp6ABoL+e58KbuW7V6rmUiu5jfgFJchVYA=
X-Received: by 2002:a05:651c:2112:b0:261:b9c1:509 with SMTP id
 a18-20020a05651c211200b00261b9c10509mr2938860ljq.39.1661005051429; Sat, 20
 Aug 2022 07:17:31 -0700 (PDT)
MIME-Version: 1.0
From: micaroni@gmail.com
Date: Sat, 20 Aug 2022 11:16:53 -0300
Message-ID: <CAHvMVPSKbG6s236uUsLmn6+i4bjRV4rDBPd4nJy3qR0ih+aTFg@mail.gmail.com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="0000000000000f879a05e6ace0a4"
X-Mailman-Approved-At: Sat, 20 Aug 2022 14:19:29 +0000
Cc: Bruno Garcia <bruno@vinteum.org>
Subject: [bitcoin-dev] Huge wallets make Bitcoin Core unusable (Daemon+CLI &
	Qt)
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 20 Aug 2022 14:17:36 -0000

--0000000000000f879a05e6ace0a4
Content-Type: text/plain; charset="UTF-8"

Hi dear devs,


1. THE ISSUE - DAEMON+CLI
========================
I had a wallet in a server production since 2017 (5 years old) and when it
reached about 273 MB, 2.079.337 transactions and 446.503 generated
addresses, the performance started to degrade exponentially.

Most of the commands, e.g. "getbalance", "walletpassphrase" and
"getreceivedbyaddress" started to timeout (more than 15 minutes delay -
default timeout). The CPU was 100% used (all 32 cores - with 150 load avg)
and the machine became almost unusable breaking everything else, with the
default config of 16 RPC threads and 15 min timeout and some attempt calls
per mi

Increasing the timeout and/or the RPC threads in the config file turns
things even worse.

Putting the wallet.dat in a very fast SSD disk and increasing the size of
the cache (I tried with 8GB) have improved but I'm not sure if it is enough.


2. TEST ON BITCOIN QT
====================
If you try to load the wallet in the "bitcoin-qt" everything gets stuck,
even the system (OS/UI) doesn't respond anymore. You click on a button and
receive the message "window doesn't respond, wait or terminate?" - if you
wait it releases after a while but it is slow and hard to use the wallet
anyway.


3. WHY IS THIS SO BAD?
=====================
This is bad because the standard client becomes almost useless for the
wallet feature:

3.1) the wallet Qt already is not so popular among end users. It doesn't
look modern, slow to first sync and hard to use. That's why people prefer
to use Electrum or Wasabi - I personally don't care but it's the sad truth;

3.2) it becomes useless now also for servers in production, forcing them to
use third party solutions for huge wallets. Even if you split in 10 wallets
it will just delay 10 times more each to degrade, postponing the problem
but not eliminating it. Not to mention the slow and daily degradation.


4. SHOULD WE GIVE UP THE WALLET FEATURE?
========================================
Then, Bitcoin Core becomes just a reference implementation and blocks
relayers, but as an application wallet itself turns into a really bad
choice. --- It leads me to the following question: if we won't invest time
on improving this, shouldn't we remove the wallet feature at all? Why keep
a wallet feature that is not useful for the end user nor the production
server? Is it useful for what then?


5. THE CURRENT "SOLUTION" IS BAD
===============================
Currently, the only "solution" for huge wallets is shameful: create a new
one and send the funds there from time to time. But when is the right time
exactly? The performance degrades suddenly or gets worse slowly for each
new address and/or tx?. And besides not being an elegant solution and "not
in the handbook", it also can break a lot of things like monitoring old
addresses and also can lead to privacy concerns unifying lots of inputs in
a big and expensive tx.


6. OTHER USER CASES?
====================
I think this could also become an issue if we have LN nodes that use the
Bitcoin Core wallet infrastructure behind to open / close many channels for
a long time.


7. FINAL THOUGHTS
=================
If moving the wallet from a HDD to a SSD improved a lot, maybe just caching
the entire wallet in memory could improve even more, but I'm afraid some
code optimization is also necessary.


8. SOME QUESTIONS
==================
8.1) Can we "optimize" a huge wallet without moving the funds to a new one?
Like a "fsck" or eqv?

8.2) Can we improve the cache usage somehow? Putting the entire wallet in
memory, for example?

8.3) Is it possible to reduce the wallet size (273 MB is too much for a HD
wallet)?

8.4) Can we tell the CLI to ignore old addresses? What if I need to watch
only the last 30 days?

8.5) How to improve the I/O treatment and/or CPU usage in the main thread
on Bitcoin-Qt to avoid window freezing on big and huge wallets?

8.6) In the last case (if it was not possible to optimize the wallet or the
CLI & Qt), can the CLI just warn the user like: "the wallet is becoming too
big and slow, execute the command 'archive'". And then, the command
"archive" could rename the current wallet to something like
"wallet.dat.archive_until_20220818", create a new "wallet.dat" and move the
funds automatically? Also, would it be nice to have an
"autoarchivehugewallets=1" in the file config?


9. POSSIBLE RELATED AND TESTS
=============================

[1] https://github.com/bitcoin/bitcoin/issues/15015
[2] https://github.com/bitcoin/bitcoin/issues/15148
[3] https://github.com/bitcoin/bitcoin/issues/16874
[4] https://github.com/bitcoin/bitcoin/pull/17135
[5] https://github.com/bitcoin/bitcoin/pull/18160
[6] https://github.com/bitpay/bitcore-node/issues/463
[7] https://github.com/RavenProject/Ravencoin/issues/499
[8] https://github.com/sugarchain-project/sugarchain/issues/106
[9]
https://bitcoin.stackexchange.com/questions/111844/loadwallet-takes-too-much-and-times-out
[10] https://bitcoin.stackexchange.com/a/45713/1761


ANOTHER POSSIBLE BUG
======================

Even if my node is 100% sync:
2022-08-20T13:11:43Z UpdateTip: new
best=00000000000000000005bba0593c2be0f1d322223501591d2b31b544e3af3d0b
height=750300 version=0x2fffe000 log2_work=93.687081 tx=758181489
date='2022-08-20T13:11:16Z' progress=1.000000 cache=4.6MiB(34964txo)

After a "loadwallet" command I am getting an old / wrong balance. The
wallet is already empty because I had moved the funds to a new one 3 or 4
days ago but it is still showing the old  balance. I didn't receive any
warning message saying the need for a rescan or something like that.

I am trying the "rescanblockchain" command but it is running and it taking
a looooooooooooooong time.




Best regards,

Felipe.

--0000000000000f879a05e6ace0a4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:arial,he=
lvetica,sans-serif;font-size:small;color:#000000">Hi dear devs,<br></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
font-size:small;color:#000000"><br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><=
br></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000">1. THE ISSUE - DAEMON+CLI<br></di=
v><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-se=
rif;font-size:small;color:#000000">=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br></div><div class=3D"gmail_default" sty=
le=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"=
>I had a wallet in a server production since 2017 (5 years old) and when it=
 reached about 273 MB, 2.079.337 transactions and 446.503 generated address=
es, the performance started to degrade exponentially.</div><div class=3D"gm=
ail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:smal=
l;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-famil=
y:arial,helvetica,sans-serif;font-size:small;color:#000000">Most of the com=
mands, e.g. &quot;getbalance&quot;, &quot;walletpassphrase&quot; and &quot;=
getreceivedbyaddress&quot; started to timeout (more than 15 minutes delay -=
 default timeout). The CPU was 100% used (all 32 cores - with 150 load avg)=
 and the machine became almost unusable breaking everything else, with the =
default config of 16 RPC threads and 15 min timeout and some attempt calls =
per mi<br></div><div class=3D"gmail_default" style=3D"font-family:arial,hel=
vetica,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gm=
ail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:smal=
l;color:#000000">Increasing the timeout and/or the RPC threads in the confi=
g file turns things even worse.</div><div class=3D"gmail_default" style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br><=
/div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans=
-serif;font-size:small;color:#000000">Putting the wallet.dat in a very fast=
 SSD disk and increasing the size of the cache (I tried with 8GB) have impr=
oved but I&#39;m not sure if it is enough.<br></div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:#000000"><br></div><div class=3D"gmail_default" style=3D"font-family:arial=
,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div class=
=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-siz=
e:small;color:#000000">2. TEST ON BITCOIN QT<br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D</div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">I=
f you try to load the wallet in the &quot;bitcoin-qt&quot; everything gets =
stuck, even the system (OS/UI) doesn&#39;t respond anymore. You click on a =
button and receive the message &quot;window doesn&#39;t respond, wait or te=
rminate?&quot; - if you wait it releases after a while but it is slow and h=
ard to use the wallet anyway.<br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><=
br></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:#000000">3. WHY IS THIS SO BAD?<br></div><div class=3D"gmail_default" styl=
e=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
font-size:small;color:#000000">This is bad because the standard client beco=
mes almost useless for the wallet feature:</div><div class=3D"gmail_default=
" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#00=
0000"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,hel=
vetica,sans-serif;font-size:small;color:#000000">3.1) the wallet Qt already=
 is not so popular among end users. It doesn&#39;t look modern, slow to fir=
st sync and hard to use. That&#39;s why people prefer to use Electrum or Wa=
sabi - I personally don&#39;t care but it&#39;s the sad truth;</div><div cl=
ass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-=
size:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D"f=
ont-family:arial,helvetica,sans-serif;font-size:small;color:#000000">3.2) i=
t becomes useless now also for servers in production, forcing them to use t=
hird party solutions for huge wallets. Even if you split in 10 wallets it w=
ill just delay 10 times more each to degrade, postponing the problem but no=
t eliminating it. Not to mention the slow and daily degradation.</div><div =
class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fon=
t-size:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D=
"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br>=
</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,san=
s-serif;font-size:small;color:#000000">4. SHOULD WE GIVE UP THE WALLET FEAT=
URE?<br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D</div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:#000000">Then, Bitcoin Core becomes just a reference implementatio=
n and blocks relayers, but as an application wallet itself turns into a rea=
lly bad choice. --- It leads me to the following question: if we won&#39;t =
invest time on improving this, shouldn&#39;t we remove the wallet feature a=
t all? Why keep a wallet feature that is not useful for the end user nor th=
e production server? Is it useful for what then?<br></div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-family=
:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div c=
lass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font=
-size:small;color:#000000">5. THE CURRENT &quot;SOLUTION&quot; IS BAD<br>=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D</div><div class=3D"gmail_default" style=3D"font-family:a=
rial,helvetica,sans-serif;font-size:small;color:#000000">Currently, the onl=
y &quot;solution&quot; for huge wallets is shameful: create a new one and s=
end the funds there from time to time. But when is the right time exactly? =
The performance degrades suddenly or gets worse slowly for each new address=
 and/or tx?. And besides not being an elegant solution and &quot;not in the=
 handbook&quot;, it also can break a lot of things like monitoring old addr=
esses and also can lead to privacy concerns unifying lots of inputs in a bi=
g and expensive tx.</div><div class=3D"gmail_default" style=3D"font-family:=
arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div cl=
ass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-=
size:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D"f=
ont-family:arial,helvetica,sans-serif;font-size:small;color:#000000">6. OTH=
ER USER CASES?<br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000">I think this could also become an=
 issue if we have LN nodes that use the Bitcoin Core wallet infrastructure =
behind to open / close many channels for a long time.<br></div><div class=
=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-siz=
e:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font=
-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div=
><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-ser=
if;font-size:small;color:#000000">7. FINAL THOUGHTS<br>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D</div>If moving the wallet from a HDD to a=
 SSD improved a lot, maybe just caching the entire wallet in memory could i=
mprove even more, but I&#39;m afraid some code optimization is also necessa=
ry.<br><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sa=
ns-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_defau=
lt" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#=
000000"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,h=
elvetica,sans-serif;font-size:small;color:#000000">8. SOME QUESTIONS</div><=
div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif=
;font-size:small;color:#000000">=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D<br></div><div class=3D"gmail_default" style=3D"font-family:ari=
al,helvetica,sans-serif;font-size:small;color:#000000">8.1) Can we &quot;op=
timize&quot; a huge wallet without moving the funds to a new one? Like a &q=
uot;fsck&quot; or eqv?</div><div class=3D"gmail_default" style=3D"font-fami=
ly:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div=
 class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fo=
nt-size:small;color:#000000">8.2) Can we improve the cache usage somehow? P=
utting the entire wallet in memory, for example?<br></div><div class=3D"gma=
il_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:#000000"><br></div><div class=3D"gmail_default" style=3D"font-family=
:arial,helvetica,sans-serif;font-size:small;color:#000000">8.3) Is it possi=
ble to reduce the wallet size (273 MB is too much for a HD wallet)?<br></di=
v><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-se=
rif;font-size:small;color:#000000"><br></div><div class=3D"gmail_default" s=
tyle=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#00000=
0">8.4) Can we tell the CLI to ignore old addresses? What if I need to watc=
h only the last 30 days?</div><div class=3D"gmail_default" style=3D"font-fa=
mily:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
font-size:small;color:#000000">8.5) How to improve the I/O treatment and/or=
 CPU usage in the main thread on Bitcoin-Qt to avoid window freezing on big=
 and huge wallets?</div><div class=3D"gmail_default" style=3D"font-family:a=
rial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div cla=
ss=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-s=
ize:small;color:#000000">8.6) In the last case (if it was not possible to o=
ptimize the wallet or the CLI &amp; Qt), can the CLI just warn the user lik=
e: &quot;the wallet is becoming too big and slow, execute the command &#39;=
archive&#39;&quot;. And then, the command &quot;archive&quot; could rename =
the current wallet to something like &quot;wallet.dat.archive_until_2022081=
8&quot;, create a new &quot;wallet.dat&quot; and move the funds automatical=
ly? Also, would it be nice to have an &quot;autoarchivehugewallets=3D1&quot=
; in the file config?</div><div class=3D"gmail_default" style=3D"font-famil=
y:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div =
class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;fon=
t-size:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D=
"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">9. P=
OSSIBLE RELATED AND TESTS</div><div class=3D"gmail_default" style=3D"font-f=
amily:arial,helvetica,sans-serif;font-size:small;color:#000000">=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:#000000">[1] <a href=3D"https://github.com/bitcoin/bitcoin/issues/15015">h=
ttps://github.com/bitcoin/bitcoin/issues/15015</a></div><div class=3D"gmail=
_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;c=
olor:#000000">[2] <a href=3D"https://github.com/bitcoin/bitcoin/issues/1514=
8">https://github.com/bitcoin/bitcoin/issues/15148</a></div><div class=3D"g=
mail_default" style=3D"font-family:arial,helvetica,sans-serif;font-size:sma=
ll;color:#000000">[3] <a href=3D"https://github.com/bitcoin/bitcoin/issues/=
16874">https://github.com/bitcoin/bitcoin/issues/16874</a></div><div class=
=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-siz=
e:small;color:#000000">[4] <a href=3D"https://github.com/bitcoin/bitcoin/pu=
ll/17135">https://github.com/bitcoin/bitcoin/pull/17135</a></div><div class=
=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-siz=
e:small;color:#000000">[5] <a href=3D"https://github.com/bitcoin/bitcoin/pu=
ll/18160">https://github.com/bitcoin/bitcoin/pull/18160</a></div><div class=
=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-siz=
e:small;color:#000000">[6] <a href=3D"https://github.com/bitpay/bitcore-nod=
e/issues/463">https://github.com/bitpay/bitcore-node/issues/463</a></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
font-size:small;color:#000000">[7] <a href=3D"https://github.com/RavenProje=
ct/Ravencoin/issues/499">https://github.com/RavenProject/Ravencoin/issues/4=
99</a><br>[8] <a href=3D"https://github.com/sugarchain-project/sugarchain/i=
ssues/106">https://github.com/sugarchain-project/sugarchain/issues/106</a><=
/div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans=
-serif;font-size:small;color:#000000">[9] <a href=3D"https://bitcoin.stacke=
xchange.com/questions/111844/loadwallet-takes-too-much-and-times-out">https=
://bitcoin.stackexchange.com/questions/111844/loadwallet-takes-too-much-and=
-times-out</a></div><div class=3D"gmail_default" style=3D"font-family:arial=
,helvetica,sans-serif;font-size:small;color:#000000">[10] <a href=3D"https:=
//bitcoin.stackexchange.com/a/45713/1761">https://bitcoin.stackexchange.com=
/a/45713/1761</a></div><div class=3D"gmail_default" style=3D"font-family:ar=
ial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div clas=
s=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font-si=
ze:small;color:#000000"><br></div><div class=3D"gmail_default" style=3D"fon=
t-family:arial,helvetica,sans-serif;font-size:small;color:#000000">ANOTHER =
POSSIBLE BUG</div><div class=3D"gmail_default" style=3D"font-family:arial,h=
elvetica,sans-serif;font-size:small;color:#000000">=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D</div><div class=3D"gmail_default=
" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#00=
0000"><br></div><div class=3D"gmail_default" style=3D"font-family:arial,hel=
vetica,sans-serif;font-size:small;color:#000000">Even if my node is 100% sy=
nc:</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000">2022-08-20T13:11:43Z UpdateTip: n=
ew best=3D00000000000000000005bba0593c2be0f1d322223501591d2b31b544e3af3d0b =
height=3D750300 version=3D0x2fffe000 log2_work=3D93.687081 tx=3D758181489 d=
ate=3D&#39;2022-08-20T13:11:16Z&#39; progress=3D1.000000 cache=3D4.6MiB(349=
64txo)</div><div class=3D"gmail_default" style=3D"font-family:arial,helveti=
ca,sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_=
default" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;co=
lor:#000000">After a &quot;loadwallet&quot; command I am getting an old / w=
rong balance. The wallet is already empty because I had moved the funds to =
a new one 3 or 4 days ago but it is still showing the old=C2=A0 balance. I =
didn&#39;t receive any warning message saying the need for a rescan or some=
thing like that.<br></div><div class=3D"gmail_default" style=3D"font-family=
:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div c=
lass=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;font=
-size:small;color:#000000">I am trying the &quot;rescanblockchain&quot; com=
mand but it is running and it taking a looooooooooooooong time.<br></div><d=
iv class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;=
font-size:small;color:#000000"><br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><=
br></div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,=
sans-serif;font-size:small;color:#000000"><br></div><div class=3D"gmail_def=
ault" style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:#000000"><br></div><div class=3D"gmail_default" style=3D"font-family:arial=
,helvetica,sans-serif;font-size:small;color:#000000">Best regards,</div><di=
v class=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif;f=
ont-size:small;color:#000000"><br></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">F=
elipe.<br></div></div>

--0000000000000f879a05e6ace0a4--