1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
|
Return-Path: <admin@multipool.us>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 8697E67
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 17 Oct 2015 04:37:33 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-yk0-f184.google.com (mail-yk0-f184.google.com
[209.85.160.184])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id AF26412E
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 17 Oct 2015 04:37:32 +0000 (UTC)
Received: by ykdz2 with SMTP id z2so420668ykd.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 16 Oct 2015 21:37:31 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to
:references:subject:mime-version:content-type;
bh=vosRhuxXwkQLdsaXFXtK1J3Mv3aeJsCbANTPlN3Uymw=;
b=X9Sgn4tHJmzm9wDyULH8eRp6jwlyT788KH5TCXEKTpQ1ni0xS2Ju1S7BBoPHzkEGiJ
4H9NdY4V+dCFiap4LWmTVhkirrNhlWhxIHbjDGQSvF2HiIV0rHzYaLoe3anEPn1ONJ8h
TPh8PaFEVxt2kjcxaJFa6eTbzLfzzUW9UazAKlDZoIH678+JTCrv9dNrJG4dCVaF4WVa
POvQEloIlnHY93IZqrE9gN145LSp+5QaNqB5kN7xO/2gQuk5HcdxShoOpI7WJ5/AaRYr
6Ze20Fbp6J/cP1vWQzUAVZH4OHGlcGp+SbkhoYx/6fGCY4XO3TnzB6MsupNYBH3LusSO
/LOw==
X-Gm-Message-State: ALoCoQn/zze/wALoBf1zWvOjClPQ3yMCpw8rwU9AmkDm8shw/GXK5FgZodmowMVNY0lh+P33w03J
X-Received: by 10.182.248.230 with SMTP id yp6mr208894obc.20.1445056651835;
Fri, 16 Oct 2015 21:37:31 -0700 (PDT)
X-Google-Already-Archived: Yes
X-Google-Doc-Id: 153533a36e6c9
X-Google-Thread-Id: ff86a7d220824d91
X-Google-Message-Url: http://groups.google.com/group/bitcoin-xt/msg/153533a36e6c9
X-Google-Thread-Url: http://groups.google.com/group/bitcoin-xt/t/ff86a7d220824d91
X-Google-Web-Client: true
Date: Fri, 16 Oct 2015 21:37:30 -0700 (PDT)
From: admin@multipool.us
To: bitcoin-xt <bitcoin-xt@googlegroups.com>
Message-Id: <1206de5a-e807-406e-9584-305a503d78ec@googlegroups.com>
In-Reply-To: <99C42DE7-814A-48F8-AB28-A5ADD77A9FD9@toom.im>
References: <99C42DE7-814A-48F8-AB28-A5ADD77A9FD9@toom.im>
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_Part_266_363891142.1445056651138"
X-Google-Token: EIqhh7EFM5Kx04Gs-3E0
X-Google-IP: 50.174.172.15
X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,HTML_MESSAGE,
RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] Memory leaks?
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Oct 2015 04:37:33 -0000
------=_Part_266_363891142.1445056651138
Content-Type: multipart/alternative;
boundary="----=_Part_267_1019226024.1445056651138"
------=_Part_267_1019226024.1445056651138
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
I can do better than that:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10625 ******** 20 0 41.310g 0.035t 51172 S 1.0 57.6 47:11.17
bitcoind
Any idea what's going on here?
$ cat /proc/10625/status|grep -i VM
VmPeak: 43647420 kB
VmSize: 43319932 kB
VmLck: 60 kB
VmPin: 0 kB
VmHWM: 38286252 kB
VmRSS: 38014300 kB
VmData: 43103812 kB
VmStk: 236 kB
VmExe: 8844 kB
VmLib: 194656 kB
VmPTE: 79932 kB
VmSwap: 0 kB
On Tuesday, October 13, 2015 at 2:56:15 PM UTC-7, Jonathan Toomim (Toomim
Bros) wrote:
>
> I just noticed that several of my running bitcoind processes were using
> around 3+ GB of RAM, even though the mempool itself seemed to be under
> control.
>
> XXXX@prime:~/bin$ ./bitcoin-cli getmempoolinfo
> {
> "size" : 1896,
> "bytes" : 37341328
> }
>
> [total memory usage not shown -- I restarted bitcoind as soon as I
> noticed, and didn't copy it down from top]
>
> 37 MB mempool, >3 GB RAM usage. Normally, when there aren't a lot of
> unconfirmed txns floating around the network, memory usage is around 600
> MB, so this is quite unusual.
>
> After restarting the process and letting it run for a few minutes, I get:
>
> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
> [###] [XXXX] 20 0 1402M 317M 49836 S 1.0 8.2 0:41.71 ./bitcoind
> -daemon
>
> XXXX@prime:~/bin$ ./bitcoin-cli getmempoolinfo
> {
> "size" : 1072,
> "bytes" : 670000
> }
>
> 0.67 MB mempool, 317 MB RAM usage. Much more reasonable.
>
>
> Here's another node I'm running that has been online longer, before
> restarting:
>
> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
> [###] [XXXX] 20 0 4961M 3540M 11080 S 2.8 45.3 8h20:11
> bin/bitcoind -daemon
>
> XXXX@feather:~$ bin/bitcoin-cli getmempoolinfo
> {
> "size" : 3045,
> "bytes" : 39656126
> }
>
> 39 MB mempool, 3540 MB total memory usage. After restarting bitcoind, I
> see:
>
> [XXXX]@feather:~$ bin/bitcoin-cli stop
> Bitcoin server stopping
> [XXXX]@feather:~$ bin/bitcoind -daemon
> Bitcoin server starting
> [XXXX]@feather:~$ sleep 10; bin/bitcoin-cli getmempoolinfo
> {
> "size" : 39,
> "bytes" : 47037
> }
>
>
> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
> [###] [XXXX] 20 0 1640M 247M 67960 S 0.0 3.2 0:05.17
> bin/bitcoind -daemon
>
>
>
>
> Does anybody have any guesses where we might be leaking memory, or what is
> using the additional 2.4 GB? I've been using minrelaytxfee=0.00003 or
> similar on my nodes. Maybe there's a leak in the minrelaytxfee code path?
> Has anyone else seen something similar?
>
> This issue appears to happen both with Bitcoin Core 0.10.1 and with
> Bitcoin XT 0.11B.
>
------=_Part_267_1019226024.1445056651138
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I can do better than that:<div><br></div><div><div><font f=
ace=3D"courier new, monospace">=C2=A0 PID USER =C2=A0 =C2=A0 =C2=A0PR =C2=
=A0NI =C2=A0 =C2=A0VIRT =C2=A0 =C2=A0RES =C2=A0 =C2=A0SHR S =C2=A0%CPU %MEM=
=C2=A0 =C2=A0 TIME+ COMMAND =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0</font></div><div><font face=3D"courier new, =
monospace">10625 ******** =C2=A020 =C2=A0 0 41.310g 0.035t =C2=A051172 S =
=C2=A0 1.0 57.6 =C2=A047:11.17 bitcoind =C2=A0 =C2=A0=C2=A0</font></div><di=
v><font face=3D"courier new, monospace"><br></font></div><div><font face=3D=
"arial, sans-serif">Any idea what's going on here?</font></div><div><fo=
nt face=3D"courier new, monospace"><br></font></div><div><font face=3D"cour=
ier new, monospace"><div>$ cat /proc/10625/status|grep -i VM</div><div>VmPe=
ak: 43647420 kB</div><div>VmSize: 43319932 kB</div><div>VmLck: =C2=A0 =C2=
=A0 =C2=A0 =C2=A060 kB</div><div>VmPin: =C2=A0 =C2=A0 =C2=A0 =C2=A0 0 kB</d=
iv><div>VmHWM: =C2=A038286252 kB</div><div>VmRSS: =C2=A038014300 kB</div><d=
iv>VmData: 43103812 kB</div><div>VmStk: =C2=A0 =C2=A0 =C2=A0 236 kB</div><d=
iv>VmExe: =C2=A0 =C2=A0 =C2=A08844 kB</div><div>VmLib: =C2=A0 =C2=A0194656 =
kB</div><div>VmPTE: =C2=A0 =C2=A0 79932 kB</div><div>VmSwap: =C2=A0 =C2=A0 =
=C2=A0 =C2=A00 kB</div><div><br></div></font></div><br>On Tuesday, October =
13, 2015 at 2:56:15 PM UTC-7, Jonathan Toomim (Toomim Bros) wrote:<blockquo=
te class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left:=
1px #ccc solid;padding-left: 1ex;"><div style=3D"word-wrap:break-word">I j=
ust noticed that several of my running bitcoind processes were using around=
3+ GB of RAM, even though the mempool itself seemed to be under control.<d=
iv><br></div><div><div style=3D"margin:0px;font-family:'Andale Mono'=
;;color:rgb(41,249,20);background-color:rgb(0,0,0)">XXXX@prime:~/bin$ ./bit=
coin-cli getmempoolinfo</div><div style=3D"margin:0px;font-family:'Anda=
le Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">{</div><div =
style=3D"margin:0px;font-family:'Andale Mono';color:rgb(41,249,20);=
background-color:rgb(0,0,0)">=C2=A0 =C2=A0 "size" : 1896,</div><d=
iv style=3D"margin:0px;font-family:'Andale Mono';color:rgb(41,249,2=
0);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 "bytes" : 37341328<=
/div><div style=3D"margin:0px;font-family:'Andale Mono';color:rgb(4=
1,249,20);background-color:rgb(0,0,0)">}</div></div><div><br></div><div>[to=
tal memory usage not shown -- I restarted bitcoind as soon as I noticed, an=
d didn't copy it down from top]</div><div><br></div><div>37 MB mempool,=
>3 GB RAM usage. Normally, when there aren't a lot of unconfirmed t=
xns floating around the network, memory usage is around 600 MB, so this is =
quite unusual.</div><div><br></div><div>After restarting the process and le=
tting it run for a few minutes, I get:</div><div><br></div><div><div style=
=3D"margin:0px;font-family:'Andale Mono';background-color:rgb(0,165=
,0)">=C2=A0=C2=A0PID USER=C2=A0 =C2=A0 =C2=A0 PRI=C2=A0 NI=C2=A0 VIRT =C2=
=A0 RES =C2=A0 SHR S CPU% <span style=3D"background-color:#00a6b2">MEM% </s=
pan>=C2=A0 TIME+=C2=A0 Command</div></div><div><div style=3D"margin:0px;fon=
t-family:'Andale Mono';color:rgb(41,249,20);background-color:rgb(0,=
0,0)">[###] [XXXX] =C2=A0 =C2=A0=C2=A020 =C2=A0 0 <span style=3D"color:#34b=
bc7">1402M=C2=A0 317M 49</span>836 S=C2=A0 1.0=C2=A0 8.2=C2=A0 0:41.71 ./bi=
tcoind -daemon</div></div><div><br></div><div><div style=3D"margin:0px;font=
-family:'Andale Mono';color:rgb(41,249,20);background-color:rgb(0,0=
,0)">XXXX@prime:~/bin$ ./bitcoin-cli getmempoolinfo</div><div style=3D"marg=
in:0px;font-family:'Andale Mono';color:rgb(41,249,20);background-co=
lor:rgb(0,0,0)">{</div><div style=3D"margin:0px;font-family:'Andale Mon=
o';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 &quo=
t;size" : 1072,</div><div style=3D"margin:0px;font-family:'Andale =
Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 &=
quot;bytes" : 670000</div><div style=3D"margin:0px;font-family:'An=
dale Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">}</div></d=
iv><div><br></div><div>0.67 MB mempool, 317 MB RAM usage. Much more reasona=
ble.</div><div><br></div><div><br></div><div>Here's another node I'=
m running that has been online longer, before restarting:</div><div><br></d=
iv><div><div style=3D"margin:0px;font-family:'Andale Mono';backgrou=
nd-color:rgb(0,165,0)">=C2=A0=C2=A0PID USER=C2=A0 =C2=A0 =C2=A0 PRI=C2=A0 N=
I=C2=A0 VIRT =C2=A0 RES =C2=A0 SHR S CPU% <span style=3D"background-color:#=
00a6b2">MEM% </span>=C2=A0 TIME+=C2=A0 Command</div><div style=3D"margin:0p=
x;font-family:'Andale Mono';color:rgb(41,249,20);background-color:r=
gb(0,0,0)">[###]=C2=A0<span style=3D"color:#000000">[XXXX]=C2=A0=C2=A0 =C2=
=A0</span> 20 =C2=A0 0 <span style=3D"color:#34bbc7">4961M 3540M 11</span>0=
80 S=C2=A0 2.8 45.3=C2=A0<span style=3D"color:#c33720"> 8h</span>20:11 bin/=
bitcoind -daemon</div></div><div><br></div><div><div style=3D"margin:0px;fo=
nt-family:'Andale Mono';color:rgb(41,249,20);background-color:rgb(0=
,0,0)">XXXX@feather:~$ bin/bitcoin-cli getmempoolinfo</div><div style=3D"ma=
rgin:0px;font-family:'Andale Mono';color:rgb(41,249,20);background-=
color:rgb(0,0,0)">{</div><div style=3D"margin:0px;font-family:'Andale M=
ono';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 &q=
uot;size" : 3045,</div><div style=3D"margin:0px;font-family:'Andal=
e Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0=
"bytes" : 39656126</div><div style=3D"margin:0px;font-family:=
9;Andale Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">}</div=
></div><div><br></div><div>39 MB mempool, 3540 MB total memory usage. After=
restarting bitcoind, I see:</div><div><br></div><div><div style=3D"margin:=
0px;font-family:'Andale Mono';color:rgb(41,249,20);background-color=
:rgb(0,0,0)">[XXXX]@feather:~$ bin/bitcoin-cli stop</div><div style=3D"marg=
in:0px;font-family:'Andale Mono';color:rgb(41,249,20);background-co=
lor:rgb(0,0,0)">Bitcoin server stopping</div><div style=3D"margin:0px;font-=
family:'Andale Mono';color:rgb(41,249,20);background-color:rgb(0,0,=
0)">[XXXX]@feather:~$ bin/bitcoind -daemon</div><div style=3D"margin:0px;fo=
nt-family:'Andale Mono';color:rgb(41,249,20);background-color:rgb(0=
,0,0)">Bitcoin server starting</div><div style=3D"margin:0px;font-family:&#=
39;Andale Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">[XXXX=
]@feather:~$ sleep 10; bin/bitcoin-cli getmempoolinfo</div><div style=3D"ma=
rgin:0px;font-family:'Andale Mono';color:rgb(41,249,20);background-=
color:rgb(0,0,0)">{</div><div style=3D"margin:0px;font-family:'Andale M=
ono';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 &q=
uot;size" : 39,</div><div style=3D"margin:0px;font-family:'Andale =
Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">=C2=A0 =C2=A0 &=
quot;bytes" : 47037</div><div style=3D"margin:0px;font-family:'And=
ale Mono';color:rgb(41,249,20);background-color:rgb(0,0,0)">}</div></di=
v><div><br></div><div><br></div><div><div style=3D"margin:0px;font-family:&=
#39;Andale Mono';background-color:rgb(0,165,0)">=C2=A0=C2=A0PID USER=C2=
=A0 =C2=A0 =C2=A0 PRI=C2=A0 NI=C2=A0 VIRT =C2=A0 RES =C2=A0 SHR S CPU% <spa=
n style=3D"background-color:#00a6b2">MEM% </span>=C2=A0 TIME+=C2=A0 Command=
</div><div style=3D"margin:0px;font-family:'Andale Mono';color:rgb(=
41,249,20);background-color:rgb(0,0,0)">[###] [XXXX] =C2=A0 =C2=A0=C2=A020 =
=C2=A0 0 <span style=3D"color:rgb(52,187,199)">1640M=C2=A0 247M 67</span>96=
0 S=C2=A0 0.0=C2=A0 3.2=C2=A0 0:05.17 bin/bitcoind -daemon</div></div><div>=
<br></div><div><br></div><div><br></div><div><br></div><div>Does anybody ha=
ve any guesses where we might be leaking memory, or what is using the addit=
ional 2.4 GB? I've been using=C2=A0<span style=3D"background-color:rgb(=
0,0,0);color:rgb(41,249,20);font-family:'Andale Mono'">minrelaytxfe=
e=3D0.00003</span>=C2=A0or similar on my nodes. Maybe there's a leak in=
the minrelaytxfee code path? Has anyone else seen something similar?</div>=
<div><br></div><div>This issue appears to happen both with Bitcoin Core 0.1=
0.1 and with Bitcoin XT 0.11B.</div></div></blockquote></div></div>
------=_Part_267_1019226024.1445056651138--
------=_Part_266_363891142.1445056651138--
|
