Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of mistfpga.net
	designates 208.91.199.217 as permitted sender)
	client-ip=208.91.199.217; envelope-from=steve@mistfpga.net;
	helo=us2.outbound.mailhostbox.com; 
Message-ID: <501001A6.90301@mistfpga.net>
Date: Wed, 25 Jul 2012 15:24:38 +0100
From: steve <steve@mistfpga.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
	rv:13.0) Gecko/20120614 Thunderbird/13.0.1
MIME-Version: 1.0
To: =?ISO-8859-1?Q?Michael_Gr=F8nager?= <gronager@mac.com>
References: <CAD2Ti29dqCYoOMcX0zcOQnpLGCxnCjYHHqMzyyq+hvcVQ2c7nQ@mail.gmail.com>
	<CAAS2fgTHrWdXHbX8oiLgnrhdC+yxL4QyPd7S4iB8RMNip_sUGg@mail.gmail.com>
	<A428177D-62AD-4712-9053-47B7ED5DBC84@mac.com>
	<500D73B9.5030806@mistfpga.net>
	<B872EE35-36F9-4F4E-8342-FFA141E433C7@mac.com>
	<500EFE00.7060200@mistfpga.net>
	<EF55BC4C-CDD4-43BD-988C-D5BC0C66F265@mac.com>
In-Reply-To: <EF55BC4C-CDD4-43BD-988C-D5BC0C66F265@mac.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Scalability issues
Precedence: list

On 25/07/2012 10:45, Michael Gr=F8nager wrote:
> Hi Steve,
>=20
> I see dramatic differences in performance on virtual machines vs=20
> running directly on the iron. I am not an expert in virtual
> machines,

They can be, it depends on how they are set up.  For reference, these
VM's used to test network stacks and file format bugs.  They do this
via debug tracing, trace into, not over.  They then dump this data in
files and it can keep up with a core2 duo laptop for file io. however
I moved to using an in ram database (4gb chunks, that then get dumped
over the network port to a db on a seperate machine)

I am not sure I have the skills to instrament this into bitcoind

>=20
> I would like to do a test keeping database log files in memory. It=20
> should not matter for durability of the wallet, as it flushes at
> each write anyway. As for the blockindex, it will remain
> consistent, but might be lagging some blocks behind at startup,
> which shouldn't really matter (except that the same block could end
> up appearing twice in the block00X files, inelegant, but not really
> a problem).
>=20
> Otherwise the system you describe (raid0 over 6 disks) should
> perform like crazy wrt disk i/o, at least on par with SSD. It is
> your virtualization I am worried about.

I will test that when I get back :)

>=20
> Have a safe trip to down under!

will do, thanks for the response. :)

>=20
> /M
>=20
> On 24/07/2012, at 21:56, steve wrote:
>=20
>> Hi Michael,
>>=20
>> from what I have noticed, bitcoin blockchain
>> download/verfication all happens in 1 thread.  (so multicores
>> doesnt really help)
>>=20
>> That said, I have never tried on an ssd.
>>=20
>> What I do have is 6 SATA 6gbs configed as RAID0 Drives. 32gb of=20
>> ram. ubuntu 64 (yeah I know), this runs upto 16 VM's (I have 4
>> of these)
>>=20
>> However I have not tried to download the blockchain on the
>> master os, just in virtulisation.  However, the dedicated
>> machines that I have been using for benchmarking the VM's against
>> is a q6600 8gb ram sata2 hdd - Win 7 (seems faster than
>> slackware...) to me it has always felt like network bandwidth was
>> the issue.  I might instrument the bitcoin-qt exe to only pick
>> low ping nodes (has someone already done this?)
>>=20
>> I guess it is time to start some benchmarking (like the gpu=20
>> comparison page)
>>=20
>> hte verification for the 5 past 5 days was negliglable. I am off
>> on a flight to australia tomorrrow, so I will set some
>> breakpoints and do some timings in a debugger.
>>=20
>> This will all happen on an e-450 (wonderful machine!)
>>=20
>> Thanks very much for your response. it would seem that I am
>> 'doing it wrong' :/
>>=20
>> cheers mate,
>>=20
>> steve
>>=20
>> (this message isnt signed because I have forgotten my password.)
>>=20
>> On 24/07/2012 09:25, Michael Gr=F8nager wrote:
>>> Hi Steve,
>>>=20
>>> 45-90 minutes - note that its numbers from March/April, so a
>>> bit longer today, but far, far away from the 12 hours.
>>>=20
>>> I am using libcoin and the bitcoind build based on this.
>>> Libcoin is based on the Satoshi client, but refactured to use
>>> an async concurrency model. I also did a minor tweeks to the
>>> db parameters. It has earlier been tested up against Satoshi
>>> bitcoin where on some OS'es it performs similarly (at least on
>>> some linuxes) and on some faster (e.g. mac).
>>>=20
>>> What is your CPU load during a block download ? (both=20
>>> initially/up to the point where verification sets in and
>>> after). The initial download is typically disk I/O bound, the=20
>>> verification stage CPU bound, though I lean to believe that
>>> even there it is disk I/O bound (at least on my system ~50% CPU
>>> load). What should be better in libcoin is the concurrency
>>> model. The Satoshi client uses a pure reentrant mutexes model,
>>> that is not generally believed to motivate the best coding
>>> practice nor performance, you might end up without the
>>> concurrency you initially strived for *). As mentioned earlier
>>> libcoin uses a pure async concurrency model (and so does
>>> libbitcoin btw).
>>>=20
>>> I would like to stress again that these numbers will depend=20
>>> largely on the system running the test - I would call my laptop
>>> a bit over the average today (MB Pro, 2.66Ghz i7 dual core,
>>> 8GBRAM, 512GB SSD). But again 12 hours - I only reach such
>>> numbers on some of my VPS'es (linode 1024) that are known for
>>> notoriously slow disk I/O. (here I have a few % CPU load during
>>> the verification indicating indeed that the disk i/o is the=20
>>> culprit).
>>>=20
>>> Cheers,
>>>=20
>>> Michael
>>>=20
>>>=20
>>> *) I like this Dave Butenhof quote: "The biggest of all the big
>>>  problems with recursive mutexes is that they encourage you to
>>>  completely lose track of your locking scheme and scope. This
>>> is deadly. Evil. It's the "thread eater". You hold locks for
>>> the absolutely shortest possible time. Period. Always. If
>>> you're calling something with a lock held simply because you
>>> don't know it's held, or because you don't know whether the
>>> callee needs the mutex, then you're holding it too long. You're
>>> aiming a shotgun at your application and pulling the trigger.
>>> You presumably started using threads to get concurrency; but
>>> you've just PREVENTED concurrency."
>>>=20
>>>=20
>>>=20
>>>=20
>>> On 23/07/2012, at 17:54, steve wrote:
>>>=20
>>> Hi Michael,
>>>=20
>>> On 23/07/2012 10:00, Michael Gr=F8nager wrote:
>>>>>> I get a full blockchain from scratch in 45 minutes on my
>>>>>>  laptop, /M
>>>>>>=20
>>> Hang on a sec, in 45 minutes you can download the entire chain=20
>>> from the genesis block?
>>>=20
>>> I have been doing extensive testing in this area and would
>>> love to know what is special about your setup (I have never had
>>> the entire chain in under 12 hours, infact it is normally
>>> closerto 24.) I have an extensive setup of test machines,
>>> everything from e4300 to phenom2x6 to i5's.
>>>=20
>>> as an example on an amd e-450 with 4gb ram, and approx 3gb/s=20
>>> internet connection it took 2 hours to sync the last 5 days.
>>>=20
>>> Maybe i am missing something important...
>>>=20
>>> Any additional information that you could provide to help me
>>> with testing would be really appreciated.
>>>=20
>>> cheers,
>>>=20
>>> steve
>>>=20
>>>>=20
>>>> --------------------------------------------------------------------=
----------
>>>>
>>>>
>>
>>>>
>>>>=20
Live Security Virtual Conference
>>>> Exclusive live event will cover all the ways today's
>>>> security and threat landscape has changed and how IT managers
>>>> can respond. Discussions will include endpoint security,
>>>> mobile security and the latest in malware threats.=20
>>>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/=20
>>>> _______________________________________________=20
>>>> Bitcoin-development mailing list=20
>>>> Bitcoin-development@lists.sourceforge.net=20
>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>>>
>>>>=20
-------------------------------------------------------------------------=
-----
>> Live Security Virtual Conference Exclusive live event will cover=20
>> all the ways today's security and threat landscape has changed
>> and how IT managers can respond. Discussions will include
>> endpoint security, mobile security and the latest in malware
>> threats.=20
>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/=20
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net=20
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>=20