Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1StRBz-0006Yx-1V for bitcoin-development@lists.sourceforge.net; Mon, 23 Jul 2012 22:33:31 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.175 as permitted sender) client-ip=209.85.214.175; envelope-from=grarpamp@gmail.com; helo=mail-ob0-f175.google.com; Received: from mail-ob0-f175.google.com ([209.85.214.175]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1StRBy-00037f-Cr for bitcoin-development@lists.sourceforge.net; Mon, 23 Jul 2012 22:33:30 +0000 Received: by obcva7 with SMTP id va7so10844509obc.34 for ; Mon, 23 Jul 2012 15:33:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.1.40 with SMTP id 8mr23060860oej.70.1343082804750; Mon, 23 Jul 2012 15:33:24 -0700 (PDT) Received: by 10.76.81.10 with HTTP; Mon, 23 Jul 2012 15:33:24 -0700 (PDT) In-Reply-To: References: Date: Mon, 23 Jul 2012 18:33:24 -0400 Message-ID: From: grarpamp To: bitcoin-development@lists.sourceforge.net Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.0 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (grarpamp[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.6 AWL AWL: From: address is in the auto white-list X-Headers-End: 1StRBy-00037f-Cr Subject: Re: [Bitcoin-development] Scalability issues X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2012 22:33:31 -0000 > You're seriously suggesting that I'm using a system > which is 720x (one month vs one hour) faster than your > P4 1.8GHz? Don't know what you're using since you've not stated it. > I find this doubtful, especially since bitcoin's sync is effectively > single threaded. Extra cores help with disk, crypto, net, etc... > month I've spent about two weeks crunching about the last month's worth of new blocks. > Your results are not expected and are not believed to be > representative. The config is reproducible, and not believed to be uncommon. > try to isolate it Mostly disk and crypto. Shall everyone instead run in bitrot and no-privacy mode? What do we do when we've got 10k trans a day coming in? 50k? 100k, 1M? When the chain gets 1M, 50M, 500M, 1B long? Forget my swamped box, these numbers are coming to others. > try to sync from a local node into tmpfs I'd bet some people using 'tmpfs' probably have it unknowingly [fall]backed to swap instead of core. Bitcoin already takes up 3GiB of disk, how many have that much free RAM? How many have 30GiB, 300GiB? > If you're building against BDB later than the recommended 4.8 > be aware that there have been performance regressions with later > versions Yes, I left out that bit of platform so here are the remaining bits... db4830, boost149, vm.kmem_size=650000000 I'm not bashing anything or anybody, just detailing a stock config that is underwater. Anybody wishing to verify can get the hardware from their junk pile and the software from freebsd.org. I'll certainly be looking at both it and different setups too. If anyone's using say Linux/BSD, BTRFS/ZFS, crypto, on i386/amd64, they could chime in with their times too. Disk is the cheapest, easiest thing for Joe to get. Think about indexing and checkpointing into said disk. I don't know what bitcoin's doing, but if it's verifying every transaction back to the root, that would seem a bit ridiculous. Joe probably won't be happy buying TiB's for bitcoind, so after that's filled (assuming there's CPU to do it), the trust model has to change. These scales are coming...