Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1WXFYi-0004Jm-5Z for bitcoin-development@lists.sourceforge.net; Mon, 07 Apr 2014 19:50:20 +0000 X-ACL-Warn: Received: from wp059.webpack.hosteurope.de ([80.237.132.66]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1WXFYg-0001Ws-JP for bitcoin-development@lists.sourceforge.net; Mon, 07 Apr 2014 19:50:20 +0000 Received: from [37.143.74.116] (helo=[192.168.0.102]); authenticated by wp059.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) id 1WXFYa-0006W9-1S; Mon, 07 Apr 2014 21:50:12 +0200 Content-Type: multipart/signed; boundary="Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Tamas Blummer In-Reply-To: Date: Mon, 7 Apr 2014 21:50:26 +0200 Message-Id: References: , <5342C833.5030906@gmail.com>, , <6D430188-CE00-44B1-BD8C-B623CF04D466@icloudtools.net>, , <6D6E55CE-2F04-4C34-BEE6-98AEF1478346@bitsofproof.com>, , <8A6DEBA4-EA59-4BAE-95CF-C964C2DBB84B@bitsofproof.com>, , <8222EAFD-813E-4046-A751-FD3D04FF6764@bitsofproof.com> To: Paul Lyon X-Mailer: Apple Mail (2.1874) X-bounce-key: webpack.hosteurope.de; tamas@bitsofproof.com; 1396900218; 21e6c4e6; X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1WXFYg-0001Ws-JP Cc: Bitcoin Development Subject: Re: [Bitcoin-development] Why are we bleeding nodes? X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 19:50:20 -0000 --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511 Content-Type: multipart/alternative; boundary="Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C" --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 You have to load headers sequantially to be able to connect them and = determine the longest chain. Blocks can be loaded in random order once you have their order given by = the headers. Computing the UTXO however will force you to at least temporarily store = the blocks unless you have plenty of RAM.=20 Regards, Tamas Blummer http://bitsofproof.com On 07.04.2014, at 21:30, Paul Lyon wrote: > I hope I'm not thread-jacking here, apologies if so, but that's the = approach I've taken with the node I'm working on. >=20 > Headers can be downloaded and stored in any order, it'll make sense of = what the winning chain is. Blocks don't need to be downloaded in any = particular order and they don't need to be saved to disk, the UTXO is = fully self-contained. That way the concern of storing blocks for seeding = (or not) is wholly separated from syncing the UTXO. This allows me to do = the initial blockchain sync in ~6 hours when I use my SSD. I only need = enough disk space to store the UTXO, and then whatever amount of block = data the user would want to store for the health of the network. >=20 > This project is a bitcoin learning exercise for me, so I can only hope = I don't have any critical design flaws in there. :) >=20 > From: tamas@bitsofproof.com > Date: Mon, 7 Apr 2014 21:20:31 +0200 > To: gmaxwell@gmail.com > CC: bitcoin-development@lists.sourceforge.net > Subject: Re: [Bitcoin-development] Why are we bleeding nodes? >=20 >=20 > Once headers are loaded first there is no reason for sequential = loading.=20 >=20 > Validation has to be sequantial, but that step can be deferred until = the blocks before a point are loaded and continous. >=20 > Tamas Blummer > http://bitsofproof.com >=20 > On 07.04.2014, at 21:03, Gregory Maxwell wrote: >=20 >> On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer = wrote: >>> therefore I guess it is more handy to return some bitmap of = pruned/full >>> blocks than ranges. >>=20 >> A bitmap also means high overhead and=97 if it's used to advertise >> non-contiguous blocks=97 poor locality, since blocks are fetched >> sequentially. >>=20 >=20 >=20 > = --------------------------------------------------------------------------= ---- Put Bad Developers to Shame Dominate Development with Jenkins = Continuous Integration Continuously Automate Build, Test & Deployment = Start a new project now. Try Jenkins in the = cloud.http://p.sf.net/sfu/13600_Cloudbees > _______________________________________________ Bitcoin-development = mailing list = Bitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lis= ts/listinfo/bitcoin-development --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 You = have to load headers sequantially to be able to connect them and = determine the longest chain.

Blocks can be loaded in = random order once you have their order given by the = headers.
Computing the UTXO however will force you to at least = temporarily store the blocks unless you have plenty of = RAM. 

Regards,

Tamas = Blummer
http://bitsofproof.com

On 07.04.2014, at 21:30, Paul Lyon <pmlyon@hotmail.ca> = wrote:

I hope I'm not = thread-jacking here, apologies if so, but that's the approach I've taken = with the node I'm working on.

Headers can be = downloaded and stored in any order, it'll make sense of what the winning = chain is. Blocks don't need to be downloaded in any particular order and = they don't need to be saved to disk, the UTXO is fully self-contained. = That way the concern of storing blocks for seeding (or not) is wholly = separated from syncing the UTXO. This allows me to do the initial = blockchain sync in ~6 hours when I use my SSD. I only need enough disk = space to store the UTXO, and then whatever amount of block data the user = would want to store for the health of the = network.

This project is a bitcoin learning = exercise for me, so I can only hope I don't have any critical design = flaws in there. :)

From: tamas@bitsofproof.com
Date: = Mon, 7 Apr 2014 21:20:31 +0200
To: gmaxwell@gmail.com
CC: bitcoin-developm= ent@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Why = are we bleeding nodes?


Once headers are = loaded first there is no reason for sequential = loading. 

Validation has to be sequantial, = but that step can be deferred until the blocks before a point are loaded = and continous.

Tamas Blummer
http://bitsofproof.com<= /div>
On 07.04.2014, at 21:03, Gregory Maxwell <gmaxwell@gmail.com> = wrote:

On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer <tamas@bitsofproof.com> = wrote:
therefore I guess it is more handy = to return some bitmap of pruned/full
blocks than = ranges.

A bitmap also means high overhead and=97 if = it's used to advertise
non-contiguous blocks=97 poor locality, since = blocks are = fetched
sequentially.



---------------= --------------------------------------------------------------- Put Bad = Developers to Shame Dominate Development with Jenkins Continuous = Integration Continuously Automate Build, Test & Deployment Start a = new project now. Try Jenkins in the cloud.http://p.sf.net/sfu/13600_Clo= udbees
_______________________________________________ = Bitcoin-development mailing list Bitcoin-developm= ent@lists.sourceforge.net= https://lists.sourceforge.net/lists/listinfo/bitcoin-development
=

= --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C-- --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTQwGCAAoJEPZykcUXcTkcbFkIAIOtdjpl3mMiacF8/0PijDrn SsPCaWT4JIHlMlNvfawRY8Qn7obCCr8yN6QKo6yJg/+hNxQq14TNoEC6a4GHke0E LzOSGeHwpBv6f0cWRHDhO+pbQt6Y1gOsfH7Qld7fNej3KdEv/ZmT5Q2/dMIBzGCx aDZ1UQqYHq4VDylAE/rpLOyZxLQddqDMY/LxRQHvTv9PjJ5nm+Ivr3wwvd+sqW0G xj0p6J9qSqcBsM/4naVDcCybUDWR1DG/qi1foQP8mnlFPLkwDwARbAraqeUzESgS ECzMVh91CGWiTnB3G0Pemt4badisA5/hN7LqV7l41Mqf4FOXMSuF9PsRoJHjExM= =Zjro -----END PGP SIGNATURE----- --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511--