diff options
author | Matthew Mitchell <matthewmitchell@godofgod.co.uk> | 2012-09-13 21:24:35 +0100 |
---|---|---|
committer | bitcoindev <bitcoindev@gnusha.org> | 2012-09-13 20:24:47 +0000 |
commit | afd8080cbfebb36192950a2f125bc6dd3a97f44f (patch) | |
tree | a527337b5f923d8f029f816de2ffdffc8449047a /ff | |
parent | 1c108c6775b6c58e1ac398cbfd471181548d6eba (diff) | |
download | pi-bitcoindev-afd8080cbfebb36192950a2f125bc6dd3a97f44f.tar.gz pi-bitcoindev-afd8080cbfebb36192950a2f125bc6dd3a97f44f.zip |
Re: [Bitcoin-development] Segmented Block Relaying BIP draft.
Diffstat (limited to 'ff')
-rw-r--r-- | ff/3933b454dcd00b5c4bc08964b5e046daebcfa4 | 165 |
1 files changed, 165 insertions, 0 deletions
diff --git a/ff/3933b454dcd00b5c4bc08964b5e046daebcfa4 b/ff/3933b454dcd00b5c4bc08964b5e046daebcfa4 new file mode 100644 index 000000000..0d522da3d --- /dev/null +++ b/ff/3933b454dcd00b5c4bc08964b5e046daebcfa4 @@ -0,0 +1,165 @@ +Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] + helo=mx.sourceforge.net) + by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) + (envelope-from + <SRS0=nh6BJu=HM=godofgod.co.uk=matthewmitchell@eigbox.net>) + id 1TCFxv-00085p-Ji for bitcoin-development@lists.sourceforge.net; + Thu, 13 Sep 2012 20:24:47 +0000 +Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of eigbox.net + designates 66.96.188.16 as permitted sender) + client-ip=66.96.188.16; + envelope-from=SRS0=nh6BJu=HM=godofgod.co.uk=matthewmitchell@eigbox.net; + helo=bosmailout16.eigbox.net; +Received: from bosmailout16.eigbox.net ([66.96.188.16]) + by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76) + id 1TCFxu-0007zf-Px for bitcoin-development@lists.sourceforge.net; + Thu, 13 Sep 2012 20:24:47 +0000 +Received: from bosmailscan20.eigbox.net ([10.20.15.20]) + by bosmailout16.eigbox.net with esmtp (Exim) id 1TCFxp-0008Jb-Lv + for bitcoin-development@lists.sourceforge.net; + Thu, 13 Sep 2012 16:24:41 -0400 +Received: from bosimpout02.eigbox.net ([10.20.55.2]) + by bosmailscan20.eigbox.net with esmtp (Exim) + id 1TCFxo-0002en-US; Thu, 13 Sep 2012 16:24:40 -0400 +Received: from bosauthsmtp06.eigbox.net ([10.20.18.6]) + by bosimpout02.eigbox.net with NO UCE + id ykQg1j00C07rX7u01kQgwH; Thu, 13 Sep 2012 16:24:41 -0400 +X-Authority-Analysis: v=2.0 cv=V8rKJ5bi c=1 sm=1 + a=Vnyysazsc9gF4v5jbSvB8A==:17 a=Goz4v7xpImgA:10 a=JhU9KSwl1l8A:10 + a=RmqW3wxksLsA:10 a=N659UExz7-8A:10 a=eGitJVp2AAAA:8 a=-aUnnjAAwhoA:10 + a=pGLkceISAAAA:8 a=FYJQRDSw8x8RavvxkmoA:9 a=pILNOxqGKmIA:10 + a=MSl-tDqOz04A:10 + a=auv_4ZWqOX5hJEMy:21 a=i7WL7bnb9U27SoyJ:21 + a=fIc3/5IyPUehxkj7BpkQ7Q==:117 +X-EN-OrigOutIP: 10.20.18.6 +X-EN-IMPSID: ykQg1j00C07rX7u01kQgwH +Received: from [109.175.173.27] + by bosauthsmtp06.eigbox.net with esmtpsa (TLSv1:AES128-SHA:128) + (Exim) id 1TCFxo-0004tV-9S; Thu, 13 Sep 2012 16:24:40 -0400 +Content-Type: text/plain; charset=windows-1252 +Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) +From: Matthew Mitchell <matthewmitchell@godofgod.co.uk> +In-Reply-To: <20120913185900.GA393@vps7135.xlshosting.net> +Date: Thu, 13 Sep 2012 21:24:35 +0100 +Content-Transfer-Encoding: quoted-printable +Message-Id: <9F7EFD92-B922-45E9-97A8-03FFC811502D@godofgod.co.uk> +References: <2104FC7F-0AE0-4C55-9987-B20EFCE19FC3@godofgod.co.uk> + <CAAS2fgSymOJ=cQnNwK9+nvRYszHHH4vtUpoQYWNNYoVaYB5Gpg@mail.gmail.com> + <19ED4257-0BCA-41C5-A533-B0AB9B500181@godofgod.co.uk> + <CAAS2fgRfXBrsFm_zdNFe7Z4FN7uQ5zSg++scng=hNrHZZV93VQ@mail.gmail.com> + <CANEZrP0HHhSXyN9pWODtTxHMfRB4B0w-eSdvNHH13WwLVgECrw@mail.gmail.com> + <FC0AF926-2CBD-49BA-A3B7-AF9D70A83CE4@godofgod.co.uk> + <CAAS2fgSd6t038Yzb-Vy34J61xob+kVqA8pK+w1+ZwJ6XtQRJww@mail.gmail.com> + <2B95CF41-4304-4D2A-9ABF-198D97B7449B@godofgod.co.uk> + <CAAS2fgQi8QFwU2M=wLiDodt3SmO48vUV5Sp3YCb1OmGJ5m=E7Q@mail.gmail.com> + <A1DC7DE8-F355-4B61-AF18-94F07DF6421E@godofgod.co.uk> + <20120913185900.GA393@vps7135.xlshosting.net> +To: Pieter Wuille <pieter.wuille@gmail.com>, + "bitcoin-development@lists.sourceforge.net" + <bitcoin-development@lists.sourceforge.net> +X-Mailer: Apple Mail (2.1486) +X-EN-UserInfo: c68a83c59c94ef03b40bb4bc312c51e4:dffc0a9b4c8a0435ad832ff5852cab82 +X-EN-AuthUser: godofgod@godofgod.co.uk +Sender: Matthew Mitchell <matthewmitchell@godofgod.co.uk> +X-EN-OrigIP: 109.175.173.27 +X-EN-OrigHost: unknown +X-Spam-Score: -1.3 (-) +X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. + See http://spamassassin.org/tag/ for more details. + -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for + sender-domain + -0.0 SPF_PASS SPF: sender matches SPF record + -0.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay + domain 0.7 AWL AWL: From: address is in the auto white-list +X-Headers-End: 1TCFxu-0007zf-Px +Subject: Re: [Bitcoin-development] Segmented Block Relaying BIP draft. +X-BeenThere: bitcoin-development@lists.sourceforge.net +X-Mailman-Version: 2.1.9 +Precedence: list +List-Id: <bitcoin-development.lists.sourceforge.net> +List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, + <mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe> +List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development> +List-Post: <mailto:bitcoin-development@lists.sourceforge.net> +List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help> +List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, + <mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe> +X-List-Received-Date: Thu, 13 Sep 2012 20:24:47 -0000 + + +On 13 Sep 2012, at 19:59, Pieter Wuille <pieter.wuille@gmail.com> wrote: + +> You want to parallellize block downloads, while at the same time = +preventing re-download of transactions that are already known. +> To do so, a requesting node would first request (for example) the 8 = +level-3 hashes, then start 8 parallel threads to download the +> transactions from presumably 8 different peers. Each thread then = +fetches the transaction id's that correspond to its own 1/8th of +> the block, and requests the transactions whose txid is not yet known. +> Comparing this with Gregory's own suggestion (just fetch the entire = +txid list at first, and then (again as parallellized as needed) +> fetch the unknown transactions from several peers), this does indeed = +have an advantage: you need to download (relatively) far less +> data before the threaded part can start. If this is what you propose, = +it is certainly meaningful, but the gains aren't very large, +> in my opinion. + +This is not fully correct. I propose downloading the roots of the = +segments when you are not looking to remove redundant transaction = +downloads. This would be the case when the node is not up-to-date and is = +unlikely to have transactions in the required blocks. When a node is = +up-to-date and can benefit from removing redundant transaction downloads = +it can switch to downloading all the transactions hashes by specifying a = +high level number. Also I wouldn't recommend using one thread per = +connection, I'd recommend using one thread for all connections. + +> However, my impression while reading your proposal was at times that = +you intend to process the different layers of the +> Merkle tree iteratively after starting the initial parallel segments. = +I don't think that is useful, as you'll need the actual +> txids anyway before deciding whether they need to be downloaded at = +all, it adds several round-trips, and once you have downloaded +> the intermediate merkle hashes for about 8 levels, you've already = +transferred more data than the transactions themselves. I think +> Gregory also assumed something like this, making him question why it's = +useful. Anyway, this whole paragraph is one assumption, so +> if it's not the case, ignore me. + +This isn't what I was suggesting. I was suggesting you only need to = +download one level. Once you have done that you verify everything for = +the hashes on that level. + +>=20 +> Can you clarify what you mean? Preferably by giving a concrete = +sequence of exchanged messages, with a given purpose? + +Well you will need to ask for the headers first. Once you do that you = +can start downloading the full blocks for the headers. The node should = +use "get blocks" to find nodes with segments of the blocks they need. = +Now for each block: + +1. Send getsiginv to a number of peers to know the segments of the = +blocks they have.=20 +2. Send gettreelevel requesting a level of the merkle tree from a peer = +that can provide it. When up-to-date use a high level to get the = +transaction hashes to find redundant data. +3. Validate the treelevel response +4. Send getsegment for each segment wanted (at the same time where = +possible) to the peers that have these segments. Skip transactions = +already known. +5. Validate the transactions in each segment received. Stop if the block = +is invalid and disconnect peers that give transactions which do not fit = +the merkle tree. +6. Revert to getdata if peers using the protocol cannot satisfy the = +block download. + +When a valid block segment is received, include the block in inv and = +headers messages for other peers using the protocol. Thus relaying can = +begin before the entire block is downloaded. + +I'm thinking about improvements to this proposal. I'll get to that = +tomorrow perhaps=85 + +Thank you everyone for the replies.= + + |