Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UWhIm-0003Zb-E6 for bitcoin-development@lists.sourceforge.net; Mon, 29 Apr 2013 06:11:04 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of outlook.com designates 65.55.116.41 as permitted sender) client-ip=65.55.116.41; envelope-from=jayf@outlook.com; helo=blu0-omc1-s30.blu0.hotmail.com; Received: from blu0-omc1-s30.blu0.hotmail.com ([65.55.116.41]) by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1UWhIl-0003Ad-6L for bitcoin-development@lists.sourceforge.net; Mon, 29 Apr 2013 06:11:04 +0000 Received: from BLU0-SMTP478 ([65.55.116.7]) by blu0-omc1-s30.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 28 Apr 2013 23:10:57 -0700 X-EIP: [fhJ4/L9TMDh9xo3afJ3FqehVakN5zZW0] X-Originating-Email: [jayf@outlook.com] Message-ID: Received: from [192.168.1.15] ([67.189.14.219]) by BLU0-SMTP478.phx.gbl over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Sun, 28 Apr 2013 23:10:56 -0700 Date: Sun, 28 Apr 2013 23:10:54 -0700 From: Jay F User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Peter Todd References: <20130429035523.GA11611@savin> In-Reply-To: <20130429035523.GA11611@savin> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 29 Apr 2013 06:10:56.0539 (UTC) FILETIME=[4FD8D6B0:01CE44A0] X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (jayf[at]outlook.com) -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [65.55.116.41 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay X-Headers-End: 1UWhIl-0003Ad-6L Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Service bits for pruned nodes X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 06:11:04 -0000 On 4/28/2013 8:55 PM, Peter Todd wrote: > On Mon, Apr 29, 2013 at 03:48:18AM +0000, John Dillon wrote: >> We can build this stuff incrementally I'll agree. It won't be the case that one >> in a thousand nodes serve up the part of the chain you need overnight. So many >> I am over engineering the solution with BitTorrent. > > I think that pretty much sums it up. > > With the block-range served in the anounce message you just need to find > an annoucement with the right range, and at worst connect to a few more > node to get what you need. One of the technologies that can be borrowed from Bittorrent (besides downloading from multiple peers at once) is analysis by clients of the part distribution, which allows a client to download and share the least-propagated parts first to maintain high availability of the whole file, even when not one individual currently has downloaded the complete file (the seed has left the swarm). Unlike Bittorrent, a partial-blockchain swarm client needs to make informed decisions about how much to download, such as rules like "until it sees at least 20 complete blockchain-equivalents in the swarm", "until it has 10% of the blockchain itself", "work backwards, all blocks from the hash tree required to verify my payments" or other minimums that might all be criteria. Bittorrent only considers directly connected peers' piecemaps when making decisions of what to download. Bitcoin, however, already has a protocol to allow peer discovery beyond the connected nodes; this could be extended to communicate what parts the peer is hosting. Careful thought into attack vectors would need to be paid in design, so that only a majority of outbound-connected peers's advertisement are able to inform consensus about part or peer availability, messages able to remove a peer or part availability from other's lists are confirmed independently without such removal verification generating DDOS traffic amplification, lying clients can be marked as discovered by the majority, etc. Such thought doesn't have to be paid if directly implementing Bittorrent, but it has the burden of centralized trackers or expensive DHT, and it also doesn't have any logic informing it besides "don't quit until I get the whole file".