Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A122CAB6 for ; Thu, 20 Apr 2017 11:27:38 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f177.google.com (mail-wr0-f177.google.com [209.85.128.177]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8D5F18D for ; Thu, 20 Apr 2017 11:27:36 +0000 (UTC) Received: by mail-wr0-f177.google.com with SMTP id w50so10217757wrc.0 for ; Thu, 20 Apr 2017 04:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to; bh=BRex3o/ZTc4ydf7x6303HXCc5sUbDLWKcZkt2D4jWl4=; b=lftyw9XqegOojXNkWD/UzAKld9lTR4VU70glXPbLm41SDfJIDAyur4BPIpUtqzKuzH giR5If8FU4nSni/9KCLK3zFzmzNQArhB8KatDnBVUtC7I7TrtooFJG6GwJ5l7zTc/M2n 1L8JA/Tv83XLzicjJilszXgvnZFIiJsUiNQ2w5IBNHbD9bf2z+7wimuMT2vnjF2VF0++ IcaP0yR+xZKzLI2f08CCBwfTYn9BW4l2ZVJwLezqcZHkHCs1y9GrtfNQ88o7e5fK2u0j tzmelTIYhVBY1yxZmA00RTjUX63PNq+ZeNz2yrPPUqwYeO7plvcQOvBalAqIguWhWkDU wtmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to; bh=BRex3o/ZTc4ydf7x6303HXCc5sUbDLWKcZkt2D4jWl4=; b=eAmqlVogSxz0ZrToqaEe61lojmdGkA3wnczTSsRaW/Nfllafl9sFWHzSFYD2d97tpl kce743GVhVM+nTOQ+qGE5FT27ntXkb7oJLdGbifQrcLh8Um45I/6wtZ1GWyg7g1DWLX/ BF7hcXV+HtuwZbqo6aKUGSzIuIdcOBjR1Mbp/5jHHLbAePLZZq1iTyz4qV12a4P3FlFo M3pRg0a78cvbQnirlS9Z7tA7xS4bwy8IILDigHH/6pGNdK/jqvTBx8dvMmTfOFKPsljn d5fSjWIscFHZkbUx4ym68AELNv+9Cp7gaf2hFk8wBulIcovafWhPAcpcjkQeiBpUAJK4 1YMg== X-Gm-Message-State: AN3rC/56uy1xhNpoCN3xSNUCSRdRBZR/WnrpKYjGfVG8/MhFerbIzdD5 CcY9tA8ZYJ+dxg== X-Received: by 10.223.164.221 with SMTP id h29mr7936395wrb.102.1492687655078; Thu, 20 Apr 2017 04:27:35 -0700 (PDT) Received: from [192.168.1.10] (ANice-654-1-153-20.w86-205.abo.wanadoo.fr. [86.205.80.20]) by smtp.googlemail.com with ESMTPSA id c5sm7212028wre.60.2017.04.20.04.27.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Apr 2017 04:27:34 -0700 (PDT) From: Aymeric Vitte To: David Vorick , Bitcoin Dev References: <19dbfef2-3791-8fe7-1c00-c4052c3d6c45@gmail.com> Message-ID: <6a8646c6-57b2-9df5-6c4d-d130cca6d454@gmail.com> Date: Thu, 20 Apr 2017 13:27:32 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.3; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------DE10D9F44A6B2AA579AE01A3" X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Small Nodes: A Better Alternative to Pruned Nodes X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 11:27:38 -0000 This is a multi-part message in MIME format. --------------DE10D9F44A6B2AA579AE01A3 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Thanks but you did not answer all the points and some of your statements look wrong, like the global ideas behind this proposal from my standpoint, which basically is inventing strange things not reusing what is already proven to be working well and could provide the same result, which at the end is not the expected one, ie increasing full nodes, it sounds like a strange workaround to prevent the centralization of the blockchain when pruning will become the default To answer some other comments in this thread, giving an incentive to run full nodes does not mean that someone setting up tomorrow 10K nodes will become rich and/or will be able to control the network, the later being not unlikely at all to happen in the current situation, the idea is more to motivate people that already have the resources to run full nodes, then we mix the concepts of optimizing the resources at no additional costs (and even decreasing costs since you get rewarded for the part that you have already paid but don't use) with the one of running nodes to protect its business For example https://gist.github.com/Ayms/aab6f8e08fef0792ab3448f542a826bf#proposal is showing some concepts where nodes can't position themselves where they like and are registered in the system by the others (but forget the proof of something as written in this gist since I think the rewards should not depend on usual miners) , so it becomes quite difficult that they position themselves where they like to possibly get the rewards, fake the system, freeride, cheat, collude in pools or setup plenty of nodes Comments below Le 19/04/2017 à 19:30, David Vorick via bitcoin-dev a écrit : > On Tue, Apr 18, 2017 at 3:43 AM, Jonas Schnelli > wrote: > > I know many torrent clients, and clients for protocols like Tor > and i2p include the ability to set both speed limits and monthly > bandwidth limits. > Yes, that's the easy part, the issue is more for the network to check that users have sufficient bandwidth and don't cheat > I worry about any type of CDN being a central point of failure. Of course > Torrenting typically relies on a DHT, which is much easier to attack > than Bitcoin's peer network. Then please explain how you would attack the bittorrent DHT and why it's "much easier" than attacking the btc network today, bittorrent is not designed for security/privacy, including its DHT, which btw is great, it's a common sign of misinformation to conclude that all DHTs are necessarily insecure > I think the bitcoin p2p network is by significant margin the best > we've got. The btc network can't be considered as a p2p network in its current form then can't be the best one for now (and if it was then we would not be in today's situation) > > Yes, there is finger-print that happens if you have nodes pick an > index. And the fingerprint gets a lot worse if you have a node pick > multiple indexes. This is another problem of your proposal, as well as fingerprinting/tracking peers based on what they have > Though, isn't it already required that nodes have some sort of IP > address or hidden service domain? I want to say that the fingerprint > created by picking an index is not a big deal, because it can be > separated from activity like transaction relaying and mining. Though, > I am not certain and perhaps it is a problem. Are you suggesting that the btc "p2p" network should be using the Tor network, and especially the nodes hosting the/(a part of) the blockchain? This is of course a very bad idea and you would not eliminate the tracking issue, a simple example is that despite ot the size of the network it's not difficult to track the peers on the bittorrent network, you might not know who is the peer but you can follow whatever he is doing, and hidding behind Tor or a VPN does not prevent this > > > > On Mon, Apr 17, 2017 at 6:14 AM, Aymeric Vitte via bitcoin-dev > > wrote: > > While I fully agree with the intent (increasing full nodes so a > big miner waking up in a bad mood can't threaten the world any > longer every day as it is now) I am not sure to get the interest > of this proposal, because: > > - it's probably not a good idea to encourage the home users to run > full nodes, there are many people running servers far from their > capacity that could easily run efficient full nodes > > Running a full node is the only way to avoid needing to trust others. > It's also how you make your opinion worthwhile for events like hard > forks and UASFs. If decentralization is the primary motivation, it > absolutely makes sense to encourage people to run their own full > nodes. Without a full node, you are at the mercy of the governance > decisions by those who do have full nodes. But if you have a full > node, you can chose to opt-out of any upgrade (example: ethereum > classic nodes). If you really know the Tor network, then you know why encouraging home users to run full nodes is probably not a good idea "Probably" because the situation is not the same for btc and indeed UASF for example is referring to "users" who today are not really "users" but intermediate nodes, so the decision finally is not made by the users > > > - if someone can't allocate 100 GB today to run a full node, then > we can't expect him to allocate more in the future > > That's why I'm proposing something to decrease the storage requirements. This is just delaying the problem, you are just proposing to store some parts of the blockchain not explaining how the peers will first setup the nodes, some parts that will of course increase... then falling back in the issue that you are trying to address > > > - this proposal is a kind of reinventing torrents, while limiting > the number of connections to something not efficient at all, I > don't see why something that is proven to be super efficient > (torrents) would be needed to be reinvented, I am not saying that > it should be used as the bittorrent network is doing but the > concepts can be reused > > It's different from torrents in that it uses specialized erasure > coding to make sure that every block is always available, even if an > adversary is running around targeting all the nodes with a particular > piece. You are reinventing something that would be achieved easily using the concepts of torrents or incremental ones (ie someone would seed the whole thing, some others some parts of it, etc) > > > - I don't get at all the concept of "archival" nodes since it's > another useless step toward centralization > > "archival" nodes are simply nodes with the full blockchain. Nobody can > bootstrap on the network without them. Today, every bitcoin-core node > is an archival node by default. What I meant is that you can't build a hierarchy of btc nodes: big nodes, medium nodes, small nodes, each node is free to decide how/if it participates to the network, so the wording of "archival" nodes for me is not adapted since it makes immediately think to centralized entities, big organizations hosting the blockchain, etc > I think the only way to increase full nodes it to design an > incentive for people to run them > > The primary incentive is the sovereignty that it gives you. Running a > Bitcoin full node gives you security and protection against political > garbage that you can't get any other way. The network does currently > depend on altruism to allow people to download the blockchain, but as > long as we can keep the resource requirements of this altruism low, I > think we can expect it to continue. This proposal attempts to keep > those requirements low. This is the usual answer but I don't believe it, people will rely on others to run full nodes and secure them, and so on... > > > > I believe that my proposal does meet all of the requirements listed by > Maxwell. I did noy read them again, some others are listed in the link above > Having a set of 8 random peers gives you a very high probability of > being able to recover every single block. You would need to connect to > at least 5 peers (and this is already >90% likely to be sufficient to > recover every block), but if you cannot connect to 5 random peers your > node is probably in trouble anyway. Highly parallel, high speed > downloads are just as possible with small nodes as with archive nodes. > It only takes a few bytes to indicate which part of the blockchain you > have, and any 2 peers have a less than 1% chance of overlapping. Again, you don't explain how you bootstrap the full nodes first, which is the main issue, and if the idea is that the pruning nodes will never desync then you should try downloading x GB to resync connecting to 5/8 peers possibly operating from home in different countries -- Zcash wallets made simple: https://github.com/Ayms/zcash-wallets Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets Get the torrent dynamic blocklist: http://peersm.com/getblocklist Check the 10 M passwords list: http://peersm.com/findmyass Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org Peersm : http://www.peersm.com torrent-live: https://github.com/Ayms/torrent-live node-Tor : https://www.github.com/Ayms/node-Tor GitHub : https://www.github.com/Ayms --------------DE10D9F44A6B2AA579AE01A3 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit

Thanks but you did not answer all the points and some of your statements look wrong, like the global ideas behind this proposal from my standpoint, which basically is inventing strange things not reusing what is already proven to be working well and could provide the same result, which at the end is not the expected one, ie increasing full nodes, it sounds like a strange workaround to prevent the centralization of the blockchain when pruning will become the default

To answer some other comments in this thread, giving an incentive to run full nodes does not mean that someone setting up tomorrow 10K nodes will become rich and/or will be able to control the network, the later being not unlikely at all to happen in the current situation, the idea is more to motivate people that already have the resources to run full nodes, then we mix the concepts of optimizing the resources at no additional costs (and even decreasing costs since you get rewarded for the part that you have already paid but don't use) with the one of running nodes to protect its business

For example https://gist.github.com/Ayms/aab6f8e08fef0792ab3448f542a826bf#proposal is showing some concepts where nodes can't position themselves where they like and are registered in the system by the others (but forget the proof of something as written in this gist since I think the rewards should not depend on usual miners) , so it becomes quite difficult that they position themselves where they like to possibly get the rewards, fake the system, freeride, cheat, collude in pools or setup plenty of nodes

Comments below


Le 19/04/2017 à 19:30, David Vorick via bitcoin-dev a écrit :
On Tue, Apr 18, 2017 at 3:43 AM, Jonas Schnelli <dev@jonasschnelli.ch> wrote:
I know many torrent clients, and clients for protocols like Tor and i2p include the ability to set both speed limits and monthly bandwidth limits.

Yes, that's the easy part, the issue is more for the network to check that users have sufficient bandwidth and don't cheat

I worry about any type of CDN being a central point of failure.

Of course

 Torrenting typically relies on a DHT, which is much easier to attack than Bitcoin's peer network.

Then please explain how you would attack the bittorrent DHT and why it's "much easier" than attacking the btc network today, bittorrent is not designed for security/privacy, including its DHT, which btw is great, it's a common sign of misinformation to conclude that all DHTs are necessarily insecure

I think the bitcoin p2p network is by significant margin the best we've got.

The btc network can't be considered as a p2p network in its current form then can't be the best one for now (and if it was then we would not be in today's situation)


Yes, there is finger-print that happens if you have nodes pick an index. And the fingerprint gets a lot worse if you have a node pick multiple indexes.

This is another problem of your proposal, as well as fingerprinting/tracking peers based on what they have

Though, isn't it already required that nodes have some sort of IP address or hidden service domain? I want to say that the fingerprint created by picking an index is not a big deal, because it can be separated from activity like transaction relaying and mining. Though, I am not certain and perhaps it is a problem.

Are you suggesting that the btc "p2p" network should be using the Tor network, and especially the nodes hosting the/(a part of) the blockchain? This is of course a very bad idea and you would not eliminate the tracking issue, a simple example is that despite ot the size of the network it's not difficult to track the peers on the bittorrent network, you might not know who is the peer but you can follow whatever he is doing, and hidding behind Tor or a VPN does not prevent this




On Mon, Apr 17, 2017 at 6:14 AM, Aymeric Vitte via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

While I fully agree with the intent (increasing full nodes so a big miner waking up in a bad mood can't threaten the world any longer every day as it is now) I am not sure to get the interest of this proposal, because:

- it's probably not a good idea to encourage the home users to run full nodes, there are many people running servers far from their capacity that could easily run efficient full nodes

Running a full node is the only way to avoid needing to trust others. It's also how you make your opinion worthwhile for events like hard forks and UASFs. If decentralization is the primary motivation, it absolutely makes sense to encourage people to run their own full nodes. Without a full node, you are at the mercy of the governance decisions by those who do have full nodes. But if you have a full node, you can chose to opt-out of any upgrade (example: ethereum classic nodes).

If you really know the Tor network, then you know why encouraging home users to run full nodes is probably not a good idea

"Probably" because the situation is not the same for btc and indeed UASF for example is referring to "users" who today are not really "users" but intermediate nodes, so the decision finally is not made by the users

 

- if someone can't allocate 100 GB today to run a full node, then we can't expect him to allocate more in the future

That's why I'm proposing something to decrease the storage requirements.

This is just delaying the problem, you are just proposing to store some parts of the blockchain not explaining how the peers will first setup the nodes, some parts that will of course increase... then falling back in the issue that you are trying to address

 

- this proposal is a kind of reinventing torrents, while limiting the number of connections to something not efficient at all, I don't see why something that is proven to be super efficient (torrents) would be needed to be reinvented, I am not saying that it should be used as the bittorrent network is doing but the concepts can be reused

It's different from torrents in that it uses specialized erasure coding to make sure that every block is always available, even if an adversary is running around targeting all the nodes with a particular piece.

You are reinventing something that would be achieved easily using the concepts of torrents or incremental ones (ie someone would seed the whole thing, some others some parts of it, etc)

 

- I don't get at all the concept of "archival" nodes since it's another useless step toward centralization

"archival" nodes are simply nodes with the full blockchain. Nobody can bootstrap on the network without them. Today, every bitcoin-core node is an archival node by default.

What I meant is that you can't build a hierarchy of btc nodes: big nodes, medium nodes, small nodes, each node is free to decide how/if it participates to the network, so the wording of "archival" nodes for me is not adapted since it makes immediately think to centralized entities, big organizations hosting the blockchain, etc

I think the only way to increase full nodes it to design an incentive for people to run them

The primary incentive is the sovereignty that it gives you. Running a Bitcoin full node gives you security and protection against political garbage that you can't get any other way. The network does currently depend on altruism to allow people to download the blockchain, but as long as we can keep the resource requirements of this altruism low, I think we can expect it to continue. This proposal attempts to keep those requirements low.

This is the usual answer but I don't believe it, people will rely on others to run full nodes and secure them, and so on...




I believe that my proposal does meet all of the requirements listed by Maxwell.

I did noy read them again, some others are listed in the link above

Having a set of 8 random peers gives you a very high probability of being able to recover every single block. You would need to connect to at least 5 peers (and this is already >90% likely to be sufficient to recover every block), but if you cannot connect to 5 random peers your node is probably in trouble anyway. Highly parallel, high speed downloads are just as possible with small nodes as with archive nodes. It only takes a few bytes to indicate which part of the blockchain you have, and any 2 peers have a less than 1% chance of overlapping.

Again, you don't explain how you bootstrap the full nodes first, which is the main issue, and if the idea is that the pruning nodes will never desync then you should try downloading x GB to resync connecting to 5/8 peers possibly operating from home in different countries

-- 
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms
--------------DE10D9F44A6B2AA579AE01A3--