Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 89E00D61 for ; Thu, 11 Feb 2016 07:03:24 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from boosthardware.localdomain (boosthardware.com [88.198.122.139]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B108ED2 for ; Thu, 11 Feb 2016 07:03:22 +0000 (UTC) Received: by boosthardware.localdomain (Postfix, from userid 48) id D62504002D; Thu, 11 Feb 2016 08:03:18 +0100 (CET) Received: from 178.73.210.16 (SquirrelMail authenticated user pshirkey@boosthardware.com) by boosthardware.com with HTTP; Thu, 11 Feb 2016 18:03:18 +1100 (EST) Message-ID: <60215.178.73.210.16.1455174198.squirrel@boosthardware.com> In-Reply-To: <56BBA879.5030309@riseup.net> References: <56BA71D4.5040200@riseup.net> <56BBA879.5030309@riseup.net> Date: Thu, 11 Feb 2016 18:03:18 +1100 (EST) From: "Patrick Shirkey" To: bitcoin-dev@lists.linuxfoundation.org User-Agent: SquirrelMail/1.4.8-5.el5.centos.10 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 12 Feb 2016 15:19:05 +0000 Subject: Re: [bitcoin-dev] Clearing up some misconceptions about full nodes X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Feb 2016 07:03:24 -0000 On Thu, February 11, 2016 8:15 am, Chris Belcher via bitcoin-dev wrote: > I've been asked to post this to this mailing list too. It's time to > clear up some misconceptions floating around about full nodes. > > === Myth: There are only about 5500 full nodes worldwide === > > This number comes from this and similar sites: https://bitnodes.21.co/ > and it measured by trying to probe every nodes on their open ports. > > Problem is, not all nodes actually have open ports that can be probed. > Either because they are behind firewalls or because their users have > configured them to not listen for connections. > > Nobody knows how many full nodes there are, since many people don't know > how to forward ports behind a firewall, and bandwidth can be costly, its > quite likely that the number of nodes with closed ports is at least > another several thousand. > > Nodes with open ports are able to upload blocks to new full nodes. In > all other ways they are the same as nodes with closed ports. But because > open-port-nodes can be measured and closed-port-nodes cannot, some > members of the bitcoin community have been mistaken into believing that > open-port-nodes are that matters. > > === Myth: This number of nodes matters and/or is too low. === > > Nodes with open ports are useful to the bitcoin network because they > help bootstrap new nodes by uploading historical blocks, they are a > measure of bandwidth capacity. Right now there is no shortage of > bandwidth capacity, and if there was it could be easily added by renting > cloud servers. > > The problem is not bandwidth or connections, but trust, security and > privacy. Let me explain. > > Full nodes are able to check that all of bitcoin's rules are being > followed. Rules like following the inflation schedule, no double > spending, no spending of coins that don't belong to the holder of the > private key and all the other rules required to make bitcoin work (e.g. > difficulty) > > Full nodes are what make bitcoin trustless. No longer do you have to > trust a financial institution like a bank or paypal, you can simply run > software on your own computer. To put simply, the only node that matters > is the one you use. > > === Myth: There is no incentive to run nodes, the network relies on > altruism === > > It is very much in the individual bitcoin's users rational self interest > to run a full node and use it as their wallet. > > Using a full node as your wallet is the only way to know for sure that > none of bitcoin's rules have been broken. Rules like no coins were spent > not belonging to the owner, that no coins were spent twice, that no > inflation happens outside of the schedule and that all the rules needed > to make the system work are followed (e.g. difficulty.) All other kinds > of wallet involve trusting a third party server. > > All these checks done by full nodes also increase the security. There > are many attacks possible against lightweight wallets that do not affect > full node wallets. > > This is not just mindless paranoia, there have been real world examples > where full node users were unaffected by turmoil in the rest of the > bitcoin ecosystem. The 4th July 2015 accidental chain fork effected many > kinds of wallets. Here is the wiki page on this event > https://en.bitcoin.it/wiki/July_2015_chain_forks#Wallet_Advice > > Notice how updated node software was completely unaffected by the fork. > All other wallets required either extra confirmations or checking that > the third-party institution was running the correct version. > > Full nodes wallets are also currently the most private way to use > Bitcoin, with nobody else learning which bitcoin addresses belong to > you. All other lightweight wallets leak information about which > addresses are yours because they must query third-party servers. The > Electrum servers will know which addresses belong to you and can link > them together. Despite bloom filtering, lightweight wallets based on > BitcoinJ do not provide much privacy against nodes who connected > directly to the wallet or wiretappers. > > For many use cases, such privacy may not be required. But an important > reason to run a full node and use it as a wallet is to get the full > privacy benefits. > > === Myth: I can just set up a node on a cloud server instance and leave > it === > > To get the benefits of running a full node, you must use it as your > wallet, preferably on hardware you control. > > Most people who do this do not use a full node as their wallet. > Unfortunately because Bitcoin has a similar name to Bittorrent, some > people believe that upload capacity is the most important thing for a > healthy network. As I've explained above: bandwidth and connections are > not a problem today, trust, security and privacy are. > > === Myth: Running a full node is not recommended, most people should use > a lightweight client === > > This was common advice in 2012, but since then the full node software > has vastly improved in terms of user experience. > > If you cannot spare the disk space to store the blockchain, you can > enable pruning as in: > https://bitcoin.org/en/release/v0.11.0#block-file-pruning. In Bitcoin > Core 0.12, pruning being enabled will leave the wallet enabled. > Altogether this should require less than 1.5GB of hard disk space. > > If you cannot spare the bandwidth to upload blocks to other nodes, there > are number of options to reduce or eliminate the bandwidth requirement > found in https://bitcoin.org/en/full-node#reduce-traffic . These include > limiting connections, bandwidth targetting and disabling listening. > Bitcoin Core 0.12 has the new option -blocksonly, where the node will > not download unconfirmed transaction and only download new blocks. This > more than halves the bandwidth usage at the expense of not seeing > unconfirmed transactions. > > Synchronizing the blockchain for a new node has improved since 2012 too. > Features like headers-first > (https://bitcoin.org/en/release/v0.10.0#faster-synchronization) and > libsecp256k1 have greatly improved the initial synchronization time. > > It can be further improved by setting -dbcache=6000 which keeps more of > the UTXO set in memory. It reduces the amount of time reading from disk > and therefore speeds up synchronization. Tests showed that the entire > blockchain can now be synchronized in less than _3 and a half hours_ > (See > https://github.com/bitcoin/bitcoin/pull/6954#issuecomment-154993958) > Note that you'll need Bitcoin Core 0.12 or later to get all these > efficiency improvements. > > === How to run a full node as your wallet === > > I think every moderate user of bitcoin would benefit by running a full > node and using it as their wallet. There are several ways to do this. > > * Run a bitcoin-qt full node (https://bitcoin.org/en/download). > > * Use wallet software that is backed by a full node e.g. Armory > (https://bitcoinarmory.com/) or JoinMarket > (https://github.com/AdamISZ/JMBinary/#jmbinary) > > * Use a lightweight wallet that connects only to your full node (e.g. > Multibit connecting only to your node running at home, Electrum > connecting only to your own Electrum server) > > So what are you waiting for? The benefits are many, the downsides are > not that bad. The more people do this, the more robust and healthy the > bitcoin ecosystem is. > > This is very useful information but from my experience it is not viable to have a full node running full time on a desktop system i.e sharing the system with a normal desktop workload. With a very powerful "Desktop" machine bitcoin-qt dominates CPU/GPU resources. Surely the majority of nodes NOT running open ports are being run on desktop systems. It's likely that the vast majority of the "normal/desktop" user base are not going to setup dedicated machines to run a full node full time. It's likely that the vast majority of full nodes that are not running open ports are used occasionally when the user wants to make a transaction or "catch up" with the blockchain. That creates a divide between those who do have the resources to contribute to the system on a full time basis (minority) and those who do not (majority). Does the power of p2p decentralization lie with the vast majority or the "wealthy" resource rich minority? How will the move to 2MB hard fork affect the vast majority of nodes? For example Debian unstable currently provides the following: apt-cache madison bitcoin-qt bitcoin-qt | 0.11.2-1 | http://ftp.lug.ro/debian/ unstable/main amd64 Packages bitcoin | 0.11.2-1 | http://ftp.lug.ro/debian/ unstable/main Sources The rollout affect of the hard fork on the entire bitcoin ecosystem is a difficult process to plan in advance. It's not viable to simply rely on press releases to encourage users to upgrade their nodes. The debacle with Pulse Audio during the mid 2000's should be a lesson for those who seek that route. Compare that to the requirements for spinning up "bitcoin-2.0" and enabling users to move their wallets to the new blockchain at their leisure. The ecosystem doesn't suffer from instant degradation. Bitcoin "brand" loyalty will ensure that users who want to move forward with the economic potential of the 2MB blocksize will be able to keep their existing funds safe while testing the waters with the new blocksize. After all Bitcoin is still the only game in town when it comes to scale and proven history of financial return. As the new blockchain builds momentum the old one will eventually become obsolete. However it may also become the digital equivalency of Silver and that is also a useful, profitable and viable alternative with a proven history of success. -- Patrick Shirkey Boost Hardware Ltd