Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UWUTZ-0006Yi-50 for bitcoin-development@lists.sourceforge.net; Sun, 28 Apr 2013 16:29:21 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.47 as permitted sender) client-ip=209.85.219.47; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f47.google.com; Received: from mail-oa0-f47.google.com ([209.85.219.47]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UWUTV-00026e-1d for bitcoin-development@lists.sourceforge.net; Sun, 28 Apr 2013 16:29:21 +0000 Received: by mail-oa0-f47.google.com with SMTP id n9so5305432oag.20 for ; Sun, 28 Apr 2013 09:29:11 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.226.136 with SMTP id rs8mr27034059obc.50.1367166551674; Sun, 28 Apr 2013 09:29:11 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.167.169 with HTTP; Sun, 28 Apr 2013 09:29:11 -0700 (PDT) In-Reply-To: References: Date: Sun, 28 Apr 2013 18:29:11 +0200 X-Google-Sender-Auth: ujnmoW7a4z2XRLETJQ9v-gooNpA Message-ID: From: Mike Hearn To: Pieter Wuille Content-Type: multipart/alternative; boundary=001a11c3018aa9592104db6e44fc X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UWUTV-00026e-1d Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Service bits for pruned nodes X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 16:29:21 -0000 --001a11c3018aa9592104db6e44fc Content-Type: text/plain; charset=UTF-8 I'd imagined that nodes would be able to pick their own ranges to keep rather than have fixed chosen intervals. "Everything or two weeks" is rather restrictive - presumably node operators are constrained by physical disk space, which means the quantity of blocks they would want to keep can vary with sizes of blocks, cost of storage, etc. Adding new fields to the addr message and relaying those fields to newer nodes means every node could advertise the height at which it pruned. I know it means a longer time before the data is available everywhere vs service bits, but it seems like most nodes won't be pruning right away anyway. There's plenty of time for upgrades. If an old node connected to a new node and getdata-d blocks that had been pruned, immediate disconnection should make the old node go find a different one. It means the combination of old node+not run for a long time might take a while before it can find a node that has what it wants, but that doesn't seem like a big deal. What is the use case for NODE_VALIDATE? Nodes that throw away blocks almost immediately? Why would a node do that? On Sun, Apr 28, 2013 at 5:51 PM, Pieter Wuille wrote: > Hello all, > > I think it is time to move forward with pruning nodes, i.e. nodes that > fully validate and relay blocks and transactions, but which do not keep > (all) historic blocks around, and thus cannot be queried for these. > > The biggest roadblock is making sure new and old nodes that start up are > able to find nodes to synchronize from. To help them find peers, I would > like to propose adding two extra service bits to the P2P protocol: > * NODE_VALIDATE: relay and validate blocks and transactions, but is only > guaranteed to answer getdata requests for (recently) relayed blocks and > transactions, and mempool transactions. > * NODE_BLOCKS_2016: can be queried for the last 2016 blocks, but without > guarantee for relaying/validating new blocks and transactions. > * NODE_NETWORK (which existed before) will imply NODE_VALIDATE and > guarantee availability of all historic blocks. > > The idea is to separate the different responsibilities of network nodes > into separate bits, so they can - at some point - be > implemented independently. Perhaps we want more than just one degree (2016 > blocks), maybe also 144 or 210000, but those can be added later if > necessary. I monitored the frequency of block depths requested from my > public node, and got this frequency distribution: > http://bitcoin.sipa.be/depth-small.png so it seems 2016 nicely matches > the set of frequently-requested blocks (indicating that few nodes are > offline for more than 2 weeks consecutively. > > I'll write a BIP to formalize this, but wanted to get an idea of how much > support there is for a change like this. > > Cheers, > > -- > Pieter > > > > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --001a11c3018aa9592104db6e44fc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I'd imagined that nodes would be able to pick their ow= n ranges to keep rather than have fixed chosen intervals. "Everything = or two weeks" is rather restrictive - presumably node operators are co= nstrained by physical disk space, which means the quantity of blocks they w= ould want to keep can vary with sizes of blocks, cost of storage, etc.

Adding new fields to the addr message and relaying tho= se fields to newer nodes means every node could advertise the height at whi= ch it pruned. I know it means a longer time before the data is available ev= erywhere vs service bits, but it seems like most nodes won't be pruning= right away anyway. There's plenty of time for upgrades. If an old node= connected to a new node and getdata-d blocks that had been pruned, immedia= te disconnection should make the old node go find a different one. It means= the combination of old node+not run for a long time might take a while bef= ore it can find a node that has what it wants, but that doesn't seem li= ke a big deal.

What is the use case for NODE_VALIDATE? Nod= es that throw away blocks almost immediately? Why would a node do that?


On Su= n, Apr 28, 2013 at 5:51 PM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
Hello all,

I think it is time to move forward with pruning nodes, i.e. nodes that f= ully validate and relay blocks and transactions, but which do not keep (all= ) historic blocks around, and thus cannot be queried for these.

The biggest roadblock is making sure new and old nodes = that start up are able to find nodes to synchronize from. To help them find= peers, I would like to propose adding two extra service bits to the P2P pr= otocol:
* NODE_VALIDATE: relay and validate blocks and transactions, but is on= ly guaranteed to answer getdata requests for (recently) relayed blocks and = transactions, and mempool transactions.
* NODE_BLOCKS_2016: can b= e queried for the last 2016 blocks, but without guarantee for relaying/vali= dating new blocks and transactions.
* NODE_NETWORK (which existed before) will imply NODE_VALIDATE and gua= rantee availability of all historic blocks.

--=C2=A0
Pieter
=C2=A0



-----------------------------------------------------------------------= -------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service=
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_= ______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


--001a11c3018aa9592104db6e44fc--