Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Wsbhc-0007bU-9c for bitcoin-development@lists.sourceforge.net; Thu, 05 Jun 2014 17:43:48 +0000 X-ACL-Warn: Received: from mail-ig0-f170.google.com ([209.85.213.170]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Wsbha-0005pn-LF for bitcoin-development@lists.sourceforge.net; Thu, 05 Jun 2014 17:43:48 +0000 Received: by mail-ig0-f170.google.com with SMTP id h3so3175703igd.3 for ; Thu, 05 Jun 2014 10:43:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=IsEQ6F7TrjVLLS6MtQMW8/IrEqMWCVyS8J+FrquOgt4=; b=kGaF5pljGBRWz0g9M000VWmtHs+ZTo28I2ad6z/uniSp8azIKVQ8vzT9P2NWleHWYO mMFOOBT8GHlP3k5L79i05Lq8Khog9w2pda6FkLcrEQeXfoFOAe04qTriJMkdMCNTcJwA eawPZ/wmsrM1ccT9XRIbtSyqyBKf1PsYCsrYcLffNqtDw0/qhxWMXMwB7kQ8h3bc8uTY HgCakOXNzXB0BaO2QprzfkSFaNniyjH2CBnYCaD6HXDbhCskN9bm856e6i6UmafgqVYf loyHV7Mu9dMNEfcF77nK2I/H9MsyxQE3dxuzSKPPgrrsaBVh+Ly1AlWJfxstpDtJQ2lr 7Z/g== X-Gm-Message-State: ALoCoQnRM75EjbUW9pzBJ87MljIchj70/sr5sZT3MiLOwIXGdMhhd56SfOqhVsFfP3UfHn3PH9wq X-Received: by 10.50.153.8 with SMTP id vc8mr22633926igb.16.1401990220744; Thu, 05 Jun 2014 10:43:40 -0700 (PDT) Received: from [192.168.2.22] (bas5-toronto47-2925108301.dsl.bell.ca. [174.89.156.77]) by mx.google.com with ESMTPSA id f9sm20018612igc.15.2014.06.05.10.43.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 05 Jun 2014 10:43:39 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46" Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) From: Richard Moore In-Reply-To: Date: Thu, 5 Jun 2014 13:43:38 -0400 Message-Id: <2ADB9019-825E-410A-ADED-A7237CBC323C@ricmoo.com> References: <34798C1C-FDA7-4A4C-B136-DBD4E59C254D@ricmoo.com> To: Mike Hearn X-Mailer: Apple Mail (2.1878.2) X-Spam-Score: 0.9 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Wsbha-0005pn-LF Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Future Feature Proposal - getgist X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 17:43:48 -0000 --Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 I was considering names like getcheckpoints() to use the terminology = that already seemed to be in place, but they were too long :) I have been using getheaders() in my thick client to quickly grab all = the headers before downloading the full blocks since I can grab more at = a time. Even with getblocks(), there is the case for a getgist() call. = Right now you call getblocks(), which can take some time to get the = corresponding inv(), at which time you can then start the call to = getdata() as well as the next call to getblocks(). With a gist, for example of segment_count 50, you could call getgist(), = then with the response, you could request 50 getblocks() each with a = block_locator of 1 hash from the gist (and optimally the stop_hash of = the next hash in the gist) to 50 different peers, providing 25,000 (50 x = 500) block hashes. Currently: >>> getblocks() <<< inv() >>> getdata() <<< block(), block(), block(), =85 (x 500) Saturates one peer, while leaving the rest un-used. Step 1 and 2 can be = repeated and dispatched to different peers, but there is still the = latency between the two calls. Gist: >>> getgist() <<< inv() >>> getblocks(), getblocks(), getblocks(), =85 (x segment_count, 1 per = peer) <<< inv(), inv(), inv(), =85 (x segment_count, 1 per peer) >>> getdata(), getdata(), getdata(), =85 (x segment_count, 1 per peer) <<< block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in = per peer) Each peer can be saturated. I will try to run some experiments this weekend to get numbers as to = whether there is actually any performance improvement using a gist, or = whether the getdata(), block() latency ends up dominating the time = anyways. RicMoo On Jun 4, 2014, at 11:42 PM, Mike Hearn wrote: > Why do you want to optimise this? getheaders is used by SPV clients = not full nodes. SPV clients like bitcoinj can and do simply ship with = gist files in them, then getheaders from the last "checkpoint" (I wish = I hadn't reused terminology like that but this is what bitcoinj calls = them). >=20 > In practice when I look at performance issues with current SPV = clients, getheaders speed is not on my radar. .=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`= =B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA> Richard Moore ~ Founder Genetic Mistakes Software inc. phone: (778) 882-6125 email: ricmoo@geneticmistakes.com www: http://GeneticMistakes.com --Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
I was considering names like getcheckpoints() = to use the terminology that already seemed to be in place, but they were = too long :)

I have been using getheaders() in my thick = client to quickly grab all the headers before downloading the full = blocks since I can grab more at a time. Even with getblocks(), there is = the case for a  getgist() call. Right now you call getblocks(), = which can take some time to get the corresponding inv(), at which time = you can then start the call to getdata() as well as the next call to = getblocks().

With a gist, for example of segment_count 50, = you could call getgist(), then with the response, you could request 50 = getblocks() each with a block_locator of 1 hash from the gist (and = optimally the stop_hash of the next hash in the gist) to 50 different = peers, providing 25,000 (50 x 500) block hashes.

Currently:
  1. >>> getblocks()
  2. <<< = inv()
  3. >>> getdata()
  4. <<< block(), = block(), block(), =85 (x 500)

Saturates one peer, = while leaving the rest un-used. Step 1 and 2 can be repeated and = dispatched to different peers, but there is still the latency between = the two calls.

Gist:
  1. >>> getgist()
  2. <<< = inv()
  3. >>> getblocks(), getblocks(), getblocks(), =85 (x = segment_count, 1 per peer)
  4. <<< inv(), inv(), inv(), =85 = (x segment_count, 1 per peer)
  5. >>> getdata(), getdata(), = getdata(), =85 (x segment_count, 1 per peer)
  6. <<< = block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in per = peer)

Each peer can be saturated.

I = will try to run some experiments this weekend to get numbers as to = whether there is actually any performance improvement using a gist, or = whether the getdata(), block() latency ends up dominating the time = anyways.


RicMoo


On Jun 4, = 2014, at 11:42 PM, Mike Hearn <mike@plan99.net> wrote:

Why do you want to optimise this? getheaders is used by SPV = clients not full nodes. SPV clients like bitcoinj can and do simply ship = with gist files in them, then getheaders from the last "checkpoint" =   (I wish I hadn't reused terminology like that but this is what = bitcoinj calls them).

In practice when I look at performance issues with = current SPV clients, getheaders speed is not on my radar.

.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4= =AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA>

Richard Moore = ~ Founder
Genetic Mistakes Software inc.
phone: (778) = 882-6125
email: ricmoo@geneticmistakes.com<= br>www: http://GeneticMistakes.com
=

= --Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46--