Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.216.182 as permitted sender)
	client-ip=209.85.216.182; envelope-from=shadders.del@gmail.com;
	helo=mail-qy0-f182.google.com; 
Message-ID: <4E663B55.9050508@gmail.com>
Date: Wed, 07 Sep 2011 01:25:09 +1000
From: Steve <shadders.del@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.21) Gecko/20110831 Thunderbird/3.1.13
MIME-Version: 1.0
To: Mike Hearn <mike@plan99.net>
References: <4E65CEE6.7030002@gmail.com>	<4E65DA06.9060403@gmail.com>	<CALxbBHUajARXc1oA-NjD+U8hW5uSqF=u4ZHHBfcmT_O8GjpNiA@mail.gmail.com>	<CANEZrP0VXDUs_mAKCVKD1Q0ijyb989oADrCN1zTZ1nnN_JQ=cQ@mail.gmail.com>	<4E661FAE.9020008@gmail.com>	<CANEZrP3=UPYkBQo6b421xaMGyP4BsGiw8DBuM8pT2ow1Vom9JQ@mail.gmail.com>	<4E662B79.5090303@gmail.com>
	<CANEZrP2Wh82sqGjZDn_M=UPufBCU4fP9zEXV_K8JpgVF8O1FCw@mail.gmail.com>
In-Reply-To: <CANEZrP2Wh82sqGjZDn_M=UPufBCU4fP9zEXV_K8JpgVF8O1FCw@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary="------------060206040500050907030804"
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Building a node crawler to map network
Precedence: list
Reply-To: shadders.del@gmail.com

This is a multi-part message in MIME format.
--------------060206040500050907030804
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Thanks for the overview Mike.  I just bailed up Gavin on IRC and between 
that convo and what you've just written I'm starting to picture a plan 
in my head... This sounds right up my alley, I wish I didn't have to go 
to bed right now as I've got a ton of ideas buzzing around I'd like to 
get started on right now.  But I'll be onto it as soon as I've got a 
free moment...

On 07/09/11 00:52, Mike Hearn wrote:
> On Tue, Sep 6, 2011 at 4:17 PM, Steve <shadders.del@gmail.com 
> <mailto:shadders.del@gmail.com>> wrote:
>
>     I'm not really understanding the use case though.  I believe most
>     bitcoind's have a default max connections of 8.  Is the goal to
>     increase this without fundamentally altering the bitcoind
>     concurrency model?
>
>
> bitcoind already uses asynchronous IO. That's not the problem.
>
> The issue came up in a conversation about scalability. If Bitcoins 
> popularity continues to grow, users are very likely to migrate away 
> from running full verifying nodes to lightweight clients, either a 
> different mode of the Satoshi client or different implementations like 
> the Android Wallet or MultiBit.
>
> Lightweight clients cannot verify thus should not relay. And they'll 
> be run by users who just want to send/receive coins from time to time, 
> so don't leave the programs running 24/7. The result could be running 
> out of sockets (like we have had problems with recently). It's 
> especially true because lightweight clients cannot check transactions 
> for themselves. If they want to show transactions appearing 
> immediately (and they do), they have to use "heard from lots of nodes" 
> as a proxy for validity. So lightweight clients are likely to be 
> socket intensive.
>
> We could solve this by just hoping that lots of people run full nodes. 
> The problem is that a full node is quite an intensive thing already, 
> it uses lots of CPU and disk seeks, and will just get more expensive 
> in future. And as transaction traffic increases, that leaves less CPU 
> time available to service thousands of connected clients. The ROI of 
> bringing up a new node decreases at the same time as the userbase 
> increases.
>
> One traditional approach to solving this is frontend proxies. 
> Jabber.com/org used this technique many years ago, and Google has also 
> used it to scale up the lockservice 
> <http://static.googleusercontent.com/external_content/untrusted_dlcp/labs.google.com/en/us/papers/chubby-osdi06.pdf> (see 
> section 3.1). It's effective because often maintaining connections to 
> thousands of clients doesn't involve much brainwork, just shifting 
> bytes around. This is especially true of Bitcoin. So if somebody is 
> running a full node already they could increase their client capacity 
> by just bringing up a frontend proxy and having it handle things like 
> outbound tx broadcasts/deduping inbound broadcasts, connection setup, 
> relaying recently found blocks etc. A well written proxy could 
> probably support tens of thousands of simultaneous clients which frees 
> up the bitcoinds time for verification and wallet manipulation.

--------------060206040500050907030804
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#ffffff" text="#000000">
    Thanks for the overview Mike.  I just bailed up Gavin on IRC and
    between that convo and what you've just written I'm starting to
    picture a plan in my head... This sounds right up my alley, I wish I
    didn't have to go to bed right now as I've got a ton of ideas
    buzzing around I'd like to get started on right now.  But I'll be
    onto it as soon as I've got a free moment...<br>
    <br>
    On 07/09/11 00:52, Mike Hearn wrote:
    <blockquote
cite="mid:CANEZrP2Wh82sqGjZDn_M=UPufBCU4fP9zEXV_K8JpgVF8O1FCw@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">On Tue, Sep 6, 2011 at 4:17 PM, Steve <span
          dir="ltr">&lt;<a moz-do-not-send="true"
            href="mailto:shadders.del@gmail.com">shadders.del@gmail.com</a>&gt;</span>
        wrote:<br>
        <blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
          0.8ex; border-left: 1px solid rgb(204, 204, 204);
          padding-left: 1ex;">
          <div bgcolor="#ffffff" text="#000000">I'm not really
            understanding the use case though.  I believe most
            bitcoind's have a default max connections of 8.  Is the goal
            to increase this without fundamentally altering the bitcoind
            concurrency model?</div>
        </blockquote>
        <div><br>
        </div>
        <div>bitcoind already uses asynchronous IO. That's not the
          problem.</div>
        <div><br>
        </div>
        <div>The issue came up in a conversation about scalability. If
          Bitcoins popularity continues to grow, users are very likely
          to migrate away from running full verifying nodes to
          lightweight clients, either a different mode of the Satoshi
          client or different implementations like the Android Wallet or
          MultiBit.</div>
        <div><br>
        </div>
        <div>Lightweight clients cannot verify thus should not relay.
          And they'll be run by users who just want to send/receive
          coins from time to time, so don't leave the programs running
          24/7. The result could be running out of sockets (like we have
          had problems with recently). It's especially true because
          lightweight clients cannot check transactions for themselves.
          If they want to show transactions appearing immediately (and
          they do), they have to use "heard from lots of nodes" as a
          proxy for validity. So lightweight clients are likely to be
          socket intensive.</div>
        <div><br>
        </div>
        <div>We could solve this by just hoping that lots of people run
          full nodes. The problem is that a full node is quite an
          intensive thing already, it uses lots of CPU and disk seeks,
          and will just get more expensive in future. And as transaction
          traffic increases, that leaves less CPU time available to
          service thousands of connected clients. The ROI of bringing up
          a new node decreases at the same time as the userbase
          increases.</div>
        <div><br>
        </div>
        <div>One traditional approach to solving this is frontend
          proxies. Jabber.com/org used this technique many years ago,
          and Google has also used it to scale up <a
            moz-do-not-send="true"
href="http://static.googleusercontent.com/external_content/untrusted_dlcp/labs.google.com/en/us/papers/chubby-osdi06.pdf">the
            lockservice</a> (see section 3.1). It's effective because
          often maintaining connections to thousands of clients doesn't
          involve much brainwork, just shifting bytes around. This is
          especially true of Bitcoin. So if somebody is running a full
          node already they could increase their client capacity by just
          bringing up a frontend proxy and having it handle things like
          outbound tx broadcasts/deduping inbound broadcasts, connection
          setup, relaying recently found blocks etc. A well written
          proxy could probably support tens of thousands of simultaneous
          clients which frees up the bitcoinds time for verification and
          wallet manipulation.</div>
      </div>
    </blockquote>
  </body>
</html>

--------------060206040500050907030804--