Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.216.175 as permitted sender)
	client-ip=209.85.216.175; envelope-from=shadders.del@gmail.com;
	helo=mail-qy0-f175.google.com; 
Message-ID: <4E662B79.5090303@gmail.com>
Date: Wed, 07 Sep 2011 00:17:29 +1000
From: Steve <shadders.del@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.21) Gecko/20110831 Thunderbird/3.1.13
MIME-Version: 1.0
To: Mike Hearn <mike@plan99.net>
References: <4E65CEE6.7030002@gmail.com>	<4E65DA06.9060403@gmail.com>	<CALxbBHUajARXc1oA-NjD+U8hW5uSqF=u4ZHHBfcmT_O8GjpNiA@mail.gmail.com>	<CANEZrP0VXDUs_mAKCVKD1Q0ijyb989oADrCN1zTZ1nnN_JQ=cQ@mail.gmail.com>	<4E661FAE.9020008@gmail.com>
	<CANEZrP3=UPYkBQo6b421xaMGyP4BsGiw8DBuM8pT2ow1Vom9JQ@mail.gmail.com>
In-Reply-To: <CANEZrP3=UPYkBQo6b421xaMGyP4BsGiw8DBuM8pT2ow1Vom9JQ@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary="------------020701080500050303030203"
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Building a node crawler to map network
Precedence: list
Reply-To: shadders.del@gmail.com

This is a multi-part message in MIME format.
--------------020701080500050303030203
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Mike,

I expect I'll be submitting patches for bitcoinj sometime in the future 
but I'm not really across it yet to the point where I'd be confident 
submitting patches right now...

This proxy sound like a good match for what I've been up to lately 
though so long as it wouldn't involve direct changes to bitcoind on my 
part.  My c/c++ skills are non-existent.

However I have been building a pool protocol using protobufs and netty 
for non-blocking IO and I'd imagine the kind of multiplexing proxy 
you're talking about could be easily implemented using netty.

I'm not really understanding the use case though.  I believe most 
bitcoind's have a default max connections of 8.  Is the goal to increase 
this without fundamentally altering the bitcoind concurrency model?  Or 
is it to provide capactity for a more hub/client oriented network?  If 
the latter then presumably this is functionality that should ideally be 
native to the client in the long term in the form of NIO?

On 06/09/11 23:31, Mike Hearn wrote:
>
>     I've looked but can't find a post like you're talking about.  Can
>     you point me to it?
>
> https://groups.google.com/forum/?pli=1#!topic/bitcoinj/LSlZdUWcCdk 
> <https://groups.google.com/forum/?pli=1#%21topic/bitcoinj/LSlZdUWcCdk>
>
>     If so then bollocks... I'm looking for something useful to do atm.
>      PoolServerJ is in a holding pattern atm as I've stabilisied all
>     the bugs I know about and am waiting for several pools to finish
>     testing and move into production so I'm twiddling thumbs trying to
>     figure out how to spend my time.
>
>
> Patches to BitCoinJ are always welcome :-)
>
> If you'd rather do your own thing, you could experiment with writing a 
> proxy that sits in front of bitcoind and multiplexes connections. 
> Gavin is concerned about socket exhaustion as users move to 
> lightweight clients. Multiplexing proxies are a battle-tested 
> technique for reducing the strain of this type of thing. BitCoinJ uses 
> thread-per-connection so wouldn't do a good job of that right now, but 
> allowing it to use a mix of async io and multi-threading would be a 
> nice improvement. It'd need some changes to bitcoind as well for a 
> really good effort, to allow for IPs to be forwarded. I'm happy to 
> discuss it more with you over on the bitcoinj list if wanted.

--------------020701080500050303030203
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#ffffff" text="#000000">
    Hi Mike,<br>
    <br>
    I expect I'll be submitting patches for bitcoinj sometime in the
    future but I'm not really across it yet to the point where I'd be
    confident submitting patches right now...<br>
    <br>
    This proxy sound like a good match for what I've been up to lately
    though so long as it wouldn't involve direct changes to bitcoind on
    my part.  My c/c++ skills are non-existent.<br>
    <br>
    However I have been building a pool protocol using protobufs and
    netty for non-blocking IO and I'd imagine the kind of multiplexing
    proxy you're talking about could be easily implemented using netty. 
    <br>
    <br>
    I'm not really understanding the use case though.  I believe most
    bitcoind's have a default max connections of 8.  Is the goal to
    increase this without fundamentally altering the bitcoind
    concurrency model?  Or is it to provide capactity for a more
    hub/client oriented network?  If the latter then presumably this is
    functionality that should ideally be native to the client in the
    long term in the form of NIO? <br>
    <br>
    On 06/09/11 23:31, Mike Hearn wrote:
    <blockquote
cite="mid:CANEZrP3=UPYkBQo6b421xaMGyP4BsGiw8DBuM8pT2ow1Vom9JQ@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">
        <blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
          0.8ex; border-left: 1px solid rgb(204, 204, 204);
          padding-left: 1ex;">
          <div class="im">I've looked but can't find a post like you're
            talking about.  Can you point me to it?<br>
          </div>
        </blockquote>
        <div> </div>
        <div><a moz-do-not-send="true"
href="https://groups.google.com/forum/?pli=1#%21topic/bitcoinj/LSlZdUWcCdk">https://groups.google.com/forum/?pli=1#!topic/bitcoinj/LSlZdUWcCdk</a></div>
        <div> </div>
        <blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
          0.8ex; border-left: 1px solid rgb(204, 204, 204);
          padding-left: 1ex;">
          <div class="im">If so then bollocks... I'm looking for
            something useful to do atm.  PoolServerJ is in a holding
            pattern atm as I've stabilisied all the bugs I know about
            and am waiting for several pools to finish testing and move
            into production so I'm twiddling thumbs trying to figure out
            how to spend my time.</div>
        </blockquote>
        <div><br>
        </div>
        <div>Patches to BitCoinJ are always welcome :-)</div>
        <div><br>
        </div>
        <div>If you'd rather do your own thing, you could experiment
          with writing a proxy that sits in front of bitcoind and
          multiplexes connections. Gavin is concerned about socket
          exhaustion as users move to lightweight clients. Multiplexing
          proxies are a battle-tested technique for reducing the strain
          of this type of thing. BitCoinJ uses thread-per-connection so
          wouldn't do a good job of that right now, but allowing it to
          use a mix of async io and multi-threading would be a nice
          improvement. It'd need some changes to bitcoind as well for a
          really good effort, to allow for IPs to be forwarded. I'm
          happy to discuss it more with you over on the bitcoinj list if
          wanted.</div>
      </div>
    </blockquote>
  </body>
</html>

--------------020701080500050303030203--