Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.210.42 as permitted sender)
	client-ip=209.85.210.42;
	envelope-from=decker.christian@gmail.com;
	helo=mail-pz0-f42.google.com; 
MIME-Version: 1.0
In-Reply-To: <4E65DA06.9060403@gmail.com>
References: <4E65CEE6.7030002@gmail.com> <4E65DA06.9060403@gmail.com>
From: Christian Decker <decker.christian@gmail.com>
Date: Tue, 6 Sep 2011 10:36:19 +0200
Message-ID: <CALxbBHUajARXc1oA-NjD+U8hW5uSqF=u4ZHHBfcmT_O8GjpNiA@mail.gmail.com>
To: shadders.del@gmail.com
Content-Type: multipart/alternative; boundary=bcaec544eefc1f244c04ac41bb4a
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Building a node crawler to map network
Precedence: list

--bcaec544eefc1f244c04ac41bb4a
Content-Type: text/plain; charset=ISO-8859-1

Hi Steve,

before attempting to hack BitcoinJ to use NIO you might want to take a look
at BitDroid (https://github.com/cdecker/BitDroid-Network), which is my
attempt to build an easily extensible network client (no crypto stuff so
far) on top of NIO and a simple publish-subscribe architecture. I build a
crawler like yours with just a single class that subscribes to events
published and closes and opens connections to crawl.

HTH,
Christian

On Tue, Sep 6, 2011 at 10:29 AM, Steve <shadders.del@gmail.com> wrote:

> **
>
>
> While I'm asking questions I'll add one more regarding the getaddr message.
>
>
>
> Talking to myself here.  I just sent this message then found this brilliant
> set of articles in the Dev & Tech forum which answers the question very
> nicely: *https://bitcointalk.org/index.php?topic=41722.0<https://bitcointalk.org/index.php?board=6.0>
> *
> Anyway just as an FYI I've been running v0.0.0.0.0.0.0.0.1 for about an
> hour.  It's only running 10 concurrent connections due to girlfriend
> complaining she couldn't watch youtube but here's some early results.
>
> New nodes: 19319 // node address discovered but no contact attempt made yet
> Contacted nodes: 754
> Uncontactable nodes: 3253
> Limbo nodes: 9 //not as exciting as it sounds, just nodes with connect in
> progress
> Total nodes: 23335 // about 5000 from initial IRC discover, the rest are
> from getaddr
>
> Versions: {
> 300=1,
> 31900=7,
> 31902=1,
> 32000=2,
> 32001=7,
> 32002=22,
> 32100=100,
> 32200=24,
> 32300=277,
> 32400=317,
> 32500=2}
>
> Fails: {
> ConnectException: Connection refused=377,
> IOException: Socket is disconnected=87,
> SocketException: Network is unreachable=2,
> ProtocolException: Error deserializing message =1,
> NoRouteToHostException: No route to host=115,
> SocketException: Connection reset=149,
> SocketTimeoutException: connect timed out=2521}
>
>
>
>
> ------------------------------------------------------------------------------
> Special Offer -- Download ArcSight Logger for FREE!
> Finally, a world-class log management solution at an even better
> price-free! And you'll get a free "Love Thy Logs" t-shirt when you
> download Logger. Secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsisghtdev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

--bcaec544eefc1f244c04ac41bb4a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Steve,<br><br>before attempting to hack BitcoinJ to use NIO you might wa=
nt to take a look at BitDroid (<a href=3D"https://github.com/cdecker/BitDro=
id-Network">https://github.com/cdecker/BitDroid-Network</a>), which is my a=
ttempt to build an easily extensible network client (no crypto stuff so far=
) on top of NIO and a simple publish-subscribe architecture. I build a craw=
ler like yours with just a single class that subscribes to events published=
 and closes and opens connections to crawl.<br>

<br>HTH,<br>Christian<br><br><div class=3D"gmail_quote">On Tue, Sep 6, 2011=
 at 10:29 AM, Steve <span dir=3D"ltr">&lt;<a href=3D"mailto:shadders.del@gm=
ail.com">shadders.del@gmail.com</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex;">

<u></u>

 =20
   =20
   =20
 =20
  <div bgcolor=3D"#ffffff" text=3D"#000000"><div class=3D"im">
    <br>
    <blockquote type=3D"cite">
      <br>
      While I&#39;m asking questions I&#39;ll add one more regarding the ge=
taddr
      message. <br>
    </blockquote>
    <br></div>
    Talking to myself here.=A0 I just sent this message then found this
    brilliant set of articles in the Dev &amp; Tech forum which answers
    the question very nicely: <b><a href=3D"https://bitcointalk.org/index.p=
hp?board=3D6.0" target=3D"_blank">https://bitcointalk.org/index.php?topic=
=3D41722.0</a><br>
    </b><br>
    Anyway just as an FYI I&#39;ve been running v0.0.0.0.0.0.0.0.1 for abou=
t
    an hour.=A0 It&#39;s only running 10 concurrent connections due to
    girlfriend complaining she couldn&#39;t watch youtube but here&#39;s so=
me
    early results.<br>
    <br>
    New nodes: 19319 // node address discovered but no contact attempt
    made yet<br>
    Contacted nodes: 754<br>
    Uncontactable nodes: 3253<br>
    Limbo nodes: 9 //not as exciting as it sounds, just nodes with
    connect in progress<br>
    Total nodes: 23335 // about 5000 from initial IRC discover, the rest
    are from getaddr<br>
    <br>
    Versions: {<br>
    300=3D1, <br>
    31900=3D7, <br>
    31902=3D1, <br>
    32000=3D2, <br>
    32001=3D7, <br>
    32002=3D22, <br>
    32100=3D100, <br>
    32200=3D24, <br>
    32300=3D277, <br>
    32400=3D317, <br>
    32500=3D2}<br>
    <br>
    Fails: {<br>
    ConnectException: Connection refused=3D377, <br>
    IOException: Socket is disconnected=3D87, <br>
    SocketException: Network is unreachable=3D2, <br>
    ProtocolException: Error deserializing message =3D1, <br>
    NoRouteToHostException: No route to host=3D115, <br>
    SocketException: Connection reset=3D149, <br>
    SocketTimeoutException: connect timed out=3D2521}<br>
    <br>
    <br>
  </div>

<br>-----------------------------------------------------------------------=
-------<br>
Special Offer -- Download ArcSight Logger for FREE!<br>
Finally, a world-class log management solution at an even better<br>
price-free! And you&#39;ll get a free &quot;Love Thy Logs&quot; t-shirt whe=
n you<br>
download Logger. Secure your free ArcSight Logger TODAY!<br>
<a href=3D"http://p.sf.net/sfu/arcsisghtdev2dev" target=3D"_blank">http://p=
.sf.net/sfu/arcsisghtdev2dev</a><br>_______________________________________=
________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
pment@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div><br>

--bcaec544eefc1f244c04ac41bb4a--