path: root/27/137543fa46c9f3d0ae08f210affe557c8b2452

Return-Path: <pindar.wong@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id C39C65B1
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 23 Jul 2015 17:41:36 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-la0-f45.google.com (mail-la0-f45.google.com
	[209.85.215.45])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B29BA112
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 23 Jul 2015 17:41:35 +0000 (UTC)
Received: by lagw2 with SMTP id w2so162128966lag.3
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu, 23 Jul 2015 10:41:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=Ki4MbdR+P2LW2x92Sj8perItZuuLGAbUe5mGIMzOmME=;
	b=i8A1DMT0eRFtpQpVj8cQHm5BVJY1wM5LGfuWA1+WdkU+MnHjPAjNvePIF0xKruKMyn
	ElzVFQ0CSdPVSCzyYSY4rdY4zfdF2ZKEeZduHRognqqNyfoKn0qEE2MYseYCeQeAO8LD
	o2+4SHDEK7uOK1Plv+LMH1OnuPgYpIOUFId+7G44of+VuD0hlsIi9B6Nm3Vty5YU6u60
	r6zb0YV/f3eCYCW3PdOAToQVgweI1mC4OTHXwJtny1RQpWaq/5212WIlOVQSBxc6zZB1
	2G34bwCQnVCY/JonVQsJIfCGIkp6+GLYJIEipDFlBysbOu6KI8ROoRcd7kkhc+SzSYSh
	Wuog==
MIME-Version: 1.0
X-Received: by 10.112.168.102 with SMTP id zv6mr9333296lbb.45.1437673294253;
	Thu, 23 Jul 2015 10:41:34 -0700 (PDT)
Received: by 10.152.133.84 with HTTP; Thu, 23 Jul 2015 10:41:33 -0700 (PDT)
In-Reply-To: <trinity-650f6539-8135-4f95-a54f-9dd0744df911-1437671671241@3capp-mailcom-bs04>
References: <trinity-c97bc41b-a953-4580-b2d2-ebdda9eb96b2-1437661199263@3capp-mailcom-bs02>
	<CADL_X_dmeyjR2PJN8oLn8EutVCu8Pn_qsP9ATRCYadx3dh4Erg@mail.gmail.com>
	<trinity-650f6539-8135-4f95-a54f-9dd0744df911-1437671671241@3capp-mailcom-bs04>
Date: Fri, 24 Jul 2015 01:41:33 +0800
Message-ID: <CAM7BtUrtfCM+P6DTGdgTTSpfq-Ot=YdTzn40dbwa_KkyoN7F7w@mail.gmail.com>
From: Pindar Wong <pindar.wong@gmail.com>
To: Slurms MacKenzie <slurms@gmx.us>
Content-Type: multipart/alternative; boundary=001a11c33d3201d4a8051b8e66eb
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE,
	RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] Bitcoin Node Speed Test
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Jul 2015 17:41:36 -0000

--001a11c33d3201d4a8051b8e66eb
Content-Type: text/plain; charset=UTF-8

This looks like the beginnings of some great analysis.

Per Peter's remarks, I think it would be productive to run the test(s) on a
simulated network with worst case network failure(s) so that we can
determine the safety margin needed.

I have potential access to h/w resources that would be available for
running such tests at the necessary scales.

Cheers,

p.


On Fri, Jul 24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> The library used isn't open source, so unfortunately not. It shouldn't be
> too hard to replicate in python-bitcoinlib or bitcoinj though.
>
> *Sent:* Thursday, July 23, 2015 at 6:55 PM
> *From:* "Jameson Lopp" <jameson.lopp@gmail.com>
> *To:* slurms@gmx.us
> *Cc:* bitcoin-dev@lists.linuxfoundation.org
> *Subject:* Re: [bitcoin-dev] Bitcoin Node Speed Test
>  Are you willing to share the code that you used to run the test?
>
> - Jameson
>
> On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> On this day, the Bitcoin network was crawled and reachable nodes surveyed
>> to find their maximum throughput in order to determine if it can safely
>> support a faster block rate. Specifically this is an attempt to prove or
>> disprove the common statement that 1MB blocks were only suitable slower
>> internet connections in 2009 when Bitcoin launched, and that connection
>> speeds have improved to the point of obviously supporting larger blocks.
>>
>>
>> The testing methodology is as follows:
>>
>>  * Nodes were randomly selected from a peers.dat, 5% of the reachable
>> nodes in the network were contacted.
>>
>>  * A random selection of blocks was downloaded from each peer.
>>
>>  * There is some bias towards higher connection speeds, very slow
>> connections (<30KB/s) timed out in order to run the test at a reasonable
>> rate.
>>
>>  * The connecting node was in Amsterdam with a 1GB NIC.
>>
>>
>> Results:
>>
>>  * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>>
>>  * 16% of connected nodes uploaded blocks faster than 10MB/s.
>>
>>  * Raw data, one line per connected node, kilobytes per second
>> http://pastebin.com/raw.php?i=6b4NuiVQ
>>
>>
>> This does not support the theory that the network has the available
>> bandwidth for increased block sizes, as in its current state 37% of nodes
>> would fail to upload a 20MB block to a single peer in under 20 seconds
>> (referencing a number quoted by Gavin). If the bar for suitability is
>> placed at taking only 1% of the block time (6 seconds) to upload one block
>> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
>> only 10% fail this metric for 1MB blocks.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

--001a11c33d3201d4a8051b8e66eb
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div>This looks like the beginnings of some grea=
t analysis.<br><br>Per Peter&#39;s remarks, I think it would be productive =
to run the test(s) on a simulated network with worst case network failure(s=
) so that we can determine the safety margin needed. <br><br></div>I have p=
otential access to h/w resources that would be available for running such t=
ests at the necessary scales.<br><br></div>Cheers,<br><br></div>p.<br><br><=
/div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Fri, Jul =
24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev <span dir=3D"ltr">&lt=
;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank"=
>bitcoin-dev@lists.linuxfoundation.org</a>&gt;</span> wrote:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex"><div><div style=3D"font-family:Verdana;font-size:12.0px=
"><div>
<div>The library used isn&#39;t open source, so unfortunately not. It shoul=
dn&#39;t be too hard to replicate in python-bitcoinlib or bitcoinj though.<=
/div>

<div>=C2=A0
<div name=3D"quote" style=3D"margin:10px 5px 5px 10px;padding:10px 0 10px 1=
0px;border-left:2px solid #c3d9e5;word-wrap:break-word">
<div style=3D"margin:0 0 10px 0"><b>Sent:</b>=C2=A0Thursday, July 23, 2015 =
at 6:55 PM<br>
<b>From:</b>=C2=A0&quot;Jameson Lopp&quot; &lt;<a href=3D"mailto:jameson.lo=
pp@gmail.com" target=3D"_blank">jameson.lopp@gmail.com</a>&gt;<br>
<b>To:</b>=C2=A0<a href=3D"mailto:slurms@gmx.us" target=3D"_blank">slurms@g=
mx.us</a><br>
<b>Cc:</b>=C2=A0<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" ta=
rget=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a><span class=3D""><=
br>
<b>Subject:</b>=C2=A0Re: [bitcoin-dev] Bitcoin Node Speed Test</span></div>

<div name=3D"quoted-content"><span class=3D"">
<div>Are you willing to share the code that you used to run the test?
<div>=C2=A0</div>

<div>- Jameson</div>
</div>

</span><div class=3D"gmail_extra">=C2=A0
<div class=3D"gmail_quote"><span class=3D"">On Thu, Jul 23, 2015 at 10:19 A=
M, slurms--- via bitcoin-dev <span>&lt;<a href=3D"http://bitcoin-dev@lists.=
linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.or=
g</a>&gt;</span> wrote:

</span><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 0.8ex;border=
-left:1.0px rgb(204,204,204) solid;padding-left:1.0ex"><span class=3D"">On =
this day, the Bitcoin network was crawled and reachable nodes surveyed to f=
ind their maximum throughput in order to determine if it can safely support=
 a faster block rate. Specifically this is an attempt to prove or disprove =
the common statement that 1MB blocks were only suitable slower internet con=
nections in 2009 when Bitcoin launched, and that connection speeds have imp=
roved to the point of obviously supporting larger blocks.<br>
<br>
<br>
The testing methodology is as follows:<br>
<br>
=C2=A0* Nodes were randomly selected from a peers.dat, 5% of the reachable =
nodes in the network were contacted.<br>
<br>
=C2=A0* A random selection of blocks was downloaded from each peer.<br>
<br>
=C2=A0* There is some bias towards higher connection speeds, very slow conn=
ections (&lt;30KB/s) timed out in order to run the test at a reasonable rat=
e.<br>
<br>
=C2=A0* The connecting node was in Amsterdam with a 1GB NIC.<br>
<br>
=C2=A0<br>
Results:<br>
<br>
=C2=A0* 37% of connected nodes failed to upload blocks faster than 1MB/s.<b=
r>
<br>
=C2=A0* 16% of connected nodes uploaded blocks faster than 10MB/s.<br>
<br>
=C2=A0* Raw data, one line per connected node, kilobytes per second <a href=
=3D"http://pastebin.com/raw.php?i=3D6b4NuiVQ" target=3D"_blank">http://past=
ebin.com/raw.php?i=3D6b4NuiVQ</a><br>
<br>
<br>
This does not support the theory that the network has the available bandwid=
th for increased block sizes, as in its current state 37% of nodes would fa=
il to upload a 20MB block to a single peer in under 20 seconds (referencing=
 a number quoted by Gavin). If the bar for suitability is placed at taking =
only 1% of the block time (6 seconds) to upload one block to one peer, then=
 69% of the network fails for 20MB blocks. For comparison, only 10% fail th=
is metric for 1MB blocks.<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
</span><a href=3D"http://bitcoin-dev@lists.linuxfoundation.org" target=3D"_=
blank">bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
target=3D"_blank">https://lists.linuxfoundation.org/mailman/listinfo/bitcoi=
n-dev</a></blockquote>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
<br>_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
<br></blockquote></div><br></div>

--001a11c33d3201d4a8051b8e66eb--