MIME-Version: 1.0
In-Reply-To: <CALOxbZtTUrZwDfy_jTbs60n=K8RKDGg5X0gkLsh-OX3ikLf1FQ@mail.gmail.com>
References: <5640F172.3010004@gmail.com> <20151109210449.GE5886@mcelrath.org>
	<CAL7-sS0Apm4O_Qi0FmY7=H580rEVD6DYjk2y+ACpZmKqUJTQwA@mail.gmail.com>
	<CALOxbZtTUrZwDfy_jTbs60n=K8RKDGg5X0gkLsh-OX3ikLf1FQ@mail.gmail.com>
Date: Tue, 10 Nov 2015 09:44:11 +0000
Message-ID: <CAE-z3OUB-se_HUvW2NLjWt=0d5sgMiPEciu0hLzr_HQN0m9fqQ@mail.gmail.com>
From: Tier Nolan <tier.nolan@gmail.com>
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a11c1344256108105242c8d98
Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream
	Compression"
Precedence: list

--001a11c1344256108105242c8d98
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

The network protocol is not quite consensus critical, but it is important.

Two implementations of the decompressor might not be bug for bug
compatible.  This (potentially) means that a block could be designed that
won't decode properly for some version of the client but would work for
another.  This would fork the network.

A "raw" network library is unlikely to have the same problem.

Rather than just compress the stream, you could compress only block
messages only.  A new "cblock" message could be created that is a
compressed block.  This shouldn't reduce efficiency by much.

If a client fails to decode a cblock, then it can ask for the block to be
re-sent as a standard "block" message.

This means that it is a pure performance improvement.  If problems occur,
then the client can just switch back to uncompressed mode for that block.

You should look into the block relay system.  This gives a larger
improvement than simply compressing the stream.  The main benefit is
latency but it means that actual blocks don't have to be sent, so gives a
potential 50% compression ratio.  Normally, a node receives all the
transactions and then those transactions are included later in the block.



On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>
>> I think 25% bandwidth savings is certainly considerable, especially for
>> people running full nodes in countries like Australia where internet
>> bandwidth is lower and there are data caps.
>>
>
> =E2=80=8BThis reinforces the idea that such trade-off decisions should be=
 be local
> and negotiated between peers, not a required feature of the network P2P.=
=E2=80=8B
>
>
> --
> Johnathan Corgan
> Corgan Labs - SDR Training and Development Services
> http://corganlabs.com
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

--001a11c1344256108105242c8d98
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div><div>The network protocol is not quite=
 consensus critical, but it is important.<br><br></div>Two implementations =
of the decompressor might not be bug for bug compatible.=C2=A0 This (potent=
ially) means that a block could be designed that won&#39;t decode properly =
for some version of the client but would work for another.=C2=A0 This would=
 fork the network.<br><br></div><div>A &quot;raw&quot; network library is u=
nlikely to have the same problem.<br></div><div><br></div>Rather than just =
compress the stream, you could compress only block messages only.=C2=A0 A n=
ew &quot;cblock&quot; message could be created that is a compressed block.=
=C2=A0 This shouldn&#39;t reduce efficiency by much.<br><br></div>If a clie=
nt fails to decode a cblock, then it can ask for the block to be re-sent as=
 a standard &quot;block&quot; message.=C2=A0 <br><br></div>This means that =
it is a pure performance improvement.=C2=A0 If problems occur, then the cli=
ent can just switch back to uncompressed mode for that block.<br><br></div>=
You should look into the block relay system.=C2=A0 This gives a larger impr=
ovement than simply compressing the stream.=C2=A0 The main benefit is laten=
cy but it means that actual blocks don&#39;t have to be sent, so gives a po=
tential 50% compression ratio.=C2=A0 Normally, a node receives all the tran=
sactions and then those transactions are included later in the block.<br><d=
iv><div><br><br></div></div></div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitc=
oin-dev <span dir=3D"ltr">&lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoun=
dation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a>&gt;=
</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .=
8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><span cla=
ss=3D""><div class=3D"gmail_default" style=3D"font-size:small">On Mon, Nov =
9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <span dir=3D"ltr">&lt;<a href=
=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitcoin=
-dev@lists.linuxfoundation.org</a>&gt;</span> wrote:<br></div></span><div c=
lass=3D"gmail_extra"><div class=3D"gmail_quote"><span class=3D""><div>=C2=
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde=
r-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">I think 25% bandwi=
dth savings is certainly considerable, especially for people running full n=
odes in countries like Australia where internet bandwidth is lower and ther=
e are data caps.</div></blockquote><div><br></div></span><div><div class=3D=
"gmail_default" style=3D"font-size:small;display:inline">=E2=80=8BThis rein=
forces the idea that such trade-off decisions should be be local and negoti=
ated between peers, not a required feature of the network P2P.=E2=80=8B</di=
v>=C2=A0</div></div><span class=3D""><div><br></div>-- <br><div><div dir=3D=
"ltr"><div><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"l=
tr"><div dir=3D"ltr"><div>Johnathan Corgan<br>Corgan Labs - SDR Training an=
d Development Services</div><div><a href=3D"http://corganlabs.com" style=3D=
"font-size:12.8px" target=3D"_blank">http://corganlabs.com</a><br></div></d=
iv></div></div></div></div></div></div></div>
</span></div></div>
<br>_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
<br></blockquote></div><br></div>

--001a11c1344256108105242c8d98--