1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
|
Return-Path: <tier.nolan@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 952FA8FE
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Nov 2015 09:44:13 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-qg0-f43.google.com (mail-qg0-f43.google.com
[209.85.192.43])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D4CA5FD
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Nov 2015 09:44:12 +0000 (UTC)
Received: by qgeb1 with SMTP id b1so113379723qge.1
for <bitcoin-dev@lists.linuxfoundation.org>;
Tue, 10 Nov 2015 01:44:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:cc
:content-type; bh=1+TuijMrRC0Bw2dfnW6mqz3KsiIFAKYLfYiMWrZHp6M=;
b=jUjO7RMSQ23bYCo9gwWZJfYrCrD/Z/aHX6B436iKiELX6tQytrM0igM58zRe47WcAd
qLh3G42PLsHNuajwaqNujJgZoFlgMzVmHwkE/C+y2Dzg120JSxTqXv2iR8GQAuMHfy0v
2XpGFU2qRZiEseHqVkZNQGaLaVrcj5wJY7FTZ8ZZXW79wqW7q/zOzEVeeHAW6ek2n73q
wIIy/hUQr8YblO/OLukonI29Hc+TLvVkaT7QE25HSWLQ819g07Hx6AXe6vmGzs305chR
/sAidH6Sq9FH0Zpi8Wr0XdrdscToGWYuP4vB4zRE/lJRWp8zuf/+ne8JtjYv30iebG28
QXpA==
MIME-Version: 1.0
X-Received: by 10.140.81.166 with SMTP id f35mr2846469qgd.63.1447148651940;
Tue, 10 Nov 2015 01:44:11 -0800 (PST)
Received: by 10.140.93.82 with HTTP; Tue, 10 Nov 2015 01:44:11 -0800 (PST)
In-Reply-To: <CALOxbZtTUrZwDfy_jTbs60n=K8RKDGg5X0gkLsh-OX3ikLf1FQ@mail.gmail.com>
References: <5640F172.3010004@gmail.com> <20151109210449.GE5886@mcelrath.org>
<CAL7-sS0Apm4O_Qi0FmY7=H580rEVD6DYjk2y+ACpZmKqUJTQwA@mail.gmail.com>
<CALOxbZtTUrZwDfy_jTbs60n=K8RKDGg5X0gkLsh-OX3ikLf1FQ@mail.gmail.com>
Date: Tue, 10 Nov 2015 09:44:11 +0000
Message-ID: <CAE-z3OUB-se_HUvW2NLjWt=0d5sgMiPEciu0hLzr_HQN0m9fqQ@mail.gmail.com>
From: Tier Nolan <tier.nolan@gmail.com>
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary=001a11c1344256108105242c8d98
X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
MALFORMED_FREEMAIL,
MISSING_HEADERS,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream
Compression"
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Nov 2015 09:44:13 -0000
--001a11c1344256108105242c8d98
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
The network protocol is not quite consensus critical, but it is important.
Two implementations of the decompressor might not be bug for bug
compatible. This (potentially) means that a block could be designed that
won't decode properly for some version of the client but would work for
another. This would fork the network.
A "raw" network library is unlikely to have the same problem.
Rather than just compress the stream, you could compress only block
messages only. A new "cblock" message could be created that is a
compressed block. This shouldn't reduce efficiency by much.
If a client fails to decode a cblock, then it can ask for the block to be
re-sent as a standard "block" message.
This means that it is a pure performance improvement. If problems occur,
then the client can just switch back to uncompressed mode for that block.
You should look into the block relay system. This gives a larger
improvement than simply compressing the stream. The main benefit is
latency but it means that actual blocks don't have to be sent, so gives a
potential 50% compression ratio. Normally, a node receives all the
transactions and then those transactions are included later in the block.
On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>
>> I think 25% bandwidth savings is certainly considerable, especially for
>> people running full nodes in countries like Australia where internet
>> bandwidth is lower and there are data caps.
>>
>
> =E2=80=8BThis reinforces the idea that such trade-off decisions should be=
be local
> and negotiated between peers, not a required feature of the network P2P.=
=E2=80=8B
>
>
> --
> Johnathan Corgan
> Corgan Labs - SDR Training and Development Services
> http://corganlabs.com
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
--001a11c1344256108105242c8d98
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><div><div><div><div>The network protocol is not quite=
consensus critical, but it is important.<br><br></div>Two implementations =
of the decompressor might not be bug for bug compatible.=C2=A0 This (potent=
ially) means that a block could be designed that won't decode properly =
for some version of the client but would work for another.=C2=A0 This would=
fork the network.<br><br></div><div>A "raw" network library is u=
nlikely to have the same problem.<br></div><div><br></div>Rather than just =
compress the stream, you could compress only block messages only.=C2=A0 A n=
ew "cblock" message could be created that is a compressed block.=
=C2=A0 This shouldn't reduce efficiency by much.<br><br></div>If a clie=
nt fails to decode a cblock, then it can ask for the block to be re-sent as=
a standard "block" message.=C2=A0 <br><br></div>This means that =
it is a pure performance improvement.=C2=A0 If problems occur, then the cli=
ent can just switch back to uncompressed mode for that block.<br><br></div>=
You should look into the block relay system.=C2=A0 This gives a larger impr=
ovement than simply compressing the stream.=C2=A0 The main benefit is laten=
cy but it means that actual blocks don't have to be sent, so gives a po=
tential 50% compression ratio.=C2=A0 Normally, a node receives all the tran=
sactions and then those transactions are included later in the block.<br><d=
iv><div><br><br></div></div></div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitc=
oin-dev <span dir=3D"ltr"><<a href=3D"mailto:bitcoin-dev@lists.linuxfoun=
dation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a>>=
</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .=
8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><span cla=
ss=3D""><div class=3D"gmail_default" style=3D"font-size:small">On Mon, Nov =
9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <span dir=3D"ltr"><<a href=
=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">bitcoin=
-dev@lists.linuxfoundation.org</a>></span> wrote:<br></div></span><div c=
lass=3D"gmail_extra"><div class=3D"gmail_quote"><span class=3D""><div>=C2=
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde=
r-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">I think 25% bandwi=
dth savings is certainly considerable, especially for people running full n=
odes in countries like Australia where internet bandwidth is lower and ther=
e are data caps.</div></blockquote><div><br></div></span><div><div class=3D=
"gmail_default" style=3D"font-size:small;display:inline">=E2=80=8BThis rein=
forces the idea that such trade-off decisions should be be local and negoti=
ated between peers, not a required feature of the network P2P.=E2=80=8B</di=
v>=C2=A0</div></div><span class=3D""><div><br></div>-- <br><div><div dir=3D=
"ltr"><div><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"l=
tr"><div dir=3D"ltr"><div>Johnathan Corgan<br>Corgan Labs - SDR Training an=
d Development Services</div><div><a href=3D"http://corganlabs.com" style=3D=
"font-size:12.8px" target=3D"_blank">http://corganlabs.com</a><br></div></d=
iv></div></div></div></div></div></div></div>
</span></div></div>
<br>_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
<br></blockquote></div><br></div>
--001a11c1344256108105242c8d98--
|
