Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 952FA8FE for ; Tue, 10 Nov 2015 09:44:13 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-qg0-f43.google.com (mail-qg0-f43.google.com [209.85.192.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D4CA5FD for ; Tue, 10 Nov 2015 09:44:12 +0000 (UTC) Received: by qgeb1 with SMTP id b1so113379723qge.1 for ; Tue, 10 Nov 2015 01:44:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=1+TuijMrRC0Bw2dfnW6mqz3KsiIFAKYLfYiMWrZHp6M=; b=jUjO7RMSQ23bYCo9gwWZJfYrCrD/Z/aHX6B436iKiELX6tQytrM0igM58zRe47WcAd qLh3G42PLsHNuajwaqNujJgZoFlgMzVmHwkE/C+y2Dzg120JSxTqXv2iR8GQAuMHfy0v 2XpGFU2qRZiEseHqVkZNQGaLaVrcj5wJY7FTZ8ZZXW79wqW7q/zOzEVeeHAW6ek2n73q wIIy/hUQr8YblO/OLukonI29Hc+TLvVkaT7QE25HSWLQ819g07Hx6AXe6vmGzs305chR /sAidH6Sq9FH0Zpi8Wr0XdrdscToGWYuP4vB4zRE/lJRWp8zuf/+ne8JtjYv30iebG28 QXpA== MIME-Version: 1.0 X-Received: by 10.140.81.166 with SMTP id f35mr2846469qgd.63.1447148651940; Tue, 10 Nov 2015 01:44:11 -0800 (PST) Received: by 10.140.93.82 with HTTP; Tue, 10 Nov 2015 01:44:11 -0800 (PST) In-Reply-To: References: <5640F172.3010004@gmail.com> <20151109210449.GE5886@mcelrath.org> Date: Tue, 10 Nov 2015 09:44:11 +0000 Message-ID: From: Tier Nolan Cc: Bitcoin Dev Content-Type: multipart/alternative; boundary=001a11c1344256108105242c8d98 X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, MALFORMED_FREEMAIL, MISSING_HEADERS,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream Compression" X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2015 09:44:13 -0000 --001a11c1344256108105242c8d98 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable The network protocol is not quite consensus critical, but it is important. Two implementations of the decompressor might not be bug for bug compatible. This (potentially) means that a block could be designed that won't decode properly for some version of the client but would work for another. This would fork the network. A "raw" network library is unlikely to have the same problem. Rather than just compress the stream, you could compress only block messages only. A new "cblock" message could be created that is a compressed block. This shouldn't reduce efficiency by much. If a client fails to decode a cblock, then it can ask for the block to be re-sent as a standard "block" message. This means that it is a pure performance improvement. If problems occur, then the client can just switch back to uncompressed mode for that block. You should look into the block relay system. This gives a larger improvement than simply compressing the stream. The main benefit is latency but it means that actual blocks don't have to be sent, so gives a potential 50% compression ratio. Normally, a node receives all the transactions and then those transactions are included later in the block. On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > > >> I think 25% bandwidth savings is certainly considerable, especially for >> people running full nodes in countries like Australia where internet >> bandwidth is lower and there are data caps. >> > > =E2=80=8BThis reinforces the idea that such trade-off decisions should be= be local > and negotiated between peers, not a required feature of the network P2P.= =E2=80=8B > > > -- > Johnathan Corgan > Corgan Labs - SDR Training and Development Services > http://corganlabs.com > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > --001a11c1344256108105242c8d98 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
The network protocol is not quite= consensus critical, but it is important.

Two implementations = of the decompressor might not be bug for bug compatible.=C2=A0 This (potent= ially) means that a block could be designed that won't decode properly = for some version of the client but would work for another.=C2=A0 This would= fork the network.

A "raw" network library is u= nlikely to have the same problem.

Rather than just = compress the stream, you could compress only block messages only.=C2=A0 A n= ew "cblock" message could be created that is a compressed block.= =C2=A0 This shouldn't reduce efficiency by much.

If a clie= nt fails to decode a cblock, then it can ask for the block to be re-sent as= a standard "block" message.=C2=A0

This means that = it is a pure performance improvement.=C2=A0 If problems occur, then the cli= ent can just switch back to uncompressed mode for that block.

= You should look into the block relay system.=C2=A0 This gives a larger impr= ovement than simply compressing the stream.=C2=A0 The main benefit is laten= cy but it means that actual blocks don't have to be sent, so gives a po= tential 50% compression ratio.=C2=A0 Normally, a node receives all the tran= sactions and then those transactions are included later in the block.



On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitc= oin-dev <bitcoin-dev@lists.linuxfoundation.org>= wrote:
On Mon, Nov = 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <bitcoin= -dev@lists.linuxfoundation.org> wrote:
=C2= =A0
I think 25% bandwi= dth savings is certainly considerable, especially for people running full n= odes in countries like Australia where internet bandwidth is lower and ther= e are data caps.

=E2=80=8BThis rein= forces the idea that such trade-off decisions should be be local and negoti= ated between peers, not a required feature of the network P2P.=E2=80=8B=C2=A0

--
Johnathan Corgan
Corgan Labs - SDR Training an= d Development Services

_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev


--001a11c1344256108105242c8d98--