Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 98C7F268 for ; Tue, 10 Nov 2015 01:59:02 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-yk0-f182.google.com (mail-yk0-f182.google.com [209.85.160.182]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D00AF8C for ; Tue, 10 Nov 2015 01:59:01 +0000 (UTC) Received: by ykdv3 with SMTP id v3so205573714ykd.0 for ; Mon, 09 Nov 2015 17:59:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=glados_cc.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=z90xfCln9MKLlZU8g2hbPsprNfbRXbHaDlo/44IkC7Q=; b=NH/T0kVgcARsUvyeYGJWcchAP8g0n+1TiSIVmE+y07Je4Ym9/6DxhSlwtQ+Tk2DSCi OalPtTlgkpPdzry99sCDYJqIIdiRGvdK2sSxDgCx57EXYgvex7uBtvceNuodU4DiChth Lx0D/YMerDVyu03uMUqtzoQtM7gvG5EUjg5qtIZpSwGZ+J3feAvY1kSkGaUxRQLDdSMN OSauV37EOyLjA7JoTFKn4qns0TunkN3wzvxT3GZvDLYnJmkhtNQMKldHPMF7ODGJSeHe +8cjML2+1VA65WnukJcaoRZYI18dVVjJdsI13hP6GEsFIKhcsJmaDLCamrP+ZyQjyze7 /bDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=z90xfCln9MKLlZU8g2hbPsprNfbRXbHaDlo/44IkC7Q=; b=DslroqV5u9wxE9mVzkOjIxOVN1x/Mv36UV4gpFPbE5pkDMLjZyicZk+9kW+EN0ATUX A8om06m2AFYp8SImm+9nkA6u6lBQJNDP+ttg850zcj0sJMyG/WNvdCdLFrk84bdNtzdi OO+TnjH7C9vdH966OYqnpn1/CWaUypOS2laQ59BI6OB8fw6QIuGD+s8mBLM1ObMuoa7L SFPkt2t1BIKWmMv8b7687SWzjsWub0o6sK+jDTTiY2aYCncfZXFaA/pE8qN8yTX9CUlJ adPqdQ63PbQwnsF14rpBWm8ZZRD708cRTkDzuxX0+lJt2AAvvrfjEhif7NSFkC8cZmgG s92w== X-Gm-Message-State: ALoCoQlE5/TPwHJEUMltRCG+Y2lj0vnZ5q5gyWrlVz0QX2Rmw2cgIMBDn3xGB7ca20ek7RsGVMWc X-Received: by 10.13.228.65 with SMTP id n62mr881672ywe.302.1447120741055; Mon, 09 Nov 2015 17:59:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.224.135 with HTTP; Mon, 9 Nov 2015 17:58:41 -0800 (PST) X-Originating-IP: [64.213.30.122] In-Reply-To: <20151109210449.GE5886@mcelrath.org> References: <5640F172.3010004@gmail.com> <20151109210449.GE5886@mcelrath.org> From: gladoscc Date: Tue, 10 Nov 2015 12:58:41 +1100 Message-ID: To: Bob McElrath Content-Type: multipart/alternative; boundary=94eb2c030c10b7eecf0524260de0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Tue, 10 Nov 2015 01:59:33 +0000 Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream Compression" X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2015 01:59:02 -0000 --94eb2c030c10b7eecf0524260de0 Content-Type: text/plain; charset=UTF-8 I think 25% bandwidth savings is certainly considerable, especially for people running full nodes in countries like Australia where internet bandwidth is lower and there are data caps. I absolutely would not dismiss 25% compression. gzip and bzip2 compression is relatively standard, and I'd consider the point of implementation complexity tradeoff to be somewhere along 5-10%. On Tue, Nov 10, 2015 at 8:04 AM, Bob McElrath via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > I would expect that since a block contains mostly hashes and crypto > signatures, > it would be almost totally incompressible. I just calculated compression > ratios: > > zlib -15% (file is LARGER) > gzip 28% > bzip2 25% > > So zlib compression is right out. How much is ~25% bandwidth savings > worth to > people? This seems not worth it to me. :-/ > > Peter Tschipper via bitcoin-dev [bitcoin-dev@lists.linuxfoundation.org] > wrote: > > This is my first time through this process so please bear with me. > > > > I opened a PR #6973 this morning for Zlib Block Compression for block > > relay and at the request of @sipa this should have a BIP associated > > with it. The idea is simple, to compress the datastream before > > sending, initially for blocks only but it could theoretically be done > > for transactions as well. Initial results show an average of 20% block > > compression and taking 90 milliseconds for a full block (on a very slow > > laptop) to compress. The savings will be mostly in terms of less > > bandwidth used, but I would expect there to be a small performance gain > > during the transmission of the blocks particularly where network latency > > is higher. > > > > I think the BIP title, if accepted should be the more generic, "Support > > for Datastream Compression" rather than the PR title of "Zlib > > Compression for block relay" since it could also be used for > > transactions as well at a later time. > > > > Thanks for your time... > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists.linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > > > !DSPAM:5640ff47206804314022622! > -- > Cheers, Bob McElrath > > "For every complex problem, there is a solution that is simple, neat, and > wrong." > -- H. L. Mencken > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --94eb2c030c10b7eecf0524260de0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I think 25% bandwidth savings is certainly considerable, e= specially for people running full nodes in countries like Australia where i= nternet bandwidth is lower and there are data caps.

I ab= solutely would not dismiss 25% compression. gzip and bzip2 compression is r= elatively standard, and I'd consider the point of implementation comple= xity tradeoff to be somewhere along 5-10%.=C2=A0

On Tue, Nov 10, 2015 at 8:04 AM,= Bob McElrath via bitcoin-dev <bitcoin-dev@lists.linux= foundation.org> wrote:
I wo= uld expect that since a block contains mostly hashes and crypto signatures,=
it would be almost totally incompressible.=C2=A0 I just calculated compress= ion ratios:

zlib=C2=A0 =C2=A0 -15%=C2=A0 =C2=A0 (file is LARGER)
gzip=C2=A0 =C2=A0 =C2=A028%
bzip2=C2=A0 =C2=A0 25%

So zlib compression is right out.=C2=A0 How much is ~25% bandwidth savings = worth to
people?=C2=A0 This seems not worth it to me.=C2=A0 :-/

Peter Tschipper via bitcoin-dev [bitcoin-dev@lists.linuxfoundation.org] wrote:
> This is my first time through this proce= ss so please bear with me.
>
> I opened a PR #6973 this morning for Zlib Block Compression for block<= br> > relay and at the request of @sipa=C2=A0 this should have a BIP associa= ted
> with it.=C2=A0 =C2=A0The idea is simple, to compress the datastream be= fore
> sending, initially for blocks only but it could theoretically be done<= br> > for transactions as well.=C2=A0 Initial results show an average of 20%= block
> compression and taking 90 milliseconds for a full block (on a very slo= w
> laptop) to compress.=C2=A0 The savings will be mostly in terms of less=
> bandwidth used, but I would expect there to be a small performance gai= n
> during the transmission of the blocks particularly where network laten= cy
> is higher.
>
> I think the BIP title, if accepted should be the more generic, "S= upport
> for Datastream Compression"=C2=A0 rather than the PR title of &qu= ot;Zlib
> Compression for block relay" since it could also be used for
> transactions as well at a later time.
>
> Thanks for your time...
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@l= ists.linuxfoundation.org
> https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev
>
>
> !DSPAM:5640ff47206804314022622!
--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, = and wrong."
=C2=A0 =C2=A0 -- H. L. Mencken

_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev

--94eb2c030c10b7eecf0524260de0--