Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 3F5F692B for ; Tue, 10 Nov 2015 16:30:59 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-qg0-f43.google.com (mail-qg0-f43.google.com [209.85.192.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 86CD71A1 for ; Tue, 10 Nov 2015 16:30:58 +0000 (UTC) Received: by qgec40 with SMTP id c40so1825760qge.2 for ; Tue, 10 Nov 2015 08:30:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=evFncqOQbIeZl9j6td/KjoRU+uxMc65xPd3gIcv9Ppg=; b=DiCwiUyoAoFNPkCGqBMENDv3TevsTuwLLkK7lrnV2frmsz9VXdSEv81SajR3+wrKFL ejuOZusMceJogT+/GPl5l+/LMOB1Gf3HMTs0RWOKP50pv8tNemTRAyYsENneJol0mZq9 KzMnNfWV7JmJenjShTZhKDYQwmSoVrj0jqYlVz/KbYJRMSO0WCuHVMxqMQwVIIipnnVv UN+ilqsrYadbmmCLRV4evbgi3498RIK1FTOm90IpQ3ApbNBEDQKaajs14VBd7+h89VWG TKdVdZCRX+b7jfg/SiUDN0RhggtEsyTR2i/eOaa8NNOCWRPRBSnPSJbzhafDHPCcjFwD WKsA== MIME-Version: 1.0 X-Received: by 10.140.181.197 with SMTP id c188mr5867384qha.4.1447173057616; Tue, 10 Nov 2015 08:30:57 -0800 (PST) Received: by 10.140.93.82 with HTTP; Tue, 10 Nov 2015 08:30:57 -0800 (PST) In-Reply-To: <5642172C.701@gmail.com> References: <5640F172.3010004@gmail.com> <20151109210449.GE5886@mcelrath.org> <5642172C.701@gmail.com> Date: Tue, 10 Nov 2015 16:30:57 +0000 Message-ID: From: Tier Nolan To: Bitcoin Dev Content-Type: multipart/alternative; boundary=001a113a95be0721d70524323ce2 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream Compression" X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2015 16:30:59 -0000 --001a113a95be0721d70524323ce2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Nov 10, 2015 at 4:11 PM, Peter Tschipper wrote: > There are better ways of sending new blocks, that's certainly true but fo= r > sending historical blocks and seding transactions I don't think so. This > PR is really designed to save bandwidth and not intended to be a huge > performance improvement in terms of time spent sending. > If the main point is for historical data, then sticking to just blocks is the best plan. Since small blocks don't compress well, you could define a "cblocks" message that handles multiple blocks (just concatenate the block messages as payload before compression). The sending peer could combine blocks so that each cblock is compressing at least 10kB of block data (or whatever is optimal). It is probably worth specifying a maximum size for network buffer reasons (either 1MB or 1 block maximum). Similarly, transactions could be combined together and compressed "ctxs". The inv messages could be modified so that you can request groups of 10-20 transactions. That would depend on how much of an improvement compressed transactions would represent. More generally, you could define a message which is a compressed message holder. That is probably to complex to be worth the effort though. > > On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev < > > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> >> >>> I think 25% bandwidth savings is certainly considerable, especially for >>> people running full nodes in countries like Australia where internet >>> bandwidth is lower and there are data caps. >>> >> >> =E2=80=8BThis reinforces the idea that such trade-off decisions should b= e be >> local and negotiated between peers, not a required feature of the networ= k >> P2P.=E2=80=8B >> >> >> -- >> Johnathan Corgan >> Corgan Labs - SDR Training and Development Services >> http://corganlabs.com >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> > > > _______________________________________________ > bitcoin-dev mailing listbitcoin-dev@lists.linuxfoundation.orghttps://list= s.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > --001a113a95be0721d70524323ce2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue, Nov 10, 2015 at 4:11 PM, Peter Tschipper <= peter.tschip= per@gmail.com> wrote:
=20 =20 =20 There are better ways of = sending new blocks, that's certainly true but for sending historical blocks and seding transactions I don't think so.=C2=A0 This PR is really designed to save bandwidth and not intended to be a huge performance improvement in terms of time spent sending.

If= the main point is for historical data, then sticking to just blocks is the= best plan.

Since small blocks don't compress well, y= ou could define a "cblocks" message that handles multiple blocks = (just concatenate the block messages as payload before compression).=C2=A0 =

The sending peer could combine blocks so that each cblock is compre= ssing at least 10kB of block data (or whatever is optimal).=C2=A0 It is pro= bably worth specifying a maximum size for network buffer reasons (either 1M= B or 1 block maximum).

Similarly, transactions could be c= ombined together and compressed "ctxs".=C2=A0 The inv messages co= uld be modified so that you can request groups of 10-20 transactions.=C2=A0= That would depend on how much of an improvement compressed transactions wo= uld represent.

More generally, you could define a messag= e which is a compressed message holder.=C2=A0 That is probably to complex t= o be worth the effort though.

=C2=A0

On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev <bitcoi= n-dev@lists.linuxfoundation.org> wrote:
On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.o= rg> wrote:
=C2=A0
I think 25% bandwidth savings is certainly considerable, especially for people running full nodes in countries like Australia where internet bandwidth is lower and there are data caps.
=E2=80=8B= This reinforces the idea that such trade-off decisions should be be local and negotiated between peers, not a required feature of the network P2P.=E2=80=8B =C2=A0

--
Johnathan Corgan
Corgan Labs - SDR Training and Development Services

_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundat= ion.org/mailman/listinfo/bitcoin-dev




_______________________________________________
bitcoin-dev mailing list
=
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoi=
n-dev


--001a113a95be0721d70524323ce2--