In-Reply-To: <CAAS2fgRoapt+z4fQd53NDGKWwZD=JLFefNCopS6u4SKrhN9A=A@mail.gmail.com>
References: <CALwsPgnnkbfUBhL=Qyspz13pnZ-6RHdaZOGvfLG34JjJRgt2Dw@mail.gmail.com>
	<B3546CB9-6A24-474C-8B56-9B1E2D33B470@mattcorallo.com>
	<CALwsPgm6xcBfLXZTNTZ40R_s3oUawE0ANZycDWpSo0cXZ+=-Vg@mail.gmail.com>
	<CAAS2fgQLNH+rivNNTFz6xt_9SxO3fFj7-3z7A-_B_2X2x-6M5w@mail.gmail.com>
	<CALwsPgm9S3UNd3bEuWreyGS7bcvSD+cXxueoD+F_D9fC=xLz2Q@mail.gmail.com>
	<CAAS2fgSXWzCv=4cF=0bwL9+udzBHSPR7goL3U_c1NjS22dpWzQ@mail.gmail.com>
	<CAKzdR-qPtPsxAXsmUX=vTkq-ro=EAmH7M8nL_Px_b4D4Z0WAXg@mail.gmail.com>
	<55C3A4BF.1010509@thinlink.com>
	<CAAS2fgRoapt+z4fQd53NDGKWwZD=JLFefNCopS6u4SKrhN9A=A@mail.gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain;
 charset=UTF-8
From: Matt Corallo <lf-lists@mattcorallo.com>
Date: Thu, 06 Aug 2015 20:50:32 +0000
To: Gregory Maxwell <gmaxwell@gmail.com>,
	Gregory Maxwell via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>,
	Tom Harding <tomh@thinlink.com>
Message-ID: <5C79A979-5D68-4A1C-B33A-177212506D24@mattcorallo.com>
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Idea: Efficient bitcoin block propagation
Precedence: list


On August 6, 2015 8:42:38 PM GMT+02:00, Gregory Maxwell via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>On Thu, Aug 6, 2015 at 6:17 PM, Tom Harding via bitcoin-dev
><bitcoin-dev@lists.linuxfoundation.org> wrote:
>>  - Will the relay network at least validate block version numbers in
>the
>> future?
>
>It already validates block version numbers.
>
>It only relays valid transactions.
>
>Although, the block relaying itself is explicitly "unvalidated" and
>the software client can only usefully be used with a mempool
>maintaining full node (otherwise it doesn't provide much value,
>because the node must wait to validate the things). ... but that
>doesn't actually mean no validation at all is performed, many
>stateless checks are performed.
>
>On Thu, Aug 6, 2015 at 5:16 PM, Sergio Demian Lerner via bitcoin-dev
><bitcoin-dev@lists.linuxfoundation.org> wrote:
>> Is there any up to date documentation about TheBlueMatt relay network
>> including what kind of block compression it is currently doing?
>(apart from
>> the source code)
>
>I don't know if Matt has an extensive writeup. But the basic
>optimization it performs is trivial.  I wouldn't call it compression,
>though it does have some analog to RTP "header compression".
>
>All it does is relay transactions verified by a local node and keeps a
>FIFO of the relayed transactions in both directions, which is
>synchronous on each side.
>
>When a block is recieved on either side, it replaces transactions with
>their indexes in the FIFO and relays it along. Transactions not in the
>fifo are escaped and sent whole. On the other side the block is
>reconstructed using the stored data and handed to the node (where the
>preforwarded transactions would have also been pre-validated).
>
>There is some more than basic elaboration for resource management
>(e.g. multiple queues for different transaction sizes)-- and more

No, just one queue, but it has a count-of-oversize-txn-limit, in addition to a size.

>recently using block templates to learn transaction priority be a bit
>more immune to spam attacks, but its fairly simple.

Except it doesn't really work :( (see https://github.com/TheBlueMatt/RelayNode/issues/12#issuecomment-128234446)

>Much better could be done about intelligently managing the queues or
>efficiently transmitting the membership sets, etc.  It's just
>basically the simplest thing that isn't completely stupid.

Patches welcome :) (read the issues list first... Rewriting the protocol from scratch is by far not the biggest win here).

Matt