Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.214.177 as permitted sender)
	client-ip=209.85.214.177; envelope-from=keziahw@gmail.com;
	helo=mail-ob0-f177.google.com; 
MIME-Version: 1.0
In-Reply-To: <CAAS2fgSObqk=rD1vtV6LZzxUuyQMh+nwGuatOq1hUaQz2od0sg@mail.gmail.com>
References: <CA+iPb=EaX=bvOjNtZ+LnYTMRLQQ9nFcrefAkBdv8eActoX_b8A@mail.gmail.com>
	<CABsx9T2PSa3MpfMMDCb8ACVF5vDOZOFLEK9zfP9PakgHA4U16w@mail.gmail.com>
	<CAPkFh0vKFnKRE-sd-Z9t1zB73VLPsiaQ3o=OYgBqqtUE4_rTaw@mail.gmail.com>
	<CA+iPb=GC7iw1LP6boyfX22oMO2k2=YcAuRhE0E3OzzJHYapsow@mail.gmail.com>
	<CAAS2fgS-KiP-tiy91Ah2hJ0pepA0OJDCG+Bv+redFtsqrUTevQ@mail.gmail.com>
	<CA+iPb=Fa4YSTjPuCfyWy0wB2XBV=Mi99G3Hb84gjy+muNDin+g@mail.gmail.com>
	<CAAS2fgSObqk=rD1vtV6LZzxUuyQMh+nwGuatOq1hUaQz2od0sg@mail.gmail.com>
From: Kaz Wesley <keziahw@gmail.com>
Date: Thu, 31 Jul 2014 15:27:59 -0700
Message-ID: <CA+iPb=FV1_0SCzcqCz+2eeQW6L18c2O2aKW4zusgNKBYirqHcA@mail.gmail.com>
To: Gregory Maxwell <gmaxwell@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Squashing redundant tx data in blocks on
 the wire
Precedence: list

> the FEC still lets you fill in the missing transactions without knowing i=
n advance all that will be missing.

I don't see why we need to solve that problem, since the protocol
already involves exchanging the information necessary to determine
(with some false positives) what a peer is missing, and needs to
continue doing so regardless of how blocks are transmitted.

Set reconciliation does have the benefit of eliminating a subset of
those false positives and offering a finer-grained mechanism for
defining what a node can choose to forget from its mempool than
remember-last-N, but if we implement it for block transmission I don't
see why we wouldn't also use it to synchronize mempool txes, and if
mempools are synchronized we don't actually need to do it as part of
block-transmission to get those benefits.

As far as I can tell, channel memory sparseblocks achieve most of the
possible bandwidth savings, and when FEC-based mempool synchronization
is implemented its benefits can be applied to the sparseblocks by
resetting the channel memory to the mutual mempool state each time
mempool differences are exchanged. Am I missing a benefit to doing FEC
at block forwarding time that can't be realized by FEC-based mempool
synchronization, implemented separately from channel-memory based
index-coding?


On Thu, Jul 31, 2014 at 2:51 PM, Gregory Maxwell <gmaxwell@gmail.com> wrote=
:
> On Thu, Jul 31, 2014 at 2:41 PM, Kaz Wesley <keziahw@gmail.com> wrote:
>>> the need to have transmitted the transaction list [..] first
>>
>> 32 bits per transaction is at least double the communication overhead
>> of the simple approach, and only offers a bound on the probability of
>> needing a round trip.
>
> "(e.g. from a reconciliation step first)" the list can be communicated
> in the space roughly equal to the size of the difference in sets plus
> coding the permutation from the permissible orderings.   If you did
> have some "simple approach" that guaranteed that some transactions
> would be present, then you could code those with indexes... the FEC
> still lets you fill in the missing transactions without knowing in
> advance all that will be missing.   (Also, the suggestion on the
> network block coding page of using part of a cryptographic permutation
> as the key means that for unknown transactions the transmission of the
> new unknown keys is always goodput=E2=80=94 doesn't add overhead)
>
> It's "only a bound" but you can pick whatever bound you want,
> including=E2=80=94 if you send data equal to the missing amount, then it'=
ll be
> always successful, but no bandwidth savings.   Though if the transport
> is unordered (e.g. UDP or non-blocking SCTP) even sending 100% of the
> missing amount can save time by eliminating a round trip that might
> otherwise be needed for a retransmission.