Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1XCyG1-0005vO-RG for bitcoin-development@lists.sourceforge.net; Thu, 31 Jul 2014 21:51:29 +0000 Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.213.170 as permitted sender) client-ip=209.85.213.170; envelope-from=gmaxwell@gmail.com; helo=mail-ig0-f170.google.com; Received: from mail-ig0-f170.google.com ([209.85.213.170]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1XCyG0-0004SN-TI for bitcoin-development@lists.sourceforge.net; Thu, 31 Jul 2014 21:51:29 +0000 Received: by mail-ig0-f170.google.com with SMTP id h3so2442504igd.1 for ; Thu, 31 Jul 2014 14:51:23 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.50.111.132 with SMTP id ii4mr171680igb.8.1406843483572; Thu, 31 Jul 2014 14:51:23 -0700 (PDT) Received: by 10.107.14.67 with HTTP; Thu, 31 Jul 2014 14:51:23 -0700 (PDT) In-Reply-To: References: Date: Thu, 31 Jul 2014 14:51:23 -0700 Message-ID: From: Gregory Maxwell To: Kaz Wesley Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gmaxwell[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1XCyG0-0004SN-TI Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 21:51:30 -0000 On Thu, Jul 31, 2014 at 2:41 PM, Kaz Wesley wrote: >> the need to have transmitted the transaction list [..] first > > 32 bits per transaction is at least double the communication overhead > of the simple approach, and only offers a bound on the probability of > needing a round trip. "(e.g. from a reconciliation step first)" the list can be communicated in the space roughly equal to the size of the difference in sets plus coding the permutation from the permissible orderings. If you did have some "simple approach" that guaranteed that some transactions would be present, then you could code those with indexes... the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. (Also, the suggestion on the network block coding page of using part of a cryptographic permutation as the key means that for unknown transactions the transmission of the new unknown keys is always goodput=E2=80=94 doesn't add overhead) It's "only a bound" but you can pick whatever bound you want, including=E2=80=94 if you send data equal to the missing amount, then it'll= be always successful, but no bandwidth savings. Though if the transport is unordered (e.g. UDP or non-blocking SCTP) even sending 100% of the missing amount can save time by eliminating a round trip that might otherwise be needed for a retransmission.