From: Tom <tomz@freedommail.ch>
To: Gregory Maxwell <greg@xiph.org>
Date: Mon, 09 May 2016 12:32:59 +0100
Message-ID: <2273040.Bd6rtJjYLF@garp>
In-Reply-To: <CAAS2fgRiSNNHA5psaUYOM6rHfjJ1aOgWhnsT8Z-pU4FBcR_65w@mail.gmail.com>
References: <5727D102.1020807@mattcorallo.com> <86058327.pdmfHP132A@kiwi>
	<CAAS2fgRiSNNHA5psaUYOM6rHfjJ1aOgWhnsT8Z-pU4FBcR_65w@mail.gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Compact Block Relay BIP
Precedence: list

On Monday 09 May 2016 10:43:02 Gregory Maxwell wrote:
> On Mon, May 9, 2016 at 9:35 AM, Tom Zander via bitcoin-dev
> 
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
> > You misunderstand the networking effects.
> > The fact that your node is required to choose which one to set the
> > announce
> > bit on implies that it needs to predict which node will have the best data
> > in the future.
> 
> Not required. It may. 

It is required, in the reference of wanting to actually use compact block 
relay.


> Testing on actual nodes in the actual network (not a "lab") shows

Apologies, I thought that the term was wider known.  "Laboratory situations" 
is used where I am from as the opposite of real-world messy and unpredictable 
situations.

So, your measurements may be true, but are not useful to decide how well it 
behaves under less optimal situations. aka "the real world".

> This also _increases_ robustness. Right now a single peer failing at
> the wrong time will delay blocks with a long time out.

If your peers that were supposed to send you a compact block fail, then you'll 
end up in exactly that same situation again.  Only with various timeouts in 
between before you get your block making it a magnitude slower.

In networking this is solved by reacting instead of predicting. The network is 
not stable. Your protocol design assumes it to be.


> > Another problem with your solution is that nodes send a much larger amount
> > of unsolicited data to peers in the form of the thin-block compared to
> > the normal inv or header-first data.
> 
> "High bandwidth" mode 

Another place where I may have explained better.
This is not about the difference about the two modes of your design.
This is about the design as a whole. As compared to current.


> > Am I to understand that you choose the solution based on the fact that
> > service bits are too expensive to extend? (if not, please respond to my
> > previous question actually answering the suggestion)
> > 
> > That sounds like a rather bad way of doing design. Maybe you can add a
> > second service bits field of message instead and then do the compact
> > blocks correctly.
> Service bits are not generally a good mechanism for negating optional
> peer-local parameters.

Service bits are exactly the right solution to indicate additional p2p 
feature-support.


> [It's a little disconcerting that you appear to be maintaining a fork
> and are unaware of this.]

ehm...


> > Wait, you didn't steal the variable length encoding from an existing
> > standard and you programmed a new one?
> 
> This is one of the two variable length encodings used for years in
> Bitcoin Core. This is just the first time it's shown up in a BIP.
>
> > Look at UTF-8 on wikipedia, you may have "invented" the same encoding that
> > IBM published in 1992.
> 
> The similarity with UTF-8 is that both are variable length and some
> control information is in the high bits. The similarity ends there.

That's all fine and well, it doesn't at any point take away from my point that 
any specification should NOT invent something new that has for decades had a 
great specification already.

If you make a spec to be used by all nodes, on the wire, don't base it on your 
proprietary implementation. Please.


> > Just the first (highest) 8 bytes of a sha256 hash.
> > 
> > The amount of collisions will not be less if you start xoring the rest.
> > The whole reason for doing this extra work is also irrelevant as a spam
> > protection.
> 
> Then you expose it to a trivial collision attack:  To find two 64 bit
> hashes that collide I need perform only roughly 2^32 computation. Then
> I can send them to the network.

No, you still need to have done a POW.

Next to that, your scheme is 2^32 computations *and* some XORs. The XORs are 
percentage wise a rounding error on the total time. So your argument also 
destroys your own addition.

> This issue is eliminated by salting the hash. 

The issue is better eliminated by not allowing nodes to send uninvited large 
messages.

I don't think we're getting anywhere.

I'm not sold on your design and I explained why. I tried explaining in this 
email some misconceptions that may have appeared after my initial emails. I 
hope things are more clear.