Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TdSQJ-0006GK-ON for bitcoin-development@lists.sourceforge.net; Tue, 27 Nov 2012 21:10:31 +0000 X-ACL-Warn: Received: from vps7135.xlshosting.net ([178.18.90.41]) by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1TdSQI-0001aH-5K for bitcoin-development@lists.sourceforge.net; Tue, 27 Nov 2012 21:10:31 +0000 Received: by vps7135.xlshosting.net (Postfix, from userid 1000) id CA60F611EA; Tue, 27 Nov 2012 22:10:23 +0100 (CET) Date: Tue, 27 Nov 2012 22:10:23 +0100 From: Pieter Wuille To: Matt Corallo Message-ID: <20121127211019.GA22701@vps7135.xlshosting.net> References: <20121121151534.GA5540@vps7135.xlshosting.net> <1353523117.1085.14.camel@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1353523117.1085.14.camel@localhost.localdomain> X-PGP-Key: http://sipa.ulyssis.org/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: 0.8 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (pieter.wuille[at]gmail.com) 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is CUSTOM_MED -0.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing list 0.0 AWL AWL: From: address is in the auto white-list X-Headers-End: 1TdSQI-0001aH-5K Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2012 21:10:31 -0000 On Wed, Nov 21, 2012 at 01:38:37PM -0500, Matt Corallo wrote: > On Wed, 2012-11-21 at 16:15 +0100, Pieter Wuille wrote: > > On Wed, Oct 24, 2012 at 05:56:07PM +0200, Mike Hearn wrote: > > > I've written a draft BIP describing the bloom filtering protocol > > > extension developed by myself and Matt. > > > > > > https://en.bitcoin.it/wiki/BIP_0037 > > > > Two comments I made on the pullreq page, which are probably better discussed here: > > * The 0xFFFFFFFF/(n-1)*i seed value seems intended to result in large bit > > differences between the different hash function's seeds. Together with the tweak, > > however, the first and the last now get seeds tweak and tweak-1. I think > > something simpler like k1*i+k2*n+tweak is better (with k1 and k2 arbitrary large > > odd 32-bit integers). > Meh, sure, whatever...I dont really think the seed values matter > significantly (Murmur3 isnt that bad of a hash function...) (and the > original algorithm wont result in a significant bit difference between > the seeds in many cases). Sure, it's nothing important, but it seems like it fails to do what it was intended for. How about just this: tweak + i*0xFBA4C795 (number optimized to give large seed differences for every tweak). If you want variation when changing the number of hash functions, just choose a different seed. > > Maybe the actual filter data in filterload can be made optional: > > if it is omitted, it's assumed to be all zeroes (though that would mean the size > > has to be specified). > > > I'm not sure here, if you are sending a filter just to use filteradd to > add things to it manually, you are doing something very, very, very > wrong... Though we could certainly do some kind of compressed bloom > filter encoding to allow for small filter loads (loading the few things > you need to filteradd right away) where you anticipate adding > significantly more filter elements down the road (can anyone even come > up with a case where you anticipate doing this?), the filter is small > enough (max 36kB) that I see little benefit for the large increase in > complexity (or is this another repeat of the merkle branch discussion?) It's probably not worth it for something that is max 36 kilobytes. If ever necessary, we can define a new message type that just lists a number of bits to be set in the server-side filter. For now, I agree that you should just send the filter as intended, and not expect to do many filteradds (though you should take the implicitly-added txids into accounted when computing the filter size). -- Pieter