Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1TtJwW-0002EW-Oc for bitcoin-development@lists.sourceforge.net; Thu, 10 Jan 2013 15:21:20 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.49 as permitted sender) client-ip=209.85.219.49; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f49.google.com; Received: from mail-oa0-f49.google.com ([209.85.219.49]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1TtJwV-0003Zn-OA for bitcoin-development@lists.sourceforge.net; Thu, 10 Jan 2013 15:21:20 +0000 Received: by mail-oa0-f49.google.com with SMTP id l10so682260oag.22 for ; Thu, 10 Jan 2013 07:21:14 -0800 (PST) MIME-Version: 1.0 Received: by 10.60.31.51 with SMTP id x19mr40893357oeh.25.1357831274405; Thu, 10 Jan 2013 07:21:14 -0800 (PST) Sender: mh.in.england@gmail.com Received: by 10.76.128.139 with HTTP; Thu, 10 Jan 2013 07:21:14 -0800 (PST) In-Reply-To: <20121127211019.GA22701@vps7135.xlshosting.net> References: <20121121151534.GA5540@vps7135.xlshosting.net> <1353523117.1085.14.camel@localhost.localdomain> <20121127211019.GA22701@vps7135.xlshosting.net> Date: Thu, 10 Jan 2013 16:21:14 +0100 X-Google-Sender-Auth: leNkaspaLux8Qi-Ne09WEt5Gp8U Message-ID: From: Mike Hearn To: Pieter Wuille Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1TtJwV-0003Zn-OA Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Draft BIP for Bloom filtering X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jan 2013 15:21:20 -0000 Here's a quick update on where we're up to. Thanks to Matts excellent work, I was able to test his bitcoinj and bitcoin-qt work together today. There are a few minor tweaks needed, but I feel like we're maybe a week away from having all the code in a mergeable state. Here is the remaining work: - There are a couple of bugfixes needed on the bitcoinj side: the fallback to downloading full blocks is problematic and needs to be deleted, there's an API change we want - Adjust the default FP rate requested by BCJ to be 0.0001, this is appropriate for the latest blocks in the chain and yields 0-5 false positives per block - Introduce a new part to the filter protocol that allows clients to control auto-expansion. This turned out to be very volatile, we saw jumps from 0-3 FPs per block to 500 in the space of 1 block, perhaps if a SatoshiDice transaction got into the filter. A simple yes/no flag can suffice for now, but a better solution would be for the client to submit templates for output scripts that would trigger auto-adding the matched outpoint - autoexpansion is only needed in the case where the input script doesn't contain any predictable data. For pay-to-address and P2SH it does, so expansion doesn't help. Matt said he'd hopefully try to look at this soon. With auto-expansion disabled, the FP rate adjusted and a bugfix on the bcj side I was able to sync a wallet using a bloom filtered chain. Although it's tight, I think this work should go into 0.8 - it'll be much more compelling to advertise it this way, we can say "Upgrade to 0.8 and help network performance for everyone". And in the case that we discover a showstopper problem, we just don't deploy the code that uses the new messages into clients.