Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1VZpdU-0000KX-4l for bitcoin-development@lists.sourceforge.net; Fri, 25 Oct 2013 22:13:40 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of petertodd.org designates 62.13.149.80 as permitted sender) client-ip=62.13.149.80; envelope-from=pete@petertodd.org; helo=outmail149080.authsmtp.com; Received: from outmail149080.authsmtp.com ([62.13.149.80]) by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1VZpdS-0004ZR-A6 for bitcoin-development@lists.sourceforge.net; Fri, 25 Oct 2013 22:13:39 +0000 Received: from mail-c235.authsmtp.com (mail-c235.authsmtp.com [62.13.128.235]) by punt10.authsmtp.com (8.14.2/8.14.2) with ESMTP id r9PMDWhR087061; Fri, 25 Oct 2013 23:13:32 +0100 (BST) Received: from petertodd.org (petertodd.org [174.129.28.249]) (authenticated bits=128) by mail.authsmtp.com (8.14.2/8.14.2/) with ESMTP id r9PMDS5j035270 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 25 Oct 2013 23:13:30 +0100 (BST) Date: Fri, 25 Oct 2013 18:13:27 -0400 From: Peter Todd To: Jeremy Spilman Message-ID: <20131025221327.GA18030@petertodd.org> References: <20131024143043.GA12658@savin> <20131024144358.GA17142@savin> <20131024145447.GA19949@savin> <20131025070708.GA5760@savin> <91968c56640bf7647325728f490b9257@localhost> <20131025161323.GA15774@petertodd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Server-Quench: aea7d945-3dc2-11e3-b802-002590a15da7 X-AuthReport-Spam: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-AuthRoute: OCd2Yg0TA1ZNQRgX IjsJECJaVQIpKltL GxAVKBZePFsRUQkR aQdMdAcUF1YAAgsB AmUbWVdeUV57WmI7 ag1VcwRfa1RMVxto VEFWR1pVCwQmQxVh e2p9V0NydABEf34+ Z0JgX3YVDRErchJ0 RktJFjlVZ3phaTUc TRJQdwFJcANIexZF O1F6ACIKLwdSbGoL NQ4vNDcwO3BTJTpY RgYVKF8UXXNDOzc/ RhYNVTsiEAUIXDky KhU6K1kaVEwcLlk/ KzMA X-Authentic-SMTP: 61633532353630.1023:706 X-AuthFastPath: 0 (Was 255) X-AuthSMTP-Origin: 174.129.28.249/587 X-AuthVirus-Status: No virus detected - but ensure you scan with your own anti-virus system. X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: petertodd.org] X-Headers-End: 1VZpdS-0004ZR-A6 Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Making fee estimation better X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 22:13:40 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 25, 2013 at 12:35:34PM -0700, Jeremy Spilman wrote: > Do you think we're at the point where wallets have to be able to > "actively bid" the fee using replacement due to block contention? If Bitcoin continues to grow we probably will be at some as-yet-unknown point in the future. > I think a fee estimation API is just a data point. Depending on the > properties of the estimator, and how that's presented in the UI, it > could serve to either increase or decrease the need for recovery. >=20 > Like you said, we already have "fee estimation" in the form of > "user, please estimate the fee!" Now we want to make fee estimation > "better", and one key aspect of better fee estimation is decreasing > the need for recovery. Techniques like signing multiple transactions > with different fee levels should become less useful the better you > are at estimating the fee. Yes, but equally all estimates are imperfect, and you can trade-off risk that your transaction will not go through initially for lower fees. Estimates can be made sufficiently conservative that they are rarely wrong - this is basically the strategy of the current system. Given that demand for blockchain space isn't "saturated" it works reasonably well for now. But without a good mechanism to recover from an initial bad estimate you have to be more conservative than is efficient. > What I find interesting is that fee estimation can look at the size > and type of the transaction, the age of the inputs, the number of > inputs versus outputs, amount of the outputs, factor in [assumptions > about] what fee policies miners are actually using, and after all > that, look at the actual competing transactions on the blockchain > and try to figure out how many of those are even real. >=20 > For example, if you just look at fee-per-KB of mempool versus > fee-per-KB of recently mined transactions, without taking into > account input age, number of inputs vs outputs, output amounts... > all the other things miner might have used to discriminate between > transactions, then I don't think you'll end up with a better fee > estimator. To a first approximation there's not much reason for miners to take anything other than fee-per-KB into account when determining what transactions to mine; you want to stuff your 1MB block full of high paying transactions. That a child tx may make a parent more profitable to mine complicates things - Gavin's current fee estimator also makes too-low-estimates in that case - and not all algorithms to do so will come to the same conclusion. (doing it perfectly is something like O(n^2), and imperfectly is O(1) but doesn't handle multiple children well) There are some second-order effects, a block is less likely to be orphaned if all transactions in it have propagated sufficiently, thus a miner should penalize very recently broadcast transactions. In addition because miners never orphan themselves large miners have a significant advantage regarding orphan-inducing effects. However those effects all tend to be miner specific, and/or only temporary. FWIW the logic behind orphans is currently rather frightening: a rational miner will, the moment they learn that a block exists via the quickly propagating block header, start working to extend that block with one that either doesn't contain any transactions, or only contains transactions they can be reasonably sure another miner didn't mine. (e.g. via exclusive tx mining contracts) This boosts their profit because they aren't wasting their effort while the rest of the block propagates, removes much of the incentive have any limit on block size, and incentivizes miners to extend chains they haven't actually validated yet. (relying on the other miners incentive not to produce an invalid block) > Contention might bump you out of a few blocks, but if the basis for > calculating the fee is fundamentally compatible with the relay > policies and the transaction-inclusion policies being run by large > mining pools, the transaction isn't dead, it's just pending. With a size-limited blocks inclusion is more a matter of supply and demand than policy. --=20 'peter'[:-1]@petertodd.org 00000000000000066c29f3319f83f1c6e912b5add5534da1b938c4c65a07b02a --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBCAAGBQJSau0HAAoJEBmcgzuo5/CFZA0H/AoNKw3FSUxugVW2N4RoM2I4 RELcuNp+ozRSpdbI6UqhUshGIuKhVkVqepy/SWBzNbAbIYcSUjK3IN4JuGhZ4BOe MfoOzBxzJ9n6y7PICVMjdifYtWjWgt5uBeu86P3Kz4/MKEqkaiEA3gbzqa9OIyOn XkXCBnczxWxDkYF9fUZvN22+hjQ4/hoA9RrsNDzRvzzYNmvFLZWELANKVvRzl/7e 6nmMJYsjRMo25oH2xnV6cbx8NpZljSQR6Tx8PjKlSCfd1lq+HvKVkMVhbWMmJFf1 UQmaVk78Kgl6Rx7vD4XRywNinWOg4wCxSUdJ4kjShTGKx/BjQv+SNW+LL7PJl/A= =Qx73 -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X--