Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of bitpay.com
	designates 74.125.82.179 as permitted sender)
	client-ip=74.125.82.179; envelope-from=jgarzik@bitpay.com;
	helo=mail-we0-f179.google.com; 
MIME-Version: 1.0
In-Reply-To: <CA+iPb=H2fkjCxS7-hYqHjFzfMh6onk5RqZMxa8zsXeTn6pQMpA@mail.gmail.com>
References: <CA+iPb=EaX=bvOjNtZ+LnYTMRLQQ9nFcrefAkBdv8eActoX_b8A@mail.gmail.com>
	<CABsx9T0ag_o_mu=5Q7Ju7s2hO3jz-o5g9FihE9h4B6+ednd2Pg@mail.gmail.com>
	<CAJHLa0NZRF+1QjSwtwjaTE07NWJ_U-O-DE24=P5eSAutMqTupg@mail.gmail.com>
	<CABsx9T2BDBNqvinVNk3FmBRWU7R8jf6Vm6NaH74te0FRCh1O-w@mail.gmail.com>
	<CAJHLa0O=eCoyvV19dWgTnYd9Di0wLLZtWmCPidc-dWqPNQv_oQ@mail.gmail.com>
	<CA+iPb=H2fkjCxS7-hYqHjFzfMh6onk5RqZMxa8zsXeTn6pQMpA@mail.gmail.com>
From: Jeff Garzik <jgarzik@bitpay.com>
Date: Fri, 18 Jul 2014 13:48:47 -0400
Message-ID: <CAJHLa0NRUdAPuKXgKDBmXOs9to7gMpHv9ECCz_hTfZpg7SVVJA@mail.gmail.com>
To: Kaz Wesley <keziahw@gmail.com>
Content-Type: text/plain; charset=UTF-8
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Squashing redundant tx data in blocks on
 the wire
Precedence: list

On a flood-fill network, you don't want to create a storm of "I
already have this" replies.

On Fri, Jul 18, 2014 at 1:39 PM, Kaz Wesley <keziahw@gmail.com> wrote:
> Peers exchanging mempool priority policies is great; that accomplishes
> the flexibility in what txes to remember that I was going for with the
> forget-filters, but much more neatly, with less overhead and some side
> benefits.
>
> Here's what I'm picturing now:
> - exchange priority policies in peer introductions
> - assign unique sequential IDs in the order the transactions were
> inved (per peer)
> - receiving a getdata for a tx updates last-known-peer-received inv to
> all invs up to the one referenced
> - include ID-last-received, last-known-peer-received in sparse block
> - reference txes in sparse block by index in receiver's
> prioritiziation with peer's sent invs up to ID-last-received and
> sender's prior invs up to last-known-peer-received
>
> Possible new messages:
> - sparseblock
> - invack message a node can send at times when it's received a bunch
> of invs it already has, so it hasn't acked with a getdata in a while
> - gettx: getdata, but using new sequential ID to save 28 bytes per tx
>
> It seems important for ordering policies to be able to be specified in
> as much detail as possible. Parameters that should be available:
> - total inputs
> - total outputs
> - bytes
> - coin days destroyed
> - net UTXO size change
> - sigops
> - is data carrier
> - is output raw multisig
> - age in mempool
> - what else?
> This parameter set should be extensible to allow for unforeseen future factors.
>
> Ordering policies should allow arbitrary algebraic combinations of
> their parameters, as well as thresholds. Boolean combinations of
> sub-policies would also be desirable. This could be implemented with a
> tx-script-like stack-based language, in which each supported tx
> property is pushed onto the stack by a particular opcode, and
> +-*//min/max/boolean operators combine them to yield the sort key.
>
> Difficult parameters:
> * Coin-days-destroyed: changes, peers need agreement on when (if?)
> it's recalculated. Probably can just not recalculate, but peers still
> need agreement on "time seen" to get CDD.
> * Age in mempool: seems intractable in terms of time, but could be
> done easily in terms of "how many txes old is this sequential ID"
>
> One potential pitfall: this allows for an environment of completely
> heterogeneous mempool policies. I think that's a good thing, but we
> need to avoid a situation where only least-common-denominator
> transactions make it farther than a hop or two, and we don't want
> nodes to have a strong preference for connecting to like-minded peers
> since clustering reduces overall connectivity. It may be worthwhile to
> add a parallel mechanism for relay policies, to differentiate between
> what a node would keep in its mempool vs. what it wouldn't even relay
> and doesn't want to see at all. Relay policies could be specified just
> like prioritization policies, but with the final stack value evaluated
> in a boolean context.
>
> An interesting additional use of policy-scripts would be a
> standardized way for miners to include a policy script in a coinbase,
> allowing miners a mechanism to advertise things like their relative
> price of sigops vs bytes. Nodes may then choose to take this
> information into account in order to optimize their mempool policies
> for likelihood of consistency with future blocks. Since policy scripts
> provide only relative information on prices of different transaction
> properties rather than an absolute fee, this should not allow miners
> to "vote fees up", although care would need to be taken they wouldn't
> be able to drive up prices by claiming common transaction types are at
> the high end of the fee scale.
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/