From rusty at rustcorp.com.au Thu May 26 20:25:36 2022 From: rusty at rustcorp.com.au (Rusty Russell) Date: Fri, 27 May 2022 05:55:36 +0930 Subject: [Lightning-dev] Gossip Propagation, Anti-spam, and Set Reconciliation In-Reply-To: <41c00f9a-ed0a-b275-45e1-21b4c5951570@mattcorallo.com> References: <87ee1pj1tj.fsf@rustcorp.com.au> <87czh8h90j.fsf@rustcorp.com.au> <87h76f10g1.fsf@rustcorp.com.au> <7853c89e-32ee-657d-be56-d2b5ab4ff5d5@mattcorallo.com> <87mtg4zoa1.fsf@rustcorp.com.au> <41c00f9a-ed0a-b275-45e1-21b4c5951570@mattcorallo.com> Message-ID: <87ee0g3utb.fsf@rustcorp.com.au> Matt Corallo writes: >>> I agree there should be *some* rough consensus, but rate-limits are a locally-enforced thing, not a >>> global one. There will always be races and updates you reject that your peers dont, no matter the >>> rate-limit, and while I agree we should have guidelines, we can't "just make them the same" - it >>> both doesn't solve the problem and means we can't change them in the future. >> >> Sure it does! It severly limits the set divergence to race conditions >> (down to block height divergence, in practice). > > Huh? There's always some line you draw, if an update happens right on the line (which they almost > certainly often will because people want to update, and they'll update every X hours to whatever the > rate limit is), then ~half the network will accept the update and half won't. I don't see how you > solve this problem. The update contains a block number. Let's say we allow an update every 100 blocks. This must be <= current block height (and presumably, newer than height - 2016). If you send an update number 600000, and then 600100, it will propagate. 600099 will not. If some nodes have 600000 and others have 600099 (because you broke the ratelimiting recommendation, *and* propagated both approx the same time), then the network will split, sure. We could be fascist and penalize nodes which do this, but that's overkill unless it actually happens a lot. Nodes which want to keep an potential update "up their sleeve" will backdate updates by 101 blocks (everyone should do this, in fact). As I said, this has a problem with block height differences, but that's explicitly included in the messages so you can ignore and wait if you want. Again, may not be a problem in practice. >> Maybe. What's a "non-update" based sketch? Some huge percentage of >> gossip is channel_update, so it's kind of the thing we want? > > Oops, maybe we're on *very* different pages, here - I mean doing sketches based on "the things that > I received since the last sync, ie all the gossip updates from the last hour" vs doing sketches > based on "the things I have, ie my full gossip store". But that requires state. Full store requires none, keeping it super-simple Though Alex has a idea for a "include even the expired entries" then "regenerate every N blocks" which avoids the problem that each change is two deltas (one remove, one add), at cost of some complexity. Cheers, Rusty.