From ZmnSCPxj at protonmail.com Sun Sep 15 12:27:11 2019 From: ZmnSCPxj at protonmail.com (ZmnSCPxj) Date: Sun, 15 Sep 2019 12:27:11 +0000 Subject: [Lightning-dev] Avoiding gossip spam: how many updates do you need? In-Reply-To: <87zhjji5xu.fsf@rustcorp.com.au> References: <87zhjji5xu.fsf@rustcorp.com.au> Message-ID: Good morning Rusty, As it happens, I already proposed a possible use-case for relatively-common `channel_update`: https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-July/002055.html In the final section I mention: > Suppose that in fact, YAijbOJA thinks that the capacity of the ZmnSCPxj<->YAijbOJA channel is too high on the YAijbOJA side. > And similarly, suppose Rene thinks the capacity of the Rene<->YAijbOJA channel is too high on the Rene side. > > Thus, both YAijbOJA and Rene would welcome the ZmnSCPxj proposal to rebalance, as it moves the capacities. > It may be that they are so welcoming of this proposal, that they are willing to waive the fee for the rebalance. > >I observe that many have already proposed "negative routing fees" in order to support rebalancing of their channels. > I also observe that routing fees are the cost used in pathfinding algorithms, and most pathfinding algorithms do not behave well with negative costs. > > But it is perfectly fine to use ***zero*** routing fees, I think. Briefly: if a channel has too much liquidity on your side, passively rebalance by broadcasting a `channel_update` with 0 routing fees. This helps JIT-Routing of nearby nodes as it allows cheaper rebalances to support. Of course, it is still desirable to rate-limit such updates. So we can do the below policy: 1. Maintain a "latest broadcast is zero" flag. 2. If the channel has been >=75% in your favor for more than 10 minutes (or whatever configuration you want), and the "latest broadcast is zero" flag is cleared, set it and broadcast a 0-fee `channel_update` and set your feerate to 0. 3. If the channel has been <75% in your favor, set your feerate to non-zero, but do not broadcast (meaning "latest broadcast is zero" flag does not change). 4. If the channel has been <75% in your favor for more than 10 minutes, and the "latest broadcast is zero" flag is set, clear it and broacast your normal `channel_update`. However this will probably still lead to more than a burst of 4 `channel_update`s per day. Regards, ZmnSCPxj Sent with ProtonMail Secure Email. ??????? Original Message ??????? On Thursday, September 5, 2019 2:59 PM, Rusty Russell wrote: > Hi all, > > The next release of c-lightning will start suppressing gossip > which comes too fast. I have implemented a simple filter which allows > each message (node_announcement or channel_update) ONCE PER DAY on > average, with a burst up to 4 times per day. We will also discard > identical gossip messages (except timestamp and signature of course). > > If people have a reason to want more frequent updates, please tell me > now! If this is tolerable, it will avoid some of the more obvious > scaling issues in future. > > With 40,000 channels and 10,000 nodes, that caps us at about 90,000 > 12 MB per day. In practice much less. > > As an aside, we're implementing other gossip reductions: > > 1. We'll start using gossip_queries for backfilling, rather than > asking three peers for a complete copy if we think we've missed > something.[1] > > 2. We will no longer ask for the last 24 hours of gossip on startup.[1] > 3. We already suppressed our own duplicate gossip, and only generated > "disable" updates if we had to return an error. > > 4. My current code has us never generating two channel_update or > node_announcement less than 300 seconds apart. > > 5. We have a PR to suppress reply gossip: if you send us a msg, we won't > return it to you. > > 6. My current code has us issuing "refresh" updates only once every > 13 days instead of the current 7. > > The main motivation is one user who has a rPi on a 4G plan. It's > painful for him to restart his node at the moment, but I am determined > to make it reasonable! > > Cheers, > Rusty. > [1] Not yet coded, but RSN. > > > Lightning-dev mailing list > Lightning-dev at lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev