From rusty at rustcorp.com.au  Thu Sep  5 06:59:57 2019
From: rusty at rustcorp.com.au (Rusty Russell)
Date: Thu, 05 Sep 2019 16:29:57 +0930
Subject: [Lightning-dev] Avoiding gossip spam: how many updates do you need?
Message-ID: <87zhjji5xu.fsf@rustcorp.com.au>

Hi all,

        The next release of c-lightning will start suppressing gossip
which comes too fast.  I have implemented a simple filter which allows
each message (node_announcement or channel_update) ONCE PER DAY on
average, with a burst up to 4 times per day.  We will also discard
identical gossip messages (except timestamp and signature of course).

If people have a reason to want more frequent updates, please tell me
now!  If this is tolerable, it will avoid some of the more obvious
scaling issues in future.

With 40,000 channels and 10,000 nodes, that caps us at about 90,000
12 MB per day.  In practice much less.

As an aside, we're implementing other gossip reductions:

1. We'll start using gossip_queries for backfilling, rather than
   asking three peers for a complete copy if we think we've missed
   something.[1]

2. We will no longer ask for the last 24 hours of gossip on startup.[1]

3. We already suppressed our own duplicate gossip, and only generated
   "disable" updates if we had to return an error.

4. My current code has us *never* generating two channel_update or
   node_announcement less than 300 seconds apart.

5. We have a PR to suppress reply gossip: if you send us a msg, we won't
   return it to you.

6. My current code has us issuing "refresh" updates only once every
   13 days instead of the current 7.

The main motivation is one user who has a rPi on a 4G plan.  It's
painful for him to restart his node at the moment, but I am determined
to make it reasonable!

Cheers,
Rusty.
[1] Not yet coded, but RSN.