From rusty at rustcorp.com.au  Fri Feb  9 01:44:24 2018
From: rusty at rustcorp.com.au (Rusty Russell)
Date: Fri, 09 Feb 2018 12:14:24 +1030
Subject: [Lightning-dev] Improving the initial gossip sync
In-Reply-To: <874lmvy4gh.fsf@gmail.com>
References: <874lmvy4gh.fsf@gmail.com>
Message-ID: <87tvurym13.fsf@rustcorp.com.au>

Hi all!

        Finally catching up.  I prefer the simplicity of the timestamp
mechanism, with a more ambitious mechanism TBA.

Deployment suggestions:

1. This should be a feature bit pair.  As usual, even == 'support this or
   disconnect', and odd == 'ok even if you don't understand'.

2. This `timestamp_routing_sync`? feature overrides `initial_routing_sync`.
   That lets you decide what old nodes do, using the older
   `initial_routing_sync` option.  Similarly, a future `fancy_sync` would
   override `timestamp_routing_sync`.

3. We can append an optional 4 byte `routing_sync_timestamp` field to to
   `init` without issues, since all lengths in there are explicit.  If you
   don't offer the `timestamp_sync` feature, this Must Be Zero (for appending
   more stuff in future).

Now, as to the proposal specifics.

I dislike the re-transmission of all old channel_announcement and
node_announcement messages, just because there's been a recent
channel_update.  Simpler to just say 'send anything >=
routing_sync_timestamp`.

Background: c-lightning internally keeps an tree of gossip in the order
we received them, keeping a 'current' pointer for each peer.  This is
very efficient (though we don't remember if a peer sent us a gossip msg
already, so uses twice the bandwidth it could).

But this isn't *quite* the same as timestamp order, so we can't just set
the 'current' pointer based on the first entry >=
`routing_sync_timestamp`; we need to actively filter.  This is still a
simple traverse, however, skipping over any entry less than
routing_sync_timestamp.

OTOH, if we need to retransmit announcements, when do we stop
retransmitting them?  If a new channel_update comes in during this time,
are we still to dump the announcements?  Do we have to remember which
ones we've sent to each peer?

If missing announcements becomes a problem, we could add a simple query
message: I think this is going to be needed for any fancy scheme anyway.

Cheers,
Rusty.