From decker.christian at gmail.com Wed Feb 28 21:38:01 2018 From: decker.christian at gmail.com (Christian Decker) Date: Wed, 28 Feb 2018 22:38:01 +0100 Subject: [Lightning-dev] Improving the initial gossip sync In-Reply-To: References: <874lmvy4gh.fsf@gmail.com> <87tvurym13.fsf@rustcorp.com.au> <87shaawft5.fsf@gmail.com> <878tbzugj0.fsf@rustcorp.com.au> <87mv0cto38.fsf@rustcorp.com.au> <87606so4bd.fsf@rustcorp.com.au> Message-ID: <878tbcztfq.fsf@gmail.com> Olaoluwa Osuntokun writes: > As I've brought up before, from my PoV, we can't append any additional > fields to the innit message as it already contains *two* variable sized > fields (and no fixed size fields). Aside from this, it seems that the > `innit` message should be simply for exchange versioning information, which > may govern exactly *which* messages are sent after it. Otherwise, by adding > _additional_ fields to the `innit` message, we paint ourselves in a corner > and can never remove it. Compared to using the `innit` message to set up the > initial session context, where we can safely add other bits to nullify or > remove certain expected messages. While I do agree that a new message with high and low watermarks for a sync controlled by the recipient is the way to go, I just don't see the issue with extending the `init` message (and I think it may be useful in future, which is why I bring it up). The two variable size fields are length prefixed so we know exactly what their size is, and where they end, so a new field added to the end can be trivially identified as such. As pointed out in my first mail, we'd have to make it mandatory for the recipient to understand the new field, since it cannot be skipped if the recipient does not, but this still doesn't preclude adding such a field. As for the overflow issue you mention, a single features bitfield is already sufficient to completely overflow the `init` message length, since it's prefix is 2 bytes, allowing for 65535 bytes for that single field alone, in a message that only has 65533 bytes of payload left. But the sender would have to be bonkers to overflow the message and then try something with the appended field. It'd overflow in the next packet since we can't even tell the recipient that we have >65535 bytes of payload, and it'd fail the HMAC check. IMHO the connection would simply be stopped right there, and the sender just found a very contorted way of closing the connection :-) In the good case however the `init` message can look something like this: - [2:gflen] - [gflen:globalfeatures] - [2:lflen] - [lflen:localfeatures] - [4:lowwatermark] - [4:highwatermark] Maybe I'm just not seeing it, and if that's the case I apologize :-) Cheers, Christian