From fabrice.drouin at acinq.fr Mon Feb 1 16:18:21 2016 From: fabrice.drouin at acinq.fr (Fabrice Drouin) Date: Mon, 1 Feb 2016 17:18:21 +0100 Subject: [Lightning-dev] Laundry list of inter-peer wire protocol changes In-Reply-To: <87d1snvhyf.fsf@rustcorp.com.au> References: <87d1snvhyf.fsf@rustcorp.com.au> Message-ID: On 27 January 2016 at 04:07, Rusty Russell wrote: > > Hi all! > > As more code emerges, I'd like to try to nail down a "1.0" > version of the inter-node protocol as much as possible. Since I'm most > familiar with my own code (c-lightning[1]), I'll use that as a basis and > discuss differences (esp. vs lnd[2]). > > For this email I'll simply list the changes or proposals I'm aware of, > then we can dissect and comment on each one: defer, accept or close. > > Direct wire format stuff > ------------------------ > > - Protobufs vs open-coded structures > - lnd open-coded their own protocol; I haven't finished reading their > code though. > - protobufs are easy to extend with new fields; with an open-coded > proto we simply need a rule that too-long packets are valid. > - protobufs are annoying for fixed-length blobs which we use a lot > (keys, signatures, hashes). > - The protocol is currently simple with very few variable-length fields. Hello, We (Pierre and I, the guys who work on the scala thingy :-) would rather use a "standard" binary format and protobuf seems to be a very good choice. Since incoming messages are encrypted, text based formats (JSON, ...) would not help that much for debugging and are not imho a good fit for binary protocols. > - Length prefix for initial key exchange > - Both lnd and c-lightning begin by exchanging a 33-byte EC key for DH. > - We should prefix with a length word, so we can extend this later > (eg. for new crypto) Agreed. > - HTLC pipelining > - lnd's protocol supports multiple in-flight HTLC negotiations; this > would allow much greater throughput, with some complexity. > - lightning-c uses a simple one-at-a-time scheme, with alternating > priority in case of simultaneous sends. Just to be sure that we understand this, you mean grouping HTLCs and sending them with one message (so just one signature will be exchanged). It becomes more complex for clients because they will have to buffer and group incoming HTLCs but the protocol and the transitions remain pretty much the same ? > Misc > ---- > - shachain vs elkrem > - We use this to generate the revocation secrets, to minimize storage > and computation for a huge number of old commitment txs. > - They're actually very similar, but elkrem is much easier to grok.[6] I like both, they're easy to implement and elkrem was indeed much easier to understand. I don't know precisely why yet but I would choose shachain though. Cheers, Fabrice