From rusty at rustcorp.com.au  Fri Feb  5 00:54:14 2016
From: rusty at rustcorp.com.au (Rusty Russell)
Date: Fri, 05 Feb 2016 11:24:14 +1030
Subject: [Lightning-dev] Protocol for multiple in-flight updates.
In-Reply-To: <87wpql9kxo.fsf@rustcorp.com.au>
References: <878u34oi9q.fsf@rustcorp.com.au> <87si1bnc4m.fsf@rustcorp.com.au>
	<20160202072809.GA6119@lightning.network>
	<877fimmy4t.fsf@rustcorp.com.au>
	<20160203013330.GA14163@lightning.network>
	<87io26becq.fsf@rustcorp.com.au>
	<20160203050135.GA1397@lightning.network>
	<87wpql9kxo.fsf@rustcorp.com.au>
Message-ID: <87io249du1.fsf@rustcorp.com.au>

OK, so this is based on my understanding of what Joseph's protocol will
look like when done, from here and perusing the lnd source.  See
proposals if this is all simply revision; it's mainly to document my
understanding.

Each HTLC has one of these states:

	ADD_PRESTAGE
	ADD_STAGED
	ADD_SIGNING_AND_REVOKING
	ADD_COMPLETE
	ADD_REJECTED

	TIMEOUT_PRESTAGE
	TIMEOUT_STAGED
	TIMEOUT_SIGNING_AND_REVOKING
	TIMEOUT_COMPLETE

	SETTLE_PRESTAGE
	SETTLE_STAGED
	SETTLE_SIGNING_AND_REVOKING
	SETTLE_COMPLETE

Each side starts with a commit tx with 0 HTLCs.  HTLCs are numbered
using a counter, but these numbers explicitly sent on the wire.
Proposed changes are processed in order.

Adding An HTLC
--------------
A: HTLC = ADD_PRESTAGE.  Send "add request { htlc details }".
B: Receive "add request".
   (If reject, reply and forget HTLC, we'll ignore this)
   If accept, HTLC = ADD_STAGED (since both sides know it), send "accept"
A: Receive "accept".  HTLC = ADD_STAGED.

Removing An HTLC
----------------
There's timeout and there's settle; the only difference is settle
requires the R value.

A: HTLC must be in ADD_COMPLETE state.  HTLC = SETTLE/TIMEOUT_PRESTAGE.
   Send "settle/timeout request { htlc id [r value] }".
B: Receive "request".  Check r value for settle, time for timeout.
   HTLC = SETTLE/TIMEOUT_STAGED (since both sides know it), send "accept"
A: Receive "accept".  HTLC = ADD_STAGED.

Committing
----------
Both sides can do this.  I can't figure out how the HTLC states work
here, since we might have sent the signature, but not received theirs.
Should there be a separate SIG_SENT and SIG_RECEIVED states?  

A: Move HTLCs all *_STAGED htlcs to *_SIGNING_AND_REVOKING, then
   generate their commit tx using all the ADD_COMPLETE and
   ADD_SIGNING_AND_REVOKING htlcs.  Sign it, and send
   "commit { staged-htlcs; sig }"
B: Receive commit, generate our commit tx using all the ADD_COMPLETE,
   and any TIMEOUT_PRESTAGE or SETTLE_PRESTAGE which are not listed
   in staged-htlcs.  Check their signature is valid for this.
   [FIXME: Does B update states here?]
   Send "commit revocation { old-revocation-preimage }" for
   the previous commit tx.
A: Receive commit revocation, move all the HTLCs we moved in
   step 1 above from *_SIGNING_AND_REVOKING to *_COMPLETE.
   (If we never create two commits in-flight at once, this is simply
   every *_SIGNING_AND_REVOKING htlc).

Proposals
=========

There's no reason to "accept" anything except an add request.  Thus the
states SETTLE_PRESTAGE and TIMEOUT_PRESTAGE are redundant.

There are also other failures (ie. upstream node rejected).  This
suggests a FAIL messages, perhaps with a reason message (which might be
encrypted back to the payer).

C-lightning only allows closes be sent from the HTLC recipient.  I
thought that would be simpler but it doesn't make any real difference
(it only applies to TIMEOUT; FAIL and SETTLE have to come from recipient
anyway).  I would defined TIMEOUT to be the responsibility of the HTLC
proposer.

In order to reduce latency, we want to be able to start staging an HTLC
on the outgoing channel before it's committed on the incoming one.  But
this means an HTLC recipient can't commit to the HTLC until the HTLC
sender does.

This has a few implications:
1) We need a new "unstage request" to unstage if the incoming HTLC fails
   to complete in reasonable time.  This requires no accept response.
2) We don't need to commit to adds and removes in order wrt each other;
   a commit message always commits to:
        - All HTLCs proposed and commited by the other side, and
        - All removes staged by the other side up to some counter, and
        - All HTLCs proposed by this side up to some counter, and
        - All removes sent by this side.
3) This implies we should count htlcs and remove requests separately;
   the htlc count could also serve as an HTLC ID which makes timeout/fail/abort
   messages shorter than using R hashes.

Oh, and it should be illegal to send a noop commit (ie. one which
doesn't change the commit tx).

That's enough email for now!
Rusty.