Date: Fri, 17 Feb 2023 22:13:05 +0000
To: Anthony Towns <aj@erisian.com.au>,
 Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
From: Pieter Wuille <bitcoin-dev@wuille.net>
Message-ID: <S1zoL4CCIDVTrjmXx2JtYhO2qjgGyNIAP6X9FXRCRKPDjoQj20VcqKFCYkmmPQkNuMyNf9zp6GFVWWC7l8dBzCogUqvzmDx9D811NPheNJ8=@wuille.net>
In-Reply-To: <Y++id6mXsscfxduH@erisian.com.au>
References: <56677685-619a-691f-d5bc-54b69fdb6ed2@bip324.com>
 <wDqcIVw-YGTsjdf5M2GO9NNRl_UQuBeka2CUQUyQ329u6u-o7RabW_7S4FD3EDfk02kUczb3bXf8LtHhKLtx773UhQ7djKOl-JPOIrXqBSc=@wuille.net>
 <JXfTBjsA71dHE3h9wkxnWXANrwTbMADO4s2w34gEvMbiduKu4PEt5t-KA3EAIz-Xs4urjBHZ15NDFZST2a7e0x_NqyJymUnEORuTp3SNfMs=@wuille.net>
 <Y3dBUXPhTskCx+Fu@erisian.com.au>
 <gSxFQedPc72pTioi9vuxvLKpaRBsnKFL4gkPKPn2G-EJgz_2Y1pYQ7cHD5SnunyCaLln7UQEHIxnopqP74LlnK__Mf9BURbJW8B5MYTZvCU=@wuille.net>
 <Y7dZctMlZtH6PEsz@erisian.com.au> <Y7vMGVQz8TjS4Cad@erisian.com.au>
 <6b83c32e-59ca-65ef-2553-d66f8d117e56@bip324.com>
 <Y++id6mXsscfxduH@erisian.com.au>
Feedback-ID: 19463299:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Cc: Dhruv M <dhruv@bip324.com>
Subject: Re: [bitcoin-dev] Refreshed BIP324
Precedence: list

On Friday, February 17th, 2023 at 10:51 AM, Anthony Towns via bitcoin-dev <=
bitcoin-dev@lists.linuxfoundation.org> wrote:

> I think it's probably less complex to close some of the doors?
=20
> 2) are short ids available/meaningful to send prior to VERACK being
> completed?

Ah, I hadn't considered this nuance. If we don't care about them being avai=
lable before VERACK negotiation, then it may be possible to introduce a way=
 to negotiate a different short id mapping table without needing a mechanis=
m for *re*-negotiating.

> Here's another approach:
>=20
> idea: we use short ids to minimise bandwidth, and don't care about
> bandwidth for long ids
>=20
> implementation:
> short id 0 is reserved for long commands. when received, we
> decode the first 12 bytes of the payload and treat them
> exactly the same as a v1 p2p message (trailing 0-bytes, etc)
> (if there's not 12 bytes of payload, it's just treated as an
> invalid command and dropped)
>=20
> short ids 1-255 are available for use as aliases of particular
> long commands
>=20
> (That's exactly compatible with p2p v1, and also avoids the temptation
> to try to choose short command names rather than descriptive ones -- the
> 0-padding to 12 bytes prevents you from saving any bandwidth that way;
> but that's what we have short ids for anyway)

I like this idea. It avoids the variable-length encoding question and relat=
ed complexity entirely for things where we admittedly don't care about the =
bandwidth impact anyway.

It may also have another (rather weak) advantage, in that it may reduce how=
 much information a passive observe may learn about application level featu=
res (sendheaders, sendaddrv2, ...) from the packet size sent (which would o=
therwise depend on command lengths), even when decoys are not in use, if no=
 short commands are included for these messages.

> > - We remove 1 byte allocations for messages that are sent at most once =
per connection per direction
>=20
> I think this leaves 32 commands that get short ids initially:
>=20
> misc: ADDR, ADDRV2, BLOCK, FEEFILTER, GETBLOCKS, GETDATA, GETHEADERS,
> HEADERS, INV, NOTFOUND, PING, PONG, TX
> bip 35/37: FILTERADD, FILTERCLEAR, FILTERLOAD, MEMPOOL, MERKLEBLOCK
> bip 152: BLOCKTXN, CMPCTBLOCK, GETBLOCKTXN
> bip 157: CFCHECKPT, CFHEADERS, CFILTER, GETCFCHCKPT, GETCFHEADERS,
> GETCFILTERS
> bip 330: RECONCILDIFF, REQRECON, REQSKETCHEXT, SENDCMPCT, SKETCH

Sounds right.

> which drops:
>=20
> VERSION, VERACK, GETADDR, SENDADDRV2, SENDHEADERS, SENDTXRCNCL,
> WTXIDRELAY

Indeed.

> compared to bip 324 currently.
>=20
> I think the things missing from the current list (and not currently in
> use by bitcoin core) are:
>=20
> bip 61: REJECT
> bip 331: GETPKGTXNS, PKGTXNS, ANCPKGINFO

Do you feel REJECT should be included?

> > - Optionally, in the implementation we can attempt to move the type id =
mapping to the p2p layer away from the transport layer. I suspect this coul=
d also be done after the implementation is merged but might be cleaner as t=
he mapping is a p2p concern.
>
> I agree that's fine, though I expect that we'll probably want to do it
> not long after bip 331 is ready for merge (or some other p2p improvement
> comes along)...

I do prefer that as well; it feels like the transport layer shouldn't be aw=
are of the different command names that exist, but this is very much just a=
n implementation issue.

Perhaps a possibility is having the transport layer translate short-command=
-number-N to the 12-byte command "\x00\x00..." + byte(N), and hand that to =
the application layer, which could then do the mapping?

Cheers,

--=20
Pieter