1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
|
Return-Path: <aj@erisian.com.au>
Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138])
by lists.linuxfoundation.org (Postfix) with ESMTP id C3926C002D
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 9 Jan 2023 08:11:16 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by smtp1.osuosl.org (Postfix) with ESMTP id 97F6D81760
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 9 Jan 2023 08:11:16 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 97F6D81760
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level:
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5
tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no
Received: from smtp1.osuosl.org ([127.0.0.1])
by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id Xaf5uIKusCnT
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 9 Jan 2023 08:11:15 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.8.0
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 32B6C81438
Received: from azure.erisian.com.au (azure.erisian.com.au [172.104.61.193])
by smtp1.osuosl.org (Postfix) with ESMTPS id 32B6C81438
for <bitcoin-dev@lists.linuxfoundation.org>;
Mon, 9 Jan 2023 08:11:14 +0000 (UTC)
Received: from aj@azure.erisian.com.au (helo=sapphire.erisian.com.au)
by azure.erisian.com.au with esmtpsa (Exim 4.92 #3 (Debian))
id 1pEnFO-0004zg-7D; Mon, 09 Jan 2023 18:11:11 +1000
Received: by sapphire.erisian.com.au (sSMTP sendmail emulation);
Mon, 09 Jan 2023 18:11:05 +1000
Date: Mon, 9 Jan 2023 18:11:05 +1000
From: Anthony Towns <aj@erisian.com.au>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Message-ID: <Y7vMGVQz8TjS4Cad@erisian.com.au>
References: <56677685-619a-691f-d5bc-54b69fdb6ed2@bip324.com>
<zxv58iXZ73hf9ge8S0QLTanW-uLzaWjNtMHuKONP9hrqS5RhwitxzfVaMH8hbi3yImgNrKme3lCuDcHYKkpxEQHyGZZHJ8xtReOcnAx3o4g=@wuille.net>
<Y2nK99fHUKxbPHmw@erisian.com.au>
<wDqcIVw-YGTsjdf5M2GO9NNRl_UQuBeka2CUQUyQ329u6u-o7RabW_7S4FD3EDfk02kUczb3bXf8LtHhKLtx773UhQ7djKOl-JPOIrXqBSc=@wuille.net>
<JXfTBjsA71dHE3h9wkxnWXANrwTbMADO4s2w34gEvMbiduKu4PEt5t-KA3EAIz-Xs4urjBHZ15NDFZST2a7e0x_NqyJymUnEORuTp3SNfMs=@wuille.net>
<Y3dBUXPhTskCx+Fu@erisian.com.au>
<gSxFQedPc72pTioi9vuxvLKpaRBsnKFL4gkPKPn2G-EJgz_2Y1pYQ7cHD5SnunyCaLln7UQEHIxnopqP74LlnK__Mf9BURbJW8B5MYTZvCU=@wuille.net>
<Y7dZctMlZtH6PEsz@erisian.com.au>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y7dZctMlZtH6PEsz@erisian.com.au>
Subject: Re: [bitcoin-dev] Refreshed BIP324
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Jan 2023 08:11:16 -0000
On Fri, Jan 06, 2023 at 09:12:50AM +1000, Anthony Towns via bitcoin-dev wrote:
> On Thu, Jan 05, 2023 at 10:06:29PM +0000, Pieter Wuille via bitcoin-dev wrote:
> > Oh, yes. I meant this as an encoding scheme, not as a (replacement for) the negotiation/coordination mechanism. There could still be an initial assignment for 1-byte encodings, and/or an explicit mechanism to negotiate other assignment, and/or nothing at all for now.
> The current implementation for 324 does the aliasing
> as part of V2TransportDeserializer::GetMessage and
> V2TransportSerializer::prepareForTransport. That makes a lot of sense,
> [...]
So I think you can make this setup work with a negotiated assignment of
shortids, perhaps starting off something like:
https://github.com/ajtowns/bitcoin/commit/6b8edd754bdcb582e293e4f5d0b41297711bdbb7
That has a 242 element array per peer giving the mappings (which
is just ~250 bytes per peer) for deserialization, which seems
workable. [0]
It also has a single global map for serialization, so we'll always shorten
CFILTER to shortid 39 for every peer that supports shortids, even, eg, for
a peer who's told us they'll send CFILTER as shortid 99 and that we should
interpret shortid 39 from them as NEWFEATUREX. That has three advantages:
* each peer can choose a mapping that minimises their own outbound
traffic, even potentially for asymmetric connections, and don't need
to coordinate with the other peer to decide a common optimal mapping
that they both use across their connection
* you don't have to have different serialization tables per-peer,
reducing memory usage / implementation complexity
* you can leave V2TransportSerializer as a `const` object, and not have
to introduce additional locking logic to be able to update its
state...
I'm not seeing a good way to introduce shortids for future one-shot
negotiation messages though (like VERSION, VERACK, SENDADDRV2,
WTXIDRELAY, SENDTXRCNCL):
* if you explicitly announce the mapping first, you're just wasting
bytes ("99=FOOBAR; 99 baz quux" vs just "FOOBAR baz quux")
* if you negotiate the tables you support between VERSION/VERACK and
then choose a mutually supported table after VERACK, that's too late
for pre-VERACK negotation messages
* announcing the tables you support as part of the VERSION message
would work, but seems a bit klunky
Also, if you did want to shift to a new table, you'd probably want to
always support sending/receiving {37, 44, 46, 47, 36} messages?
I guess I still kind-of think it'd make more sense to just reserve
shortids for post-VERACK messages that are going to be sent more
than once per connection... At that point, even if you don't have any
table in common with your peer, just following VERACK with an immediate
announcement of each shortid you want to use and its meaning would still
make reasonable sense.
If we included the ability to define your own shortids concurrently
with bip324 rollout, then I think nodes could always have a static set
of shortids they use for all their peers for outbound messages, which,
as above, seems like it would make for simpler implementations.
ie, you might send:
VERSION
SHORTIDTBLS ["","awesomeshortids"]
WTXIDRELAY
SENDADDRV2
SENDPACKAGES 1
VERACK
SHORTID "" [(52,"getpkgtxns"), (53, "pkgtxns"), (54, "ancpkginfo")]
...but you'd do all that long form, and only switch to shortids for
messages after you've declared exactly what your shortids are going to
be.
(where "" is the table name for bip324's table, and "awesomeshortids"
is an updated table that includes the package relay commands already,
perhaps)
Cheers,
aj
[0] m_deserializer is used from the SocketHandler thread in
CNode::ReceiveMsgBytes(), but the p2p protocol is managed from the
MessageHandler thread; with multiple messages potentially deserialized
into vRecvMsg() at once -- but that means that if the first message
redefines shortid decoding, and the second message uses one of the
redefined shortids, it will have already been decoded incorrectly.
So that would need some futzing about still.
|
