diff options
| author | matejcik <jan.matejek@satoshilabs.com> | 2018-07-04 15:19:11 +0200 |
|---|---|---|
| committer | bitcoindev <bitcoindev@gnusha.org> | 2018-07-04 13:19:18 +0000 |
| commit | 67ad070f01da03b0cd5b1841b03e05ca6be5d4b0 (patch) | |
| tree | 8156a98f4453a8cd695fcbfe88bb9d43b8a6e9d2 | |
| parent | 3f8befecafd5bd802604817971c70eaeccc9c928 (diff) | |
| download | pi-bitcoindev-67ad070f01da03b0cd5b1841b03e05ca6be5d4b0.tar.gz pi-bitcoindev-67ad070f01da03b0cd5b1841b03e05ca6be5d4b0.zip | |
Re: [bitcoin-dev] BIP 174 thoughts
| -rw-r--r-- | 4e/9593589eb07fe44b616e57e2dda59cadf53386 | 683 |
1 files changed, 683 insertions, 0 deletions
diff --git a/4e/9593589eb07fe44b616e57e2dda59cadf53386 b/4e/9593589eb07fe44b616e57e2dda59cadf53386 new file mode 100644 index 000000000..21b54434a --- /dev/null +++ b/4e/9593589eb07fe44b616e57e2dda59cadf53386 @@ -0,0 +1,683 @@ +Return-Path: <jan.matejek@satoshilabs.com> +Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org + [172.17.192.35]) + by mail.linuxfoundation.org (Postfix) with ESMTPS id 72C56CB2 + for <bitcoin-dev@lists.linuxfoundation.org>; + Wed, 4 Jul 2018 13:19:18 +0000 (UTC) +X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 +Received: from mail.sldev.cz (mail.sldev.cz [88.208.115.66]) + by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E013276A + for <bitcoin-dev@lists.linuxfoundation.org>; + Wed, 4 Jul 2018 13:19:16 +0000 (UTC) +Received: from localhost (localhost [127.0.0.1]) + by mail.sldev.cz (Postfix) with ESMTP id 74131E105A; + Wed, 4 Jul 2018 13:19:14 +0000 (UTC) +X-Virus-Scanned: Debian amavisd-new at mail.sldev.cz +Received: from mail.sldev.cz ([127.0.0.1]) + by localhost (mail.sldev.cz [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id c8KbxARtKyOu; Wed, 4 Jul 2018 13:19:12 +0000 (UTC) +Received: from [10.8.0.37] (unknown [10.8.0.37]) + by mail.sldev.cz (Postfix) with ESMTPSA id 42854E0E15; + Wed, 4 Jul 2018 13:19:12 +0000 (UTC) +To: Achow101 <achow101-lists@achow101.com>, + Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> +References: <CAPg+sBhGMxXatsyCAqeboQKH8ASSFAfiXzxyXR9UrNFnah5PPw@mail.gmail.com> + <ljk5Z_a3KK6DHfmPJxI8o9W2CkwszkUG34h0i1MTGU4ss8r3BTQ3GnTtDTfWF6J7ZqcSAmejzrr11muWqYN-_wnWw_0NFn5_lggNnjI0_Rc=@achow101.com> + <f8f5b1e3-692a-fc1e-2ad3-c4ad4464957f@satoshilabs.com> + <TGyS7Azu3inMQFv9QFn8USr9v2m5QbhDRmiOI-4FWwscUeuIB9rA7mCmZA4-kwCJOMAx92fO7XICHtE7ES_QmIYLDy6RHof1WLALskGUYAc=@achow101.com> + <c32dc90d-9919-354b-932c-f93fe329760b@satoshilabs.com> + <CAPg+sBhhYuMi6E1in7wZovX7R7M=450cm6vxaGC1Sxr=cJAZsw@mail.gmail.com> + <881def14-696c-3207-cf6c-49f337ccf0d1@satoshilabs.com> + <CAPg+sBg4MCOoMDBVQ2eZ=p3iS3dq506Jh4vUNBmmM20a6uCwYw@mail.gmail.com> + <95137ba3-1662-b75d-e55f-893d64c76059@satoshilabs.com> + <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com> +From: matejcik <jan.matejek@satoshilabs.com> +Openpgp: preference=signencrypt +Autocrypt: addr=jan.matejek@satoshilabs.com; prefer-encrypt=mutual; keydata= + xsFNBFqFmMgBEADPJ8NULpuu0nwox/tIfo+slGfcXZLUEZstNoaY9QgNuILJRtoJ6xZy8rQf + S7iQlkaZcrpMJYdZtkRHvndkceBxesCG8io6tsU+t2SK6AvaW0FG95a9shFM/U9/JVO/QmBi + IuQzbiE2XTZ/JStyEp4zpuyJqX1o9gzS/4MBXwj7Rzk8u+fHI28h96HILC2a0mC+c2gJ7f5t + o/w+vxFZmk06COK08W5+odb9I8mjs0uf7jgTUEFrfwi6oCoTFmSon7cOy/WTieClwF/vUKuJ + DBAtsMh2rxh8IHyH8xpR+Ay/K6jUWqeb3P2csQqMXmquYG/qdaHjQgxyuoJFbn+nT6jNGVQZ + MjpZkMrGnjLccecaXlgx/rZK6ElCZ1PDHKOTW7A1YY1/eG7TWYnVv1ehQLueAoqyyfiEutsK + E5jGbR0AmNjCahpeK7dxj+8g8TXpVsH207xJ+mqOm5RYqlX4OzfVvcnoHhlRIOu85i4I9rWm + 1u/pP6uJFnBCKtuhhbmXCxM6wF7W5U6EVW3yymsPmSoVoaR024vffE3L5jZSsDMRxY6fDXNm + ljRnOpT3l3d+kMVdAM3CdDCgmV87fdo4PAaGDfnmufGue/Gp0RiLCe/Wsm4DgIIa5UK6DmzD + q0B6i9y/GJSPUChzZ8y7fYzuyXdpk/13gV2NRsskg9oXJVd1vQARAQABzSZtYXRlamNpayA8 + amFuLm1hdGVqZWtAc2F0b3NoaWxhYnMuY29tPsLBfQQTAQgAJwUCWoWYyAIbIwUJCWYBgAUL + CQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRDGf7EG5O0XHoU0D/4+fTbt4KELEtnpkirDH4mQ + Vt3KtKJrI/gp/3u+r6jUWMv2V9iRFMs09GAVBmE2DkXXIlfaT1P0QfwVSpHC4k5lwKwSCSyS + MUgBbQGPOiYMCgMQ+in4vjlqWWcx6jjlgxQctQHRrVG5jyi7BSb0jwG8rcYtx8SAYkN4joG/ + oy2zMbq6qu+Vsl+xR5WwWF2mcUUyiVo7dSwNy+1PaeygOR9xAWkM8J42ckLfJgvyLSviBKnU + 9rgg94ryEDAMNUL5yJUygQmUM/jdpyBpBycRbWMB+zIYDPVGnFj4vN8Hs9DyGUHVb2OqSW+q + VPxD7U9m9z6J3NnY9HpaFX1DD8leK3TebpyYaeODY5jyk7retuLrMq+W4kJU0290xzlWa9sU + wa7lTWw63pelfPUKZ+mjhSFQSZBqiuNv67CBd/UmoqMWSDrCWj+3IFQxReFbh47Wl4MUX2cK + cLocYkBzDck7hH4YfK6jJ++teN6RKXr7P1y6EI25WEfJxWK9say7x/FRkNW0s98MxtOuwEsm + /vHqHQQanAT4R5l+Rr7XfU7fpmH0As98qD81lc3RHbrxEXgA0ks2VuCxBWsPpzaHUFPOcE9H + hsg1jSEDi/Mo6D4e2ap7FYXDgZiKye9WnSdPlVBqJxqinDDgSBv5wzKaEGQS0MKrF9myS7d0 + pBSy1Dr6IWOegM7BTQRahZjIARAAwwT6h4IFvs/hmY9KHiX/GIbvybQUU71ZWYRE2KKo5E2c + ZXBJj7SiDtU80bS+NCSeF2c0i4xOYgZlIYMqlgS8k1zfdBt/JHmG3tm1JgohVj+pm42RfBAF + d0y05zz5wysQOw1M4WlWKZH0ameM+0/AGqspeZushWay8Q4yx1dO/6MeyPy/NwE/MKEsCOPV + aN28DndN3iKOyriCQt/IhG/n6ORPRGyei3JYqxsnpW36BOmSPWJ7Qj2pFw53p5coPOEDL8mN + Ique0LJZ3zVFVMa4i7HtqIEnYO+ZnKx2G8aLsHEir2pzBv6tMwlgETcUTVfK1ePN7OzhYy4q + a38hMWzk0db2V+gOlAu6SuAi1ANkcPhCPUWxPIvXiNdd9iwe5gOzFy0FoZxj22rFwgUX8wcc + cfWStgoE1MGE9G5zrqc01R0x7by8BOFkImAwTyJ9vq4jG+w7Npky3PhoHPgCT5knV7Q91U2I + TqPOQBcMda0B+4LOaElb1sXqe44dHwcg4dMVngaea5xL7winSqU2Gtm6pqFAGut5F7JiYhPb + dGUHJPMS67ONkKe5ARu/Z/r9XoFe2TxpkvNJ/+QJQ3PCiJ6ya31ij6HOIfFbZr3xlTyU/DvG + SejIvDK/SnJMw+/x60bYAshYBp0uQgih1ugtoZh7cnKj3KfhlpXT0mL8rsl1QHsAEQEAAcLB + ZQQYAQgADwUCWoWYyAIbDAUJCWYBgAAKCRDGf7EG5O0XHs2xD/92sa5L6gafP/rRKfo9u3/w + s+7E/kKPgG4VGDeirLo8hbinCjPr0cfZ7OgDDvp0zy6lTdZc2tcHsEbiPqblzaSZimV5Y3EQ + eIzz0UhY6YdDELr8pvdnB8qnOJHXgWmZTRYkRgxFOWI3v4STmOYZQ7MFv0kHBfV3htCjYTHS + Qx2jQO4CTbcSEbkVwNv56OiZroabrHRf0WUSyzElf13P/MRFjUJFYYZDqc0iOWUh4QeXbFiY + fLYpOCtm0nqaDdG1VD4jMpKq1FKBvTw4id1i7pONENd4BB7ytnDvKGdVI6oDnGUBsc5VUrEa + h1PbbshNMbRtFigeMe8998jWhK4jQzeuDr0FSBlhxbluGfyMUgk7s6aBC9BOsdDkgtJk1Fd/ + j9sWOj8Pxzc4lMQRfygm+QxxLdqa36Qh3oK+jfK7362CXlqBfb9ryerjfFGY4VqMBzQ+BFtj + lYZSdVzGWlmLD9D88wzeByIZMScQPvrXSFwPO2/TuOQNCo0VHcgHpNFzeMRK2eT8bhry+dlq + U+0Kxy2gQijw9j/EZlqR3w053EwUrfAAmHHeYPimXK4pc8oSw0s1A6hQO7Vc0SgblF8taFTM + UhRR7xZg+l5vybAgrDYVL75b9CDscZqd7WVmZx+xU23sUG6SaxXI7PV6bPuMug0fD3SAsieu + +vypQ3jCcUKGrA== +Message-ID: <c7a4476b-8643-3ddd-723b-1ff8b8910e36@satoshilabs.com> +Date: Wed, 4 Jul 2018 15:19:11 +0200 +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 + Thunderbird/52.8.0 +MIME-Version: 1.0 +In-Reply-To: <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com> +Content-Type: multipart/signed; micalg=pgp-sha256; + protocol="application/pgp-signature"; + boundary="3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE" +X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham + version=3.3.1 +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on + smtp1.linux-foundation.org +X-Mailman-Approved-At: Thu, 05 Jul 2018 07:57:46 +0000 +Subject: Re: [bitcoin-dev] BIP 174 thoughts +X-BeenThere: bitcoin-dev@lists.linuxfoundation.org +X-Mailman-Version: 2.1.12 +Precedence: list +List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> +List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> +List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> +List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> +List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> +List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, + <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> +X-List-Received-Date: Wed, 04 Jul 2018 13:19:18 -0000 + +This is an OpenPGP/MIME signed message (RFC 4880 and 3156) +--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE +Content-Type: multipart/mixed; boundary="6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE"; + protected-headers="v1" +From: matejcik <jan.matejek@satoshilabs.com> +To: Achow101 <achow101-lists@achow101.com>, + Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org> +Cc: Pieter Wuille <pieter.wuille@gmail.com>, tomas.susanka@satoshilabs.com +Message-ID: <c7a4476b-8643-3ddd-723b-1ff8b8910e36@satoshilabs.com> +Subject: Re: [bitcoin-dev] BIP 174 thoughts +References: <CAPg+sBhGMxXatsyCAqeboQKH8ASSFAfiXzxyXR9UrNFnah5PPw@mail.gmail.com> + <ljk5Z_a3KK6DHfmPJxI8o9W2CkwszkUG34h0i1MTGU4ss8r3BTQ3GnTtDTfWF6J7ZqcSAmejzrr11muWqYN-_wnWw_0NFn5_lggNnjI0_Rc=@achow101.com> + <f8f5b1e3-692a-fc1e-2ad3-c4ad4464957f@satoshilabs.com> + <TGyS7Azu3inMQFv9QFn8USr9v2m5QbhDRmiOI-4FWwscUeuIB9rA7mCmZA4-kwCJOMAx92fO7XICHtE7ES_QmIYLDy6RHof1WLALskGUYAc=@achow101.com> + <c32dc90d-9919-354b-932c-f93fe329760b@satoshilabs.com> + <CAPg+sBhhYuMi6E1in7wZovX7R7M=450cm6vxaGC1Sxr=cJAZsw@mail.gmail.com> + <881def14-696c-3207-cf6c-49f337ccf0d1@satoshilabs.com> + <CAPg+sBg4MCOoMDBVQ2eZ=p3iS3dq506Jh4vUNBmmM20a6uCwYw@mail.gmail.com> + <95137ba3-1662-b75d-e55f-893d64c76059@satoshilabs.com> + <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com> +In-Reply-To: <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com> + +--6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE +Content-Type: text/plain; charset=utf-8 +Content-Language: en-US +Content-Transfer-Encoding: quoted-printable + +hello, + +we still have some concerns about the BIP as currently proposed - not +about the format or data contents, but more about strictness and +security properties. I have raised some in the previous e-mails, but +they might have been lost in the overall talk about format. + +* Choosing from duplicate keys when combining. +We believe that "choose whichever value it wishes" is not a good +resolution strategy. We propose to either change this to "in case of +conflicts, software MUST reject the conflicting PSBTs", or explain in +more detail why picking at random is a safe choice. + +* Signing records with unknown keys. +There's been some talk about this at start, but there should be a clear +strategy for Signers when unknown fields are encountered. We intend to +implement the rule: "will not sign an input with any unknown fields +present". +Maybe it is worth codifying this behavior in the standard, or maybe +there should be a way to mark a field as "optional" so that strict +Signers know they can _safely_ ignore the unknown field. + + +And two minor points: + +* Fields with empty keys. +This might be inferred from the definition, but is probably worth +spelling out explicitly: If a field definition states that the key data +is empty, an implementation MUST enforce this and reject PSBTs that +contain non-empty data. +We suggest adding something to the effect of: +"If a key or value data in a field doesn't match the specified format, +the PSBT is invalid. In particular, if key data is specified as "none" +but the key contains data beyond the type specifier, implementation MUST +reject the PSBT." +(not sure about the languge, this should of course allow processing +unknown fields) + +* "Combiner can detect inconsistencies" +Added in response to this comment [1], the current wording looks like +it's describing what the Combiner is _capable of_, as opposed to +prescribing what the combiner is _allowed to_ do. +We suggest changing to something like: +"For every field type that the Combiner understands, it MAY also refuse +to combine PSBTs that have inconsistencies in that field, or cause a +conflict when combined." + +regards +m. + +[1] https://github.com/bitcoin/bips/pull/694#discussion_r199232318 + +On 29.6.2018 21:12, Achow101 wrote: +> Hi, +>=20 +> I do not think that protobuf is the way to go for this. Not only is it = +another dependency +> which many wallets do not want to add (e.g. Armory has not added BIP 70= + support because +> of its dependency on protobuf), but it is a more drastic change than th= +e currently proposed +> changes. The point of this email thread isn't to rewrite and design a n= +ew BIP (which is effectively +> what is currently going on). The point is to modify and improve the cur= +rent one. In particular, +> we do not want such drastic changes that people who have already implem= +ented the current +> BIP would have to effectively rewrite everything from scratch again. +>=20 +> I believe that this discussion has become bikeshedding and is really no= + longer constructive. Neither +> of us are going to convince the other to use or not use protobuf. ASeei= +ng how no one else +> has really participated in this discussion about protobuf and key uniqu= +eness, I do not think +> that these suggested changes are really necessary nor useful to others.= + It boils down to personal preference +> rather than technical merit. As such, I have opened a PR to the BIPs re= +po (https://github.com/bitcoin/bips/pull/694) +> which contains the changes that I proposed in an earlier email. +>=20 +> Additionally, because there have been no objections to the currently pr= +oposed changes, I propose +> to move the BIP from Draft to Proposed status. +>=20 +> Andrew +>=20 +>=20 +> =E2=80=8B=E2=80=8B +>=20 +> =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Origina= +l Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90= + +>=20 +> On June 29, 2018 2:53 AM, matejcik via bitcoin-dev <bitcoin-dev@lists.l= +inuxfoundation.org> wrote: +>=20 +>> =E2=80=8B=E2=80=8B +>> +>> Short version: +>> +>> - I propose that conflicting "values" for the same "key" are conside= +red +>> =20 +>> invalid. +>> =20 +>> - Let's not optimize for invalid data. +>> - Given that, there's an open question on how to handle invalid data= + +>> =20 +>> when encountered +>> =20 +>> In general, I don't think it's possible to enforce correctness at = +the +>> =20 +>> format level. You still need application level checks - and that c= +alls +>> =20 +>> into question what we gain by trying to do this on the format leve= +l. +>> =20 +>> Long version: +>> =20 +>> Let's look at this from a different angle. +>> =20 +>> There are roughly two possible "modes" for the format with regard = +to +>> =20 +>> possibly-conflicting data. Call them "permissive" and "restrictive= +". +>> =20 +>> The spec says: +>> =20 +>> """ +>> =20 +>> Keys within each scope should never be duplicated; all keys in the= + +>> =20 +>> format are unique. PSBTs containing duplicate keys are invalid. Ho= +wever +>> =20 +>> implementors will still need to handle events where keys are dupli= +cated +>> =20 +>> when combining transactions with duplicated fields. In this event,= + the +>> =20 +>> software may choose whichever value it wishes. +>> =20 +>> """ +>> =20 +>> The last sentence of this paragraph sets the mode to permissive: +>> =20 +>> duplicate values are pretty much OK. If you see them, just pick on= +e. +>> =20 +>> You seem to argue that Combiners, in particular simple ones that d= +on't +>> =20 +>> understand field semantics, should merge keys permissively, but +>> =20 +>> deduplicate values restrictively. +>> =20 +>> IOW: if you receive two different values for the same key, just pi= +ck +>> =20 +>> whichever, but $deity forbid you include both! +>> =20 +>> This choice doesn't make sense to me. +>> =20 +>> What would make sense is fully restrictive mode: receiving two +>> =20 +>> different values for the same key is a fatal condition with no rec= +overy. +>> =20 +>> If you have a non-deterministic scheme, put a differentiator in th= +e key. +>> =20 +>> Or all the data, for that matter. +>> =20 +>> (Incidentally, this puts key-aware and keyless Combiners on the sa= +me +>> =20 +>> footing. As long as all participants uphold the protocol, differen= +t +>> =20 +>> value =3D different key =3D different full record.) +>> =20 +>> Given that, it's nice to have the Combiner perform the task of det= +ecting +>> =20 +>> this and failing. But not at all necessary. As the quoted paragrap= +h +>> =20 +>> correctly notes, consumers still need to handle PSBTs with duplica= +te keys. +>> =20 +>> (In this context, your implied permissive/restrictive Combiner is +>> =20 +>> optimized for dealing with invalid data. That seems like a wrong +>> =20 +>> optimization.) +>> =20 +>> A reasonable point to decide is whether the handling at the consum= +er +>> =20 +>> should be permissive or restrictive. Personally I'm OK with either= +=2E I'd +>> =20 +>> go with the following change: +>> =20 +>> """ +>> =20 +>> In this event, the software MAY reject the transaction as invalid.= + If it +>> =20 +>> decides to accept it, it MUST choose the last value encountered. +>> =20 +>> """ +>> =20 +>> (deterministic way of choosing, instead of "whichever you like") +>> =20 +>> We could also drop the first part, explicitly allowing consumers t= +o +>> =20 +>> pick, and simplifying the Combiner algorithm to `sort -u`. +>> =20 +>> Note that this sort of "picking" will probably be implicit. I'd ex= +pect +>> =20 +>> the consumer to look like this: +>> =20 +>> +>> for key, value in parse(nextRecord()): +>> data[key] =3D value +>> =20 +>> +>> Or we could drop the second part and switch MAY to MUST, for a fully +>> +>> restrictive mode - which, funnily enough, still lets the Combiner work= + +>> +>> as `sort -u`. +>> +>> To see why, remember that distinct values for the same key are not +>> +>> allowed in fully restrictive mode. If a Combiner encounters two +>> +>> conflicting values F(1) and F(2), it should fail -- but if it doesn't,= + +>> +>> it includes both and the same failure WILL happen on the fully +>> +>> restrictive consumer. +>> +>> This was (or is) my point of confusion re Combiners: the permissive ke= +y +>> +>> - restrictive value mode of operation doesn't seem to help subsequen= +t +>> =20 +>> consumers in any way. +>> =20 +>> Now, for the fully restrictive consumer, the key-value model is in= +deed +>> =20 +>> advantageous (and this is the only scenario that I can imagine in = +which +>> =20 +>> it is advantageous), because you can catch key duplication on the = +parser +>> =20 +>> level. +>> =20 +>> But as it turns out, it's not enough. Consider the following recor= +ds: +>> =20 +>> key(<PSBT_IN_REDEEM_SCRIPT> + abcde), value(<some redeem script>) +>> =20 +>> +>> and: +>> +>> key(<PSBT_IN_REDEEM_SCRIPT> + fghij), value(<some other redeem script>= +) +>> +>> A purely syntactic Combiner simply can't handle this case. The +>> +>> restrictive consumer needs to know whether the key is supposed to be +>> +>> repeating or not. +>> +>> We could fix this, e.g., by saying that repeating types must have high= + +>> +>> bit set and non-repeating must not. We also don't have to, because the= + +>> +>> worst failure here is that a consumer passes an invalid record to a +>> +>> subsequent one and the failure happens one step later. +>> +>> At this point it seems weird to be concerned about the "unique key" +>> +>> correctness, which is a very small subset of possibly invalid inputs. = +As +>> +>> a strict safety measure, I'd instead propose that a consumer MUST NOT +>> +>> operate on inputs or outputs, unless it understand ALL included fields= + - +>> +>> IOW, if you're signing a particular input, all fields in said input ar= +e +>> +>> mandatory. This prevents a situation where a simple Signer processes a= +n +>> +>> input incorrectly based on incomplete set of fields, while still +>> +>> allowing Signers with different capabilities within the same PSBT. +>> +>> (The question here is whether to have either a flag or a reserved rang= +e +>> +>> for "optional fields" that can be safely ignored by consumers that don= +'t +>> +>> understand them, but provide data for consumers who do.) +>> +>>>> To repeat and restate my central question: Why is it important, +>>>> +>>>> that an agent which doesn't understand a particular field +>>>> +>>>> structure, can nevertheless make decisions about its inclusion or +>>>> +>>>> omission from the result (based on a repeated prefix)? +>>> +>>> Again, because otherwise you may need a separate Combiner for each +>>> +>>> type of script involved. That would be unfortunate, and is very +>>> +>>> easily avoided. +>> +>> This is still confusing to me, and I would really like to get to the +>> +>> same page on this particular thing, because a lot of the debate hinges= + +>> +>> on it. I think I covered most of it above, but there are still pieces = +to +>> +>> clarify. +>> +>> As I understand it, the Combiner role (actually all the roles) is most= +ly +>> +>> an algorithm, with the implication that it can be performed +>> +>> independently by a separate agent, say a network node. +>> +>> So there's two types of Combiners: +>> +>> a) Combiner as a part of an intelligent consumer -- the usual scenario= + +>> +>> is a Creator/Combiner/Finalizer/Extractor being one participant, and +>> +>> Updater/Signers as other participants. +>> +>> In this case, the discussion of "simple Combiners" is actually talking= + +>> +>> about intelligent Combiners which don't understand new fields and must= + +>> +>> correctly pass them on. I argue that this can safely be done without +>> +>> loss of any important properties. +>> +>> b) Combiner as a separate service, with no understanding of semantics.= + +>> +>> Although parts of the debate seem to assume this scenario, I don't thi= +nk +>> +>> it's worth considering. Again, do you have an usecase in mind for it? +>> +>> You also insist on enforcing a limited form of correctness on the +>> +>> Combiner level, but that is not worth it IMHO, as discussed above. +>> +>> Or am I missing something else? +>> +>>> Perhaps you want to avoid signing with keys that are already signed +>>> +>>> with? If you need to derive all the keys before even knowing what +>>> +>>> was already signed with, you've already performed 80% of the work. +>> +>> This wouldn't concern me at all, honestly. If the user sends an alread= +y +>> +>> signed PSBT to the same signer, IMHO it is OK to sign again; the +>> +>> slowdown is a fault of the user/workflow. You could argue that signing= + +>> +>> again is the valid response. Perhaps the Signer should even "consume" +>> +>> its keys and not pass them on after producing a signature? That seems +>> +>> like a sensible rule. +>> +>>> To your point: proto v2 afaik has no way to declare "whole record +>>> +>>> uniqueness", so either you drop that (which I think is unacceptable +>>> +>>> - see the copy/sign/combine argument above), or you deal with it in= + +>>> =20 +>>> your application code. +>>> =20 +>> +>> Yes. My argument is that "whole record uniqueness" isn't in fact an +>> +>> important property, because you need application-level checks anyway. +>> +>> Additionally, protobuf provides awareness of which fields are repeated= + +>> +>> and which aren't, and implicitly implements the "pick last" resolution= + +>> +>> strategy for duplicates. +>> +>> The simplest possible protobuf-based Combiner will: +>> +>> - assume all fields are repeating +>> - concatenate and parse +>> - deduplicate and reserialize. +>> =20 +>> More knowledgeable Combiner will intelligently handle non-repeatin= +g +>> =20 +>> fields, but still has to assume that unknown fields are repeating = +and +>> =20 +>> use the above algorithm. +>> =20 +>> For "pick last" strategy, a consumer can simply parse the message = +and +>> =20 +>> perform appropriate application-level checks. +>> =20 +>> For "hard-fail" strategy, it must parse all fields as repeating an= +d +>> =20 +>> check that there's only one of those that are supposed to be uniqu= +e. +>> =20 +>> This is admittedly more work, and yes, protobuf is not perfectly s= +uited +>> =20 +>> for this task. +>> =20 +>> But: +>> =20 +>> One, this work must be done by hand anyway, if we go with a custom= + +>> =20 +>> hand-parsed format. There is a protobuf implementation for every +>> =20 +>> conceivable platform, we'll never have the same amount of BIP174 p= +arsing +>> =20 +>> code. +>> =20 +>> (And if you're hand-writing a parser in order to avoid the depende= +ncy, +>> =20 +>> you can modify it to do the checks at parser level. Note that this= + is +>> =20 +>> not breaking the format! The modifed parser will consume well-form= +ed +>> =20 +>> protobuf and reject that which is valid protobuf but invalid bip17= +4 - a +>> =20 +>> correct behavior for a bip174 parser.) +>> =20 +>> Two, it is my opinion that this is worth it in order to have a sta= +ndard, +>> =20 +>> well described, well studied and widely implemented format. +>> =20 +>> Aside: I ha that there is no advantage to a record-set based +>> =20 +>> custom format by itself, so IMHO the choice is between protobuf vs= + +>> =20 +>> a custom key-value format. Additionally, it's even possible to imp= +lement +>> =20 +>> a hand-parsable key-value format in terms of protobuf -- again, ar= +guing +>> =20 +>> that "standardness" of protobuf is valuable in itself. +>> =20 +>> regards +>> =20 +>> m. +>> =20 +>> +>> bitcoin-dev mailing list +>> +>> bitcoin-dev@lists.linuxfoundation.org +>> +>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev +>=20 +>=20 + + +--6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE-- + +--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE +Content-Type: application/pgp-signature; name="signature.asc" +Content-Description: OpenPGP digital signature +Content-Disposition: attachment; filename="signature.asc" + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v2 + +iQIcBAEBCAAGBQJbPMlPAAoJEMZ/sQbk7RcecqkP/0jnDmSnq0xYZhIzEFGRpGvW +beoY/A+Q3JtOVyAzWUkdK/rsljGx+E6akmQ1bLtERy4xOxPsUYfYz+vCpOSkBdVw +mTJsuxlvL1N06VjRwwoDx77HqLIH+3mA727DZQnXvFhRATtSiimCeTKQvJI6sNw1 +RaKd6bpLOflkvbRVhErXUj41FfTnYDVMkpyIuEi8cCJoOKiOREDNh6cjdr5J/AE4 +6BDPdHC/qBkmB7G6j+g7CBttv1sCynmPs1K6h7HHGRwcABzInTCu2IcWv3zNL/K7 +IFBnNiV6FAubIRrwy5UO593OLWvynWU79wXjMb53PuclCleiA9Q8EZ5/5oqMYkWI +V5q7w9bsOVY5nHGi0rdFHUeUVMGzA8ercSweK8QvINFt8PiNcR5l2lZGPyipBSMd +1WEwysOqry72xyAP/DUPIAzSipsdKDW8weFjufGl3nixPL6SsBwWv5Y2Ma6aQY32 +uC9rUptGwIHIX4R/oEjB4nFdXV+Br9HD/L1Vy9nAIpiW0uNJhQotcZpsi/dOLvbI +93LlMG91DdcdVPMPgbdimmzF2YN3rlnw9XtgZv84UVwD1G11jbbO/Eq3QaNBqA9y +Czmz7U500KiuOESXCF1uJTqWgT36gnOIwrO1EA17bSPLOUaG8RJ4ej/6k5VaaVDB +wilwxP+KQgLvLFBLqsjO +=39hj +-----END PGP SIGNATURE----- + +--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE-- + |