path: root/4e/9593589eb07fe44b616e57e2dda59cadf53386

Return-Path: <jan.matejek@satoshilabs.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id 72C56CB2
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Wed,  4 Jul 2018 13:19:18 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mail.sldev.cz (mail.sldev.cz [88.208.115.66])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E013276A
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Wed,  4 Jul 2018 13:19:16 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
	by mail.sldev.cz (Postfix) with ESMTP id 74131E105A;
	Wed,  4 Jul 2018 13:19:14 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at mail.sldev.cz
Received: from mail.sldev.cz ([127.0.0.1])
	by localhost (mail.sldev.cz [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id c8KbxARtKyOu; Wed,  4 Jul 2018 13:19:12 +0000 (UTC)
Received: from [10.8.0.37] (unknown [10.8.0.37])
	by mail.sldev.cz (Postfix) with ESMTPSA id 42854E0E15;
	Wed,  4 Jul 2018 13:19:12 +0000 (UTC)
To: Achow101 <achow101-lists@achow101.com>,
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
References: <CAPg+sBhGMxXatsyCAqeboQKH8ASSFAfiXzxyXR9UrNFnah5PPw@mail.gmail.com>
	<ljk5Z_a3KK6DHfmPJxI8o9W2CkwszkUG34h0i1MTGU4ss8r3BTQ3GnTtDTfWF6J7ZqcSAmejzrr11muWqYN-_wnWw_0NFn5_lggNnjI0_Rc=@achow101.com>
	<f8f5b1e3-692a-fc1e-2ad3-c4ad4464957f@satoshilabs.com>
	<TGyS7Azu3inMQFv9QFn8USr9v2m5QbhDRmiOI-4FWwscUeuIB9rA7mCmZA4-kwCJOMAx92fO7XICHtE7ES_QmIYLDy6RHof1WLALskGUYAc=@achow101.com>
	<c32dc90d-9919-354b-932c-f93fe329760b@satoshilabs.com>
	<CAPg+sBhhYuMi6E1in7wZovX7R7M=450cm6vxaGC1Sxr=cJAZsw@mail.gmail.com>
	<881def14-696c-3207-cf6c-49f337ccf0d1@satoshilabs.com>
	<CAPg+sBg4MCOoMDBVQ2eZ=p3iS3dq506Jh4vUNBmmM20a6uCwYw@mail.gmail.com>
	<95137ba3-1662-b75d-e55f-893d64c76059@satoshilabs.com>
	<RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com>
From: matejcik <jan.matejek@satoshilabs.com>
Openpgp: preference=signencrypt
Autocrypt: addr=jan.matejek@satoshilabs.com; prefer-encrypt=mutual; keydata=
	xsFNBFqFmMgBEADPJ8NULpuu0nwox/tIfo+slGfcXZLUEZstNoaY9QgNuILJRtoJ6xZy8rQf
	S7iQlkaZcrpMJYdZtkRHvndkceBxesCG8io6tsU+t2SK6AvaW0FG95a9shFM/U9/JVO/QmBi
	IuQzbiE2XTZ/JStyEp4zpuyJqX1o9gzS/4MBXwj7Rzk8u+fHI28h96HILC2a0mC+c2gJ7f5t
	o/w+vxFZmk06COK08W5+odb9I8mjs0uf7jgTUEFrfwi6oCoTFmSon7cOy/WTieClwF/vUKuJ
	DBAtsMh2rxh8IHyH8xpR+Ay/K6jUWqeb3P2csQqMXmquYG/qdaHjQgxyuoJFbn+nT6jNGVQZ
	MjpZkMrGnjLccecaXlgx/rZK6ElCZ1PDHKOTW7A1YY1/eG7TWYnVv1ehQLueAoqyyfiEutsK
	E5jGbR0AmNjCahpeK7dxj+8g8TXpVsH207xJ+mqOm5RYqlX4OzfVvcnoHhlRIOu85i4I9rWm
	1u/pP6uJFnBCKtuhhbmXCxM6wF7W5U6EVW3yymsPmSoVoaR024vffE3L5jZSsDMRxY6fDXNm
	ljRnOpT3l3d+kMVdAM3CdDCgmV87fdo4PAaGDfnmufGue/Gp0RiLCe/Wsm4DgIIa5UK6DmzD
	q0B6i9y/GJSPUChzZ8y7fYzuyXdpk/13gV2NRsskg9oXJVd1vQARAQABzSZtYXRlamNpayA8
	amFuLm1hdGVqZWtAc2F0b3NoaWxhYnMuY29tPsLBfQQTAQgAJwUCWoWYyAIbIwUJCWYBgAUL
	CQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRDGf7EG5O0XHoU0D/4+fTbt4KELEtnpkirDH4mQ
	Vt3KtKJrI/gp/3u+r6jUWMv2V9iRFMs09GAVBmE2DkXXIlfaT1P0QfwVSpHC4k5lwKwSCSyS
	MUgBbQGPOiYMCgMQ+in4vjlqWWcx6jjlgxQctQHRrVG5jyi7BSb0jwG8rcYtx8SAYkN4joG/
	oy2zMbq6qu+Vsl+xR5WwWF2mcUUyiVo7dSwNy+1PaeygOR9xAWkM8J42ckLfJgvyLSviBKnU
	9rgg94ryEDAMNUL5yJUygQmUM/jdpyBpBycRbWMB+zIYDPVGnFj4vN8Hs9DyGUHVb2OqSW+q
	VPxD7U9m9z6J3NnY9HpaFX1DD8leK3TebpyYaeODY5jyk7retuLrMq+W4kJU0290xzlWa9sU
	wa7lTWw63pelfPUKZ+mjhSFQSZBqiuNv67CBd/UmoqMWSDrCWj+3IFQxReFbh47Wl4MUX2cK
	cLocYkBzDck7hH4YfK6jJ++teN6RKXr7P1y6EI25WEfJxWK9say7x/FRkNW0s98MxtOuwEsm
	/vHqHQQanAT4R5l+Rr7XfU7fpmH0As98qD81lc3RHbrxEXgA0ks2VuCxBWsPpzaHUFPOcE9H
	hsg1jSEDi/Mo6D4e2ap7FYXDgZiKye9WnSdPlVBqJxqinDDgSBv5wzKaEGQS0MKrF9myS7d0
	pBSy1Dr6IWOegM7BTQRahZjIARAAwwT6h4IFvs/hmY9KHiX/GIbvybQUU71ZWYRE2KKo5E2c
	ZXBJj7SiDtU80bS+NCSeF2c0i4xOYgZlIYMqlgS8k1zfdBt/JHmG3tm1JgohVj+pm42RfBAF
	d0y05zz5wysQOw1M4WlWKZH0ameM+0/AGqspeZushWay8Q4yx1dO/6MeyPy/NwE/MKEsCOPV
	aN28DndN3iKOyriCQt/IhG/n6ORPRGyei3JYqxsnpW36BOmSPWJ7Qj2pFw53p5coPOEDL8mN
	Ique0LJZ3zVFVMa4i7HtqIEnYO+ZnKx2G8aLsHEir2pzBv6tMwlgETcUTVfK1ePN7OzhYy4q
	a38hMWzk0db2V+gOlAu6SuAi1ANkcPhCPUWxPIvXiNdd9iwe5gOzFy0FoZxj22rFwgUX8wcc
	cfWStgoE1MGE9G5zrqc01R0x7by8BOFkImAwTyJ9vq4jG+w7Npky3PhoHPgCT5knV7Q91U2I
	TqPOQBcMda0B+4LOaElb1sXqe44dHwcg4dMVngaea5xL7winSqU2Gtm6pqFAGut5F7JiYhPb
	dGUHJPMS67ONkKe5ARu/Z/r9XoFe2TxpkvNJ/+QJQ3PCiJ6ya31ij6HOIfFbZr3xlTyU/DvG
	SejIvDK/SnJMw+/x60bYAshYBp0uQgih1ugtoZh7cnKj3KfhlpXT0mL8rsl1QHsAEQEAAcLB
	ZQQYAQgADwUCWoWYyAIbDAUJCWYBgAAKCRDGf7EG5O0XHs2xD/92sa5L6gafP/rRKfo9u3/w
	s+7E/kKPgG4VGDeirLo8hbinCjPr0cfZ7OgDDvp0zy6lTdZc2tcHsEbiPqblzaSZimV5Y3EQ
	eIzz0UhY6YdDELr8pvdnB8qnOJHXgWmZTRYkRgxFOWI3v4STmOYZQ7MFv0kHBfV3htCjYTHS
	Qx2jQO4CTbcSEbkVwNv56OiZroabrHRf0WUSyzElf13P/MRFjUJFYYZDqc0iOWUh4QeXbFiY
	fLYpOCtm0nqaDdG1VD4jMpKq1FKBvTw4id1i7pONENd4BB7ytnDvKGdVI6oDnGUBsc5VUrEa
	h1PbbshNMbRtFigeMe8998jWhK4jQzeuDr0FSBlhxbluGfyMUgk7s6aBC9BOsdDkgtJk1Fd/
	j9sWOj8Pxzc4lMQRfygm+QxxLdqa36Qh3oK+jfK7362CXlqBfb9ryerjfFGY4VqMBzQ+BFtj
	lYZSdVzGWlmLD9D88wzeByIZMScQPvrXSFwPO2/TuOQNCo0VHcgHpNFzeMRK2eT8bhry+dlq
	U+0Kxy2gQijw9j/EZlqR3w053EwUrfAAmHHeYPimXK4pc8oSw0s1A6hQO7Vc0SgblF8taFTM
	UhRR7xZg+l5vybAgrDYVL75b9CDscZqd7WVmZx+xU23sUG6SaxXI7PV6bPuMug0fD3SAsieu
	+vypQ3jCcUKGrA==
Message-ID: <c7a4476b-8643-3ddd-723b-1ff8b8910e36@satoshilabs.com>
Date: Wed, 4 Jul 2018 15:19:11 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.8.0
MIME-Version: 1.0
In-Reply-To: <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE"
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham
	version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Thu, 05 Jul 2018 07:57:46 +0000
Subject: Re: [bitcoin-dev] BIP 174 thoughts
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Wed, 04 Jul 2018 13:19:18 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE
Content-Type: multipart/mixed; boundary="6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE";
 protected-headers="v1"
From: matejcik <jan.matejek@satoshilabs.com>
To: Achow101 <achow101-lists@achow101.com>,
 Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Cc: Pieter Wuille <pieter.wuille@gmail.com>, tomas.susanka@satoshilabs.com
Message-ID: <c7a4476b-8643-3ddd-723b-1ff8b8910e36@satoshilabs.com>
Subject: Re: [bitcoin-dev] BIP 174 thoughts
References: <CAPg+sBhGMxXatsyCAqeboQKH8ASSFAfiXzxyXR9UrNFnah5PPw@mail.gmail.com>
 <ljk5Z_a3KK6DHfmPJxI8o9W2CkwszkUG34h0i1MTGU4ss8r3BTQ3GnTtDTfWF6J7ZqcSAmejzrr11muWqYN-_wnWw_0NFn5_lggNnjI0_Rc=@achow101.com>
 <f8f5b1e3-692a-fc1e-2ad3-c4ad4464957f@satoshilabs.com>
 <TGyS7Azu3inMQFv9QFn8USr9v2m5QbhDRmiOI-4FWwscUeuIB9rA7mCmZA4-kwCJOMAx92fO7XICHtE7ES_QmIYLDy6RHof1WLALskGUYAc=@achow101.com>
 <c32dc90d-9919-354b-932c-f93fe329760b@satoshilabs.com>
 <CAPg+sBhhYuMi6E1in7wZovX7R7M=450cm6vxaGC1Sxr=cJAZsw@mail.gmail.com>
 <881def14-696c-3207-cf6c-49f337ccf0d1@satoshilabs.com>
 <CAPg+sBg4MCOoMDBVQ2eZ=p3iS3dq506Jh4vUNBmmM20a6uCwYw@mail.gmail.com>
 <95137ba3-1662-b75d-e55f-893d64c76059@satoshilabs.com>
 <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com>
In-Reply-To: <RdSjdFhvANrG9ve8bXVnqs68ih5_iVK11jdOAL6WoMI2358TdylR3H2SyGHQfByKwMYYOfIJIOq0l6clYf-az8_D_D-D7cByzqbyYt1nV4c=@achow101.com>

--6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable

hello,

we still have some concerns about the BIP as currently proposed - not
about the format or data contents, but more about strictness and
security properties. I have raised some in the previous e-mails, but
they might have been lost in the overall talk about format.

* Choosing from duplicate keys when combining.
We believe that "choose whichever value it wishes" is not a good
resolution strategy. We propose to either change this to "in case of
conflicts, software MUST reject the conflicting PSBTs", or explain in
more detail why picking at random is a safe choice.

* Signing records with unknown keys.
There's been some talk about this at start, but there should be a clear
strategy for Signers when unknown fields are encountered. We intend to
implement the rule: "will not sign an input with any unknown fields
present".
Maybe it is worth codifying this behavior in the standard, or maybe
there should be a way to mark a field as "optional" so that strict
Signers know they can _safely_ ignore the unknown field.


And two minor points:

* Fields with empty keys.
This might be inferred from the definition, but is probably worth
spelling out explicitly: If a field definition states that the key data
is empty, an implementation MUST enforce this and reject PSBTs that
contain non-empty data.
We suggest adding something to the effect of:
"If a key or value data in a field doesn't match the specified format,
the PSBT is invalid. In particular, if key data is specified as "none"
but the key contains data beyond the type specifier, implementation MUST
reject the PSBT."
(not sure about the languge, this should of course allow processing
unknown fields)

* "Combiner can detect inconsistencies"
Added in response to this comment [1], the current wording looks like
it's describing what the Combiner is _capable of_, as opposed to
prescribing what the combiner is _allowed to_ do.
We suggest changing to something like:
"For every field type that the Combiner understands, it MAY also refuse
to combine PSBTs that have inconsistencies in that field, or cause a
conflict when combined."

regards
m.

[1] https://github.com/bitcoin/bips/pull/694#discussion_r199232318

On 29.6.2018 21:12, Achow101 wrote:
> Hi,
>=20
> I do not think that protobuf is the way to go for this. Not only is it =
another dependency
> which many wallets do not want to add (e.g. Armory has not added BIP 70=
 support because
> of its dependency on protobuf), but it is a more drastic change than th=
e currently proposed
> changes. The point of this email thread isn't to rewrite and design a n=
ew BIP (which is effectively
> what is currently going on). The point is to modify and improve the cur=
rent one. In particular,
> we do not want such drastic changes that people who have already implem=
ented the current
> BIP would have to effectively rewrite everything from scratch again.
>=20
> I believe that this discussion has become bikeshedding and is really no=
 longer constructive. Neither
> of us are going to convince the other to use or not use protobuf. ASeei=
ng how no one else
> has really participated in this discussion about protobuf and key uniqu=
eness, I do not think
> that these suggested changes are really necessary nor useful to others.=
 It boils down to personal preference
> rather than technical merit. As such, I have opened a PR to the BIPs re=
po (https://github.com/bitcoin/bips/pull/694)
> which contains the changes that I proposed in an earlier email.
>=20
> Additionally, because there have been no objections to the currently pr=
oposed changes, I propose
> to move the BIP from Draft to Proposed status.
>=20
> Andrew
>=20
>=20
> =E2=80=8B=E2=80=8B
>=20
> =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Origina=
l Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=

>=20
> On June 29, 2018 2:53 AM, matejcik via bitcoin-dev <bitcoin-dev@lists.l=
inuxfoundation.org> wrote:
>=20
>> =E2=80=8B=E2=80=8B
>>
>> Short version:
>>
>> -   I propose that conflicting "values" for the same "key" are conside=
red
>>    =20
>>     invalid.
>>    =20
>> -   Let's not optimize for invalid data.
>> -   Given that, there's an open question on how to handle invalid data=

>>    =20
>>     when encountered
>>    =20
>>     In general, I don't think it's possible to enforce correctness at =
the
>>    =20
>>     format level. You still need application level checks - and that c=
alls
>>    =20
>>     into question what we gain by trying to do this on the format leve=
l.
>>    =20
>>     Long version:
>>    =20
>>     Let's look at this from a different angle.
>>    =20
>>     There are roughly two possible "modes" for the format with regard =
to
>>    =20
>>     possibly-conflicting data. Call them "permissive" and "restrictive=
".
>>    =20
>>     The spec says:
>>    =20
>>     """
>>    =20
>>     Keys within each scope should never be duplicated; all keys in the=

>>    =20
>>     format are unique. PSBTs containing duplicate keys are invalid. Ho=
wever
>>    =20
>>     implementors will still need to handle events where keys are dupli=
cated
>>    =20
>>     when combining transactions with duplicated fields. In this event,=
 the
>>    =20
>>     software may choose whichever value it wishes.
>>    =20
>>     """
>>    =20
>>     The last sentence of this paragraph sets the mode to permissive:
>>    =20
>>     duplicate values are pretty much OK. If you see them, just pick on=
e.
>>    =20
>>     You seem to argue that Combiners, in particular simple ones that d=
on't
>>    =20
>>     understand field semantics, should merge keys permissively, but
>>    =20
>>     deduplicate values restrictively.
>>    =20
>>     IOW: if you receive two different values for the same key, just pi=
ck
>>    =20
>>     whichever, but $deity forbid you include both!
>>    =20
>>     This choice doesn't make sense to me.
>>    =20
>>     What would make sense is fully restrictive mode: receiving two
>>    =20
>>     different values for the same key is a fatal condition with no rec=
overy.
>>    =20
>>     If you have a non-deterministic scheme, put a differentiator in th=
e key.
>>    =20
>>     Or all the data, for that matter.
>>    =20
>>     (Incidentally, this puts key-aware and keyless Combiners on the sa=
me
>>    =20
>>     footing. As long as all participants uphold the protocol, differen=
t
>>    =20
>>     value =3D different key =3D different full record.)
>>    =20
>>     Given that, it's nice to have the Combiner perform the task of det=
ecting
>>    =20
>>     this and failing. But not at all necessary. As the quoted paragrap=
h
>>    =20
>>     correctly notes, consumers still need to handle PSBTs with duplica=
te keys.
>>    =20
>>     (In this context, your implied permissive/restrictive Combiner is
>>    =20
>>     optimized for dealing with invalid data. That seems like a wrong
>>    =20
>>     optimization.)
>>    =20
>>     A reasonable point to decide is whether the handling at the consum=
er
>>    =20
>>     should be permissive or restrictive. Personally I'm OK with either=
=2E I'd
>>    =20
>>     go with the following change:
>>    =20
>>     """
>>    =20
>>     In this event, the software MAY reject the transaction as invalid.=
 If it
>>    =20
>>     decides to accept it, it MUST choose the last value encountered.
>>    =20
>>     """
>>    =20
>>     (deterministic way of choosing, instead of "whichever you like")
>>    =20
>>     We could also drop the first part, explicitly allowing consumers t=
o
>>    =20
>>     pick, and simplifying the Combiner algorithm to `sort -u`.
>>    =20
>>     Note that this sort of "picking" will probably be implicit. I'd ex=
pect
>>    =20
>>     the consumer to look like this:
>>    =20
>>
>>     for key, value in parse(nextRecord()):
>>       data[key] =3D value
>>    =20
>>
>> Or we could drop the second part and switch MAY to MUST, for a fully
>>
>> restrictive mode - which, funnily enough, still lets the Combiner work=

>>
>> as `sort -u`.
>>
>> To see why, remember that distinct values for the same key are not
>>
>> allowed in fully restrictive mode. If a Combiner encounters two
>>
>> conflicting values F(1) and F(2), it should fail -- but if it doesn't,=

>>
>> it includes both and the same failure WILL happen on the fully
>>
>> restrictive consumer.
>>
>> This was (or is) my point of confusion re Combiners: the permissive ke=
y
>>
>> -   restrictive value mode of operation doesn't seem to help subsequen=
t
>>    =20
>>     consumers in any way.
>>    =20
>>     Now, for the fully restrictive consumer, the key-value model is in=
deed
>>    =20
>>     advantageous (and this is the only scenario that I can imagine in =
which
>>    =20
>>     it is advantageous), because you can catch key duplication on the =
parser
>>    =20
>>     level.
>>    =20
>>     But as it turns out, it's not enough. Consider the following recor=
ds:
>>    =20
>>     key(<PSBT_IN_REDEEM_SCRIPT> + abcde), value(<some redeem script>)
>>    =20
>>
>> and:
>>
>> key(<PSBT_IN_REDEEM_SCRIPT> + fghij), value(<some other redeem script>=
)
>>
>> A purely syntactic Combiner simply can't handle this case. The
>>
>> restrictive consumer needs to know whether the key is supposed to be
>>
>> repeating or not.
>>
>> We could fix this, e.g., by saying that repeating types must have high=

>>
>> bit set and non-repeating must not. We also don't have to, because the=

>>
>> worst failure here is that a consumer passes an invalid record to a
>>
>> subsequent one and the failure happens one step later.
>>
>> At this point it seems weird to be concerned about the "unique key"
>>
>> correctness, which is a very small subset of possibly invalid inputs. =
As
>>
>> a strict safety measure, I'd instead propose that a consumer MUST NOT
>>
>> operate on inputs or outputs, unless it understand ALL included fields=
 -
>>
>> IOW, if you're signing a particular input, all fields in said input ar=
e
>>
>> mandatory. This prevents a situation where a simple Signer processes a=
n
>>
>> input incorrectly based on incomplete set of fields, while still
>>
>> allowing Signers with different capabilities within the same PSBT.
>>
>> (The question here is whether to have either a flag or a reserved rang=
e
>>
>> for "optional fields" that can be safely ignored by consumers that don=
't
>>
>> understand them, but provide data for consumers who do.)
>>
>>>> To repeat and restate my central question: Why is it important,
>>>>
>>>> that an agent which doesn't understand a particular field
>>>>
>>>> structure, can nevertheless make decisions about its inclusion or
>>>>
>>>> omission from the result (based on a repeated prefix)?
>>>
>>> Again, because otherwise you may need a separate Combiner for each
>>>
>>> type of script involved. That would be unfortunate, and is very
>>>
>>> easily avoided.
>>
>> This is still confusing to me, and I would really like to get to the
>>
>> same page on this particular thing, because a lot of the debate hinges=

>>
>> on it. I think I covered most of it above, but there are still pieces =
to
>>
>> clarify.
>>
>> As I understand it, the Combiner role (actually all the roles) is most=
ly
>>
>> an algorithm, with the implication that it can be performed
>>
>> independently by a separate agent, say a network node.
>>
>> So there's two types of Combiners:
>>
>> a) Combiner as a part of an intelligent consumer -- the usual scenario=

>>
>> is a Creator/Combiner/Finalizer/Extractor being one participant, and
>>
>> Updater/Signers as other participants.
>>
>> In this case, the discussion of "simple Combiners" is actually talking=

>>
>> about intelligent Combiners which don't understand new fields and must=

>>
>> correctly pass them on. I argue that this can safely be done without
>>
>> loss of any important properties.
>>
>> b) Combiner as a separate service, with no understanding of semantics.=

>>
>> Although parts of the debate seem to assume this scenario, I don't thi=
nk
>>
>> it's worth considering. Again, do you have an usecase in mind for it?
>>
>> You also insist on enforcing a limited form of correctness on the
>>
>> Combiner level, but that is not worth it IMHO, as discussed above.
>>
>> Or am I missing something else?
>>
>>> Perhaps you want to avoid signing with keys that are already signed
>>>
>>> with? If you need to derive all the keys before even knowing what
>>>
>>> was already signed with, you've already performed 80% of the work.
>>
>> This wouldn't concern me at all, honestly. If the user sends an alread=
y
>>
>> signed PSBT to the same signer, IMHO it is OK to sign again; the
>>
>> slowdown is a fault of the user/workflow. You could argue that signing=

>>
>> again is the valid response. Perhaps the Signer should even "consume"
>>
>> its keys and not pass them on after producing a signature? That seems
>>
>> like a sensible rule.
>>
>>> To your point: proto v2 afaik has no way to declare "whole record
>>>
>>> uniqueness", so either you drop that (which I think is unacceptable
>>>
>>> -   see the copy/sign/combine argument above), or you deal with it in=

>>>    =20
>>>     your application code.
>>>    =20
>>
>> Yes. My argument is that "whole record uniqueness" isn't in fact an
>>
>> important property, because you need application-level checks anyway.
>>
>> Additionally, protobuf provides awareness of which fields are repeated=

>>
>> and which aren't, and implicitly implements the "pick last" resolution=

>>
>> strategy for duplicates.
>>
>> The simplest possible protobuf-based Combiner will:
>>
>> -   assume all fields are repeating
>> -   concatenate and parse
>> -   deduplicate and reserialize.
>>    =20
>>     More knowledgeable Combiner will intelligently handle non-repeatin=
g
>>    =20
>>     fields, but still has to assume that unknown fields are repeating =
and
>>    =20
>>     use the above algorithm.
>>    =20
>>     For "pick last" strategy, a consumer can simply parse the message =
and
>>    =20
>>     perform appropriate application-level checks.
>>    =20
>>     For "hard-fail" strategy, it must parse all fields as repeating an=
d
>>    =20
>>     check that there's only one of those that are supposed to be uniqu=
e.
>>    =20
>>     This is admittedly more work, and yes, protobuf is not perfectly s=
uited
>>    =20
>>     for this task.
>>    =20
>>     But:
>>    =20
>>     One, this work must be done by hand anyway, if we go with a custom=

>>    =20
>>     hand-parsed format. There is a protobuf implementation for every
>>    =20
>>     conceivable platform, we'll never have the same amount of BIP174 p=
arsing
>>    =20
>>     code.
>>    =20
>>     (And if you're hand-writing a parser in order to avoid the depende=
ncy,
>>    =20
>>     you can modify it to do the checks at parser level. Note that this=
 is
>>    =20
>>     not breaking the format! The modifed parser will consume well-form=
ed
>>    =20
>>     protobuf and reject that which is valid protobuf but invalid bip17=
4 - a
>>    =20
>>     correct behavior for a bip174 parser.)
>>    =20
>>     Two, it is my opinion that this is worth it in order to have a sta=
ndard,
>>    =20
>>     well described, well studied and widely implemented format.
>>    =20
>>     Aside: I ha that there is no advantage to a record-set based
>>    =20
>>     custom format by itself, so IMHO the choice is between protobuf vs=

>>    =20
>>     a custom key-value format. Additionally, it's even possible to imp=
lement
>>    =20
>>     a hand-parsable key-value format in terms of protobuf -- again, ar=
guing
>>    =20
>>     that "standardness" of protobuf is valuable in itself.
>>    =20
>>     regards
>>    =20
>>     m.
>>    =20
>>
>> bitcoin-dev mailing list
>>
>> bitcoin-dev@lists.linuxfoundation.org
>>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>=20
>=20


--6Da0OhUKrXVTK0gqZvZGfShkP4sl8D8pE--

--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJbPMlPAAoJEMZ/sQbk7RcecqkP/0jnDmSnq0xYZhIzEFGRpGvW
beoY/A+Q3JtOVyAzWUkdK/rsljGx+E6akmQ1bLtERy4xOxPsUYfYz+vCpOSkBdVw
mTJsuxlvL1N06VjRwwoDx77HqLIH+3mA727DZQnXvFhRATtSiimCeTKQvJI6sNw1
RaKd6bpLOflkvbRVhErXUj41FfTnYDVMkpyIuEi8cCJoOKiOREDNh6cjdr5J/AE4
6BDPdHC/qBkmB7G6j+g7CBttv1sCynmPs1K6h7HHGRwcABzInTCu2IcWv3zNL/K7
IFBnNiV6FAubIRrwy5UO593OLWvynWU79wXjMb53PuclCleiA9Q8EZ5/5oqMYkWI
V5q7w9bsOVY5nHGi0rdFHUeUVMGzA8ercSweK8QvINFt8PiNcR5l2lZGPyipBSMd
1WEwysOqry72xyAP/DUPIAzSipsdKDW8weFjufGl3nixPL6SsBwWv5Y2Ma6aQY32
uC9rUptGwIHIX4R/oEjB4nFdXV+Br9HD/L1Vy9nAIpiW0uNJhQotcZpsi/dOLvbI
93LlMG91DdcdVPMPgbdimmzF2YN3rlnw9XtgZv84UVwD1G11jbbO/Eq3QaNBqA9y
Czmz7U500KiuOESXCF1uJTqWgT36gnOIwrO1EA17bSPLOUaG8RJ4ej/6k5VaaVDB
wilwxP+KQgLvLFBLqsjO
=39hj
-----END PGP SIGNATURE-----

--3bZxgmNGoGtr5e1cqwCkXH1h8bWmzFVbE--