path: root/af/0f341ac5d85ab5a6044e50170cd8748f67bb4e

Delivery-date: Tue, 02 Jul 2024 04:27:12 -0700
Received: from mail-yb1-f190.google.com ([209.85.219.190])
	by mail.fairlystable.org with esmtps  (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
	(Exim 4.94.2)
	(envelope-from <bitcoindev+bncBDL4XL646QOBBB6IR62AMGQE53XBRYA@googlegroups.com>)
	id 1sObfD-0001Iy-54
	for bitcoindev@gnusha.org; Tue, 02 Jul 2024 04:27:12 -0700
Received: by mail-yb1-f190.google.com with SMTP id 3f1490d57ef6-e036150efc6sf6836586276.2
        for <bitcoindev@gnusha.org>; Tue, 02 Jul 2024 04:27:10 -0700 (PDT)
ARC-Seal: i=2; a=rsa-sha256; t=1719919625; cv=pass;
        d=google.com; s=arc-20160816;
        b=lyEQiShgOv+p5AdEviOTAHhrCOJbLPTbQgXmC0/suPZJjjVGv7VIqSGt01UD4cUyHM
         4a9j/+h6XggX80SIGkRzhHSLOGjonmG32UM5PKgBJ4yGR2Sah7g2UKVvRdbrm5oWeIpU
         U5+0Lq+SbpUCaqgucF5wDrKxabaodR0FNYGDpkyfLQUmBtN845nTLnmXETY+SMMZxWE2
         YmdRUWCqKtc8ur8YVim56CvJ1xlqQSgnwJmedEkNzoO9KQrIuPPWF7WOOytdd4njm+R/
         j9mvtl03uJlog90FmSkMesm1JjmoJQKfKGwEhQw/eXl0i6GnMmDaCR5UqAIJ7KuEX24r
         HdTg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:reply-to:mime-version:feedback-id
         :references:in-reply-to:message-id:subject:cc:from:to:date
         :dkim-signature;
        bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=;
        fh=B8gD5/RfkTpsJzruBx/3IjkAQefkQFBaKnXLFpcB0Nw=;
        b=NFeqUhzMdorJ3eBZ4QobKp9Kh7GF9iUdVN3M7wEVMUqDjOExZw7xICVmx/oCFEIPc7
         OT8RJBKxCfN50Hg/M/g3AOeOA02Lj3hKMBUOMm/8UiOIW+0f5dUPsLG9uBtRYJnlnj7Y
         pfynD7pU9fi8r5JfQ2MpdY/JI4723eLTGfSUst36HhIF3lcNkEi8N4uLgvTclrfCu9PV
         XzedpbpMHuLp+8Q8O1WDmTBNIVQjhRrX5ut6VgCtoJunTDspQShmEyL3FEPwYYEjpHqf
         bXCF43sPG9X1RIqsyuNsyUEjY7ssoApmZC0ViJLGdTSNBJdaCDFEdBpsAB6PDJHkBZ5F
         GpSw==;
        darn=gnusha.org
ARC-Authentication-Results: i=2; gmr-mx.google.com;
       dkim=pass header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb;
       spf=pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com;
       dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=googlegroups.com; s=20230601; t=1719919625; x=1720524425; darn=gnusha.org;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:reply-to
         :x-original-authentication-results:x-original-sender:mime-version
         :feedback-id:references:in-reply-to:message-id:subject:cc:from:to
         :date:from:to:cc:subject:date:message-id:reply-to;
        bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=;
        b=DjpT1TAtkphUb99oFvEAmA8KwKi6dRxQ5PqUfEuQ1XInG4lG9ahJ67hKI4tZVC/Z8/
         2UB5R7/ZWuGRdcM8Yjxx78jJnHXM2R6ZzskcDzxNp+Od7M6fMKRshQzSHsrrIsU3sEBy
         2/CJnFvmy47QPqvAHeRhO2Ju9VGrLbYDPpGQN7Li4TlL8XLR08/AoW4ysLPecDnln1b1
         EiiD1HEpn7AB0O80SVQGBKnTjJmheBdkTQ4O7BMjdzLTnsXP+kv17qAa6rCNCY24jU6c
         ojw7hBxSROvjNSu5HEX1a3dnJziVNU2hWOVi6Di//KgWI9W9D55cE7yNrn/CeKzkXkQ4
         +y/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1719919625; x=1720524425;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:reply-to
         :x-original-authentication-results:x-original-sender:mime-version
         :feedback-id:references:in-reply-to:message-id:subject:cc:from:to
         :date:x-beenthere:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VhrZDUoDExOECQRhvvxZLhS8pTd1dg8uVSdhg+U2ijw=;
        b=C3c7xnSOJXhiw4vPabGSXJ1Chc70Zzc0WSY5VTEmymNUc/8LwRsOGhFLxcQRBiOA2u
         M6P+Vn73uI/oywSXMd2E7MZI79auA8B5vRBWSkT0Py+3JN201MrFXc/jKlaxIRvlm33A
         +reV2/aYIngGImbXRn9Flq3MeTbs/9rAb2SD0LzK39vrjogOj8bjv+vkIZ7zVOY9is7R
         T3yOSrP/hvX7/8uQZR2LW0Tm//BgHQkgDSP1SVtczBK7mKLhdQTAmxXsQAVw5Q8y1uph
         IM6ipfqMkLhpCwhqVWH3/KT1Qof7kugvii6vW0WzY3Kk7z06iUkqlionX0Vdla1xnutc
         /gLQ==
X-Forwarded-Encrypted: i=2; AJvYcCWg2LbjKYrdST19nuOh7QRjKdSoZJn6lzJhywJ8bdO6d0uPyBdwWskBqmWy6SP7x96tn/tbxOQKiy9YfVsUDnYG56Eh8nE=
X-Gm-Message-State: AOJu0Yw/PYlpbtpiTyk9YuJ3SUZihy1JAXkhs5K3t0W+Sg3mHKSWYSaH
	iW4RR29mDfVhZcbhTM0Ew0ghNfJhpLMw6zzPxGndtTADYS0C9VfZ
X-Google-Smtp-Source: AGHT+IGIoFVbXsWyr03DUceJ7Pfy4N5KZyTUDElOr38kZbWRRSBogf3ZtJdhSH0+nbH9TsjgZqBIow==
X-Received: by 2002:a25:b198:0:b0:e03:5b27:70bc with SMTP id 3f1490d57ef6-e039234a4fdmr3693880276.45.1719919624574;
        Tue, 02 Jul 2024 04:27:04 -0700 (PDT)
X-BeenThere: bitcoindev@googlegroups.com
Received: by 2002:ad4:5bec:0:b0:6b5:577:fbee with SMTP id 6a1803df08f44-6b59fcb6f75ls73677346d6.1.-pod-prod-06-us;
 Tue, 02 Jul 2024 04:27:03 -0700 (PDT)
X-Received: by 2002:a05:6214:19ea:b0:6b5:b491:830a with SMTP id 6a1803df08f44-6b5b71242e7mr1380596d6.13.1719919623419;
        Tue, 02 Jul 2024 04:27:03 -0700 (PDT)
Received: by 2002:a05:620a:917:b0:79d:5863:c65b with SMTP id af79cd13be357-79d99efb7fdms85a;
        Tue, 2 Jul 2024 03:24:00 -0700 (PDT)
X-Received: by 2002:a05:651c:889:b0:2ec:440c:4e1c with SMTP id 38308e7fff4ca-2ee5e380e02mr61771641fa.11.1719915838459;
        Tue, 02 Jul 2024 03:23:58 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1719915838; cv=none;
        d=google.com; s=arc-20160816;
        b=yBh14/rnljjbamJKSxX5mfEPdAVau5XcNkhGgPiYDXxEuL3mK0uqa3hg6Bq1PCbTwY
         HrGs9Wcg6Ey1oD/5dDvf9DMw/7MEwAcLM4njUYqcfPReBiNJW71i5tjEg+LCD22sUEIG
         k8XNGgLi4i/uMSnOtubJQ3kGGJWPLDeESFxowhXP4nmfOMvz4/zAGheQejxD9zL64bMV
         JYNhfybpXxPiV3WyoAYhrmucYqR9vJGhU9CZcB7DxeS75RDXWVpN6n+hez049ChnnU5l
         NOTzOgFGETyN4jKYwqw0U0Z6skj17YvwJbh8EGBzSQsL0TF9D/5T9ol2JIkPNjbn3hgg
         vlkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=mime-version:feedback-id:references:in-reply-to:message-id:subject
         :cc:from:to:date:dkim-signature;
        bh=SMWemyCP6c3lRHcbKXYFEG8Ep22IBd/uhhRPq6YQnX4=;
        fh=VUyRMGDsLDyKXHBc8DWjokFBiSMTvXavinKdBJZhUls=;
        b=tncEpbi66UWtlBrFgQ9wwrLjl6v8FAHvY9jsZwUF/ZCmQUKLCFQitovqWp5e+eSjVh
         yfuQJIoUk7KDyJVrlwXO7AShPvSD8u+Z+ojtu0i2pfKbJFfKise/za/QCN2PtxHR+7J7
         tT2pO+hcg53VlGjWFxMPvxTJJShLuZAC2o17IW8A2UaQSrWydeGxeXN8bIm3/YdTXIKR
         Xfgr//dLknz1jwAnxe0X9/Vk1ClIVE5GuS6w4MYT0P2SLe8/p6p/mZx6FsPvZ9G/mTe8
         pKyicCEpYtCAzAeXshpzVCoWSvhsdDdh1+IXGYkbjkMo9NKEIyRhNJdN2sYMuGBXqKq3
         m36w==;
        dara=google.com
ARC-Authentication-Results: i=1; gmr-mx.google.com;
       dkim=pass header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb;
       spf=pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com;
       dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com
Received: from mail-40141.protonmail.ch (mail-40141.protonmail.ch. [185.70.40.141])
        by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-425787f3b39si3304545e9.1.2024.07.02.03.23.58
        for <bitcoindev@googlegroups.com>
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Tue, 02 Jul 2024 03:23:58 -0700 (PDT)
Received-SPF: pass (google.com: domain of darosior@protonmail.com designates 185.70.40.141 as permitted sender) client-ip=185.70.40.141;
Date: Tue, 02 Jul 2024 10:23:51 +0000
To: Eric Voskuil <eric@voskuil.org>
From: "'Antoine Poinsot' via Bitcoin Development Mailing List" <bitcoindev@googlegroups.com>
Cc: Bitcoin Development Mailing List <bitcoindev@googlegroups.com>
Subject: Re: [bitcoindev] Re: Great Consensus Cleanup Revival
Message-ID: <wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFeOZT7sHDlyrXwJ5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw=@protonmail.com>
In-Reply-To: <9a4c4151-36ed-425a-a535-aa2837919a04n@googlegroups.com>
References: <gnM89sIQ7MhDgI62JciQEGy63DassEv7YZAMhj0IEuIo0EdnafykF6RH4OqjTTHIHsIoZvC2MnTUzJI7EfET4o-UQoD-XAQRDcct994VarE=@protonmail.com> <72e83c31-408f-4c13-bff5-bf0789302e23n@googlegroups.com> <heKH68GFJr4Zuf6lBozPJrb-StyBJPMNvmZL0xvKFBnBGVA3fVSgTLdWc-_8igYWX8z3zCGvzflH-CsRv0QCJQcfwizNyYXlBJa_Kteb2zg=@protonmail.com> <5b0331a5-4e94-465d-a51d-02166e2c1937n@googlegroups.com> <yt1O1F7NiVj-WkmnYeta1fSqCYNFx8h6OiJaTBmwhmJ2MWAZkmmjPlUST6FM7t6_-2NwWKdglWh77vcnEKA8swiAnQCZJY2SSCAh4DOKt2I=@protonmail.com> <be78e733-6e9f-4f4e-8dc2-67b79ddbf677n@googlegroups.com> <jJLDrYTXvTgoslhl1n7Fk9-pL1mMC-0k6gtoniQINmioJpzgtqrJ_WqyFZkLltsCUusnQ4jZ6HbvRC-mGuaUlDi3kcqcFHALd10-JQl-FMY=@protonmail.com> <9a4c4151-36ed-425a-a535-aa2837919a04n@googlegroups.com>
Feedback-ID: 7060259:user:proton
X-Pm-Message-ID: 662ea3888d61a9c66124cb4e384213808b6d4377
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8"
X-Original-Sender: darosior@protonmail.com
X-Original-Authentication-Results: gmr-mx.google.com;       dkim=pass
 header.i=@protonmail.com header.s=protonmail3 header.b=gx0mzASb;
       spf=pass (google.com: domain of darosior@protonmail.com designates
 185.70.40.141 as permitted sender) smtp.mailfrom=darosior@protonmail.com;
       dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=protonmail.com
X-Original-From: Antoine Poinsot <darosior@protonmail.com>
Reply-To: Antoine Poinsot <darosior@protonmail.com>
Precedence: list
Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com
List-ID: <bitcoindev.googlegroups.com>
X-Google-Group-Id: 786775582512
List-Post: <https://groups.google.com/group/bitcoindev/post>, <mailto:bitcoindev@googlegroups.com>
List-Help: <https://groups.google.com/support/>, <mailto:bitcoindev+help@googlegroups.com>
List-Archive: <https://groups.google.com/group/bitcoindev
List-Subscribe: <https://groups.google.com/group/bitcoindev/subscribe>, <mailto:bitcoindev+subscribe@googlegroups.com>
List-Unsubscribe: <mailto:googlegroups-manage+786775582512+unsubscribe@googlegroups.com>,
 <https://groups.google.com/group/bitcoindev/subscribe>
X-Spam-Score: -1.0 (-)

This is a multi-part message in MIME format.

--b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

>>> This does not produce unmalleable block hashes. Duplicate tx hash malle=
ation remains in either case, to the same effect. Without a resolution to b=
oth issues this is an empty promise.
>> Duplicate txids have been invalid since 2012 (CVE-2012-2459).
>
> I think again here you may have misunderstood me. I was not making a poin=
t pertaining to BIP30.

No, in fact you did. CVE-2012-2459 is unrelated to BIP30, it's the duplicat=
e txids malleability found by forrestv in 2012. It's the one you are talkin=
g about thereafter and the one relevant for the purpose of this discussion.

For future reference, the full disclosure of CVE-2012-2459 can be found the=
re: https://bitcointalk.org/?topic=3D102395.

> The proposal does not enable that objective, it is already the case. No m=
alleated block is a valid block.

You are right. The advantage i initially mentioned about how making 64-byte=
s transactions invalid could help caching block failures at an earlier stag=
e is incorrect.

Best,
Antoine Poinsot
On Friday, June 28th, 2024 at 7:14 PM, Eric Voskuil <eric@voskuil.org> wrot=
e:

>>> It is not clear to me how determining the coinbase size can be done at =
an earlier stage of validation than detection of the non-null coinbase.
>> My point wasn't about checking the coinbase size, it was about being abl=
e to cache the hash of a (non-malleated) invalid block as permanently inval=
id to avoid re-downloading and re-validating it.
>
> This I understood, but I think you misunderstood me. Your point was speci=
fically that, "it would let node implementations cache block failures at an=
 earlier stage of validation." Since you have not addressed that aspect I a=
ssume you agree with my assertion above that the proposed rule does not act=
ually achieve this.
>
> Regarding the question of checking coinbase size, the issue is of detecti=
ng (or preventing) hashes mallied via the 64 byte tx technique. A rule agai=
nst 64 byte txs would allow this determination by checking the coinbase alo=
ne. If the coinbase is 64 bytes the block is invalid, if it is not the bloc=
k hash cannot have been mallied (all txs must have been 64 bytes, see previ=
ous reference).
>
> In that case if the block is invalid the invalidity can be cached. But bl=
ock invalidity cannot actually be cached until the block is fully validated=
. A rule to prohibit *all* 64 byte txs is counterproductive as it only adds=
 additional checks on typically thousands of txs per block, serving no purp=
ose.
>
>>> It seems to me that introducing an arbitrary tx size validity may creat=
e more potential implementation bugs than it resolves.
>> The potential for implementation bugs is a fair point to raise, but in t=
his case i don't think it's a big concern. Verifying no transaction in a bl=
ock is 64 bytes is as simple a check as you can get.
>
> You appear to be making the assumption that the check is performed after =
the block is fully parsed (contrary to your "earlier" criterion above). The=
 only way to determine the tx sizes is to parse each tx for witness marker,=
 input count, output count, input script sizes, output script sizes, witnes=
s sizes, and skipping over the header, several constants, and associated bu=
ffers. Doing this "early" to detect malleation is an extraordinarily comple=
x and costly process. On the other hand, as I pointed out, a rational imple=
mentation would only do this early check for the coinbase.
>
> Yet even determining the size of the coinbase is significantly more compl=
ex and costly than checking its first input point against null. That check =
(which is already necessary for validation) resolves the malleation questio=
n, can be performed on the raw unparsed block buffer by simply skipping hea=
der, version, reading input count and witness marker as necessary, offsetti=
ng to the 36 byte point buffer, and performing a byte comparison against [0=
000000000000000000000000000000000000000000000000000000000000000ffffffff].
>
> This is:
>
> (1) earlier
> (2) faster
> (3) simpler
> (4) already consensus
>
>>> And certainly anyone implementing such a verifier must know many intric=
acies of the protocol.
>> They need to know some, but i don't think it's reasonable to expect them=
 to realize the merkle tree construction is such that an inner node may be =
confused with a 64 bytes transaction.
>
> A protocol developer needs to understand that the hash of an invalid bloc=
k cannot be cached unless at least the coinbase has been restricted in size=
 (under the proposal) -or- that the coinbase is a null point (presently or =
under the proposal). In the latter case the check is already performed in v=
alidation, so there is no way a block would presently be cached as invalid =
without checking it. The proposal adds a redundant check, even if limited t=
o just the coinbase. [He must also understand the second type of malleabili=
ty, discussed below.]
>
> If this proposed rule was to activate we would implement it in a late sta=
ge tx.check, after txs/blocks had been fully deserialized. We would not che=
ck it an all in the case where the block is under checkpoint or milestone (=
"assume valid"). In this case we would retain the early null point malleati=
on check (along with the hash duplication malleation check) that we present=
ly have, would validate tx commitments, and commit the block. In other word=
s, the proposal adds unnecessary late stage checks only. Implementing it ot=
herwise would just add complexity and hurt performance.
>
>>> I do not see this. I see a very ugly perpetual seam which will likely r=
esult in unexpected complexities over time.
>> What makes you think making 64 bytes transactions invalid could result i=
n unexpected complexities? And why do you think it's likely?
>
> As described above, it's later, slower, more complex, unnecessarily broad=
, and a consensus change. Beyond that it creates an arbitrary size limit - =
not a lower or upper bound, but a slice out of the domain. Discontinuities =
are inherent complexities in computing. The "unexpected" part speaks for it=
self.
>
>>> This does not produce unmalleable block hashes. Duplicate tx hash malle=
ation remains in either case, to the same effect. Without a resolution to b=
oth issues this is an empty promise.
>> Duplicate txids have been invalid since 2012 (CVE-2012-2459).
>
> I think again here you may have misunderstood me. I was not making a poin=
t pertaining to BIP30. I was referring to the other form of block hash mall=
eability, which results from duplicating sets of trailing txs in a single b=
lock (see previous reference). This malleation vector remains, even with in=
valid 64 byte txs. As I pointed out, this has the "same effect" as the 64 b=
yte tx issue. Merkle hashing the set of txs is insufficient to determine id=
entity. In one case the coinbase must be checked (null point or size) and i=
n the other case the set of tx hashes must be checked for trailing duplicat=
ed sets. [Core performs this second check within the Merkle hashing algorit=
hm (with far more comparisons than necessary), though this can be performed=
 earlier and independently to avoid any hashing in the malleation case.]
>
> I would also point out in the interest of correctness that Core reverted =
its BIP30 soft fork implementation as a consequence of the BIP90 hard fork,=
 following and requiring the BIP34 soft fork that presumably precluded it b=
ut didn't, so it is no longer the case that duplicate tx hashes are invalid=
 in implementation. As you have proposed in this rollup, this requires fixi=
ng again.
>
>> If 64 bytes transactions are also made invalid, this would make it impos=
sible for two valid blocks to have the same hash.
>
> Aside from the BIP30/34/90 issue addressed above, it is already "impossib=
le" (cannot be stronger than computationally infeasible) for two *valid* bl=
ocks to have the same hash. The proposal does not enable that objective, it=
 is already the case. No malleated block is a valid block.
>
> The proposal aims only to make it earlier or easier or faster to check fo=
r block hash malleation. And as I've pointed out above, it doesn't achieve =
those objectives. Possibly the perception that this would be the case is a =
consequence of implementation details, but as I have shown above, it is not=
 in fact the case.
>
> Given either type of malleation, the malleated block can be determined to=
 be invalid by a context free check. But this knowledge cannot ever be cach=
ed against the block hash, since the same hash may be valid. Invalidity can=
 only be cached once a non-mallied block is validated and determined to be =
invalid. Block hash malleations are and will remain invalid blocks with or =
without the proposal, and it will continue to be necessary to avoid caching=
 invalid against the malleation. As you said:
>
>> it was about being able to cache the hash of a (non-malleated) invalid b=
lock as permanently invalid to avoid re-downloading and re-validating it.
>
> This is already the case, and requires validating the full non-malleated =
block. Adding a redundant invalidity check doesn't improve this in any way.
>
> Best,
> Eric
>
> --
> You received this message because you are subscribed to the Google Groups=
 "Bitcoin Development Mailing List" group.
> To unsubscribe from this group and stop receiving emails from it, send an=
 email to bitcoindev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgi=
d/bitcoindev/9a4c4151-36ed-425a-a535-aa2837919a04n%40googlegroups.com.

--=20
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to bitcoindev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/=
bitcoindev/wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFeOZT7sHDlyrX=
wJ5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw%3D%40protonmail.com.

--b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<blockquote style=3D"border-color: rgb(200, 200, 200); border-left: 3px sol=
id rgb(200, 200, 200); padding-left: 10px; color: rgb(102, 102, 102);"><div=
>&gt;&gt; This does not produce unmalleable block hashes. Duplicate tx hash=
=20
malleation remains in either case, to the same effect. Without a=20
resolution to both issues this is an empty promise.<br></div><div>&gt; Dupl=
icate txids have been invalid since 2012 (CVE-2012-2459).<br></div><div><br=
>I think again here you may have misunderstood me. I was not making a point=
 pertaining to BIP30.<br></div></blockquote><div style=3D"font-family: Aria=
l, sans-serif; font-size: 14px; color: rgb(0, 0, 0); background-color: rgb(=
255, 255, 255);"><br></div><div style=3D"font-family: Arial, sans-serif; fo=
nt-size: 14px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">=
No, in fact you did. CVE-2012-2459 is unrelated to BIP30, it's the duplicat=
e txids malleability found by forrestv in 2012. It's the one you are talkin=
g about thereafter and the one relevant for the purpose of this discussion.=
<br></div><div style=3D"font-family: Arial, sans-serif; font-size: 14px; co=
lor: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><br></div><div st=
yle=3D"font-family: Arial, sans-serif; font-size: 14px; color: rgb(0, 0, 0)=
; background-color: rgb(255, 255, 255);">For future reference, the full dis=
closure of&nbsp;CVE-2012-2459 can be found there: <span><a target=3D"_blank=
" rel=3D"noreferrer nofollow noopener" href=3D"https://bitcointalk.org/?top=
ic=3D102395">https://bitcointalk.org/?topic=3D102395</a></span>.</div><div =
style=3D"font-family: Arial, sans-serif; font-size: 14px; color: rgb(0, 0, =
0); background-color: rgb(255, 255, 255);"><br></div><blockquote style=3D"b=
order-color: rgb(200, 200, 200); border-left: 3px solid rgb(200, 200, 200);=
 padding-left: 10px; color: rgb(102, 102, 102);"><div style=3D"font-family:=
 Arial, sans-serif; font-size: 14px; color: rgb(0, 0, 0); background-color:=
 rgb(255, 255, 255);">The proposal does not enable that objective, it is al=
ready the case. No malleated block is a valid block.</div></blockquote><div=
 style=3D""><br></div><div style=3D"">You are right. The advantage i initia=
lly mentioned about how making 64-bytes transactions invalid could help cac=
hing block failures at an earlier stage is incorrect.</div><div style=3D"">=
<br></div><div style=3D"">Best,<br>Antoine Poinsot<br></div><div class=3D"p=
rotonmail_quote">
        On Friday, June 28th, 2024 at 7:14 PM, Eric Voskuil &lt;eric@voskui=
l.org&gt; wrote:<br>
        <blockquote class=3D"protonmail_quote" type=3D"cite">
            &gt;&gt; It is not clear to me how determining the coinbase siz=
e can be done at an earlier stage of validation than detection of the non-n=
ull coinbase.<br>&gt; My point wasn't about checking the coinbase size, it =
was about being able to cache the hash of a (non-malleated) invalid block a=
s permanently invalid to avoid re-downloading and re-validating it.<br><br>=
This I understood, but I think you misunderstood me. Your point was specifi=
cally that, "it would let node implementations cache block failures at an e=
arlier stage of validation." Since you have not addressed that aspect I ass=
ume you agree with my assertion above that the proposed rule does not actua=
lly achieve this.<br><br>Regarding the question of checking coinbase size, =
the issue is of detecting (or preventing) hashes mallied via the 64 byte tx=
 technique. A rule against 64 byte txs would allow this determination by ch=
ecking the coinbase alone. If the coinbase is 64 bytes the block is invalid=
, if it is not the block hash cannot have been mallied (all txs must have b=
een 64 bytes, see previous reference).<br><br>In that case if the block is =
invalid the invalidity can be cached. But block invalidity cannot actually =
be cached until the block is fully validated. A rule to prohibit *all* 64 b=
yte txs is counterproductive as it only adds additional checks on typically=
 thousands of txs per block, serving no purpose.<br><br>&gt;&gt; It seems t=
o me that introducing an arbitrary tx size validity may create more potenti=
al implementation bugs than it resolves.<br>&gt; The potential for implemen=
tation bugs is a fair point to raise, but in this case i don't think it's a=
 big concern. Verifying no transaction in a block is 64 bytes is as simple =
a check as you can get.<br><br>You appear to be making the assumption that =
the check is performed after the block is fully parsed (contrary to your "e=
arlier" criterion above). The only way to determine the tx sizes is to pars=
e each tx for witness marker, input count, output count, input script sizes=
, output script sizes, witness sizes, and skipping over the header, several=
 constants, and associated buffers. Doing this "early" to detect malleation=
 is an extraordinarily complex and costly process. On the other hand, as I =
pointed out, a rational implementation would only do this early check for t=
he coinbase.<br><br>Yet even determining the size of the coinbase is signif=
icantly more complex and costly than checking its first input point against=
 null. That check (which is already necessary for validation) resolves the =
malleation question, can be performed on the raw unparsed block buffer by s=
imply skipping header, version, reading input count and witness marker as n=
ecessary, offsetting to the 36 byte point buffer, and performing a byte com=
parison against [0000000000000000000000000000000000000000000000000000000000=
000000ffffffff].<br><br>This is:<br><br>(1) earlier<br>(2) faster<br>(3) si=
mpler<br>(4) already consensus<br><br>&gt;&gt; And certainly anyone impleme=
nting such a verifier must know many intricacies of the protocol.<br>&gt; T=
hey need to know some, but i don't think it's reasonable to expect them to =
realize the merkle tree construction is such that an inner node may be conf=
used with a 64 bytes transaction.<br><br>A protocol developer needs to unde=
rstand that the hash of an invalid block cannot be cached unless at least t=
he coinbase has been restricted in size (under the proposal) -or- that the =
coinbase is a null point (presently or under the proposal). In the latter c=
ase the check is already performed in validation, so there is no way a bloc=
k would presently be cached as invalid without checking it. The proposal ad=
ds a redundant check, even if limited to just the coinbase. [He must also u=
nderstand the second type of malleability, discussed below.]<br><br>If this=
 proposed rule was to activate we would implement it in a late stage tx.che=
ck, after txs/blocks had been fully deserialized. We would not check it an =
all in the case where the block is under checkpoint or milestone ("assume v=
alid"). In this case we would retain the early null point malleation check =
(along with the hash duplication malleation check) that we presently have, =
would validate tx commitments, and commit the block. In other words, the pr=
oposal adds unnecessary late stage checks only. Implementing it otherwise w=
ould just add complexity and hurt performance.<br><br>&gt;&gt; I do not see=
 this. I see a very ugly perpetual seam which will likely result in unexpec=
ted complexities over time.<br>&gt; What makes you think making 64 bytes tr=
ansactions invalid could result in unexpected complexities? And why do you =
think it's likely?<br><br>As described above, it's later, slower, more comp=
lex, unnecessarily broad, and a consensus change. Beyond that it creates an=
 arbitrary size limit - not a lower or upper bound, but a slice out of the =
domain. Discontinuities are inherent complexities in computing. The "unexpe=
cted" part speaks for itself.<br><br>&gt;&gt; This does not produce unmalle=
able block hashes. Duplicate tx hash malleation remains in either case, to =
the same effect. Without a resolution to both issues this is an empty promi=
se.<br>&gt; Duplicate txids have been invalid since 2012 (CVE-2012-2459).<b=
r><br>I think again here you may have misunderstood me. I was not making a =
point pertaining to BIP30. I was referring to the other form of block hash =
malleability, which results from duplicating sets of trailing txs in a sing=
le block (see previous reference). This malleation vector remains, even wit=
h invalid 64 byte txs. As I pointed out, this has the "same effect" as the =
64 byte tx issue. Merkle hashing the set of txs is insufficient to determin=
e identity. In one case the coinbase must be checked (null point or size) a=
nd in the other case the set of tx hashes must be checked for trailing dupl=
icated sets. [Core performs this second check within the Merkle hashing alg=
orithm (with far more comparisons than necessary), though this can be perfo=
rmed earlier and independently to avoid any hashing in the malleation case.=
]<br><br>I would also point out in the interest of correctness that Core re=
verted its BIP30 soft fork implementation as a consequence of the BIP90 har=
d fork, following and requiring the BIP34 soft fork that presumably preclud=
ed it but didn't, so it is no longer the case that duplicate tx hashes are =
invalid in implementation. As you have proposed in this rollup, this requir=
es fixing again.<br><br>&gt; If 64 bytes transactions are also made invalid=
, this would make it impossible for two valid blocks to have the same hash.=
<br><br>Aside from the BIP30/34/90 issue addressed above, it is already "im=
possible" (cannot be stronger than computationally infeasible) for two *val=
id* blocks to have the same hash. The proposal does not enable that objecti=
ve, it is already the case. No malleated block is a valid block.<br><br>The=
 proposal aims only to make it earlier or easier or faster to check for blo=
ck hash malleation. And as I've pointed out above, it doesn't achieve those=
 objectives. Possibly the perception that this would be the case is a conse=
quence of implementation details, but as I have shown above, it is not in f=
act the case.<br><br>Given either type of malleation, the malleated block c=
an be determined to be invalid by a context free check. But this knowledge =
cannot ever be cached against the block hash, since the same hash may be va=
lid. Invalidity can only be cached once a non-mallied block is validated an=
d determined to be invalid. Block hash malleations are and will remain inva=
lid blocks with or without the proposal, and it will continue to be necessa=
ry to avoid caching invalid against the malleation. As you said:<br><br>&gt=
; it was about being able to cache the hash of a (non-malleated) invalid bl=
ock as permanently invalid to avoid re-downloading and re-validating it.<br=
><br>This is already the case, and requires validating the full non-malleat=
ed block. Adding a redundant invalidity check doesn't improve this in any w=
ay.<br><br>Best,<br>Eric

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:bitcoindev+unsubscribe@googlegroups.com" rel=3D"n=
oreferrer nofollow noopener" target=3D"_blank">bitcoindev+unsubscribe@googl=
egroups.com</a>.<br>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/bitcoindev/9a4c4151-36ed-425a-a535-aa2837919a04n%40googlegroups.=
com" rel=3D"noreferrer nofollow noopener" target=3D"_blank">https://groups.=
google.com/d/msgid/bitcoindev/9a4c4151-36ed-425a-a535-aa2837919a04n%40googl=
egroups.com</a>.<br>

        </blockquote><br>
    </div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;Bitcoin Development Mailing List&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:bitcoindev+unsubscribe@googlegroups.com">bitcoind=
ev+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/bitcoindev/wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFe=
OZT7sHDlyrXwJ5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw%3D%40protonmail.com?=
utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/b=
itcoindev/wg_er0zMhAF9ERoYXmxI6aB7rc97Cum6PQj4UOELapsHVBBVWktFeOZT7sHDlyrXw=
J5o5s9iMb2LW2Od-qacywsh-86p5Q7dP3XjWASXcMw%3D%40protonmail.com</a>.<br />

--b1_Imppqfpa05uoQJeuGY4iXaW2SkKgtL17uJOzJMt8--