Delivery-date: Tue, 28 May 2024 15:29:26 -0700
Received-SPF: pass (google.com: domain of fjahr@protonmail.com designates 185.70.43.16 as permitted sender) client-ip=185.70.43.16;
Date: Tue, 28 May 2024 22:01:05 +0000
To: Bitcoin Development Mailing List <bitcoindev@googlegroups.com>
From: "'Fabian' via Bitcoin Development Mailing List" <bitcoindev@googlegroups.com>
Subject: [bitcoindev] BIP for Testnet 4
Message-ID: <a6e3VPsXJf9p3gt_FmNF_Up-wrFuNMKTN30-xCSDHBKXzXnSpVflIZIj2NQ8Wos4PhQCzI2mWEMvIms_FAEs7rQdL15MpC_Phmu_fnR9iTg=@protonmail.com>
Feedback-ID: 5067558:user:proton
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="b1_tHLKC78ARIKRUpKZKO4WjS96BkIj4vUl0CGgsNY4"
Reply-To: Fabian <fjahr@protonmail.com>
Precedence: list
Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com

This is a multi-part message in MIME format.

--b1_tHLKC78ARIKRUpKZKO4WjS96BkIj4vUl0CGgsNY4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello list,

a potential reset or replacement of Testnet 3 has been discussed on this li=
st previously here: https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/=
9yCPo3uUBwAJ

The discussion continued in the accompanying bitcoin core pull request (htt=
ps://github.com/bitcoin/bitcoin/pull/29775) which has been tested in the pa=
st couple of weeks and adopted by several projects on an experimental basis=
, such as https://mempool.space/testnet4 for example.

I have now formalized the current rules and genesis block implemented in th=
e PR as BIP draft: https://github.com/bitcoin/bips/pull/1601

-----------

=3D=3D Abstract =3D=3D

A new test network with the goal to replace Testnet 3. This network comes w=
ith small but important improvements of the consensus rules, that should ma=
ke it impractical to attack the network using only CPU mining.

=3D=3D Motivation =3D=3D

Quoting the original mailing list post from Jameson Lopp:

"Testnet3 has been running for 13 years. It's on block 2.5 million somethin=
g and the block reward is down to ~0.014 TBTC, so mining is not doing a gre=
at job at distributing testnet coins anymore.

The reason the block height is insanely high is due to a rather amusing edg=
e case bug that causes the difficulty to regularly get reset to 1, which ca=
uses a bit of havoc. If you want a deep dive into the quirk: https://blog.l=
opp.net/the-block-storms-of-bitcoins-testnet/

Testnet3 is being actively used for scammy airdrops; those of us who tend t=
o be generous with our testnet coins are getting hounded by non-developers =
chasing cheap gains.

As a result, TBTC is being actively bought and sold; one could argue that t=
he fundamental principle of testnet coins having no value has been broken."

Since then the issue with block storms has been further demonstrated on Tes=
tnet 3 when three years' worth of blocks were mined in a few weeks while re=
ndering the network practically unusable at the same time.

=3D=3D Specification =3D=3D

Consensus of Testnet 4 follows the same rules as Testnet 3 with the excepti=
on of the two new rules detailed below.

This means that the existing 20 min difficulty exception in Testnet 3 is ex=
plicitly kept in place, meaning that a block with a timestamp 20 minutes fu=
rther into the future from the current time is allowed to have a minimum di=
fficulty of 1 instead of whatever the actual level currently is.

=3D=3D=3D Block Storm Fix =3D=3D=3D

When the next work required is calculated for the first block in a new diff=
iculty period, the difficulty of the last block of the previous difficulty =
period is only used as the base for this calculation if its difficulty is n=
ot 1. If its difficulty is 1, all blocks in the previous difficulty period =
are checked in reverse order if any of them have a different difficulty tha=
n 1. When a different difficulty is encountered, it is assumed to be the ac=
tual difficulty of the network and used as the base for the calculation of =
the new difficulty level. Only if all blocks from the previous difficulty p=
eriod have had a difficulty of 1 (possibly by the use of the 20-min rule), =
this is also used as the base for the calculation of the next difficulty pe=
riod.

For the avoidance of doubt, no matter which block in the previous difficult=
y period provides the actual difficulty used as the basis for the calculati=
on, the timestamp of the last block is always the one that is used as the e=
nd time of the difficulty period.

=3D=3D=3D Time Warp Fix =3D=3D=3D

To protect against the time warp attack, the following rule proposed as par=
t of The Great Consensus Cleanup is enforced: "The nTime field of each bloc=
k whose height, mod 2016, is 0 must be greater than or equal to the nTime f=
ield of the immediately prior block minus 600. For the avoidance of doubt, =
such blocks must still comply with existing Median-Time-Past nTime restrict=
ions."

=3D=3D Rationale =3D=3D

The applied changes were the result of discussions on the mailing list and =
the PR. The selected changes try to strike a balance between minimal change=
s to the network (keeping it as close to mainnet as possible) while making =
it more robust against attackers that try to disrupt the network. Several a=
lternative designs were considered:

* For the block storm fix an alternative fix could have been to prevent the=
 last block in a difficulty period from applying the existing difficulty ex=
ception. Both solutions were deemed acceptable and there was no clear prefe=
rence among reviewers.
* Removal of the 20-min difficulty exception was discussed but dismissed si=
nce several reviewers insisted that it was a useful feature allowing non-st=
andard transactions to be mined with just a CPU.
* Increase of minimum difficulty was discussed but dismissed as it would ca=
tegorically prevent participation in the network using a CPU miner.
* Increase of the delay in the 20 min difficulty exception was suggested bu=
t did not receive significant support.
* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was d=
ismissed as it had led to confusion among layer-2s that had used testnet fo=
r transaction propagation tests and expected it to behave similar to mainne=
t.
* Motivating miners to re-org min difficulty blocks was suggested but this =
may still be done after Testnet 4 is deployed, so it can be considered out =
of scope for this BIP.
* Persisting the real difficulty in the version field was suggested to prev=
ent exploits of the 20-min rule in an even more robust way, but did not rec=
eive a critical level of support since it would be a more invasive change.

=3D=3D Network Parameters =3D=3D

=3D=3D=3D Consensus Rules =3D=3D=3D

All consensus rules active on mainnet at the time of this proposal are enfo=
rced from block 1, the newest of these rules being the Taproot softfork.

=3D=3D=3D Genesis Block =3D=3D=3D

* Message: <code>03/May/2024 000000000000000000001ebd58c244970b3aa9d783bb00=
1011fbe8ea8e98e00e</code>
* Pubkey: <code>00000000000000000000000000000000000000000000000000000000000=
0000000</code>
* Time stamp: 1714777860
* Nonce: 393743547
* Difficulty: <code>0x1d00ffff</code>
* Version: 1

The resulting genesis block hash is <code>00000000da84f2bafbbc53dee25a72ae5=
07ff4914b867c565be350b0da8bf043</code>, and the block hex is <code>01000000=
010000000000000000000000000000000000000000000000000000000000000000ffffffff5=
504ffff001d01044c4c30332f4d61792f323032342030303030303030303030303030303030=
303030303165626435386332343439373062336161396437383362623030313031316662653=
865613865393865303065ffffffff0100f2052a010000002321000000000000000000000000=
000000000000000000000000000000000000000000ac00000000</code>.

=3D=3D=3D Message Start =3D=3D=3D

The message start is defined as <code>0x1c163f28</code>. These four bytes w=
ere randomly generated and have no special meaning.

=3D=3D Compatibility =3D=3D

This specification is backward compatible in the sense that existing softwa=
re can use Testnet 4 out of the box, assuming support for Testnet 3 already=
 exists.

Simply by adding the network parameters for Testnet 4 (magic number, etc.),=
 a client can connect to and use Testnet 4 without further modifications. T=
he block headers have valid proof of work, so clients can trivially check t=
hat blocks are "probably" valid.

However, without the implementation of the changes detailed in Specificatio=
ns, a client could follow a chain that does not follow the rules. Any fully=
 validating node should check these rules and reject blocks that fail to fo=
llow them. Clients should either validate these rules or connect to trusted=
 peers that do full validation.

=3D=3D Reference implementation =3D=3D

Pull request at https://github.com/bitcoin/bitcoin/pull/29775

=3D=3D References =3D=3D

# Original mailing list thread: https://groups.google.com/g/bitcoindev/c/9b=
L00vRj7OU/m/kFPaQCzmBwAJ
# Blog post on block storm bug: https://blog.lopp.net/the-block-storms-of-b=
itcoins-testnet/
# Consensus Cleanup BIP draft: https://github.com/TheBlueMatt/bips/blob/cle=
anup-softfork/bip-XXXX.mediawiki
# Consensus Cleanup PR draft: https://github.com/bitcoin/bitcoin/pull/15482

=3D=3D Copyright =3D=3D

This document is licensed under the Creative Commons CC0 1.0 Universal lice=
nse.

--=20
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to bitcoindev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/=
bitcoindev/a6e3VPsXJf9p3gt_FmNF_Up-wrFuNMKTN30-xCSDHBKXzXnSpVflIZIj2NQ8Wos4=
PhQCzI2mWEMvIms_FAEs7rQdL15MpC_Phmu_fnR9iTg%3D%40protonmail.com.

--b1_tHLKC78ARIKRUpKZKO4WjS96BkIj4vUl0CGgsNY4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div style=3D"font-family: Arial, sans-serif; font-size: 14px;">Hello list,=
</div><div style=3D"font-family: Arial, sans-serif; font-size: 14px;"><br><=
/div><div style=3D"font-family: Arial, sans-serif; font-size: 14px;">a pote=
ntial reset or replacement of Testnet 3 has been discussed on this list pre=
viously here:&nbsp;<span><a target=3D"_blank" rel=3D"noreferrer nofollow no=
opener" href=3D"https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCP=
o3uUBwAJ">https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBw=
AJ</a></span><br><span></span></div><div style=3D"font-family: Arial, sans-=
serif; font-size: 14px;"><br></div><div style=3D"font-family: Arial, sans-s=
erif; font-size: 14px;">The discussion continued in the&nbsp;<span>accompan=
ying</span> bitcoin core pull request (<span style=3D"scrollbar-width: thin=
; background-color: rgb(255, 255, 255);"><a target=3D"_blank" rel=3D"norefe=
rrer nofollow noopener" href=3D"https://github.com/bitcoin/bitcoin/pull/297=
75" style=3D"scrollbar-width: thin; color: blue;">https://github.com/bitcoi=
n/bitcoin/pull/29775</a>)</span> which has been tested in the past couple o=
f weeks and adopted by several projects on an experimental basis, such as&n=
bsp;<span><a target=3D"_blank" rel=3D"noreferrer nofollow noopener" href=3D=
"https://mempool.space/testnet4">https://mempool.space/testnet4</a>&nbsp;fo=
r example.</span><br><span></span></div><div style=3D"font-family: Arial, s=
ans-serif; font-size: 14px;"><span><br></span></div><div style=3D"font-fami=
ly: Arial, sans-serif; font-size: 14px;"> I have now formalized t<span styl=
e=3D"display: inline !important; background-color: rgb(255, 255, 255);">he =
current rules and genesis block implemented in the PR as BIP draft</span>:&=
nbsp;<span><a target=3D"_blank" rel=3D"noreferrer nofollow noopener" href=
=3D"https://github.com/bitcoin/bips/pull/1601">https://github.com/bitcoin/b=
ips/pull/1601</a></span></div><div style=3D"font-family: Arial, sans-serif;=
 font-size: 14px;"><br></div><div style=3D"font-family: Arial, sans-serif; =
font-size: 14px;">-----------</div><div style=3D"font-family: Arial, sans-s=
erif; font-size: 14px;"><span></span><div><span>=3D=3D Abstract =3D=3D</spa=
n></div><div><br></div><div><span>A new test network with the goal to repla=
ce Testnet 3. This network comes with small but important improvements of t=
he consensus rules, that should make it impractical to attack the network u=
sing only CPU mining.</span></div><div><br></div><div><span>=3D=3D Motivati=
on =3D=3D</span></div><div><br></div><div><span>Quoting the original mailin=
g list post from Jameson Lopp:</span></div><div><br></div><div><span>"Testn=
et3 has been running for 13 years. It's on block 2.5 million something and =
the block reward is down to ~0.014 TBTC, so mining is not doing a great job=
 at distributing testnet coins anymore.</span></div><div><br></div><div><sp=
an>The reason the block height is insanely high is due to a rather amusing =
edge case bug that causes the difficulty to regularly get reset to 1, which=
 causes a bit of havoc. If you want a deep dive into the quirk: <a target=
=3D"_blank" rel=3D"noreferrer nofollow noopener" href=3D"https://blog.lopp.=
net/the-block-storms-of-bitcoins-testnet/">https://blog.lopp.net/the-block-=
storms-of-bitcoins-testnet/</a></span></div><div><br></div><div><span>Testn=
et3 is being actively used for scammy airdrops; those of us who tend to be =
generous with our testnet coins are getting hounded by non-developers chasi=
ng cheap gains.</span></div><div><br></div><div><span>As a result, TBTC is =
being actively bought and sold; one could argue that the fundamental princi=
ple of testnet coins having no value has been broken."</span></div><div><br=
></div><div><span>Since then the issue with block storms has been further d=
emonstrated on Testnet 3 when three years' worth of blocks were mined in a =
few weeks while rendering the network practically unusable at the same time=
.</span></div><div><br></div><div><span>=3D=3D Specification =3D=3D</span><=
/div><div><br></div><div><span>Consensus of Testnet 4 follows the same rule=
s as Testnet 3 with the exception of the two new rules detailed below.</spa=
n></div><div><br></div><div><span>This means that the existing 20 min diffi=
culty exception in Testnet 3 is explicitly kept in place, meaning that a bl=
ock with a timestamp 20 minutes further into the future from the current ti=
me is allowed to have a minimum difficulty of 1 instead of whatever the act=
ual level currently is.</span></div><div><br></div><div><span>=3D=3D=3D Blo=
ck Storm Fix =3D=3D=3D</span></div><div><br></div><div><span>When the next =
work required is calculated for the first block in a new difficulty period,=
 the difficulty of the last block of the previous difficulty period is only=
 used as the base for this calculation if its difficulty is not 1. If its d=
ifficulty is 1, all blocks in the previous difficulty period are checked in=
 reverse order if any of them have a different difficulty than 1. When a di=
fferent difficulty is encountered, it is assumed to be the actual difficult=
y of the network and used as the base for the calculation of the new diffic=
ulty level. Only if all blocks from the previous difficulty period have had=
 a difficulty of 1 (possibly by the use of the 20-min rule), this is also u=
sed as the base for the calculation of the next difficulty period.</span></=
div><div><br></div><div><span>For the avoidance of doubt, no matter which b=
lock in the previous difficulty period provides the actual difficulty used =
as the basis for the calculation, the timestamp of the last block is always=
 the one that is used as the end time of the difficulty period.</span></div=
><div><br></div><div><span>=3D=3D=3D Time Warp Fix =3D=3D=3D</span></div><d=
iv><br></div><div><span>To protect against the time warp attack, the follow=
ing rule proposed as part of The Great Consensus Cleanup is enforced: "The =
nTime field of each block whose height, mod 2016, is 0 must be greater than=
 or equal to the nTime field of the immediately prior block minus 600. For =
the avoidance of doubt, such blocks must still comply with existing Median-=
Time-Past nTime restrictions."</span></div><div><br></div><div><span>=3D=3D=
 Rationale =3D=3D</span></div><div><br></div><div><span>The applied changes=
 were the result of discussions on the mailing list and the PR. The selecte=
d changes try to strike a balance between minimal changes to the network (k=
eeping it as close to mainnet as possible) while making it more robust agai=
nst attackers that try to disrupt the network. Several alternative designs =
were considered:</span></div><div><br></div><div><span>* For the block stor=
m fix an alternative fix could have been to prevent the last block in a dif=
ficulty period from applying the existing difficulty exception. Both soluti=
ons were deemed acceptable and there was no clear preference among reviewer=
s.</span></div><div><span>* Removal of the 20-min difficulty exception was =
discussed but dismissed since several reviewers insisted that it was a usef=
ul feature allowing non-standard transactions to be mined with just a CPU.<=
/span></div><div><span>* Increase of minimum difficulty was discussed but d=
ismissed as it would categorically prevent participation in the network usi=
ng a CPU miner.</span></div><div><span>* Increase of the delay in the 20 mi=
n difficulty exception was suggested but did not receive significant suppor=
t.</span></div><div><span>* Re-enabling &lt;code&gt;acceptnonstdtxn&lt;/cod=
e&gt; in bitcoin core by default was dismissed as it had led to confusion a=
mong layer-2s that had used testnet for transaction propagation tests and e=
xpected it to behave similar to mainnet.</span></div><div><span>* Motivatin=
g miners to re-org min difficulty blocks was suggested but this may still b=
e done after Testnet 4 is deployed, so it can be considered out of scope fo=
r this BIP.</span></div><div><span>* Persisting the real difficulty in the =
version field was suggested to prevent exploits of the 20-min rule in an ev=
en more robust way, but did not receive a critical level of support since i=
t would be a more invasive change.</span></div><div><br></div><div><span>=
=3D=3D Network Parameters =3D=3D</span></div><div><br></div><div><span>=3D=
=3D=3D Consensus Rules =3D=3D=3D</span></div><div><br></div><div><span>All =
consensus rules active on mainnet at the time of this proposal are enforced=
 from block 1, the newest of these rules being the Taproot softfork.</span>=
</div><div><br></div><div><span>=3D=3D=3D Genesis Block =3D=3D=3D</span></d=
iv><div><br></div><div><span>* Message: &lt;code&gt;03/May/2024 00000000000=
0000000001ebd58c244970b3aa9d783bb001011fbe8ea8e98e00e&lt;/code&gt;</span></=
div><div><span>* Pubkey: &lt;code&gt;00000000000000000000000000000000000000=
0000000000000000000000000000&lt;/code&gt;</span></div><div><span>* Time sta=
mp: 1714777860</span></div><div><span>* Nonce: 393743547</span></div><div><=
span>* Difficulty: &lt;code&gt;0x1d00ffff&lt;/code&gt;</span></div><div><sp=
an>* Version: 1</span></div><div><br></div><div><span>The resulting genesis=
 block hash is &lt;code&gt;00000000da84f2bafbbc53dee25a72ae507ff4914b867c56=
5be350b0da8bf043&lt;/code&gt;, and the block hex is &lt;code&gt;01000000010=
000000000000000000000000000000000000000000000000000000000000000ffffffff5504=
ffff001d01044c4c30332f4d61792f323032342030303030303030303030303030303030303=
030303165626435386332343439373062336161396437383362623030313031316662653865=
613865393865303065ffffffff0100f2052a010000002321000000000000000000000000000=
000000000000000000000000000000000000000ac00000000&lt;/code&gt;.</span></div=
><div><br></div><div><span>=3D=3D=3D Message Start =3D=3D=3D</span></div><d=
iv><br></div><div><span>The message start is defined as &lt;code&gt;0x1c163=
f28&lt;/code&gt;. These four bytes were randomly generated and have no spec=
ial meaning.</span></div><div><br></div><div><span>=3D=3D Compatibility =3D=
=3D</span></div><div><br></div><div><span>This specification is backward co=
mpatible in the sense that existing software can use Testnet 4 out of the b=
ox, assuming support for Testnet 3 already exists.</span></div><div><br></d=
iv><div><span>Simply by adding the network parameters for Testnet 4 (magic =
number, etc.), a client can connect to and use Testnet 4 without further mo=
difications. The block headers have valid proof of work, so clients can tri=
vially check that blocks are "probably" valid.</span></div><div><br></div><=
div><span>However, without the implementation of the changes detailed in Sp=
ecifications, a client could follow a chain that does not follow the rules.=
 Any fully validating node should check these rules and reject blocks that =
fail to follow them. Clients should either validate these rules or connect =
to trusted peers that do full validation.</span></div><div><br></div><div><=
span>=3D=3D Reference implementation =3D=3D</span></div><div><br></div><div=
><span>Pull request at <a target=3D"_blank" rel=3D"noreferrer nofollow noop=
ener" href=3D"https://github.com/bitcoin/bitcoin/pull/29775">https://github=
.com/bitcoin/bitcoin/pull/29775</a></span></div><div><br></div><div><span>=
=3D=3D References =3D=3D</span></div><div><br></div><div><span># Original m=
ailing list thread: <a target=3D"_blank" rel=3D"noreferrer nofollow noopene=
r" href=3D"https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/kFPaQCzmB=
wAJ">https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/kFPaQCzmBwAJ</a=
></span></div><div><span># Blog post on block storm bug: <a target=3D"_blan=
k" rel=3D"noreferrer nofollow noopener" href=3D"https://blog.lopp.net/the-b=
lock-storms-of-bitcoins-testnet/">https://blog.lopp.net/the-block-storms-of=
-bitcoins-testnet/</a></span></div><div><span># Consensus Cleanup BIP draft=
: <a target=3D"_blank" rel=3D"noreferrer nofollow noopener" href=3D"https:/=
/github.com/TheBlueMatt/bips/blob/cleanup-softfork/bip-XXXX.mediawiki">http=
s://github.com/TheBlueMatt/bips/blob/cleanup-softfork/bip-XXXX.mediawiki</a=
></span></div><div><span># Consensus Cleanup PR draft: <a target=3D"_blank"=
 rel=3D"noreferrer nofollow noopener" href=3D"https://github.com/bitcoin/bi=
tcoin/pull/15482">https://github.com/bitcoin/bitcoin/pull/15482</a></span><=
/div><div><br></div><div><span>=3D=3D Copyright =3D=3D</span></div><div><br=
></div><div><span>This document is licensed under the Creative Commons CC0 =
1.0 Universal license.</span></div></div><div class=3D"protonmail_signature=
_block" style=3D"font-family: Arial, sans-serif; font-size: 14px;">
</div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;Bitcoin Development Mailing List&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:bitcoindev+unsubscribe@googlegroups.com">bitcoind=
ev+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/bitcoindev/a6e3VPsXJf9p3gt_FmNF_Up-wrFuNMKTN30-xCSDHBKXzXnSpVflI=
ZIj2NQ8Wos4PhQCzI2mWEMvIms_FAEs7rQdL15MpC_Phmu_fnR9iTg%3D%40protonmail.com?=
utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/b=
itcoindev/a6e3VPsXJf9p3gt_FmNF_Up-wrFuNMKTN30-xCSDHBKXzXnSpVflIZIj2NQ8Wos4P=
hQCzI2mWEMvIms_FAEs7rQdL15MpC_Phmu_fnR9iTg%3D%40protonmail.com</a>.<br />

--b1_tHLKC78ARIKRUpKZKO4WjS96BkIj4vUl0CGgsNY4--