MIME-Version: 1.0
In-Reply-To: <87mvwqb132.fsf@rustcorp.com.au>
References: <87mvwqb132.fsf@rustcorp.com.au>
From: Btc Drak <btcdrak@gmail.com>
Date: Wed, 16 Sep 2015 16:53:08 +0100
Message-ID: <CADJgMztpJ2+GHMoHdP0Zt27BWBhVX1LaKO+CEP=q0rvoimgNSw@mail.gmail.com>
To: Rusty Russell <rusty@rustcorp.com.au>
Content-Type: multipart/alternative; boundary=089e01227b8aa45e80051fdf4cec
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>,
	Pieter Wuille <pieter.wuille@gmail.com>, Greg Maxwell <greg@xiph.org>
Subject: Re: [bitcoin-dev] [BIP Proposal] Version bits with timeout and
	delay.
Precedence: list

--089e01227b8aa45e80051fdf4cec
Content-Type: text/plain; charset=UTF-8

Rusty,

I think you've covered all the issues discussed now. +1 for submitting to
BIPs repo to get an official number.

Are you planning to write the implementation?



On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi all,
>
> Those who've seen the original versionbits bip, this adds:
>    1) Percentage checking only on retarget period boundaries.
>    2) 1 retarget period between 95% and activation.
>    3) A stronger suggestion for timeout value selection.
>
> https://gist.github.com/rustyrussell/47eb08093373f71f87de
>
> And pasted below, de-formatted a little.
>
> Thanks,
> Rusty.
>
>   BIP: ??
>   Title: Version bits with timeout and delay
>   Author: Pieter Wuille <pieter.wuille@gmail.com>, Peter Todd <
> pete@petertodd.org>, Greg Maxwell <greg@xiph.org>, Rusty Russell <
> rusty@rustcorp.com.au>
>   Status: Draft
>   Type: Informational Track
>   Created: 2015-10-04
>
> ==Abstract==
>
> This document specifies a proposed change to the semantics of the
> 'version' field in Bitcoin blocks, allowing multiple backward-compatible
> changes (further called called "soft forks") being deployed in parallel. It
> relies on interpreting the version field as a bit vector, where each bit
> can be used to track an independent change. These are tallied each retarget
> period. Once the consensus change succeeds or times out, there is a
> "fallow" pause after which the bit can be reused for later changes.
>
> ==Motivation==
>
> BIP 34 introduced a mechanism for doing soft-forking changes without
> predefined flag timestamp (or flag block height), instead relying on
> measuring miner support indicated by a higher version number in block
> headers. As it relies on comparing version numbers as integers however, it
> only supports one single change being rolled out at once, requiring
> coordination between proposals, and does not allow for permanent rejection:
> as long as one soft fork is not fully rolled out, no future one can be
> scheduled.
>
> In addition, BIP 34 made the integer comparison (nVersion >= 2) a
> consensus rule after its 95% threshold was reached, removing 2^31 +2 values
> from the set of valid version numbers (all negative numbers, as nVersion is
> interpreted as a signed integer, as well as 0 and 1). This indicates
> another downside this approach: every upgrade permanently restricts the set
> of allowed nVersion field values. This approach was later reused in BIP 66,
> which further removed nVersion = 2 as valid option. As will be shown
> further, this is unnecessary.
>
> ==Specification==
>
> ===Mechanism===
>
> '''Bit flags'''
> We are permitting several independent soft forks to be deployed in
> parallel. For each, a bit B is chosen from the set {0,1,2,...,28}, which is
> not currently in use for any other ongoing soft fork. Miners signal intent
> to enforce the new rules associated with the proposed soft fork by setting
> bit 1<sup>B</sup> in nVersion to 1 in their blocks.
>
> '''High bits'''
> The highest 3 bits are set to 001, so the range of actually possible
> nVersion values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two
> future upgrades for different mechanisms (top bits 010 and 011), while
> complying to the constraints set by BIP34 and BIP66. Having more than 29
> available bits for parallel soft forks does not add anything anyway, as the
> (nVersion >= 3) requirement already makes that impossible.
>
> '''States'''
> With every softfork proposal we associate a state BState, which begins
> at ''defined'', and can be ''locked-in'', ''activated'',
> or ''failed''.  Transitions are considered after each
> retarget period.
>
> '''Soft Fork Support'''
> Software which supports the change should begin by setting B in all blocks
> mined until it is resolved.
>
>  if (BState == defined) {
>      SetBInBlock();
>  }
>
> '''Success: Lock-in Threshold'''
> If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks
> within a retarget period, it is considered ''locked-in''.  Miners should
> stop setting bit B.
>
>  if (NextBlockHeight % 2016 == 0) {
>     if (BState == defined && Previous2016BlocksCountB() >= 1916) {
>         BState = locked-in;
>         BActiveHeight = NextBlockHeight + 2016;
>     }
>  }
>
> '''Success: Activation Delay'''
> The consensus rules related to ''locked-in'' soft fork will be enforced in
> the second retarget period; ie. there is a one retarget period in
> which the remaining 5% can upgrade.  At the that activation block and
> after, the bit B may be reused for a different soft fork.
>
>  if (BState == locked-in && NextBlockHeight == BActiveHeight) {
>     BState = activated;
>     ApplyRulesForBFromNextBlock();
>     /* B can be reused, immediately */
>  }
>
> '''Failure: Timeout'''
> A soft fork proposal should include a ''timeout''.  This is measured
> as the beginning of a calendar year as per this table (suggested
> three years from drafting the soft fork proposal):
>
> Timeout Year    >= Seconds              Timeout Year    >= Seconds
> 2018            1514764800              2026            1767225600
> 2019            1546300800              2027            1798761600
> 2020            1577836800              2028            1830297600
> 2021            1609459200              2029            1861920000
> 2022            1640995200              2030            1893456000
> 2023            1672531200              2031            1924992000
> 2024            1704067200              2032            1956528000
> 2025            1735689600              2033            1988150400
>
> If the soft fork still not ''locked-in'' and the
> GetMedianTimePast() of a block following a retarget period is at or
> past this timeout, miners should cease setting this bit.
>
>  if (NextBlockHeight % 2016 == 0) {
>     if (BState == defined && GetMedianTimePast(nextblock) >= BFinalYear) {
>          BState = failed;
>     }
>  }
>
> After another retarget period (to allow detection of buggy miners),
> the bit may be reused.
>
> '''Warning system'''
> To support upgrade warnings, an extra "unknown upgrade" is tracked, using
> the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask
> will be non-zero whenever an unexpected bit is set in nVersion.  Whenever
> lock-in for the unknown upgrade is detected, the software should warn
> loudly about the upcoming soft fork.  It should warn even more loudly after
> the next retarget period.
>
> '''Forks'''
> It should be noted that the states are maintained along block chain
> branches, but may need recomputation when a reorganization happens.
>
> ===Support for future changes===
>
> The mechanism described above is very generic, and variations are possible
> for future soft forks. Here are some ideas that can be taken into account.
>
> '''Modified thresholds'''
> The 95% threshold (based on in BIP 34) does not have to be maintained for
> eternity, but changes should take the effect on the warning system into
> account. In particular, having a lock-in threshold that is incompatible
> with the one used for the warning system may have long-term effects, as the
> warning system cannot rely on a permanently detectable condition anymore.
>
> '''Conflicting soft forks'''
> At some point, two mutually exclusive soft forks may be proposed. The
> naive way to deal with this is to never create software that implements
> both, but that is a making a bet that at least one side is guaranteed to
> lose. Better would be to encode "soft fork X cannot be locked-in" as
> consensus rule for the conflicting soft fork - allowing software that
> supports both, but can never trigger conflicting changes.
>
> '''Multi-stage soft forks'''
> Soft forks right now are typically treated as booleans: they go from an
> inactive to an active state in blocks. Perhaps at some point there is
> demand for a change that has a larger number of stages, with additional
> validation rules that get enabled one by one. The above mechanism can be
> adapted to support this, by interpreting a combination of bits as an
> integer, rather than as isolated bits. The warning system is compatible
> with this, as (nVersion & ~nExpectedVersion) will always be non-zero for
> increasing integers.
>
> == Rationale ==
>
> The failure timeout allows eventual reuse of bits even if a soft fork was
> never activated, so it's clear that the new use of the bit refers to a
> new BIP.  It's deliberately very course grained, to take into account
> reasonable development and deployment delays.  There are unlikely to be
> enough failed proposals to cause a bit shortage.
>
> The fallow period at the conclusion of a soft fork attempt allows some
> detection of buggy clients, and allows time for warnings and software
> upgrades for successful soft forks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--089e01227b8aa45e80051fdf4cec
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Rusty,=C2=A0<div><br></div><div>I think you&#39;ve covered=
 all the issues discussed now. +1 for submitting to BIPs repo to get an off=
icial number.</div><div><br></div><div>Are you planning to write the implem=
entation?</div><div><br></div><div><br></div></div><div class=3D"gmail_extr=
a"><br><div class=3D"gmail_quote">On Sun, Sep 13, 2015 at 7:56 PM, Rusty Ru=
ssell via bitcoin-dev <span dir=3D"ltr">&lt;<a href=3D"mailto:bitcoin-dev@l=
ists.linuxfoundation.org" target=3D"_blank">bitcoin-dev@lists.linuxfoundati=
on.org</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"=
margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
Those who&#39;ve seen the original versionbits bip, this adds:<br>
=C2=A0 =C2=A01) Percentage checking only on retarget period boundaries.<br>
=C2=A0 =C2=A02) 1 retarget period between 95% and activation.<br>
=C2=A0 =C2=A03) A stronger suggestion for timeout value selection.<br>
<br>
<a href=3D"https://gist.github.com/rustyrussell/47eb08093373f71f87de" rel=
=3D"noreferrer" target=3D"_blank">https://gist.github.com/rustyrussell/47eb=
08093373f71f87de</a><br>
<br>
And pasted below, de-formatted a little.<br>
<br>
Thanks,<br>
Rusty.<br>
<br>
=C2=A0 BIP: ??<br>
=C2=A0 Title: Version bits with timeout and delay<br>
=C2=A0 Author: Pieter Wuille &lt;<a href=3D"mailto:pieter.wuille@gmail.com"=
>pieter.wuille@gmail.com</a>&gt;, Peter Todd &lt;<a href=3D"mailto:pete@pet=
ertodd.org">pete@petertodd.org</a>&gt;, Greg Maxwell &lt;<a href=3D"mailto:=
greg@xiph.org">greg@xiph.org</a>&gt;, Rusty Russell &lt;<a href=3D"mailto:r=
usty@rustcorp.com.au">rusty@rustcorp.com.au</a>&gt;<br>
=C2=A0 Status: Draft<br>
=C2=A0 Type: Informational Track<br>
=C2=A0 Created: 2015-10-04<br>
<br>
=3D=3DAbstract=3D=3D<br>
<br>
This document specifies a proposed change to the semantics of the &#39;vers=
ion&#39; field in Bitcoin blocks, allowing multiple backward-compatible cha=
nges (further called called &quot;soft forks&quot;) being deployed in paral=
lel. It relies on interpreting the version field as a bit vector, where eac=
h bit can be used to track an independent change. These are tallied each re=
target period. Once the consensus change succeeds or times out, there is a =
&quot;fallow&quot; pause after which the bit can be reused for later change=
s.<br>
<br>
=3D=3DMotivation=3D=3D<br>
<br>
BIP 34 introduced a mechanism for doing soft-forking changes without predef=
ined flag timestamp (or flag block height), instead relying on measuring mi=
ner support indicated by a higher version number in block headers. As it re=
lies on comparing version numbers as integers however, it only supports one=
 single change being rolled out at once, requiring coordination between pro=
posals, and does not allow for permanent rejection: as long as one soft for=
k is not fully rolled out, no future one can be scheduled.<br>
<br>
In addition, BIP 34 made the integer comparison (nVersion &gt;=3D 2) a cons=
ensus rule after its 95% threshold was reached, removing 2^31 +2 values fro=
m the set of valid version numbers (all negative numbers, as nVersion is in=
terpreted as a signed integer, as well as 0 and 1). This indicates another =
downside this approach: every upgrade permanently restricts the set of allo=
wed nVersion field values. This approach was later reused in BIP 66, which =
further removed nVersion =3D 2 as valid option. As will be shown further, t=
his is unnecessary.<br>
<br>
=3D=3DSpecification=3D=3D<br>
<br>
=3D=3D=3DMechanism=3D=3D=3D<br>
<br>
&#39;&#39;&#39;Bit flags&#39;&#39;&#39;<br>
We are permitting several independent soft forks to be deployed in parallel=
. For each, a bit B is chosen from the set {0,1,2,...,28}, which is not cur=
rently in use for any other ongoing soft fork. Miners signal intent to enfo=
rce the new rules associated with the proposed soft fork by setting bit 1&l=
t;sup&gt;B&lt;/sup&gt; in nVersion to 1 in their blocks.<br>
<br>
&#39;&#39;&#39;High bits&#39;&#39;&#39;<br>
The highest 3 bits are set to 001, so the range of actually possible nVersi=
on values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two future u=
pgrades for different mechanisms (top bits 010 and 011), while complying to=
 the constraints set by BIP34 and BIP66. Having more than 29 available bits=
 for parallel soft forks does not add anything anyway, as the (nVersion &gt=
;=3D 3) requirement already makes that impossible.<br>
<br>
&#39;&#39;&#39;States&#39;&#39;&#39;<br>
With every softfork proposal we associate a state BState, which begins<br>
at &#39;&#39;defined&#39;&#39;, and can be &#39;&#39;locked-in&#39;&#39;, &=
#39;&#39;activated&#39;&#39;,<br>
or &#39;&#39;failed&#39;&#39;.=C2=A0 Transitions are considered after each<=
br>
retarget period.<br>
<br>
&#39;&#39;&#39;Soft Fork Support&#39;&#39;&#39;<br>
Software which supports the change should begin by setting B in all blocks<=
br>
mined until it is resolved.<br>
<br>
=C2=A0if (BState =3D=3D defined) {<br>
=C2=A0 =C2=A0 =C2=A0SetBInBlock();<br>
=C2=A0}<br>
<br>
&#39;&#39;&#39;Success: Lock-in Threshold&#39;&#39;&#39;<br>
If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks<br>
within a retarget period, it is considered &#39;&#39;locked-in&#39;&#39;.=
=C2=A0 Miners should<br>
stop setting bit B.<br>
<br>
=C2=A0if (NextBlockHeight % 2016 =3D=3D 0) {<br>
=C2=A0 =C2=A0 if (BState =3D=3D defined &amp;&amp; Previous2016BlocksCountB=
() &gt;=3D 1916) {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 BState =3D locked-in;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 BActiveHeight =3D NextBlockHeight + 2016;<br>
=C2=A0 =C2=A0 }<br>
=C2=A0}<br>
<br>
&#39;&#39;&#39;Success: Activation Delay&#39;&#39;&#39;<br>
The consensus rules related to &#39;&#39;locked-in&#39;&#39; soft fork will=
 be enforced in<br>
the second retarget period; ie. there is a one retarget period in<br>
which the remaining 5% can upgrade.=C2=A0 At the that activation block and<=
br>
after, the bit B may be reused for a different soft fork.<br>
<br>
=C2=A0if (BState =3D=3D locked-in &amp;&amp; NextBlockHeight =3D=3D BActive=
Height) {<br>
=C2=A0 =C2=A0 BState =3D activated;<br>
=C2=A0 =C2=A0 ApplyRulesForBFromNextBlock();<br>
=C2=A0 =C2=A0 /* B can be reused, immediately */<br>
=C2=A0}<br>
<br>
&#39;&#39;&#39;Failure: Timeout&#39;&#39;&#39;<br>
A soft fork proposal should include a &#39;&#39;timeout&#39;&#39;.=C2=A0 Th=
is is measured<br>
as the beginning of a calendar year as per this table (suggested<br>
three years from drafting the soft fork proposal):<br>
<br>
Timeout Year=C2=A0 =C2=A0 &gt;=3D Seconds=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 Timeout Year=C2=A0 =C2=A0 &gt;=3D Seconds<br>
2018=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1514764800=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2026=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1767225600<br>
2019=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1546300800=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2027=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1798761600<br>
2020=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1577836800=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2028=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1830297600<br>
2021=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1609459200=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2029=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1861920000<br>
2022=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1640995200=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2030=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1893456000<br>
2023=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1672531200=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2031=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1924992000<br>
2024=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1704067200=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2032=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1956528000<br>
2025=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1735689600=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2033=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 1988150400<br>
<br>
If the soft fork still not &#39;&#39;locked-in&#39;&#39; and the<br>
GetMedianTimePast() of a block following a retarget period is at or<br>
past this timeout, miners should cease setting this bit.<br>
<br>
=C2=A0if (NextBlockHeight % 2016 =3D=3D 0) {<br>
=C2=A0 =C2=A0 if (BState =3D=3D defined &amp;&amp; GetMedianTimePast(nextbl=
ock) &gt;=3D BFinalYear) {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0BState =3D failed;<br>
=C2=A0 =C2=A0 }<br>
=C2=A0}<br>
<br>
After another retarget period (to allow detection of buggy miners),<br>
the bit may be reused.<br>
<br>
&#39;&#39;&#39;Warning system&#39;&#39;&#39;<br>
To support upgrade warnings, an extra &quot;unknown upgrade&quot; is tracke=
d, using the &quot;implicit bit&quot; mask =3D (block.nVersion &amp; ~expec=
tedVersion) !=3D 0. Mask will be non-zero whenever an unexpected bit is set=
 in nVersion.=C2=A0 Whenever lock-in for the unknown upgrade is detected, t=
he software should warn loudly about the upcoming soft fork.=C2=A0 It shoul=
d warn even more loudly after the next retarget period.<br>
<br>
&#39;&#39;&#39;Forks&#39;&#39;&#39;<br>
It should be noted that the states are maintained along block chain<br>
branches, but may need recomputation when a reorganization happens.<br>
<br>
=3D=3D=3DSupport for future changes=3D=3D=3D<br>
<br>
The mechanism described above is very generic, and variations are possible =
for future soft forks. Here are some ideas that can be taken into account.<=
br>
<br>
&#39;&#39;&#39;Modified thresholds&#39;&#39;&#39;<br>
The 95% threshold (based on in BIP 34) does not have to be maintained for e=
ternity, but changes should take the effect on the warning system into acco=
unt. In particular, having a lock-in threshold that is incompatible with th=
e one used for the warning system may have long-term effects, as the warnin=
g system cannot rely on a permanently detectable condition anymore.<br>
<br>
&#39;&#39;&#39;Conflicting soft forks&#39;&#39;&#39;<br>
At some point, two mutually exclusive soft forks may be proposed. The naive=
 way to deal with this is to never create software that implements both, bu=
t that is a making a bet that at least one side is guaranteed to lose. Bett=
er would be to encode &quot;soft fork X cannot be locked-in&quot; as consen=
sus rule for the conflicting soft fork - allowing software that supports bo=
th, but can never trigger conflicting changes.<br>
<br>
&#39;&#39;&#39;Multi-stage soft forks&#39;&#39;&#39;<br>
Soft forks right now are typically treated as booleans: they go from an ina=
ctive to an active state in blocks. Perhaps at some point there is demand f=
or a change that has a larger number of stages, with additional validation =
rules that get enabled one by one. The above mechanism can be adapted to su=
pport this, by interpreting a combination of bits as an integer, rather tha=
n as isolated bits. The warning system is compatible with this, as (nVersio=
n &amp; ~nExpectedVersion) will always be non-zero for increasing integers.=
<br>
<br>
=3D=3D Rationale =3D=3D<br>
<br>
The failure timeout allows eventual reuse of bits even if a soft fork was<b=
r>
never activated, so it&#39;s clear that the new use of the bit refers to a<=
br>
new BIP.=C2=A0 It&#39;s deliberately very course grained, to take into acco=
unt<br>
reasonable development and deployment delays.=C2=A0 There are unlikely to b=
e<br>
enough failed proposals to cause a bit shortage.<br>
<br>
The fallow period at the conclusion of a soft fork attempt allows some<br>
detection of buggy clients, and allows time for warnings and software<br>
upgrades for successful soft forks.<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div><br></div>

--089e01227b8aa45e80051fdf4cec--