Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 04443123C for ; Wed, 16 Sep 2015 15:53:33 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com [209.85.212.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F2829EA for ; Wed, 16 Sep 2015 15:53:28 +0000 (UTC) Received: by wicfx3 with SMTP id fx3so79840225wic.1 for ; Wed, 16 Sep 2015 08:53:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=YDgDfGag4v7WzLCLWQRVmsHl8Rp7kkupUPG+snT5YGs=; b=DDPgZ1uvr72jF4MPxnYlM+t2rvMHjzf6si2BCEVKPIFRXlmrwKSdAfRORaH1eDx9ep gobWT9X1XtIbuliT3a+gGBdiS8JC8r2WWGPjH0hey3erDsk0KskeyhkXaJy5F2aX+4cM gjYjCD3c0ETFj/MuLQfNQ9ahWC6bQyId3OJIVW0pqi+mpk8Z9FS7XEESs8m4SFoUT+Cr fc5+31VINDikI3WCap37I85n35O4Sd6q8pGHWjc1YEL3fRf5jH4uKABhGux+MQYL5aec WuJPk9Xw4AbkBipXwlJRL8sxpLOV8EDZL/RuEccbMgpO2MiNZeoouK9f6+AQz5vbJxl3 ZweA== X-Received: by 10.194.121.131 with SMTP id lk3mr22547081wjb.77.1442418807560; Wed, 16 Sep 2015 08:53:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.21.200 with HTTP; Wed, 16 Sep 2015 08:53:08 -0700 (PDT) In-Reply-To: <87mvwqb132.fsf@rustcorp.com.au> References: <87mvwqb132.fsf@rustcorp.com.au> From: Btc Drak Date: Wed, 16 Sep 2015 16:53:08 +0100 Message-ID: To: Rusty Russell Content-Type: multipart/alternative; boundary=089e01227b8aa45e80051fdf4cec X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HK_RANDOM_ENVFROM, HK_RANDOM_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , Pieter Wuille , Greg Maxwell Subject: Re: [bitcoin-dev] [BIP Proposal] Version bits with timeout and delay. X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2015 15:53:33 -0000 --089e01227b8aa45e80051fdf4cec Content-Type: text/plain; charset=UTF-8 Rusty, I think you've covered all the issues discussed now. +1 for submitting to BIPs repo to get an official number. Are you planning to write the implementation? On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi all, > > Those who've seen the original versionbits bip, this adds: > 1) Percentage checking only on retarget period boundaries. > 2) 1 retarget period between 95% and activation. > 3) A stronger suggestion for timeout value selection. > > https://gist.github.com/rustyrussell/47eb08093373f71f87de > > And pasted below, de-formatted a little. > > Thanks, > Rusty. > > BIP: ?? > Title: Version bits with timeout and delay > Author: Pieter Wuille , Peter Todd < > pete@petertodd.org>, Greg Maxwell , Rusty Russell < > rusty@rustcorp.com.au> > Status: Draft > Type: Informational Track > Created: 2015-10-04 > > ==Abstract== > > This document specifies a proposed change to the semantics of the > 'version' field in Bitcoin blocks, allowing multiple backward-compatible > changes (further called called "soft forks") being deployed in parallel. It > relies on interpreting the version field as a bit vector, where each bit > can be used to track an independent change. These are tallied each retarget > period. Once the consensus change succeeds or times out, there is a > "fallow" pause after which the bit can be reused for later changes. > > ==Motivation== > > BIP 34 introduced a mechanism for doing soft-forking changes without > predefined flag timestamp (or flag block height), instead relying on > measuring miner support indicated by a higher version number in block > headers. As it relies on comparing version numbers as integers however, it > only supports one single change being rolled out at once, requiring > coordination between proposals, and does not allow for permanent rejection: > as long as one soft fork is not fully rolled out, no future one can be > scheduled. > > In addition, BIP 34 made the integer comparison (nVersion >= 2) a > consensus rule after its 95% threshold was reached, removing 2^31 +2 values > from the set of valid version numbers (all negative numbers, as nVersion is > interpreted as a signed integer, as well as 0 and 1). This indicates > another downside this approach: every upgrade permanently restricts the set > of allowed nVersion field values. This approach was later reused in BIP 66, > which further removed nVersion = 2 as valid option. As will be shown > further, this is unnecessary. > > ==Specification== > > ===Mechanism=== > > '''Bit flags''' > We are permitting several independent soft forks to be deployed in > parallel. For each, a bit B is chosen from the set {0,1,2,...,28}, which is > not currently in use for any other ongoing soft fork. Miners signal intent > to enforce the new rules associated with the proposed soft fork by setting > bit 1B in nVersion to 1 in their blocks. > > '''High bits''' > The highest 3 bits are set to 001, so the range of actually possible > nVersion values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two > future upgrades for different mechanisms (top bits 010 and 011), while > complying to the constraints set by BIP34 and BIP66. Having more than 29 > available bits for parallel soft forks does not add anything anyway, as the > (nVersion >= 3) requirement already makes that impossible. > > '''States''' > With every softfork proposal we associate a state BState, which begins > at ''defined'', and can be ''locked-in'', ''activated'', > or ''failed''. Transitions are considered after each > retarget period. > > '''Soft Fork Support''' > Software which supports the change should begin by setting B in all blocks > mined until it is resolved. > > if (BState == defined) { > SetBInBlock(); > } > > '''Success: Lock-in Threshold''' > If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks > within a retarget period, it is considered ''locked-in''. Miners should > stop setting bit B. > > if (NextBlockHeight % 2016 == 0) { > if (BState == defined && Previous2016BlocksCountB() >= 1916) { > BState = locked-in; > BActiveHeight = NextBlockHeight + 2016; > } > } > > '''Success: Activation Delay''' > The consensus rules related to ''locked-in'' soft fork will be enforced in > the second retarget period; ie. there is a one retarget period in > which the remaining 5% can upgrade. At the that activation block and > after, the bit B may be reused for a different soft fork. > > if (BState == locked-in && NextBlockHeight == BActiveHeight) { > BState = activated; > ApplyRulesForBFromNextBlock(); > /* B can be reused, immediately */ > } > > '''Failure: Timeout''' > A soft fork proposal should include a ''timeout''. This is measured > as the beginning of a calendar year as per this table (suggested > three years from drafting the soft fork proposal): > > Timeout Year >= Seconds Timeout Year >= Seconds > 2018 1514764800 2026 1767225600 > 2019 1546300800 2027 1798761600 > 2020 1577836800 2028 1830297600 > 2021 1609459200 2029 1861920000 > 2022 1640995200 2030 1893456000 > 2023 1672531200 2031 1924992000 > 2024 1704067200 2032 1956528000 > 2025 1735689600 2033 1988150400 > > If the soft fork still not ''locked-in'' and the > GetMedianTimePast() of a block following a retarget period is at or > past this timeout, miners should cease setting this bit. > > if (NextBlockHeight % 2016 == 0) { > if (BState == defined && GetMedianTimePast(nextblock) >= BFinalYear) { > BState = failed; > } > } > > After another retarget period (to allow detection of buggy miners), > the bit may be reused. > > '''Warning system''' > To support upgrade warnings, an extra "unknown upgrade" is tracked, using > the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask > will be non-zero whenever an unexpected bit is set in nVersion. Whenever > lock-in for the unknown upgrade is detected, the software should warn > loudly about the upcoming soft fork. It should warn even more loudly after > the next retarget period. > > '''Forks''' > It should be noted that the states are maintained along block chain > branches, but may need recomputation when a reorganization happens. > > ===Support for future changes=== > > The mechanism described above is very generic, and variations are possible > for future soft forks. Here are some ideas that can be taken into account. > > '''Modified thresholds''' > The 95% threshold (based on in BIP 34) does not have to be maintained for > eternity, but changes should take the effect on the warning system into > account. In particular, having a lock-in threshold that is incompatible > with the one used for the warning system may have long-term effects, as the > warning system cannot rely on a permanently detectable condition anymore. > > '''Conflicting soft forks''' > At some point, two mutually exclusive soft forks may be proposed. The > naive way to deal with this is to never create software that implements > both, but that is a making a bet that at least one side is guaranteed to > lose. Better would be to encode "soft fork X cannot be locked-in" as > consensus rule for the conflicting soft fork - allowing software that > supports both, but can never trigger conflicting changes. > > '''Multi-stage soft forks''' > Soft forks right now are typically treated as booleans: they go from an > inactive to an active state in blocks. Perhaps at some point there is > demand for a change that has a larger number of stages, with additional > validation rules that get enabled one by one. The above mechanism can be > adapted to support this, by interpreting a combination of bits as an > integer, rather than as isolated bits. The warning system is compatible > with this, as (nVersion & ~nExpectedVersion) will always be non-zero for > increasing integers. > > == Rationale == > > The failure timeout allows eventual reuse of bits even if a soft fork was > never activated, so it's clear that the new use of the bit refers to a > new BIP. It's deliberately very course grained, to take into account > reasonable development and deployment delays. There are unlikely to be > enough failed proposals to cause a bit shortage. > > The fallow period at the conclusion of a soft fork attempt allows some > detection of buggy clients, and allows time for warnings and software > upgrades for successful soft forks. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --089e01227b8aa45e80051fdf4cec Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Rusty,=C2=A0

I think you've covered= all the issues discussed now. +1 for submitting to BIPs repo to get an off= icial number.

Are you planning to write the implem= entation?



On Sun, Sep 13, 2015 at 7:56 PM, Rusty Ru= ssell via bitcoin-dev <bitcoin-dev@lists.linuxfoundati= on.org> wrote:
Hi all,

Those who've seen the original versionbits bip, this adds:
=C2=A0 =C2=A01) Percentage checking only on retarget period boundaries.
=C2=A0 =C2=A02) 1 retarget period between 95% and activation.
=C2=A0 =C2=A03) A stronger suggestion for timeout value selection.

https://gist.github.com/rustyrussell/47eb= 08093373f71f87de

And pasted below, de-formatted a little.

Thanks,
Rusty.

=C2=A0 BIP: ??
=C2=A0 Title: Version bits with timeout and delay
=C2=A0 Author: Pieter Wuille <pieter.wuille@gmail.com>, Peter Todd <pete@petertodd.org>, Greg Maxwell <greg@xiph.org>, Rusty Russell <rusty@rustcorp.com.au>
=C2=A0 Status: Draft
=C2=A0 Type: Informational Track
=C2=A0 Created: 2015-10-04

=3D=3DAbstract=3D=3D

This document specifies a proposed change to the semantics of the 'vers= ion' field in Bitcoin blocks, allowing multiple backward-compatible cha= nges (further called called "soft forks") being deployed in paral= lel. It relies on interpreting the version field as a bit vector, where eac= h bit can be used to track an independent change. These are tallied each re= target period. Once the consensus change succeeds or times out, there is a = "fallow" pause after which the bit can be reused for later change= s.

=3D=3DMotivation=3D=3D

BIP 34 introduced a mechanism for doing soft-forking changes without predef= ined flag timestamp (or flag block height), instead relying on measuring mi= ner support indicated by a higher version number in block headers. As it re= lies on comparing version numbers as integers however, it only supports one= single change being rolled out at once, requiring coordination between pro= posals, and does not allow for permanent rejection: as long as one soft for= k is not fully rolled out, no future one can be scheduled.

In addition, BIP 34 made the integer comparison (nVersion >=3D 2) a cons= ensus rule after its 95% threshold was reached, removing 2^31 +2 values fro= m the set of valid version numbers (all negative numbers, as nVersion is in= terpreted as a signed integer, as well as 0 and 1). This indicates another = downside this approach: every upgrade permanently restricts the set of allo= wed nVersion field values. This approach was later reused in BIP 66, which = further removed nVersion =3D 2 as valid option. As will be shown further, t= his is unnecessary.

=3D=3DSpecification=3D=3D

=3D=3D=3DMechanism=3D=3D=3D

'''Bit flags'''
We are permitting several independent soft forks to be deployed in parallel= . For each, a bit B is chosen from the set {0,1,2,...,28}, which is not cur= rently in use for any other ongoing soft fork. Miners signal intent to enfo= rce the new rules associated with the proposed soft fork by setting bit 1&l= t;sup>B</sup> in nVersion to 1 in their blocks.

'''High bits'''
The highest 3 bits are set to 001, so the range of actually possible nVersi= on values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two future u= pgrades for different mechanisms (top bits 010 and 011), while complying to= the constraints set by BIP34 and BIP66. Having more than 29 available bits= for parallel soft forks does not add anything anyway, as the (nVersion >= ;=3D 3) requirement already makes that impossible.

'''States'''
With every softfork proposal we associate a state BState, which begins
at ''defined'', and can be ''locked-in'', &= #39;'activated'',
or ''failed''.=C2=A0 Transitions are considered after each<= br> retarget period.

'''Soft Fork Support'''
Software which supports the change should begin by setting B in all blocks<= br> mined until it is resolved.

=C2=A0if (BState =3D=3D defined) {
=C2=A0 =C2=A0 =C2=A0SetBInBlock();
=C2=A0}

'''Success: Lock-in Threshold'''
If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks
within a retarget period, it is considered ''locked-in''.= =C2=A0 Miners should
stop setting bit B.

=C2=A0if (NextBlockHeight % 2016 =3D=3D 0) {
=C2=A0 =C2=A0 if (BState =3D=3D defined && Previous2016BlocksCountB= () >=3D 1916) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 BState =3D locked-in;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 BActiveHeight =3D NextBlockHeight + 2016;
=C2=A0 =C2=A0 }
=C2=A0}

'''Success: Activation Delay'''
The consensus rules related to ''locked-in'' soft fork will= be enforced in
the second retarget period; ie. there is a one retarget period in
which the remaining 5% can upgrade.=C2=A0 At the that activation block and<= br> after, the bit B may be reused for a different soft fork.

=C2=A0if (BState =3D=3D locked-in && NextBlockHeight =3D=3D BActive= Height) {
=C2=A0 =C2=A0 BState =3D activated;
=C2=A0 =C2=A0 ApplyRulesForBFromNextBlock();
=C2=A0 =C2=A0 /* B can be reused, immediately */
=C2=A0}

'''Failure: Timeout'''
A soft fork proposal should include a ''timeout''.=C2=A0 Th= is is measured
as the beginning of a calendar year as per this table (suggested
three years from drafting the soft fork proposal):

Timeout Year=C2=A0 =C2=A0 >=3D Seconds=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 Timeout Year=C2=A0 =C2=A0 >=3D Seconds
2018=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1514764800=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2026=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1767225600
2019=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1546300800=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2027=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1798761600
2020=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1577836800=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2028=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1830297600
2021=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1609459200=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2029=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1861920000
2022=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1640995200=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2030=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1893456000
2023=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1672531200=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2031=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1924992000
2024=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1704067200=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2032=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1956528000
2025=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1735689600=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2033=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1988150400

If the soft fork still not ''locked-in'' and the
GetMedianTimePast() of a block following a retarget period is at or
past this timeout, miners should cease setting this bit.

=C2=A0if (NextBlockHeight % 2016 =3D=3D 0) {
=C2=A0 =C2=A0 if (BState =3D=3D defined && GetMedianTimePast(nextbl= ock) >=3D BFinalYear) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0BState =3D failed;
=C2=A0 =C2=A0 }
=C2=A0}

After another retarget period (to allow detection of buggy miners),
the bit may be reused.

'''Warning system'''
To support upgrade warnings, an extra "unknown upgrade" is tracke= d, using the "implicit bit" mask =3D (block.nVersion & ~expec= tedVersion) !=3D 0. Mask will be non-zero whenever an unexpected bit is set= in nVersion.=C2=A0 Whenever lock-in for the unknown upgrade is detected, t= he software should warn loudly about the upcoming soft fork.=C2=A0 It shoul= d warn even more loudly after the next retarget period.

'''Forks'''
It should be noted that the states are maintained along block chain
branches, but may need recomputation when a reorganization happens.

=3D=3D=3DSupport for future changes=3D=3D=3D

The mechanism described above is very generic, and variations are possible = for future soft forks. Here are some ideas that can be taken into account.<= br>
'''Modified thresholds'''
The 95% threshold (based on in BIP 34) does not have to be maintained for e= ternity, but changes should take the effect on the warning system into acco= unt. In particular, having a lock-in threshold that is incompatible with th= e one used for the warning system may have long-term effects, as the warnin= g system cannot rely on a permanently detectable condition anymore.

'''Conflicting soft forks'''
At some point, two mutually exclusive soft forks may be proposed. The naive= way to deal with this is to never create software that implements both, bu= t that is a making a bet that at least one side is guaranteed to lose. Bett= er would be to encode "soft fork X cannot be locked-in" as consen= sus rule for the conflicting soft fork - allowing software that supports bo= th, but can never trigger conflicting changes.

'''Multi-stage soft forks'''
Soft forks right now are typically treated as booleans: they go from an ina= ctive to an active state in blocks. Perhaps at some point there is demand f= or a change that has a larger number of stages, with additional validation = rules that get enabled one by one. The above mechanism can be adapted to su= pport this, by interpreting a combination of bits as an integer, rather tha= n as isolated bits. The warning system is compatible with this, as (nVersio= n & ~nExpectedVersion) will always be non-zero for increasing integers.=

=3D=3D Rationale =3D=3D

The failure timeout allows eventual reuse of bits even if a soft fork was never activated, so it's clear that the new use of the bit refers to a<= br> new BIP.=C2=A0 It's deliberately very course grained, to take into acco= unt
reasonable development and deployment delays.=C2=A0 There are unlikely to b= e
enough failed proposals to cause a bit shortage.

The fallow period at the conclusion of a soft fork attempt allows some
detection of buggy clients, and allows time for warnings and software
upgrades for successful soft forks.
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev

--089e01227b8aa45e80051fdf4cec--