Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 50F27722 for ; Tue, 10 Oct 2017 02:19:16 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f182.google.com (mail-pf0-f182.google.com [209.85.192.182]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 7BEA8422 for ; Tue, 10 Oct 2017 02:19:13 +0000 (UTC) Received: by mail-pf0-f182.google.com with SMTP id b85so4503514pfj.13 for ; Mon, 09 Oct 2017 19:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=friedenbach-org.20150623.gappssmtp.com; s=20150623; h=from:content-transfer-encoding:mime-version:date:subject:message-id :references:in-reply-to:to; bh=UrHpNpiGXYad+c8/Fg7n5ibh6E8sVm/lIk07W7Zw7L0=; b=mmKNwZlh/1oC2DUE9qQ/8lE3+LXTGiaw1PpGBfmIVl94cNuwEufHRVDqsR2tL7jG2s E5DvSE+UvqFBR9foNdulf+W6lL35n+9ayBeY8bD956w53u6Umr0ase9SaAuDJUjmqSzL R56Yv2oHjMSg60OeLvp3OoldrvR90k29KepvOszhm54rCvopPqWKxj3jWtlLIGGGNr5G a52HvtNgJxUnxq+Ew/vWYEQCJaOsGwI3v4NwbUdrFAklE3vWD9VG31nmO8ByS1Eq9imJ TkN4us+Zas+X6CXV8WeuMbq7ICIJ5b08u0x75Ht0mHh2dYvMz4UESEkI6sadzZ8bNaNr 1zZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version:date :subject:message-id:references:in-reply-to:to; bh=UrHpNpiGXYad+c8/Fg7n5ibh6E8sVm/lIk07W7Zw7L0=; b=mvLkyAkXl+JUro2FDKLS4ZQOLtOU2wzKXAE2dtZta2Ozsvt648OCTlVOc6PUcU0mnL O9ceF1XMxWq/RoAXij7xJaxEeTCR07GPYRZJGrzyRsdKYx3L/NbEO7tO2s8WVznIwExO 74MqWiroZ9nYNh8AlpgpJJYcCKEd5A0tLxvnSA9iMkBTRuyKZUfQjKfSBTdkBn5ioSS/ vie1O+Y1iWR23ieG8BfFyFlP9epx/p0eGhZCP/qFghUcz9HERQXLccIA7wmWn+A0PEIk TmISPu2UMAlkubF8TPlvOIh/RYci53GWUpYrujv92OuJAFPsw4FKiDXqK9MPgVWfvmuo mQSA== X-Gm-Message-State: AMCzsaVM8jxbCg+TGFsqIlgdS/7XwGxUMNPxqNpU0hcLu5S+UC8xd8GN Q2DsLR4+JdbgXg4Xwo83dM837FEZtFA= X-Google-Smtp-Source: AOwi7QAwm/rEKxFD4wrFp6UAc6R/bLaC9/oroSPTgNjKJ8lIGM6No8wmgP0SQwdzpoiRhUReGFPbmg== X-Received: by 10.98.213.194 with SMTP id d185mr11951623pfg.107.1507601952951; Mon, 09 Oct 2017 19:19:12 -0700 (PDT) Received: from ?IPv6:2601:646:8080:1291:649b:a2e9:4254:d68f? ([2601:646:8080:1291:649b:a2e9:4254:d68f]) by smtp.gmail.com with ESMTPSA id q69sm6294526pfg.127.2017.10.09.19.19.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Oct 2017 19:19:12 -0700 (PDT) From: Mark Friedenbach Content-Type: multipart/alternative; boundary=Apple-Mail-0858C34E-84BB-4B7C-B1DC-EDBB070403C4 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Mon, 9 Oct 2017 19:19:11 -0700 Message-Id: References: <1213518291.4328204.1507589852818.ref@mail.yahoo.com> <1213518291.4328204.1507589852818@mail.yahoo.com> In-Reply-To: <1213518291.4328204.1507589852818@mail.yahoo.com> To: Scott Roberts , Bitcoin Protocol Discussion X-Mailer: iPhone Mail (15A402) X-Spam-Status: No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWit2x fork? (reformatted text) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2017 02:19:16 -0000 --Apple-Mail-0858C34E-84BB-4B7C-B1DC-EDBB070403C4 Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: quoted-printable The problem of fast acting but non vulnerable difficulty adjustment algorith= ms is interesting. I would certainly like to see this space further explored= , and even have some ideas myself. However without commenting on the technical merits of this specific proposal= , I think it must be said upfront that the stated goal is not good. The larg= est technical concern (ignoring governance) over B2X is that it is a rushed,= poorly reviewed hard fork. Hard forks should not be rushed, and they should= receive more than the usual level of expert and community review. I=A1=AFm that light, doing an even more rushed hard fork on an even newer id= ea with even less review would be hypocritical at best. I would suggest refr= aming as a hardfork wishlist research problem for the next properly planned h= ard fork, if one occurs. You might also find the hardfork research group a m= ore accommodating venue for this discussion: https://bitcoinhardforkresearch.github.io/ > On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev wrote: >=20 > Sorry, my previous email did not have the plain text I intended. >=20 > Background:=20 >=20 > The bitcoin difficulty algorithm does not seem to be a good one. If there=20= > is a fork due to miners seeking maximum profit without due regard to=20 > security, users, and nodes, the "better" coin could end up being the=20 > minority chain. If 90% of hashrate is really going to at least initially g= o=20 > towards using SegWit2x, BTC would face 10x delays in confirmations=20 > until the next difficulty adjustment, negatively affecting its price relat= ive=20 > to BTC1, causing further delays from even more miner abandonment=20 > (until the next adjustment). The 10% miners remaining on BTC do not=20 > inevitably lose by staying to endure 10x delays because they have 10x=20 > less competition, and the same situation applies to BTC1 miners. If the=20= > prices are the same and stable, all seems well for everyone, other things=20= > aside. But if the BTC price does not fall to reflect the decreased hashrat= e,=20 > he situation seems to be a big problem for both coins: BTC1 miners will=20= > jump back to BTC when the difficulty adjustment occurs, initiating a=20 > potentially never-ending oscillation between the two coins, potentially=20= > worse than what BCH is experiencing. They will not issue coins too fast=20= > like BCH because that is a side effect of the asymmetry in BCH's rise and=20= > fall algorithm.=20 >=20 > Solution:=20 >=20 > Hard fork to implement a new difficulty algorithm that uses a simple rolli= ng=20 > average with a much smaller window. Many small coins have done this as=20= > a way to stop big miners from coming on and then suddenly leaving, leaving= =20 > constant miners stuck with a high difficulty for the rest of a (long) aver= aging=20 > window. Even better, adjust the reward based on recent solvetimes to=20 > motivate more mining (or less) if the solvetimes are too slow (or too fast= ).=20 > This will keep keep coin issuance rate perfectly on schedule with real tim= e.=20 >=20 > I recommend the following for Bitcoin, as fast, simple, and better than an= y=20 > other difficulty algorithm I'm aware of. This is the result of a lot of w= ork the=20 > past year.=20 >=20 > =3D=3D=3D Begin difficulty algorithm =3D=3D=3D=20 > # Zawy v6 difficulty algorithm (modified for bitcoin)=20 > # Unmodified Zawy v6 for alt coins:=20 > # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.htm= l=20 > # All my failed attempts at something better:=20 > # https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800b= e910ce345668a=20 > #=20 > # Keep negative solvetimes to correct bad timestamps.=20 > # Do not be tempted to use:=20 > # next_D =3D sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];=20 > # ST=3D Solvetime, TS =3D timestamp=20 >=20 > # set constants until next hard fork:=20 >=20 > T=3D600; # coin's TargetSolvetime=20 > N=3D30; # Averaging window. Smoother than N=3D15, faster response than N=3D= 60.=20 > X=3D5;=20 > limit =3D X^(2/N); # limit rise and fall in case of timestamp manipulation= =20 > adjust =3D 1/(1+0.67/N); # keeps avg solvetime on track=20 >=20 > # begin difficulty algorithm=20 >=20 > avg_ST=3D0; avg_D=3D0;=20 > for ( i=3Dheight; i > height-N; i--) { # go through N most recent block= s=20 > avg_ST +=3D (TS[i] - TS[i-1]) / N;=20 > avg_D +=3D D[i]/N;=20 > }=20 > avg_ST =3D T*limit if avg_ST > T*limit;=20 > avg_ST =3D T/limit if avg_ST < T/limit;=20 >=20 > next_D =3D avg_D * T / avg_ST * adjust;=20 >=20 > # Tim Olsen suggested changing reward to protect against hash attacks.=20 > # Karbowanek coin suggested something similar.=20 > # I could not find anything better than the simplest idea below.=20 > # It was a great surprise that coin issuance rate came out perfect.=20 > # BaseReward =3D coins per block=20 >=20 > next_reward =3D BaseReward * avg_ST / T;=20 >=20 > =3D=3D=3D=3D=3D=3D=3D end algo =3D=3D=3D=3D=20 >=20 > Due to the limit and keeping negative solvetimes in a true average,=20 > timestamp errors resulting in negative solvetimes are corrected in the nex= t=20 > block. Otherwise, one would need to do like Zcash and cause a 5-block=20 > delay in the response by resorting to the median of past 11 blocks (MPT)=20= > as the most recent timestamp, offsetting the timestamps from their=20 > corresponding difficulties by 5 blocks. (it does not cause an averaging=20= > problem, but it does cause a 5-block delay in the response.) > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --Apple-Mail-0858C34E-84BB-4B7C-B1DC-EDBB070403C4 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable The problem of fast acting but non vulnerab= le difficulty adjustment algorithms is interesting. I would certainly like t= o see this space further explored, and even have some ideas myself.

=
However without commenting on the technical merits of this specif= ic proposal, I think it must be said upfront that the stated goal is not goo= d. The largest technical concern (ignoring governance) over B2X is that it i= s a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and t= hey should receive more than the usual level of expert and community review.=

I=E2=80=99m that light, doing an even more rushed h= ard fork on an even newer idea with even less review would be hypocritical a= t best. I would suggest reframing as a hardfork wishlist research problem fo= r the next properly planned hard fork, if one occurs. You might also find th= e hardfork research group a more accommodating venue for this discussion:


On Oct 9= , 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org= > wrote:

Sorry, my prev= ious email did not have the plain text I intended.
Background:

The bitcoin difficulty= algorithm does not seem to be a good one. If there
is a fo= rk due to miners seeking maximum profit without due regard to
security, users, and nodes, the "better" coin could end up being the
minority chain. If 90% of hashrate is really going to at least i= nitially go
towards using SegWit2x, BTC would face 10x dela= ys in confirmations
until the next difficulty adjustment, n= egatively affecting its price relative
to BTC1, causing fur= ther delays from even more miner abandonment
(until the nex= t adjustment). The 10% miners remaining on BTC do not
inevi= tably lose by staying to endure 10x delays because they have 10x
= less competition, and the same situation applies to BTC1 miners. If th= e
prices are the same and stable, all seems well for everyo= ne, other things
aside. But if the BTC price does not fall t= o reflect the decreased hashrate,
he situation seems to be a= big problem for both coins: BTC1 miners will
jump back to B= TC when the difficulty adjustment occurs, initiating a
pote= ntially never-ending oscillation between the two coins, potentially <= br>worse than what BCH is experiencing.  They will not issue coin= s too fast
like BCH because that is a side effect of the as= ymmetry in BCH's rise and
fall algorithm.
=
Solution:

Hard fork to im= plement a new difficulty algorithm that uses a simple rolling
average with a much smaller window.  Many small coins have done this= as
a way to stop big miners from coming on and then sudden= ly leaving, leaving
constant miners stuck with a high diffi= culty for the rest of a (long) averaging
window.  Even= better, adjust the reward based on recent solvetimes to
mo= tivate more mining (or less) if the solvetimes are too slow (or too fast). <= /span>
This will keep keep coin issuance rate perfectly on schedule= with real time.

I recommend the following= for Bitcoin, as fast, simple, and better than any
other di= fficulty algorithm I'm aware of.  This is the result of a lot of work t= he
past year.

=3D=3D=3D B= egin difficulty algorithm =3D=3D=3D
# Zawy v6 difficulty al= gorithm (modified for bitcoin)
# Unmodified Zawy v6 for alt= coins:
# http://zawy1.blogspot.com/2017/07/best-= difficulty-algorithm-zawy-v1b.html
# All my failed atte= mpts at something better:
# https://g= ithub.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a=
#
# Keep negative solvetimes to corre= ct bad timestamps.
# Do not be tempted to use:
<= span># next_D =3D sum(last N Ds) * T / [max(last N TSs) - min(last N TSs]; <= /span>
# ST=3D Solvetime, TS =3D timestamp
=
# set constants until next hard fork:

= T=3D600; # coin's TargetSolvetime
N=3D30; # Averaging= window. Smoother than N=3D15, faster response than N=3D60.
X=3D5;
limit =3D X^(2/N); # limit rise and fall in case of= timestamp manipulation
adjust =3D 1/(1+0.67/N);  # ke= eps avg solvetime on track

# begin difficu= lty algorithm

avg_ST=3D0; avg_D=3D0;
for ( i=3Dheight;  i > height-N;  i--) {  # go= through N most recent blocks
avg_ST +=3D (TS[i] - TS[i-1])= / N;
avg_D +=3D D[i]/N;
}
avg_ST =3D T*limit if avg_ST > T*limit;
avg_ST =3D T/l= imit if avg_ST < T/limit;

next_D =3D av= g_D * T / avg_ST * adjust;

# Tim Olsen sug= gested changing reward to protect against hash attacks.
# K= arbowanek coin suggested something similar.
# I could not f= ind anything better than the simplest idea below.
# It was a= great surprise that coin issuance rate came out perfect.
#= BaseReward =3D coins per block

next_rewar= d =3D BaseReward * avg_ST / T;

=3D=3D=3D=3D= =3D=3D=3D end algo =3D=3D=3D=3D

Due to the= limit and keeping negative solvetimes in a true average,
t= imestamp errors resulting in negative solvetimes are corrected in the next <= /span>
block. Otherwise, one would need to do like Zcash and cause a= 5-block
delay in the response by resorting to the median o= f past 11 blocks (MPT)
as the most recent timestamp, offset= ting the timestamps from their
corresponding difficulties b= y 5 blocks. (it does not cause an averaging
problem, but it= does cause a 5-block delay in the response.)
______________= _________________________________
bitcoin-dev mailing list
bitc= oin-dev@lists.linuxfoundation.org
https://lists.linuxfoun= dation.org/mailman/listinfo/bitcoin-dev
= --Apple-Mail-0858C34E-84BB-4B7C-B1DC-EDBB070403C4--