path: root/9a/9f385f7be5e5e2e70bf46fe23f5547d36f8b3d

Return-Path: <jlrubin@mit.edu>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 304E8C000D
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue,  7 Sep 2021 16:44:35 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp2.osuosl.org (Postfix) with ESMTP id 11E5D4018D
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue,  7 Sep 2021 16:44:35 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -4.199
X-Spam-Level: 
X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5
 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from smtp2.osuosl.org ([127.0.0.1])
 by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 9c6j_UwVl2Uu
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue,  7 Sep 2021 16:44:33 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0
Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])
 by smtp2.osuosl.org (Postfix) with ESMTPS id 1EA4140150
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue,  7 Sep 2021 16:44:32 +0000 (UTC)
Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com
 [209.85.208.179]) (authenticated bits=0)
 (User authenticated as jlrubin@ATHENA.MIT.EDU)
 by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 187GiUPa028162
 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT)
 for <bitcoin-dev@lists.linuxfoundation.org>; Tue, 7 Sep 2021 12:44:31 -0400
Received: by mail-lj1-f179.google.com with SMTP id h1so17553946ljl.9
 for <bitcoin-dev@lists.linuxfoundation.org>;
 Tue, 07 Sep 2021 09:44:31 -0700 (PDT)
X-Gm-Message-State: AOAM530NP6EeCzckERoXljJ7q6G39wtML/ecT0wrUfj8OkJI4a37jk3H
 F9Vg4FiiwCMEmfjVO5+Kq4epg3iSHTkacIdst7k=
X-Google-Smtp-Source: ABdhPJx+I2IV+vqTdVKi7apFMG3ufUMIh6G9fgNoq07VZkV0jpFWmrkJF908WkxHGVOBlD/SLA0416zFdJMxwk3kZ8s=
X-Received: by 2002:a2e:9049:: with SMTP id n9mr15552078ljg.425.1631033069773; 
 Tue, 07 Sep 2021 09:44:29 -0700 (PDT)
MIME-Version: 1.0
References: <83272afb-ed87-15b6-e02c-16bb1102beb4@gmail.com>
In-Reply-To: <83272afb-ed87-15b6-e02c-16bb1102beb4@gmail.com>
From: Jeremy <jlrubin@mit.edu>
Date: Tue, 7 Sep 2021 09:44:17 -0700
X-Gmail-Original-Message-ID: <CAD5xwhh7j5maMOs6TGHAAKz2_HqU0RYv9C1F_wPOMFrZftPdAQ@mail.gmail.com>
Message-ID: <CAD5xwhh7j5maMOs6TGHAAKz2_HqU0RYv9C1F_wPOMFrZftPdAQ@mail.gmail.com>
To: 0xB10C <0xb10c@gmail.com>,
 Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="000000000000bdab0a05cb6a7ac4"
Subject: Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on
 approach and parameters
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, 
 <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Sep 2021 16:44:35 -0000

--000000000000bdab0a05cb6a7ac4
Content-Type: text/plain; charset="UTF-8"

If you make the to be reorged flag 2 bits, 1 bit can mark final block and
the other can mark to be reorged.

That way the nodes opting into reorg can see the reorg and ignore the final
blocks (until a certain time? Or until it's via a reorg?), and the nodes
wanting not to see reorgs get continuous service without disruption

On Tue, Sep 7, 2021, 9:12 AM 0xB10C via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hello,
>
> tl;dr: We want to make reorgs on SigNet a reality and are looking for
> feedback on approach and parameters.
>
> One of the ideas for SigNet is the possibility for it to be reliably
> unreliable, for example, planned chain reorganizations. These have not
> been implemented yet.
>
> My summerofbitcoin.org mentee Nikhil Bartwal and I have been looking at
> implementing support for reorgs on SigNet. We are looking for feedback
> on which approach and parameters to use. Please consider answering the
> questions below if you or your company is interested in chain
> reorganizations on SigNet.
>
> With feedback from AJ and Kalle Alm (thanks again!), we came up with two
> scenarios that could be implemented in the current SigNet miner script
> [0]. Both would trigger automatically in a fixed block interval.
> Scenario 1 simulates a race scenario where two chains compete for D
> blocks. Scenario 2 simulates a chain rollback where the top D blocks get
> replaced by a chain that outgrows the earlier branch.
>
> AJ proposed to allow SigNet users to opt-out of reorgs in case they
> explicitly want to remain unaffected. This can be done by setting a
> to-be-reorged version bit flag on the blocks that won't end up in the
> most work chain. Node operators could choose not to accept to-be-reorged
> SigNet blocks with this flag set via a configuration argument.
>
> The reorg-interval X very much depends on the user's needs. One could
> argue that there should be, for example, three reorgs per day, each 48
> blocks apart. Such a short reorg interval allows developers in all time
> zones to be awake during one or two reorgs per day. Developers don't
> need to wait for, for example, a week until they can test their reorgs
> next. However, too frequent reorgs could hinder other SigNet users.
>
> We propose that the reorg depth D is deterministically random between a
> minimum and a maximum based on, e.g., the block hash or the nonce of the
> last block before the reorg. Compared to a local randint() based
> implementation, this allows reorg-handling tests and external tools to
> calculate the expected reorg depth.
>
> # Scenario 1: Race between two chains
>
> For this scenario, at least two nodes and miner scripts need to be
> running. An always-miner A continuously produces blocks and rejects
> blocks with the to-be-reorged version bit flag set. And a race-miner R
> that only mines D blocks at the start of each interval and then waits X
> blocks. A and R both have the same hash rate. Assuming both are well
> connected to the network, it's random which miner will first mine and
> propagate a block. In the end, the A miner chain will always win the race.
>
> # Scenario 2: Chain rollback
>
> This scenario only requires one miner and Bitcoin Core node but also
> works in a multiminer setup. The miners mine D blocks with the
> to-be-reorged version bit flag set at the start of the interval. After
> allowing the block at height X+D to propagate, they invalidate the block
> at height X+1 and start mining on block X again. This time without
> setting the to-be-reorged version bit flag. Non-miner nodes will reorg
> to the new tip at height X+D+1, and the first-seen branch stalls.
>
> # Questions
>
>     1. How do you currently test your applications reorg handling? Do
>        the two discussed scenarios (race and chain rollback) cover your
>        needs? Are we missing something you'd find helpful?
>
>     2. How often should reorgs happen on the default SigNet? Should
>        there be multiple reorgs a day (e.g., every 48 or 72 blocks
>        assuming 144 blocks per day) as your engineers need to be awake?
>        Do you favor less frequent reorgs (once per week or month)? Why?
>
>     3. How deep should the reorgs be on average? Do you want to test
>        deeper reorgs (10+ blocks) too?
>
>
> # Next Steps
>
> We will likely implement Scenario 1, the race between two chains, first.
> We'll set up a public test-SigNet along with a faucet, block explorer,
> and a block tree visualization. If there is interest in the second
> approach, chain rollbacks can be implemented too. Future work will add
> the possibility to include conflicting transactions in the two branches.
> After enough testing, the default SigNet can start to do periodical
> reorgs, too.
>
> Thanks,
> 0xB10C
>
> [0]: https://github.com/bitcoin/bitcoin/blob/master/contrib/signet/miner
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--000000000000bdab0a05cb6a7ac4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">If you make the to be reorged flag 2 bits, 1 bit can mark=
 final block and the other can mark to be reorged.<div dir=3D"auto"><br></d=
iv><div dir=3D"auto">That way the nodes opting into reorg can see the reorg=
 and ignore the final blocks (until a certain time? Or until it&#39;s via a=
 reorg?), and the nodes wanting not to see reorgs get continuous service wi=
thout disruption</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr"=
 class=3D"gmail_attr">On Tue, Sep 7, 2021, 9:12 AM 0xB10C via bitcoin-dev &=
lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lis=
ts.linuxfoundation.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e=
x">Hello,<br>
<br>
tl;dr: We want to make reorgs on SigNet a reality and are looking for<br>
feedback on approach and parameters.<br>
<br>
One of the ideas for SigNet is the possibility for it to be reliably<br>
unreliable, for example, planned chain reorganizations. These have not<br>
been implemented yet.<br>
<br>
My <a href=3D"http://summerofbitcoin.org" rel=3D"noreferrer noreferrer" tar=
get=3D"_blank">summerofbitcoin.org</a> mentee Nikhil Bartwal and I have bee=
n looking at<br>
implementing support for reorgs on SigNet. We are looking for feedback<br>
on which approach and parameters to use. Please consider answering the<br>
questions below if you or your company is interested in chain<br>
reorganizations on SigNet.<br>
<br>
With feedback from AJ and Kalle Alm (thanks again!), we came up with two<br=
>
scenarios that could be implemented in the current SigNet miner script<br>
[0]. Both would trigger automatically in a fixed block interval.<br>
Scenario 1 simulates a race scenario where two chains compete for D<br>
blocks. Scenario 2 simulates a chain rollback where the top D blocks get<br=
>
replaced by a chain that outgrows the earlier branch.<br>
<br>
AJ proposed to allow SigNet users to opt-out of reorgs in case they<br>
explicitly want to remain unaffected. This can be done by setting a<br>
to-be-reorged version bit flag on the blocks that won&#39;t end up in the<b=
r>
most work chain. Node operators could choose not to accept to-be-reorged<br=
>
SigNet blocks with this flag set via a configuration argument.<br>
<br>
The reorg-interval X very much depends on the user&#39;s needs. One could<b=
r>
argue that there should be, for example, three reorgs per day, each 48<br>
blocks apart. Such a short reorg interval allows developers in all time<br>
zones to be awake during one or two reorgs per day. Developers don&#39;t<br=
>
need to wait for, for example, a week until they can test their reorgs<br>
next. However, too frequent reorgs could hinder other SigNet users.<br>
<br>
We propose that the reorg depth D is deterministically random between a<br>
minimum and a maximum based on, e.g., the block hash or the nonce of the<br=
>
last block before the reorg. Compared to a local randint() based<br>
implementation, this allows reorg-handling tests and external tools to<br>
calculate the expected reorg depth.<br>
<br>
# Scenario 1: Race between two chains<br>
<br>
For this scenario, at least two nodes and miner scripts need to be<br>
running. An always-miner A continuously produces blocks and rejects<br>
blocks with the to-be-reorged version bit flag set. And a race-miner R<br>
that only mines D blocks at the start of each interval and then waits X<br>
blocks. A and R both have the same hash rate. Assuming both are well<br>
connected to the network, it&#39;s random which miner will first mine and<b=
r>
propagate a block. In the end, the A miner chain will always win the race.<=
br>
<br>
# Scenario 2: Chain rollback<br>
<br>
This scenario only requires one miner and Bitcoin Core node but also<br>
works in a multiminer setup. The miners mine D blocks with the<br>
to-be-reorged version bit flag set at the start of the interval. After<br>
allowing the block at height X+D to propagate, they invalidate the block<br=
>
at height X+1 and start mining on block X again. This time without<br>
setting the to-be-reorged version bit flag. Non-miner nodes will reorg<br>
to the new tip at height X+D+1, and the first-seen branch stalls.<br>
<br>
# Questions<br>
<br>
=C2=A0=C2=A0=C2=A0 1. How do you currently test your applications reorg han=
dling? Do<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0the two discussed scenarios (race and chain roll=
back) cover your<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0needs? Are we missing something you&#39;d find h=
elpful?<br>
<br>
=C2=A0=C2=A0=C2=A0 2. How often should reorgs happen on the default SigNet?=
 Should<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0there be multiple reorgs a day (e.g., every 48 o=
r 72 blocks<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0assuming 144 blocks per day) as your engineers n=
eed to be awake?<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0Do you favor less frequent reorgs (once per week=
 or month)? Why?<br>
<br>
=C2=A0 =C2=A0 3. How deep should the reorgs be on average? Do you want to t=
est<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0deeper reorgs (10+ blocks) too?<br>
<br>
<br>
# Next Steps<br>
<br>
We will likely implement Scenario 1, the race between two chains, first.<br=
>
We&#39;ll set up a public test-SigNet along with a faucet, block explorer,<=
br>
and a block tree visualization. If there is interest in the second<br>
approach, chain rollbacks can be implemented too. Future work will add<br>
the possibility to include conflicting transactions in the two branches.<br=
>
After enough testing, the default SigNet can start to do periodical<br>
reorgs, too.<br>
<br>
Thanks,<br>
0xB10C<br>
<br>
[0]: <a href=3D"https://github.com/bitcoin/bitcoin/blob/master/contrib/sign=
et/miner" rel=3D"noreferrer noreferrer" target=3D"_blank">https://github.co=
m/bitcoin/bitcoin/blob/master/contrib/signet/miner</a> <br>
<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank" =
rel=3D"noreferrer">bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer noreferrer" target=3D"_blank">https://lists.linuxfoundati=
on.org/mailman/listinfo/bitcoin-dev</a><br>
</blockquote></div>

--000000000000bdab0a05cb6a7ac4--