path: root/df/b4258976566e09198336602f1ae2387efe4246

Return-Path: <jl2012@xbt.hk>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id 86243723
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Sat, 14 Jan 2017 21:15:02 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from sender163-mail.zoho.com (sender163-mail.zoho.com
	[74.201.84.163])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6BC3D138
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Sat, 14 Jan 2017 21:15:00 +0000 (UTC)
Received: from pn-206-59.itsc.cuhk.edu.hk (pn-206-59.itsc.cuhk.edu.hk
	[137.189.206.59]) by mx.zohomail.com
	with SMTPS id 1484428499723252.64572643518;
	Sat, 14 Jan 2017 13:14:59 -0800 (PST)
From: Johnson Lau <jl2012@xbt.hk>
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812"
Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Date: Sun, 15 Jan 2017 05:14:55 +0800
References: <FB8593E6-3CD7-46D5-8FC8-A73A0EF1AE9A@xbt.hk>
To: bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
In-Reply-To: <FB8593E6-3CD7-46D5-8FC8-A73A0EF1AE9A@xbt.hk>
Message-Id: <5CDE542F-204F-4988-838F-F438D30C7D99@xbt.hk>
X-Mailer: Apple Mail (2.3259)
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE,
	MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] Forcenet: an experimental network with a new
 header format
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jan 2017 21:15:02 -0000


--Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

I created a second version of forcenet with more experimental features =
and stopped my forcenet1 node.

1. It has a new header format: Height (4), BIP9 signalling field (4), =
hardfork signalling field (2), Hash TMR (32), Hash WMR (32), Merkle sum =
root (32), number of tx (4), prev hash (32), timestamp (4), nBits (4), =
nonce1 (4), nonce2 (4), nonce3 (compactSize + variable), merkle branches =
leading to header C (compactSize + 32 bit hashes)

2. Anti-tx-replay. If, after masking the highest byte, the tx nVersion =
is >=3D3, the sighash for both segwit and non-segwit outputs is =
calculated with BIP143, except 0x2000000 is added to the nHashType. Such =
signatures are invalid for legacy nodes. But since they are non-std due =
the nVersion, they won=E2=80=99t be relayed nor validated by legacy =
nodes. This also removes the O(n^2) sighash problem when spending =
non-segwit outputs. (anti-replay is a long story and I will discuss in a =
separate post/BIP)

3. Block sighashlimit =
(https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki =
<https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki>). =
Due to point 2, SigHashSize is counted only for legacy non-segwit inputs =
(with masked tx nVersion < 3). We have to support legacy signature to =
make sure time-locked txs made before the hard fork are still valid.

4. A totally new way to define tx weight. Tx weight is the maximum of =
the following metrics:
a. SigHashSize (see the bip in point 3)
b. Witness serialised size * 2 * 90
c. Adjusted size * 90. Adjusted size =3D tx weight (BIP141) + (number of =
non-OP_RETURN outputs - number of inputs) * 41 * 4
d. nSigOps * 50 * 90. All SigOps are equal (no witness scaling). For =
non-segwit txs, the sigops in output scriptPubKey are not counted, while =
the sigops in input scriptPubKey are counted.

90 is the scaling factor for SigHashSize, to maintain the 1:90 ratio =
(see the BIP in point 3)
50 is the scaling factor for nSigOps, maintaining the 1:50 ratio in =
BIP141

Rationale for adjusted size: 4 is witness scaling factor. 41 is the =
minimum size for an input (32 hash + 4 index + 4 nSequence + 1 =
scriptSig). This requires people to pre-pay majority of the fee of =
spending an UTXO. It makes creation of UTXO more expensive, while =
spending of UTXO cheaper, creates a strong incentive to limit the growth =
of UTXO set.

Rationale for taking the maximum of different metrics: this indirectly =
set an upper block resources for _every_ metrics, while making the tx =
fee estimation a linear function. Currently, there are 2 block resources =
limits: block weight and nSigOp cost (BIP141). However, since users do =
not know what the other txs are included in the next block, it is =
difficult to determine whether tx weight of nSigOp cost is a more =
important factor in determining the tx fee. (This is not a real problem =
now, because weight is more important in most cases). With an unified =
definition of tx weight, the fee estimation becomes a linear problem.

Translating to new metric, the current BIP141 limit is 360,000,000. This =
is equivalent to 360MB of sighashing, 2MB of serialised size, 4MB of =
adjusted size, or 80000 nSigOp.

Any new block-level limit metrics could be added to tx weight using soft =
forks.

5. Smooth halving: the reward of the last 2016 blocks in a halving cycle =
will be reduced by 25%, which is contributed to the first 2016 blocks of =
the new halving cycle. (different parameters for forcenet) This makes a =
more graceful transition but we will lose some fun around halving.

6. A new coinbase tx format. BIP34 is removed. Coinbase tx may have more =
than 1 input. The prevout hash of first input must be the hash of =
previous block, and index must be 0xffffffff. The other inputs (if any) =
must come from UTXOs with valid signatures. Spending of previous =
coinbase outputs in a coinbase tx is exempted from the 100 block =
maturity requirement. Therefore, miners of an earlier block may pay =
other miners to convince them to confirm their blocks.

7. Merkle sum tree: it allows generating of fraud-proof for fee and =
weight. A special softfork (bit 15) is defined. When this softfork is =
activated, the full node will not validate the sum tree. This is needed =
because when the definition of tx weight is changed through a softfork =
(e.g. a new script version introducing new sigop), olds nodes won=E2=80=99=
t know the new rules and will find the sum tree invalid. Disabling the =
sum tree validation won=E2=80=99t degrade the security of a full node by =
more than an usual softfork, because the full node would still validate =
all other known rules.

However, it is still not possible to create fraud proof for spending of =
non-existing UTXO. This requires commitment of the block height of =
inputs, and the tx index in the block. I=E2=80=99m not quire sure how =
this could be implemented because a re-org may change such info (I think =
validation is easy but mining is more tricky)

How to join: codes at https://github.com/jl2012/bitcoin/tree/forcenet2 =
<https://github.com/jl2012/bitcoin/tree/forcenet2> , start with =
"bitcoind =E2=80=94forcenet" .
Connection: I=E2=80=99m running a node at 8333.info <http://8333.info/> =
with default port (39901)
Mining: there is only basic internal mining support. To use the internal =
miner, writeup a shell script to repeatedly call =E2=80=9Cbitcoin-cli =
=E2=80=94forcenet generate 1=E2=80=9D

jl2012=

--Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">I created a second version of forcenet with =
more experimental features and stopped my forcenet1 node.</div><div =
class=3D""><br class=3D""></div><div class=3D"">1. It has a new header =
format: Height (4), BIP9 signalling field (4), hardfork signalling field =
(2), Hash TMR (32), Hash WMR (32), Merkle sum root (32), number of tx =
(4), prev hash (32), timestamp (4), nBits (4), nonce1 (4), nonce2 (4), =
nonce3 (compactSize + variable), merkle branches leading to header C =
(compactSize + 32 bit hashes)</div><div class=3D""><br =
class=3D""></div><div class=3D"">2. Anti-tx-replay. If, after masking =
the highest byte, the tx nVersion is &gt;=3D3, the sighash for both =
segwit and non-segwit outputs is calculated with BIP143, except =
0x2000000 is added to the nHashType. Such signatures are invalid for =
legacy nodes. But since they are non-std due the nVersion, they won=E2=80=99=
t be relayed nor validated by legacy nodes. This also removes the O(n^2) =
sighash problem when spending non-segwit outputs. (anti-replay is a long =
story and I will discuss in a separate post/BIP)</div><div class=3D""><br =
class=3D""></div><div class=3D"">3. Block sighashlimit (<a =
href=3D"https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki"=
 =
class=3D"">https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawi=
ki</a>). Due to point 2, SigHashSize is counted only for legacy =
non-segwit inputs (with masked tx nVersion &lt; 3). We have to support =
legacy signature to make sure time-locked txs made before the hard fork =
are still valid.</div><div class=3D""><br class=3D""></div><div =
class=3D"">4. A totally new way to define tx weight. Tx weight is the =
maximum of the following metrics:</div><div class=3D"">a. SigHashSize =
(see the bip in point 3)</div><div class=3D"">b. Witness serialised size =
* 2 * 90</div><div class=3D"">c. Adjusted size * 90. Adjusted size =3D =
tx weight (BIP141) + (number of non-OP_RETURN outputs - number of =
inputs) * 41 * 4</div><div class=3D"">d. nSigOps * 50 * 90. All SigOps =
are equal (no witness scaling). For non-segwit txs, the sigops in output =
scriptPubKey are not counted, while the sigops in input scriptPubKey are =
counted.</div><div class=3D""><br class=3D""></div><div class=3D"">90 is =
the scaling factor for SigHashSize, to maintain the 1:90 ratio (see the =
BIP in point 3)</div><div class=3D"">50 is the scaling factor for =
nSigOps, maintaining the 1:50 ratio in BIP141</div><div class=3D""><br =
class=3D""></div><div class=3D"">Rationale for adjusted size: 4 is =
witness scaling factor. 41 is the minimum size for an input (32 hash + 4 =
index + 4 nSequence + 1 scriptSig). This requires people to pre-pay =
majority of the fee of spending an UTXO. It makes creation of UTXO more =
expensive, while spending of UTXO cheaper, creates a strong incentive to =
limit the growth of UTXO set.</div><div class=3D""><br =
class=3D""></div><div class=3D"">Rationale for taking the maximum of =
different metrics: this indirectly set an upper block resources for =
_every_ metrics, while making the tx fee estimation a linear function. =
Currently, there are 2 block resources limits: block weight and nSigOp =
cost (BIP141). However, since users do not know what the other txs are =
included in the next block, it is difficult to determine whether tx =
weight of nSigOp cost is a more important factor in determining the tx =
fee. (This is not a real problem now, because weight is more important =
in most cases). With an unified definition of tx weight, the fee =
estimation becomes a linear problem.</div><div class=3D""><br =
class=3D""></div><div class=3D"">Translating to new metric, the current =
BIP141 limit is 360,000,000. This is equivalent to 360MB of sighashing, =
2MB of serialised size, 4MB of adjusted size, or 80000 nSigOp.</div><div =
class=3D""><br class=3D""></div><div class=3D"">Any new block-level =
limit metrics could be added to tx weight using soft forks.</div><div =
class=3D""><br class=3D""></div><div class=3D"">5. Smooth halving: the =
reward of the last 2016 blocks in a halving cycle will be reduced by =
25%, which is contributed to the first 2016 blocks of the new halving =
cycle. (different parameters for forcenet) This makes a more graceful =
transition but we will lose some fun around halving.</div><div =
class=3D""><br class=3D""></div><div class=3D"">6. A new coinbase tx =
format. BIP34 is removed. Coinbase tx may have more than 1 input. The =
prevout hash of first input must be the hash of previous block, and =
index must be 0xffffffff. The other inputs (if any) must come from UTXOs =
with valid signatures. Spending of previous coinbase outputs in a =
coinbase tx is exempted from the 100 block maturity requirement. =
Therefore, miners of an earlier block may pay other miners to convince =
them to confirm their blocks.</div><div class=3D""><br =
class=3D""></div><div class=3D"">7. Merkle sum tree: it allows =
generating of fraud-proof for fee and weight. A special softfork (bit =
15) is defined. When this softfork is activated, the full node will not =
validate the sum tree. This is needed because when the definition of tx =
weight is changed through a softfork (e.g. a new script version =
introducing new sigop), olds nodes won=E2=80=99t know the new rules and =
will find the sum tree invalid. Disabling the sum tree validation =
won=E2=80=99t degrade the security of a full node by more than an usual =
softfork, because the full node would still validate all other known =
rules.</div><div class=3D""><br class=3D""></div><div class=3D"">However, =
it is still not possible to create fraud proof for spending of =
non-existing UTXO. This requires commitment of the block height of =
inputs, and the tx index in the block. I=E2=80=99m not quire sure how =
this could be implemented because a re-org may change such info (I think =
validation is easy but mining is more tricky)</div><div class=3D""><br =
class=3D""></div><div class=3D"">How to join: codes at&nbsp;<a =
href=3D"https://github.com/jl2012/bitcoin/tree/forcenet2" =
class=3D"">https://github.com/jl2012/bitcoin/tree/forcenet2</a>&nbsp;, =
start with "bitcoind =E2=80=94forcenet" .<br class=3D"">Connection: =
I=E2=80=99m running a node at&nbsp;<a href=3D"http://8333.info" =
class=3D"">8333.info</a>&nbsp;with default port (39901)<br =
class=3D"">Mining: there is only basic internal mining support. To use =
the internal miner, writeup a shell script to repeatedly call =
=E2=80=9Cbitcoin-cli =E2=80=94forcenet generate 1=E2=80=9D</div><div =
class=3D""><br class=3D""></div><div class=3D"">jl2012</div></body></html>=

--Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812--