Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 86243723 for ; Sat, 14 Jan 2017 21:15:02 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6BC3D138 for ; Sat, 14 Jan 2017 21:15:00 +0000 (UTC) Received: from pn-206-59.itsc.cuhk.edu.hk (pn-206-59.itsc.cuhk.edu.hk [137.189.206.59]) by mx.zohomail.com with SMTPS id 1484428499723252.64572643518; Sat, 14 Jan 2017 13:14:59 -0800 (PST) From: Johnson Lau Content-Type: multipart/alternative; boundary="Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812" Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Date: Sun, 15 Jan 2017 05:14:55 +0800 References: To: bitcoin-dev In-Reply-To: Message-Id: <5CDE542F-204F-4988-838F-F438D30C7D99@xbt.hk> X-Mailer: Apple Mail (2.3259) X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Forcenet: an experimental network with a new header format X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jan 2017 21:15:02 -0000 --Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I created a second version of forcenet with more experimental features = and stopped my forcenet1 node. 1. It has a new header format: Height (4), BIP9 signalling field (4), = hardfork signalling field (2), Hash TMR (32), Hash WMR (32), Merkle sum = root (32), number of tx (4), prev hash (32), timestamp (4), nBits (4), = nonce1 (4), nonce2 (4), nonce3 (compactSize + variable), merkle branches = leading to header C (compactSize + 32 bit hashes) 2. Anti-tx-replay. If, after masking the highest byte, the tx nVersion = is >=3D3, the sighash for both segwit and non-segwit outputs is = calculated with BIP143, except 0x2000000 is added to the nHashType. Such = signatures are invalid for legacy nodes. But since they are non-std due = the nVersion, they won=E2=80=99t be relayed nor validated by legacy = nodes. This also removes the O(n^2) sighash problem when spending = non-segwit outputs. (anti-replay is a long story and I will discuss in a = separate post/BIP) 3. Block sighashlimit = (https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki = ). = Due to point 2, SigHashSize is counted only for legacy non-segwit inputs = (with masked tx nVersion < 3). We have to support legacy signature to = make sure time-locked txs made before the hard fork are still valid. 4. A totally new way to define tx weight. Tx weight is the maximum of = the following metrics: a. SigHashSize (see the bip in point 3) b. Witness serialised size * 2 * 90 c. Adjusted size * 90. Adjusted size =3D tx weight (BIP141) + (number of = non-OP_RETURN outputs - number of inputs) * 41 * 4 d. nSigOps * 50 * 90. All SigOps are equal (no witness scaling). For = non-segwit txs, the sigops in output scriptPubKey are not counted, while = the sigops in input scriptPubKey are counted. 90 is the scaling factor for SigHashSize, to maintain the 1:90 ratio = (see the BIP in point 3) 50 is the scaling factor for nSigOps, maintaining the 1:50 ratio in = BIP141 Rationale for adjusted size: 4 is witness scaling factor. 41 is the = minimum size for an input (32 hash + 4 index + 4 nSequence + 1 = scriptSig). This requires people to pre-pay majority of the fee of = spending an UTXO. It makes creation of UTXO more expensive, while = spending of UTXO cheaper, creates a strong incentive to limit the growth = of UTXO set. Rationale for taking the maximum of different metrics: this indirectly = set an upper block resources for _every_ metrics, while making the tx = fee estimation a linear function. Currently, there are 2 block resources = limits: block weight and nSigOp cost (BIP141). However, since users do = not know what the other txs are included in the next block, it is = difficult to determine whether tx weight of nSigOp cost is a more = important factor in determining the tx fee. (This is not a real problem = now, because weight is more important in most cases). With an unified = definition of tx weight, the fee estimation becomes a linear problem. Translating to new metric, the current BIP141 limit is 360,000,000. This = is equivalent to 360MB of sighashing, 2MB of serialised size, 4MB of = adjusted size, or 80000 nSigOp. Any new block-level limit metrics could be added to tx weight using soft = forks. 5. Smooth halving: the reward of the last 2016 blocks in a halving cycle = will be reduced by 25%, which is contributed to the first 2016 blocks of = the new halving cycle. (different parameters for forcenet) This makes a = more graceful transition but we will lose some fun around halving. 6. A new coinbase tx format. BIP34 is removed. Coinbase tx may have more = than 1 input. The prevout hash of first input must be the hash of = previous block, and index must be 0xffffffff. The other inputs (if any) = must come from UTXOs with valid signatures. Spending of previous = coinbase outputs in a coinbase tx is exempted from the 100 block = maturity requirement. Therefore, miners of an earlier block may pay = other miners to convince them to confirm their blocks. 7. Merkle sum tree: it allows generating of fraud-proof for fee and = weight. A special softfork (bit 15) is defined. When this softfork is = activated, the full node will not validate the sum tree. This is needed = because when the definition of tx weight is changed through a softfork = (e.g. a new script version introducing new sigop), olds nodes won=E2=80=99= t know the new rules and will find the sum tree invalid. Disabling the = sum tree validation won=E2=80=99t degrade the security of a full node by = more than an usual softfork, because the full node would still validate = all other known rules. However, it is still not possible to create fraud proof for spending of = non-existing UTXO. This requires commitment of the block height of = inputs, and the tx index in the block. I=E2=80=99m not quire sure how = this could be implemented because a re-org may change such info (I think = validation is easy but mining is more tricky) How to join: codes at https://github.com/jl2012/bitcoin/tree/forcenet2 = , start with = "bitcoind =E2=80=94forcenet" . Connection: I=E2=80=99m running a node at 8333.info = with default port (39901) Mining: there is only basic internal mining support. To use the internal = miner, writeup a shell script to repeatedly call =E2=80=9Cbitcoin-cli = =E2=80=94forcenet generate 1=E2=80=9D jl2012= --Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
I created a second version of forcenet with = more experimental features and stopped my forcenet1 node.

1. It has a new header = format: Height (4), BIP9 signalling field (4), hardfork signalling field = (2), Hash TMR (32), Hash WMR (32), Merkle sum root (32), number of tx = (4), prev hash (32), timestamp (4), nBits (4), nonce1 (4), nonce2 (4), = nonce3 (compactSize + variable), merkle branches leading to header C = (compactSize + 32 bit hashes)

2. Anti-tx-replay. If, after masking = the highest byte, the tx nVersion is >=3D3, the sighash for both = segwit and non-segwit outputs is calculated with BIP143, except = 0x2000000 is added to the nHashType. Such signatures are invalid for = legacy nodes. But since they are non-std due the nVersion, they won=E2=80=99= t be relayed nor validated by legacy nodes. This also removes the O(n^2) = sighash problem when spending non-segwit outputs. (anti-replay is a long = story and I will discuss in a separate post/BIP)

3. Block sighashlimit (https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawi= ki). Due to point 2, SigHashSize is counted only for legacy = non-segwit inputs (with masked tx nVersion < 3). We have to support = legacy signature to make sure time-locked txs made before the hard fork = are still valid.

4. A totally new way to define tx weight. Tx weight is the = maximum of the following metrics:
a. SigHashSize = (see the bip in point 3)
b. Witness serialised size = * 2 * 90
c. Adjusted size * 90. Adjusted size =3D = tx weight (BIP141) + (number of non-OP_RETURN outputs - number of = inputs) * 41 * 4
d. nSigOps * 50 * 90. All SigOps = are equal (no witness scaling). For non-segwit txs, the sigops in output = scriptPubKey are not counted, while the sigops in input scriptPubKey are = counted.

90 is = the scaling factor for SigHashSize, to maintain the 1:90 ratio (see the = BIP in point 3)
50 is the scaling factor for = nSigOps, maintaining the 1:50 ratio in BIP141

Rationale for adjusted size: 4 is = witness scaling factor. 41 is the minimum size for an input (32 hash + 4 = index + 4 nSequence + 1 scriptSig). This requires people to pre-pay = majority of the fee of spending an UTXO. It makes creation of UTXO more = expensive, while spending of UTXO cheaper, creates a strong incentive to = limit the growth of UTXO set.

Rationale for taking the maximum of = different metrics: this indirectly set an upper block resources for = _every_ metrics, while making the tx fee estimation a linear function. = Currently, there are 2 block resources limits: block weight and nSigOp = cost (BIP141). However, since users do not know what the other txs are = included in the next block, it is difficult to determine whether tx = weight of nSigOp cost is a more important factor in determining the tx = fee. (This is not a real problem now, because weight is more important = in most cases). With an unified definition of tx weight, the fee = estimation becomes a linear problem.

Translating to new metric, the current = BIP141 limit is 360,000,000. This is equivalent to 360MB of sighashing, = 2MB of serialised size, 4MB of adjusted size, or 80000 nSigOp.

Any new block-level = limit metrics could be added to tx weight using soft forks.

5. Smooth halving: the = reward of the last 2016 blocks in a halving cycle will be reduced by = 25%, which is contributed to the first 2016 blocks of the new halving = cycle. (different parameters for forcenet) This makes a more graceful = transition but we will lose some fun around halving.

6. A new coinbase tx = format. BIP34 is removed. Coinbase tx may have more than 1 input. The = prevout hash of first input must be the hash of previous block, and = index must be 0xffffffff. The other inputs (if any) must come from UTXOs = with valid signatures. Spending of previous coinbase outputs in a = coinbase tx is exempted from the 100 block maturity requirement. = Therefore, miners of an earlier block may pay other miners to convince = them to confirm their blocks.

7. Merkle sum tree: it allows = generating of fraud-proof for fee and weight. A special softfork (bit = 15) is defined. When this softfork is activated, the full node will not = validate the sum tree. This is needed because when the definition of tx = weight is changed through a softfork (e.g. a new script version = introducing new sigop), olds nodes won=E2=80=99t know the new rules and = will find the sum tree invalid. Disabling the sum tree validation = won=E2=80=99t degrade the security of a full node by more than an usual = softfork, because the full node would still validate all other known = rules.

However, = it is still not possible to create fraud proof for spending of = non-existing UTXO. This requires commitment of the block height of = inputs, and the tx index in the block. I=E2=80=99m not quire sure how = this could be implemented because a re-org may change such info (I think = validation is easy but mining is more tricky)

How to join: codes at https://github.com/jl2012/bitcoin/tree/forcenet2 , = start with "bitcoind =E2=80=94forcenet" .
Connection: = I=E2=80=99m running a node at 8333.info with default port (39901)
Mining: there is only basic internal mining support. To use = the internal miner, writeup a shell script to repeatedly call = =E2=80=9Cbitcoin-cli =E2=80=94forcenet generate 1=E2=80=9D

jl2012
= --Apple-Mail=_34C4717A-065F-4577-A879-E5E3686A6812--