From: Damian Williamson <willtech@live.com.au>
To: "bitcoin-dev@lists.linuxfoundation.org"
	<bitcoin-dev@lists.linuxfoundation.org>
Thread-Topic: BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
	Ordering Transactions In Blocks
Thread-Index: AQHTb5z9LKaNJrfjBkOvllhZtrqBGg==
Date: Thu, 7 Dec 2017 21:01:43 +0000
Message-ID: <PS2P216MB01794ABD544248B27BF0DFD99D330@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
Accept-Language: en-AU, en-US
Content-Language: en-AU
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [C9UP2Hm1bBDnjgpCIuU9wivrDXO2Tmsr]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; HK2APC01HT017;
	6:4GxWogGjd860LvEcwjEaAjAGpT4z4qNUzRI8Pp+nUq64GytvUE7z4BXtuUOj8gSr5zur63md9FuLWwhdDqiOgJK7/YL196kQuPB0qn8/EKHOsvVlBjjwhxs8ibU4+vEfvHdBD6pMqqoNx9PT8KZrurzjXejwpUaIH30EevZH2+Tzr05UZQy5ME9Hj2xMIrv8wW79Ju21JeQtEHwbg1GSBtrngKwzcAE3tBE7spUO8OhSgzsgcmLVXwUX4oTbF6HWTQm0Rp4XbkuezVrUWjuyNDViFDuJnXp0dBPItmwlZILsrthxXXnGEm13SbDwwX1JaIjvdpI0ouV7iOSiRXrrXBHyGVmurGXgrMWbnbsXUhs=;
	5:lwi+6azTfP7W/8sV9b5u81XoYumFjE7Pf7BqajgX50TgImJl57njXhUq7qVhTSEDuMMv/8vug3KCHD39agVUwgqfqDqfP9ZUVyOS+18pu1CUO2dSN/ObfYfksoppdQjjJ0dd/t0rcktj1x+1bkubCgN+8YeH7hHeRKrqS4ASIZ8=;
	24:yhe01ZuqdMJMzTj7C/vtvQQ9kAQtueZDahcqhNhGpomlB5mVJzle493xIedmcgO4TP3JyxZXiKA8NYu2nGE4hhcnmnsiJ4LpjLTLL0BDzU8=;
	7:oDlB0P3eFGikIC6vLOIDYyLkS8jTWvH/gmr5sPPG5J0H06kg2LiDFam2ZcEPExByhk95Y5b0plH8jjb04ZaVYb4c9XdX+dVqXt7jyGVQO0B/+lQuG9/Dwjjw0ukfxjeSIQU23i6mRUAqpGpLvyRYWTTwHWzf08cARE92/nZXj/HUPSuwGBy3sf6TSwAEgfkAE0OC0I6jQZC63XIaHbyDliEBMQxYHam1f9Rug0e9crCwUV/Ilyrbh3kCcL1oJBJL
x-incomingheadercount: 44
x-eopattributedmessage: 0
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
	RULEID:(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125374)(1603101448)(1701031045);
	SRVR:HK2APC01HT017; 
x-ms-traffictypediagnostic: HK2APC01HT017:
x-ms-office365-filtering-correlation-id: b12b5b10-4926-4a86-b28c-08d53db5b8ae
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031);
	SRVR:HK2APC01HT017; BCL:0; PCL:0;
	RULEID:(100000803101)(100110400095); SRVR:HK2APC01HT017; 
x-forefront-prvs: 05143A8241
x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT;
	SFP:1901; SCL:1; SRVR:HK2APC01HT017;
	H:PS2P216MB0179.KORP216.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:;
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_PS2P216MB01794ABD544248B27BF0DFD99D330PS2P216MB0179KORP_"
MIME-Version: 1.0
X-OriginatorOrg: outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b12b5b10-4926-4a86-b28c-08d53db5b8ae
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Dec 2017 21:01:43.8687 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2APC01HT017
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Thu, 07 Dec 2017 21:02:44 +0000
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
 Priority For Ordering Transactions In Blocks
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Dec 2017 21:01:48 -0000

--_000_PS2P216MB01794ABD544248B27BF0DFD99D330PS2P216MB0179KORP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Good afternoon,

The need for this proposal:

We all must learn to admit that transaction bandwidth is still lurking as a=
 serious issue for the operation, reliability, safety, consumer acceptance,=
 uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day target=
 confirmation from the fee recommendation. That transaction has still not c=
onfirmed after now more than six days - even waiting twice as long seems qu=
ite reasonable to me. That transaction is a valid transaction; it is not ru=
bbish, junk or, spam. Under the current model with transaction bandwidth li=
mitation, the longer a transaction waits, the less likely it is ever to con=
firm due to rising transaction numbers and being pushed back by transaction=
s with rising fees.

I argue that no transactions are rubbish or junk, only some zero fee transa=
ctions might be spam. Having an ever-increasing number of valid transaction=
s that do not confirm as more new transactions with higher fees are created=
 is the opposite of operating a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not conf=
irm. Even a business choosing a modest fee has no guarantee that their vali=
d transaction will not be shuffled down by new transactions to the realm of=
 never confirming after it is created. Consumers also will not accept this =
model as Bitcoin expands. If Bitcoin cannot be a reliable payment system fo=
r confirmed transactions then consumers, by and large, will simply not acce=
pt the model once they understand. Bitcoin will be a dirty payment system, =
and this will kill the value of Bitcoin.

Under the current system, a minority of transactions will eventually be the=
 lucky few who have fees high enough to escape being pushed down the list.

Once there are more than x transactions (transaction bandwidth limit) every=
 ten minutes, only those choosing twenty-minute confirmation (2 blocks) wil=
l have initially at most a fifty percent chance of ever having their paymen=
t confirm. Presently, not even using fee recommendations can ensure a suffi=
ciently high fee is paid to ensure transaction confirmation.

I also argue that the current auction model for limited transaction bandwid=
th is wrong, is not suitable for a reliable transaction system and, is wron=
g for Bitcoin. All transactions must confirm in due time. Currently, Bitcoi=
n is not a safe way to send payments.

I do not believe that consumers and business are against paying fees, even =
high fees. What is required is operational reliability.

This great issue needs to be resolved for the safety and reliability of Bit=
coin. The time to resolve issues in commerce is before they become great bi=
g issues. The time to resolve this issue is now. We must have the foresight=
 to identify and resolve problems before they trip us over.  Simply doublin=
g block sizes every so often is reactionary and is not a reliable permanent=
 solution. I have written a BIP proposal for a technical solution but, need=
 your help to write it up to an acceptable standard to be a full BIP.

I have formatted the following with markdown which is human readable so, I =
hope nobody minds. I have done as much with this proposal as I feel that I =
am able so far but continue to take your feedback.

# BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering Transactio=
ns In Blocks

## The problem:
Everybody wants value. Miners want to maximize revenue from fees (and we pr=
esume, to minimize block size). Consumers need transaction reliability and,=
 (we presume) want low fees.

The current transaction bandwidth limit is a limiting factor for both. As t=
he operational safety of transactions is limited, so is consumer confidence=
 as they realize the issue and, accordingly, uptake is limited. Fees are ar=
tificially inflated due to bandwidth limitations while failing to provide a=
 full confirmation service for all transactions.

Current fee recommendations provide no satisfaction for transaction reliabi=
lity and, as Bitcoin scales, this will worsen.

Bitcoin must be a fully scalable and reliable service, providing full trans=
action confirmation for every valid transaction.

The possibility to send a transaction with a fee lower than one that is acc=
eptable to allow eventual transaction confirmation should be removed from t=
he protocol and also from the user interface.

## Solution summary:
Provide each transaction with an individual transaction priority each time =
before choosing transactions to include in the current block, the priority =
being a function of the fee paid (on a curve), and the time waiting in the =
transaction pool (also on a curve) out to n days (n=3D60 ?). The transactio=
n priority to serve as the likelihood of a transaction being included in th=
e current block, and for determining the order in which transactions are tr=
ied to see if they will be included.

Use a target block size. Determine the target block size using; current tra=
nsaction pool size x ( 1 / (144 x n days ) ) =3D number of transactions to =
be included in the current block. Broadcast the next target block size with=
 the current block when it is solved so that nodes know the next target blo=
ck size for the block that they are building on.

The curves used for the priority of transactions would have to be appropria=
te. Perhaps a mathematician with experience in probability can develop the =
right formulae. My thinking is a steep curve. I suppose that the probabilit=
y of all transactions should probably account for a sufficient number of in=
clusions that the target block size is met although, it may not always be. =
As a suggestion, consider including some zero fee transactions to pad, high=
est BTC value first?

**Explanation of the operation of priority:**
> If transaction priority is, for example, a number between one (low) and o=
ne-hundred (high) it can be directly understood as the percentage chance in=
 one-hundred of a transaction being included in the block. Using probabilit=
y or likelihood infers that there is some function of random. If random (10=
0) < transaction priority then the transaction is included.

>To break it down further, if both the fee on a curve value and the time wa=
iting on a curve value are each a number between one and one-hundred, a rud=
imentary method may be to simply multiply those two numbers, to find the pr=
iority number. For example, a middle fee transaction waiting thirty days (i=
f n =3D 60 days) may have a value of five for each part  (yes, just five, t=
he values are on a curve). When multiplied that will give a priority value =
of twenty-five, or,  a twenty-five percent chance at that moment of being i=
ncluded in the block; it will likely be included in one of the next four bl=
ocks, getting more likely each chance. If it is still not included then the=
 value of time waiting will be higher, making for more probability. A very =
low fee transaction would have a value for the fee of one. It would not be =
until near sixty-days that the particular low fee transaction has a high li=
kelihood of being included in the block.

I am not concerned with low (or high) transaction fees, the primary reason =
for addressing the issue is to ensure transactional reliability and scalabi=
lity while having each transaction confirm in due time.

## Pros:
* Maximizes transaction reliability.
* Fully scalable.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because=
 of reliability, in time confidence and overall uptake are greater; therefo=
re, more transactions.
* Market determines fee paid for transaction priority.
* Fee recommendations work all the way out to 30 days or greater.
* Provides additional block entropy; greater security since there is less p=
robability of predicting the next block.

## Cons:
* Could initially lower total transaction fees per block.
* Must be first be programmed.

## Solution operation:
This is a simplistic view of the operation. The actual operation will need =
to be determined in a spec for the programmer.

1. Determine the target block size for the current block.
2. Assign a transaction priority to each transaction in the pool.
3. Select transactions to include in the current block using probability in=
 transaction priority order until the target block size is met.
5. Solve block.
6. Broadcast the next target block size with the current block when it is s=
olved.
7. Block is received.
8. Block verification process.
9. Accept/reject block based on verification result.
10. Repeat.

## Closing comments:
It may be possible to verify blocks conform to the proposal by showing that=
 the probability for all transactions included in the block statistically c=
onforms to a probability distribution curve, *if* the individual transactio=
n priority can be recreated. I am not that deep into the mathematics; howev=
er, it may also be possible to use a similar method to do this just based o=
n the fee, that statistically, the blocks conform to a fee distribution. An=
y zero fee transactions would have to be ignored. This solution needs a cle=
ver mathematician.

I implore, at the very least, that we use some method that validates full t=
ransaction reliability and enables scalability of block sizes. If not this =
proposal, an alternative.

Regards,
Damian Williamson

--_000_PS2P216MB01794ABD544248B27BF0DFD99D330PS2P216MB0179KORP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"><!-- P {margin-top:0;margi=
n-bottom:0;} --></style>
</head>
<body dir=3D"ltr">
<div id=3D"divtagdefaultwrapper" style=3D"font-size: 12pt; color: rgb(0, 0,=
 0); font-family: Calibri,Helvetica,sans-serif,&quot;EmojiFont&quot;,&quot;=
Apple Color Emoji&quot;,&quot;Segoe UI Emoji&quot;,NotoColorEmoji,&quot;Seg=
oe UI Symbol&quot;,&quot;Android Emoji&quot;,EmojiSymbols;" dir=3D"ltr">
<p style=3D"margin-top:0;margin-bottom:0"></p>
<div>Good afternoon,<br>
<br>
The need for this proposal:<br>
<br>
We all must learn to admit that transaction bandwidth is still lurking as a=
 serious issue for the operation, reliability, safety, consumer acceptance,=
 uptake and, for the value of Bitcoin.<br>
<br>
I recently sent a payment which was not urgent so; I chose three-day target=
 confirmation from the fee recommendation. That transaction has still not c=
onfirmed after now more than six days - even waiting twice as long seems qu=
ite reasonable to me. That transaction
 is a valid transaction; it is not rubbish, junk or, spam. Under the curren=
t model with transaction bandwidth limitation, the longer a transaction wai=
ts, the less likely it is ever to confirm due to rising transaction numbers=
 and being pushed back by transactions
 with rising fees.<br>
<br>
I argue that no transactions are rubbish or junk, only some zero fee transa=
ctions might be spam. Having an ever-increasing number of valid transaction=
s that do not confirm as more new transactions with higher fees are created=
 is the opposite of operating a
 robust, reliable transaction system.<br>
<br>
Business cannot operate with a model where transactions may or may not conf=
irm. Even a business choosing a modest fee has no guarantee that their vali=
d transaction will not be shuffled down by new transactions to the realm of=
 never confirming after it is created.
 Consumers also will not accept this model as Bitcoin expands. If Bitcoin c=
annot be a reliable payment system for confirmed transactions then consumer=
s, by and large, will simply not accept the model once they understand. Bit=
coin will be a dirty payment system,
 and this will kill the value of Bitcoin.<br>
<br>
Under the current system, a minority of transactions will eventually be the=
 lucky few who have fees high enough to escape being pushed down the list.<=
br>
<br>
Once there are more than x transactions (transaction bandwidth limit) every=
 ten minutes, only those choosing twenty-minute confirmation (2 blocks) wil=
l have initially at most a fifty percent chance of ever having their paymen=
t confirm. Presently, not even using
 fee recommendations can ensure a sufficiently high fee is paid to ensure t=
ransaction confirmation.<br>
<br>
I also argue that the current auction model for limited transaction bandwid=
th is wrong, is not suitable for a reliable transaction system and, is wron=
g for Bitcoin. All transactions must confirm in due time. Currently, Bitcoi=
n is not a safe way to send payments.<br>
<br>
I do not believe that consumers and business are against paying fees, even =
high fees. What is required is operational reliability.<br>
<br>
This great issue needs to be resolved for the safety and reliability of Bit=
coin. The time to resolve issues in commerce is before they become great bi=
g issues. The time to resolve this issue is now. We must have the foresight=
 to identify and resolve problems
 before they trip us over.&nbsp; Simply doubling block sizes every so often=
 is reactionary and is not a reliable permanent solution. I have written a =
BIP proposal for a technical solution but, need your help to write it up to=
 an acceptable standard to be a full
 BIP.<br>
<br>
I have formatted the following with markdown which is human readable so, I =
hope nobody minds. I have done as much with this proposal as I feel that I =
am able so far but continue to take your feedback.<br>
<br>
# BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering Transactio=
ns In Blocks<br>
<br>
## The problem:<br>
Everybody wants value. Miners want to maximize revenue from fees (and we pr=
esume, to minimize block size). Consumers need transaction reliability and,=
 (we presume) want low fees.<br>
<br>
The current transaction bandwidth limit is a limiting factor for both. As t=
he operational safety of transactions is limited, so is consumer confidence=
 as they realize the issue and, accordingly, uptake is limited. Fees are ar=
tificially inflated due to bandwidth
 limitations while failing to provide a full confirmation service for all t=
ransactions.<br>
<br>
Current fee recommendations provide no satisfaction for transaction reliabi=
lity and, as Bitcoin scales, this will worsen.<br>
<br>
Bitcoin must be a fully scalable and reliable service, providing full trans=
action confirmation for every valid transaction.<br>
<br>
The possibility to send a transaction with a fee lower than one that is acc=
eptable to allow eventual transaction confirmation should be removed from t=
he protocol and also from the user interface.<br>
<br>
## Solution summary:<br>
Provide each transaction with an individual transaction priority each time =
before choosing transactions to include in the current block, the priority =
being a function of the fee paid (on a curve), and the time waiting in the =
transaction pool (also on a curve)
 out to n days (n=3D60 ?). The transaction priority to serve as the likelih=
ood of a transaction being included in the current block, and for determini=
ng the order in which transactions are tried to see if they will be include=
d.
<br>
<br>
Use a target block size. Determine the target block size using; current tra=
nsaction pool size x ( 1 / (144 x n days ) ) =3D number of transactions to =
be included in the current block. Broadcast the next target block size with=
 the current block when it is solved
 so that nodes know the next target block size for the block that they are =
building on.<br>
<br>
The curves used for the priority of transactions would have to be appropria=
te. Perhaps a mathematician with experience in probability can develop the =
right formulae. My thinking is a steep curve. I suppose that the probabilit=
y of all transactions should probably
 account for a sufficient number of inclusions that the target block size i=
s met although, it may not always be. As a suggestion, consider including s=
ome zero fee transactions to pad, highest BTC value first?<br>
<br>
**Explanation of the operation of priority:**<br>
&gt; If transaction priority is, for example, a number between one (low) an=
d one-hundred (high) it can be directly understood as the percentage chance=
 in one-hundred of a transaction being included in the block. Using probabi=
lity or likelihood infers that there
 is some function of random. If random (100) &lt; transaction priority then=
 the transaction is included.<br>
<br>
&gt;To break it down further, if both the fee on a curve value and the time=
 waiting on a curve value are each a number between one and one-hundred, a =
rudimentary method may be to simply multiply those two numbers, to find the=
 priority number. For example, a middle
 fee transaction waiting thirty days (if n =3D 60 days) may have a value of=
 five for each part&nbsp; (yes, just five, the values are on a curve). When=
 multiplied that will give a priority value of twenty-five, or,&nbsp; a twe=
nty-five percent chance at that moment of being
 included in the block; it will likely be included in one of the next four =
blocks, getting more likely each chance. If it is still not included then t=
he value of time waiting will be higher, making for more probability. A ver=
y low fee transaction would have
 a value for the fee of one. It would not be until near sixty-days that the=
 particular low fee transaction has a high likelihood of being included in =
the block.<br>
<br>
I am not concerned with low (or high) transaction fees, the primary reason =
for addressing the issue is to ensure transactional reliability and scalabi=
lity while having each transaction confirm in due time.<br>
<br>
## Pros:<br>
* Maximizes transaction reliability.<br>
* Fully scalable.<br>
* Maximizes possibility for consumer and business uptake.<br>
* Maximizes total fees paid per block without reducing reliability; because=
 of reliability, in time confidence and overall uptake are greater; therefo=
re, more transactions.<br>
* Market determines fee paid for transaction priority.<br>
* Fee recommendations work all the way out to 30 days or greater.<br>
* Provides additional block entropy; greater security since there is less p=
robability of predicting the next block.<br>
<br>
## Cons:<br>
* Could initially lower total transaction fees per block.<br>
* Must be first be programmed.<br>
<br>
## Solution operation:<br>
This is a simplistic view of the operation. The actual operation will need =
to be determined in a spec for the programmer.<br>
<br>
1. Determine the target block size for the current block.<br>
2. Assign a transaction priority to each transaction in the pool.<br>
3. Select transactions to include in the current block using probability in=
 transaction priority order until the target block size is met.<br>
5. Solve block.<br>
6. Broadcast the next target block size with the current block when it is s=
olved.<br>
7. Block is received.<br>
8. Block verification process.<br>
9. Accept/reject block based on verification result.<br>
10. Repeat.<br>
<br>
## Closing comments:<br>
It may be possible to verify blocks conform to the proposal by showing that=
 the probability for all transactions included in the block statistically c=
onforms to a probability distribution curve, *if* the individual transactio=
n priority can be recreated. I am
 not that deep into the mathematics; however, it may also be possible to us=
e a similar method to do this just based on the fee, that statistically, th=
e blocks conform to a fee distribution. Any zero fee transactions would hav=
e to be ignored. This solution needs
 a clever mathematician.<br>
<br>
I implore, at the very least, that we use some method that validates full t=
ransaction reliability and enables scalability of block sizes. If not this =
proposal, an alternative.<br>
<br>
Regards,<br>
Damian Williamson</div>
<p></p>
</div>
</body>
</html>

--_000_PS2P216MB01794ABD544248B27BF0DFD99D330PS2P216MB0179KORP_--