path: root/df/f73c5e1a4ae45338c4d056b52ee9fec232e69c

Return-Path: <criley@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id D86FBBA9
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Mon, 18 Dec 2017 12:10:18 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-it0-f48.google.com (mail-it0-f48.google.com
	[209.85.214.48])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 40BC71A6
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Mon, 18 Dec 2017 12:10:16 +0000 (UTC)
Received: by mail-it0-f48.google.com with SMTP id u62so28085605ita.2
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Mon, 18 Dec 2017 04:10:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to; 
	bh=zLhhMpfZ/7uI6dyPuHxP24CHylSJLSoFarHP4ICPDPk=;
	b=NP0kEqKvrC7mw8IsUTxpkFcrZ54SKhvBazjniJs1gaEbNGKuL0jm6V+Hl3S06LVjpS
	hDeHetafBBrsE4aW2Qfhv3mMxJi46McbzkmkZ5dx0R26SBD6QpFOLZ1UyzAPZUO3jNW3
	99V86YxvZvJxfOVFWmsd98an9REm4CN5RbwOpyoE5MtRH13+JbX/E+8b6xdx5yM3J1Ii
	UuhdxVuO/XRwz/pwTD1c06+VG7BN3DzF9t+PBk5p3EHiiJdhrVwOMJdtzMepUK/8ID6E
	Ch437lsm5P6whRLpUsSyxOTP5s/DoxzihSx3gsijeDtjf/YtP1S9iCH2yK59UHXV+jyr
	JEhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to;
	bh=zLhhMpfZ/7uI6dyPuHxP24CHylSJLSoFarHP4ICPDPk=;
	b=NZ/o6Kot5n6FWOBv/TMT+D4jatTSsw6lOXr1Gsx9PVYRIhu3kzbZj9l/9Ah4NPgKbW
	InBjHcT4gp2h9jpjsCcK9X1DfzAL35zpJNsJSCTUNdiUzxJ7isTK2WUAK6eG/jLovt8D
	q7GSYs5qkt4x9H6PVe+lcSP24YoXKoKmlEXVbBw9SnBUjmqTyXWcLn3aSJxT9dj2UJpy
	4dIwq0anQcIv9GaT3/z+MYttL3xrSVGSREd5HfjWTbME5QN5anaFO5IjCKYXdRaas65u
	zqK/+7O1dfhENLosPlkuADjxYdh3ZkL9iQCs6+VGErMvLLLl+Gd22eQKizFhCmULMVoU
	4H4g==
X-Gm-Message-State: AKGB3mLAg7k8gqr6Cmjeq6dkcEXgyAnZUtuYBZILb56p5wjPRQkX1lRR
	4Dsv6VVAk5HgAfEmFj+aoGFHbuQxMjVisRm4pRz7p3xO
X-Google-Smtp-Source: ACJfBounSkcLzBYvejls3t87+6uMI1pqa1kNOd7Bo7U2r7aE+GeWrVd/sMPP7f1Nn/7UjXrfnPdh4r3Fx692Ck5MLWM=
X-Received: by 10.36.101.16 with SMTP id u16mr19430814itb.90.1513599015421;
	Mon, 18 Dec 2017 04:10:15 -0800 (PST)
MIME-Version: 1.0
Received: by 10.79.12.65 with HTTP; Mon, 18 Dec 2017 04:09:34 -0800 (PST)
In-Reply-To: <PS2P216MB0179544A6503C2992190CEB69D0B0@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
References: <PS2P216MB01794ABD544248B27BF0DFD99D330@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
	<PS2P216MB01795BFC05612E021CCEDD7C9D0B0@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
	<5upGmF0IhXUWhcikhdV-e9Pqg-kXfUuXe0kOpGxumie_TO2jLvCgTZ5c6vgBRtaqkL6DmOJb1YftK0osufB5RkhW7YhAhhCI0zBTH3YcORY=@protonmail.com>
	<PS2P216MB0179544A6503C2992190CEB69D0B0@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
From: Chris Riley <criley@gmail.com>
Date: Mon, 18 Dec 2017 07:09:34 -0500
Message-ID: <CAL5BAw0pQJg8MjopRH5ReBVHmJR0bYud=E6fkFuY=-3hMAumEw@mail.gmail.com>
To: Damian Williamson <willtech@live.com.au>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="001a113f6514a5a01505609c3b1f"
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
	RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Mon, 18 Dec 2017 12:12:32 +0000
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
 Priority For Ordering Transactions In Blocks
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Dec 2017 12:10:19 -0000

--001a113f6514a5a01505609c3b1f
Content-Type: text/plain; charset="UTF-8"

Regarding "problem" #2 where you say "How do we ensure that all valid
transactions are eventually included in the blockchain?":  I do not believe
that all people would (a) agree this is a problem or (b) that we do want to
*ENSURE* that *ALL* valid transactions are eventually included in the
blockchain.  There are many *valid* transactions that oftentimes miners do
not (and should not) wish to require be confirmed and included in the
blockchain.  Spam transactions for example can be valid, but used to attack
bitcoin by using no or low fee.  Any valid transaction MAY be included by a
miner, but requiring it in some fashion at this point would open the
network to other attack vectors.  Perhaps you meant it a different way.


On Fri, Dec 15, 2017 at 3:59 PM, Damian Williamson via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> There are really two separate problems to solve.
>
>
> How does Bitcoin scale with fixed block size?
> How do we ensure that all valid transactions are eventually included in
the blockchain?
>
>
> Those are the two issues that the proposal attempts to address. It makes
sense to resolve these two problems together. Using the proposed system for
variable block sizes would solve the first problem but there would still be
a whole bunch of never confirming transactions. I am not sure how to
reliably solve the second problem at scale without first solving the first.
>
>
> >* Every node has a (potentially) different mempool, you can't use it to
decide consensus values like the max block size.
>
>
> I do not suggest a consensus. Depending on which node solves a block the
value for next block size will be different. The consensus would be that
blocks will adhere to the next block size value transmitted with the
current block. It is easy to verify that the consensus is being adhered to
once in place.
>
> >* Increasing the entropy in a block to make it more unpredictable
doesn't really make sense.
>
> Not a necessary function, just an effect of using a probability-based
distribution.
>
> >* Bitcoin should be roughly incentive compatible. Your proposal
explicits asks miners to ignore their best interests, and confirm
transactions by "priority".  What are you going to do if a "malicious"
miner decides to go after their profits and order by what makes them the
most money. Add "ordered by priority" as a consensus requirement? And even
if you miners can still sort their mempool by fee, and then order the top
1MB by priority.
>
> I entirely agree with your sentiment that Bitcoin must be incentive
compatible. It is necessary.
>
> It is in only miners immediate interest to make the most profitable block
from the available transaction pool. As with so many other things, it is
necessary to partially ignore short-term gain for long-term benefit. It is
in miners and everybody's long-term interest to have a reliable transaction
service. A busy transaction service that confirms lots of transactions per
hour will become more profitable as demand increases and more users are
prepared to pay for priority. As it is there is currently no way to fully
scale because of the transaction bandwidth limit and that is problematic.
If all valid transactions must eventually confirm then there must be a way
to resolve that problem.
>
> Bitcoin deliberately removes traditional scale by ensuring blocks take
ten minutes on average to solve, an ingenious idea and, incentive
compatible but, fixed block sizes leaves us with a problem to solve when we
want to scale.
>
> >If you could find a good solution that would allow you to know if miners
were following your rule or not (and thus ignore it if it doesn't) then you
wouldn't even need bitcoin in the first place.
>
> I am confident that the math to verify blocks based on the proposal can
be developed (and I think it will not be too complex for a mathematician
with the relevant experience), however, I am nowhere near experienced
enough with probability and statistical analysis to do it. Yes, if Bitcoin
doesn't then it might make another great opportunity for an altcoin but I
am not even nearly interested in promoting any altcoins.
>
>
> If not the proposal that I have put forward, then, hopefully, someone can
come up with a better solution. The important thing is that the issues are
resolved.
>
>
> Regards,
>
> Damian Williamson
>
>
>
> ________________________________
> From: Rhavar <rhavar@protonmail.com>
> Sent: Saturday, 16 December 2017 3:38 AM
> To: Damian Williamson
> Cc: Bitcoin Protocol Discussion
> Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use
Transaction Priority For Ordering Transactions In Blocks
>
> > I understand that there would be technical issues to resolve in
implementation, but, are there no fundamental errors?
>
> Unfortunately your proposal is really fundamentally broken, on a few
levels. I think you might need to do a bit more research into how bitcoin
works before coming up with such improvements =)
>
> But just some quick notes:
>
> * Every node has a (potentially) different mempool, you can't use it to
decide consensus values like the max block size.
>
> * Increasing the entropy in a block to make it more unpredictable doesn't
really make sense.
>
> * Bitcoin should be roughly incentive compatible. Your proposal explicits
asks miners to ignore their best interests, and confirm transactions by
"priority".  What are you going to do if a "malicious" miner decides to go
after their profits and order by what makes them the most money. Add
"ordered by priority" as a consensus requirement? And even if you miners
can still sort their mempool by fee, and then order the top 1MB by priority.
>
> If you could find a good solution that would allow you to know if miners
were following your rule or not (and thus ignore it if it doesn't) then you
wouldn't even need bitcoin in the first place.
>
>
>
>
> -Ryan
>
>
> -------- Original Message --------
> Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
Priority For Ordering Transactions In Blocks
> Local Time: December 15, 2017 3:42 AM
> UTC Time: December 15, 2017 9:42 AM
> From: bitcoin-dev@lists.linuxfoundation.org
> To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
>
>
>
> I should not take it that the lack of critical feedback to this revised
proposal is a glowing endorsement. I understand that there would be
technical issues to resolve in implementation, but, are there no
fundamental errors?
>
> I suppose that it if is difficult to determine how long a transaction has
been waiting in the pool then, each node could simply keep track of when a
transaction was first seen. This may have implications for a verify
routine, however, for example, if a node was offline, how should it
differentiate how long each transaction was waiting in that case? If a node
was restarted daily would it always think that all transactions had been
waiting in the pool less than one day If each node keeps the current
transaction pool in a file and updates it, as transactions are included in
blocks and, as new transactions appear in the pool, then that would go some
way to alleviate the issue, apart from entirely new nodes. There should be
no reason the contents of a transaction pool files cannot be shared without
agreement as to the transaction pool between nodes, just as nodes transmit
new transactions freely.
>
> It has been questioned why miners could not cheat. For the question of
how many transactions to include in a block, I say it is a standoff and
miners will conform to the proposal, not wanting to leave transactions with
valid fees standing, and, not wanting to shrink the transaction pool. In
any case, if miners shrink the transaction pool then I am not immediately
concerned since it provides a more efficient service. For the question of
including transactions according to the proposal, I say if it is possible
to keep track of how long transactions are waiting in the pool so that they
can be included on a probability curve then it is possible to verify that
blocks conform to the proposal, since the input is a probability, the
output should conform to a probability curve.
>
>
> If someone has the necessary skill, would anyone be willing to develop
the math necessary for the proposal?
>
> Regards,
> Damian Williamson
>
>
> ________________________________
>
> From: bitcoin-dev-bounces@lists.linuxfoundation.org <
bitcoin-dev-bounces@lists.linuxfoundation.org> on behalf of Damian
Williamson via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
> Sent: Friday, 8 December 2017 8:01 AM
> To: bitcoin-dev@lists.linuxfoundation.org
> Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
Priority For Ordering Transactions In Blocks
>
>
>
> Good afternoon,
>
> The need for this proposal:
>
> We all must learn to admit that transaction bandwidth is still lurking as
a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.
>
> I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has still
not confirmed after now more than six days - even waiting twice as long
seems quite reasonable to me. That transaction is a valid transaction; it
is not rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely it is
ever to confirm due to rising transaction numbers and being pushed back by
transactions with rising fees.
>
> I argue that no transactions are rubbish or junk, only some zero fee
transactions might be spam. Having an ever-increasing number of valid
transactions that do not confirm as more new transactions with higher fees
are created is the opposite of operating a robust, reliable transaction
system.
>
> Business cannot operate with a model where transactions may or may not
confirm. Even a business choosing a modest fee has no guarantee that their
valid transaction will not be shuffled down by new transactions to the
realm of never confirming after it is created. Consumers also will not
accept this model as Bitcoin expands. If Bitcoin cannot be a reliable
payment system for confirmed transactions then consumers, by and large,
will simply not accept the model once they understand. Bitcoin will be a
dirty payment system, and this will kill the value of Bitcoin.
>
> Under the current system, a minority of transactions will eventually be
the lucky few who have fees high enough to escape being pushed down the
list.
>
> Once there are more than x transactions (transaction bandwidth limit)
every ten minutes, only those choosing twenty-minute confirmation (2
blocks) will have initially at most a fifty percent chance of ever having
their payment confirm. Presently, not even using fee recommendations can
ensure a sufficiently high fee is paid to ensure transaction confirmation.
>
> I also argue that the current auction model for limited transaction
bandwidth is wrong, is not suitable for a reliable transaction system and,
is wrong for Bitcoin. All transactions must confirm in due time. Currently,
Bitcoin is not a safe way to send payments.
>
> I do not believe that consumers and business are against paying fees,
even high fees. What is required is operational reliability.
>
> This great issue needs to be resolved for the safety and reliability of
Bitcoin. The time to resolve issues in commerce is before they become great
big issues. The time to resolve this issue is now. We must have the
foresight to identify and resolve problems before they trip us over.
Simply doubling block sizes every so often is reactionary and is not a
reliable permanent solution. I have written a BIP proposal for a technical
solution but, need your help to write it up to an acceptable standard to be
a full BIP.
>
> I have formatted the following with markdown which is human readable so,
I hope nobody minds. I have done as much with this proposal as I feel that
I am able so far but continue to take your feedback.
>
> # BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering
Transactions In Blocks
>
> ## The problem:
> Everybody wants value. Miners want to maximize revenue from fees (and we
presume, to minimize block size). Consumers need transaction reliability
and, (we presume) want low fees.
>
> The current transaction bandwidth limit is a limiting factor for both. As
the operational safety of transactions is limited, so is consumer
confidence as they realize the issue and, accordingly, uptake is limited.
Fees are artificially inflated due to bandwidth limitations while failing
to provide a full confirmation service for all transactions.
>
> Current fee recommendations provide no satisfaction for transaction
reliability and, as Bitcoin scales, this will worsen.
>
> Bitcoin must be a fully scalable and reliable service, providing full
transaction confirmation for every valid transaction.
>
> The possibility to send a transaction with a fee lower than one that is
acceptable to allow eventual transaction confirmation should be removed
from the protocol and also from the user interface.
>
> ## Solution summary:
> Provide each transaction with an individual transaction priority each
time before choosing transactions to include in the current block, the
priority being a function of the fee paid (on a curve), and the time
waiting in the transaction pool (also on a curve) out to n days (n=60 ?).
The transaction priority to serve as the likelihood of a transaction being
included in the current block, and for determining the order in which
transactions are tried to see if they will be included.
>
> Use a target block size. Determine the target block size using; current
transaction pool size x ( 1 / (144 x n days ) ) = number of transactions to
be included in the current block. Broadcast the next target block size with
the current block when it is solved so that nodes know the next target
block size for the block that they are building on.
>
> The curves used for the priority of transactions would have to be
appropriate. Perhaps a mathematician with experience in probability can
develop the right formulae. My thinking is a steep curve. I suppose that
the probability of all transactions should probably account for a
sufficient number of inclusions that the target block size is met although,
it may not always be. As a suggestion, consider including some zero fee
transactions to pad, highest BTC value first?
>
> **Explanation of the operation of priority:**
> > If transaction priority is, for example, a number between one (low) and
one-hundred (high) it can be directly understood as the percentage chance
in one-hundred of a transaction being included in the block. Using
probability or likelihood infers that there is some function of random. If
random (100) < transaction priority then the transaction is included.
>
> >To break it down further, if both the fee on a curve value and the time
waiting on a curve value are each a number between one and one-hundred, a
rudimentary method may be to simply multiply those two numbers, to find the
priority number. For example, a middle fee transaction waiting thirty days
(if n = 60 days) may have a value of five for each part  (yes, just five,
the values are on a curve). When multiplied that will give a priority value
of twenty-five, or,  a twenty-five percent chance at that moment of being
included in the block; it will likely be included in one of the next four
blocks, getting more likely each chance. If it is still not included then
the value of time waiting will be higher, making for more probability. A
very low fee transaction would have a value for the fee of one. It would
not be until near sixty-days that the particular low fee transaction has a
high likelihood of being included in the block.
>
> I am not concerned with low (or high) transaction fees, the primary
reason for addressing the issue is to ensure transactional reliability and
scalability while having each transaction confirm in due time.
>
> ## Pros:
> * Maximizes transaction reliability.
> * Fully scalable.
> * Maximizes possibility for consumer and business uptake.
> * Maximizes total fees paid per block without reducing reliability;
because of reliability, in time confidence and overall uptake are greater;
therefore, more transactions.
> * Market determines fee paid for transaction priority.
> * Fee recommendations work all the way out to 30 days or greater.
> * Provides additional block entropy; greater security since there is less
probability of predicting the next block.
>
> ## Cons:
> * Could initially lower total transaction fees per block.
> * Must be first be programmed.
>
> ## Solution operation:
> This is a simplistic view of the operation. The actual operation will
need to be determined in a spec for the programmer.
>
> 1. Determine the target block size for the current block.
> 2. Assign a transaction priority to each transaction in the pool.
> 3. Select transactions to include in the current block using probability
in transaction priority order until the target block size is met.
> 5. Solve block.
> 6. Broadcast the next target block size with the current block when it is
solved.
> 7. Block is received.
> 8. Block verification process.
> 9. Accept/reject block based on verification result.
> 10. Repeat.
>
> ## Closing comments:
> It may be possible to verify blocks conform to the proposal by showing
that the probability for all transactions included in the block
statistically conforms to a probability distribution curve, *if* the
individual transaction priority can be recreated. I am not that deep into
the mathematics; however, it may also be possible to use a similar method
to do this just based on the fee, that statistically, the blocks conform to
a fee distribution. Any zero fee transactions would have to be ignored.
This solution needs a clever mathematician.
>
> I implore, at the very least, that we use some method that validates full
transaction reliability and enables scalability of block sizes. If not this
proposal, an alternative.
>
> Regards,
> Damian Williamson
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--001a113f6514a5a01505609c3b1f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Regarding &quot;problem&quot; #2 where you say &quot;How d=
o we ensure that all valid transactions are eventually included in the bloc=
kchain?&quot;: =C2=A0I do not believe that all people would (a) agree this =
is a problem or (b) that we do want to *ENSURE* that *ALL* valid transactio=
ns are eventually included in the blockchain.=C2=A0 There are many *valid* =
transactions that oftentimes miners do not (and should not) wish to require=
 be confirmed and included in the blockchain.=C2=A0 Spam transactions for e=
xample can be valid, but used to attack bitcoin by using no or low fee.=C2=
=A0 Any valid transaction MAY be included by a miner, but requiring it in s=
ome fashion at this point would open the network to other attack vectors.=
=C2=A0 Perhaps you meant it a different way.<div><br><br>On Fri, Dec 15, 20=
17 at 3:59 PM, Damian Williamson via bitcoin-dev &lt;<a href=3D"mailto:bitc=
oin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfoundation.org</a=
>&gt; wrote:<br>&gt;<br>&gt; There are really two separate problems to solv=
e.<br>&gt;<br>&gt;<br>&gt; How does Bitcoin scale with fixed block size?<br=
>&gt; How do we ensure that all valid transactions are eventually included =
in the blockchain?<br>&gt;<br>&gt;<br>&gt; Those are the two issues that th=
e proposal attempts to address. It makes sense to resolve these two problem=
s together. Using the proposed system for variable block sizes would solve =
the first problem but there would still be a whole bunch of never confirmin=
g transactions. I am not sure how to reliably solve the second problem at s=
cale without first solving the first.<br>&gt;<br>&gt;<br>&gt; &gt;* Every n=
ode has a (potentially) different mempool, you can&#39;t use it to decide c=
onsensus values like the max block size. <br>&gt;<br>&gt;<br>&gt; I do not =
suggest a consensus. Depending on which node solves a block the value for n=
ext block size will be different. The consensus would be that blocks will a=
dhere to the next block size value transmitted with the current block. It i=
s easy to verify that the consensus is being adhered to once in place.<br>&=
gt; =C2=A0<br>&gt; &gt;* Increasing the entropy in a block to make it more =
unpredictable doesn&#39;t really make sense. <br>&gt;<br>&gt; Not a necessa=
ry function, just an effect of using a probability-based distribution.<br>&=
gt;<br>&gt; &gt;* Bitcoin should be roughly incentive compatible. Your prop=
osal explicits asks miners to ignore their best interests, and confirm tran=
sactions by &quot;priority&quot;.=C2=A0 What are you going to do if a &quot=
;malicious&quot; miner decides to go after their profits and order by what =
makes them the most money. Add &quot;ordered by priority&quot; as a consens=
us requirement? And even if you miners can still sort their mempool by fee,=
 and then order the top 1MB by priority.<br>&gt;<br>&gt; I entirely agree w=
ith your sentiment that Bitcoin must be incentive compatible. It is necessa=
ry.<br>&gt;<br>&gt; It is in only miners immediate interest to make the mos=
t profitable block from the available transaction pool. As with so many oth=
er things, it is necessary to partially ignore short-term gain for long-ter=
m benefit. It is in miners and everybody&#39;s long-term interest to have a=
 reliable transaction service. A busy transaction service that confirms lot=
s of transactions per hour will become more profitable as demand increases =
and more users are prepared to pay for priority. As it is there is currentl=
y no way to fully scale because of the transaction bandwidth limit and that=
 is problematic. If all valid transactions must eventually confirm then the=
re must be a way to resolve that problem.<br>&gt;<br>&gt; Bitcoin deliberat=
ely removes traditional scale by ensuring blocks take ten minutes on averag=
e to solve, an ingenious idea and, incentive compatible but, fixed block si=
zes leaves us with a problem to solve when we want to scale.<br>&gt;<br>&gt=
; &gt;If you could find a good solution that would allow you to know if min=
ers were following your rule or not (and thus ignore it if it doesn&#39;t) =
then you wouldn&#39;t even need bitcoin in the first place.<br>&gt;<br>&gt;=
 I am confident that the math to verify blocks based on the proposal can be=
 developed (and I think it will not be too complex for a mathematician with=
 the relevant experience), however, I am nowhere near experienced enough wi=
th probability and statistical analysis to do it. Yes, if Bitcoin doesn&#39=
;t then it might make another great opportunity for an altcoin but I am not=
 even nearly interested in promoting any altcoins.<br>&gt;<br>&gt;<br>&gt; =
If not the proposal that I have put forward, then, hopefully, someone can c=
ome up with a better solution. The important thing is that the issues are r=
esolved.<br>&gt;<br>&gt;<br>&gt; Regards,<br>&gt;<br>&gt; Damian Williamson=
<br>&gt;<br>&gt;<br>&gt;<br>&gt; ________________________________<br>&gt; F=
rom: Rhavar &lt;<a href=3D"mailto:rhavar@protonmail.com">rhavar@protonmail.=
com</a>&gt;<br>&gt; Sent: Saturday, 16 December 2017 3:38 AM<br>&gt; To: Da=
mian Williamson<br>&gt; Cc: Bitcoin Protocol Discussion<br>&gt; Subject: Re=
: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority =
For Ordering Transactions In Blocks<br>&gt; =C2=A0<br>&gt; &gt; I understan=
d that there would be technical issues to resolve in implementation, but, a=
re there no fundamental errors?<br>&gt;<br>&gt; Unfortunately your proposal=
 is really fundamentally broken, on a few levels. I think you might need to=
 do a bit more research into how bitcoin works before coming up with such i=
mprovements =3D)<br>&gt;<br>&gt; But just some quick notes:<br>&gt;<br>&gt;=
 * Every node has a (potentially) different mempool, you can&#39;t use it t=
o decide consensus values like the max block size. <br>&gt;<br>&gt; * Incre=
asing the entropy in a block to make it more unpredictable doesn&#39;t real=
ly make sense. <br>&gt;<br>&gt; * Bitcoin should be roughly incentive compa=
tible. Your proposal explicits asks miners to ignore their best interests, =
and confirm transactions by &quot;priority&quot;.=C2=A0 What are you going =
to do if a &quot;malicious&quot; miner decides to go after their profits an=
d order by what makes them the most money. Add &quot;ordered by priority&qu=
ot; as a consensus requirement? And even if you miners can still sort their=
 mempool by fee, and then order the top 1MB by priority.<br>&gt;<br>&gt; If=
 you could find a good solution that would allow you to know if miners were=
 following your rule or not (and thus ignore it if it doesn&#39;t) then you=
 wouldn&#39;t even need bitcoin in the first place.<br>&gt;<br>&gt;<br>&gt;=
<br>&gt;<br>&gt; -Ryan<br>&gt;<br>&gt;<br>&gt; -------- Original Message --=
------<br>&gt; Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use=
 Transaction Priority For Ordering Transactions In Blocks<br>&gt; Local Tim=
e: December 15, 2017 3:42 AM<br>&gt; UTC Time: December 15, 2017 9:42 AM<br=
>&gt; From: <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoi=
n-dev@lists.linuxfoundation.org</a><br>&gt; To: Bitcoin Protocol Discussion=
 &lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@l=
ists.linuxfoundation.org</a>&gt;<br>&gt;<br>&gt;<br>&gt;<br>&gt; I should n=
ot take it that the lack of critical feedback to this revised proposal is a=
 glowing endorsement. I understand that there would be technical issues to =
resolve in implementation, but, are there no fundamental errors?<br>&gt;<br=
>&gt; I suppose that it if is difficult to determine how long a transaction=
 has been waiting in the pool then, each node could simply keep track of wh=
en a transaction was first seen. This may have implications for a verify ro=
utine, however, for example, if a node was offline, how should it different=
iate how long each transaction was waiting in that case? If a node was rest=
arted daily would it always think that all transactions had been waiting in=
 the pool less than one day If each node keeps the current transaction pool=
 in a file and updates it, as transactions are included in blocks and, as n=
ew transactions appear in the pool, then that would go some way to alleviat=
e the issue, apart from entirely new nodes. There should be no reason the c=
ontents of a transaction pool files cannot be shared without agreement as t=
o the transaction pool between nodes, just as nodes transmit new transactio=
ns freely.<br>&gt;<br>&gt; It has been questioned why miners could not chea=
t. For the question of how many transactions to include in a block, I say i=
t is a standoff and miners will conform to the proposal, not wanting to lea=
ve transactions with valid fees standing, and, not wanting to shrink the tr=
ansaction pool. In any case, if miners shrink the transaction pool then I a=
m not immediately concerned since it provides a more efficient service. For=
 the question of including transactions according to the proposal, I say if=
 it is possible to keep track of how long transactions are waiting in the p=
ool so that they can be included on a probability curve then it is possible=
 to verify that blocks conform to the proposal, since the input is a probab=
ility, the output should conform to a probability curve.<br>&gt;<br>&gt;<br=
>&gt; If someone has the necessary skill, would anyone be willing to develo=
p the math necessary for the proposal?<br>&gt;<br>&gt; Regards,<br>&gt; Dam=
ian Williamson<br>&gt;<br>&gt;<br>&gt; ________________________________<br>=
&gt;<br>&gt; From: <a href=3D"mailto:bitcoin-dev-bounces@lists.linuxfoundat=
ion.org">bitcoin-dev-bounces@lists.linuxfoundation.org</a> &lt;<a href=3D"m=
ailto:bitcoin-dev-bounces@lists.linuxfoundation.org">bitcoin-dev-bounces@li=
sts.linuxfoundation.org</a>&gt; on behalf of Damian Williamson via bitcoin-=
dev &lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-de=
v@lists.linuxfoundation.org</a>&gt;<br>&gt; Sent: Friday, 8 December 2017 8=
:01 AM<br>&gt; To: <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org"=
>bitcoin-dev@lists.linuxfoundation.org</a><br>&gt; Subject: [bitcoin-dev] B=
IP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Tran=
sactions In Blocks<br>&gt; =C2=A0<br>&gt;<br>&gt;<br>&gt; Good afternoon,<b=
r>&gt;<br>&gt; The need for this proposal:<br>&gt;<br>&gt; We all must lear=
n to admit that transaction bandwidth is still lurking as a serious issue f=
or the operation, reliability, safety, consumer acceptance, uptake and, for=
 the value of Bitcoin.<br>&gt;<br>&gt; I recently sent a payment which was =
not urgent so; I chose three-day target confirmation from the fee recommend=
ation. That transaction has still not confirmed after now more than six day=
s - even waiting twice as long seems quite reasonable to me. That transacti=
on is a valid transaction; it is not rubbish, junk or, spam. Under the curr=
ent model with transaction bandwidth limitation, the longer a transaction w=
aits, the less likely it is ever to confirm due to rising transaction numbe=
rs and being pushed back by transactions with rising fees.<br>&gt;<br>&gt; =
I argue that no transactions are rubbish or junk, only some zero fee transa=
ctions might be spam. Having an ever-increasing number of valid transaction=
s that do not confirm as more new transactions with higher fees are created=
 is the opposite of operating a robust, reliable transaction system.<br>&gt=
;<br>&gt; Business cannot operate with a model where transactions may or ma=
y not confirm. Even a business choosing a modest fee has no guarantee that =
their valid transaction will not be shuffled down by new transactions to th=
e realm of never confirming after it is created. Consumers also will not ac=
cept this model as Bitcoin expands. If Bitcoin cannot be a reliable payment=
 system for confirmed transactions then consumers, by and large, will simpl=
y not accept the model once they understand. Bitcoin will be a dirty paymen=
t system, and this will kill the value of Bitcoin.<br>&gt;<br>&gt; Under th=
e current system, a minority of transactions will eventually be the lucky f=
ew who have fees high enough to escape being pushed down the list.<br>&gt;<=
br>&gt; Once there are more than x transactions (transaction bandwidth limi=
t) every ten minutes, only those choosing twenty-minute confirmation (2 blo=
cks) will have initially at most a fifty percent chance of ever having thei=
r payment confirm. Presently, not even using fee recommendations can ensure=
 a sufficiently high fee is paid to ensure transaction confirmation.<br>&gt=
;<br>&gt; I also argue that the current auction model for limited transacti=
on bandwidth is wrong, is not suitable for a reliable transaction system an=
d, is wrong for Bitcoin. All transactions must confirm in due time. Current=
ly, Bitcoin is not a safe way to send payments.<br>&gt;<br>&gt; I do not be=
lieve that consumers and business are against paying fees, even high fees. =
What is required is operational reliability.<br>&gt;<br>&gt; This great iss=
ue needs to be resolved for the safety and reliability of Bitcoin. The time=
 to resolve issues in commerce is before they become great big issues. The =
time to resolve this issue is now. We must have the foresight to identify a=
nd resolve problems before they trip us over.=C2=A0 Simply doubling block s=
izes every so often is reactionary and is not a reliable permanent solution=
. I have written a BIP proposal for a technical solution but, need your hel=
p to write it up to an acceptable standard to be a full BIP.<br>&gt;<br>&gt=
; I have formatted the following with markdown which is human readable so, =
I hope nobody minds. I have done as much with this proposal as I feel that =
I am able so far but continue to take your feedback.<br>&gt;<br>&gt; # BIP =
Proposal: UTPFOTIB - Use Transaction Priority For Ordering Transactions In =
Blocks<br>&gt;<br>&gt; ## The problem:<br>&gt; Everybody wants value. Miner=
s want to maximize revenue from fees (and we presume, to minimize block siz=
e). Consumers need transaction reliability and, (we presume) want low fees.=
<br>&gt;<br>&gt; The current transaction bandwidth limit is a limiting fact=
or for both. As the operational safety of transactions is limited, so is co=
nsumer confidence as they realize the issue and, accordingly, uptake is lim=
ited. Fees are artificially inflated due to bandwidth limitations while fai=
ling to provide a full confirmation service for all transactions.<br>&gt;<b=
r>&gt; Current fee recommendations provide no satisfaction for transaction =
reliability and, as Bitcoin scales, this will worsen.<br>&gt;<br>&gt; Bitco=
in must be a fully scalable and reliable service, providing full transactio=
n confirmation for every valid transaction.<br>&gt;<br>&gt; The possibility=
 to send a transaction with a fee lower than one that is acceptable to allo=
w eventual transaction confirmation should be removed from the protocol and=
 also from the user interface.<br>&gt;<br>&gt; ## Solution summary:<br>&gt;=
 Provide each transaction with an individual transaction priority each time=
 before choosing transactions to include in the current block, the priority=
 being a function of the fee paid (on a curve), and the time waiting in the=
 transaction pool (also on a curve) out to n days (n=3D60 ?). The transacti=
on priority to serve as the likelihood of a transaction being included in t=
he current block, and for determining the order in which transactions are t=
ried to see if they will be included.<br>&gt;<br>&gt; Use a target block si=
ze. Determine the target block size using; current transaction pool size x =
( 1 / (144 x n days ) ) =3D number of transactions to be included in the cu=
rrent block. Broadcast the next target block size with the current block wh=
en it is solved so that nodes know the next target block size for the block=
 that they are building on.<br>&gt;<br>&gt; The curves used for the priorit=
y of transactions would have to be appropriate. Perhaps a mathematician wit=
h experience in probability can develop the right formulae. My thinking is =
a steep curve. I suppose that the probability of all transactions should pr=
obably account for a sufficient number of inclusions that the target block =
size is met although, it may not always be. As a suggestion, consider inclu=
ding some zero fee transactions to pad, highest BTC value first?<br>&gt;<br=
>&gt; **Explanation of the operation of priority:**<br>&gt; &gt; If transac=
tion priority is, for example, a number between one (low) and one-hundred (=
high) it can be directly understood as the percentage chance in one-hundred=
 of a transaction being included in the block. Using probability or likelih=
ood infers that there is some function of random. If random (100) &lt; tran=
saction priority then the transaction is included.<br>&gt;<br>&gt; &gt;To b=
reak it down further, if both the fee on a curve value and the time waiting=
 on a curve value are each a number between one and one-hundred, a rudiment=
ary method may be to simply multiply those two numbers, to find the priorit=
y number. For example, a middle fee transaction waiting thirty days (if n =
=3D 60 days) may have a value of five for each part =C2=A0(yes, just five, =
the values are on a curve). When multiplied that will give a priority value=
 of twenty-five, or, =C2=A0a twenty-five percent chance at that moment of b=
eing included in the block; it will likely be included in one of the next f=
our blocks, getting more likely each chance. If it is still not included th=
en the value of time waiting will be higher, making for more probability. A=
 very low fee transaction would have a value for the fee of one. It would n=
ot be until near sixty-days that the particular low fee transaction has a h=
igh likelihood of being included in the block.<br>&gt;<br>&gt; I am not con=
cerned with low (or high) transaction fees, the primary reason for addressi=
ng the issue is to ensure transactional reliability and scalability while h=
aving each transaction confirm in due time.<br>&gt;<br>&gt; ## Pros:<br>&gt=
; * Maximizes transaction reliability.<br>&gt; * Fully scalable.<br>&gt; * =
Maximizes possibility for consumer and business uptake.<br>&gt; * Maximizes=
 total fees paid per block without reducing reliability; because of reliabi=
lity, in time confidence and overall uptake are greater; therefore, more tr=
ansactions.<br>&gt; * Market determines fee paid for transaction priority.<=
br>&gt; * Fee recommendations work all the way out to 30 days or greater.<b=
r>&gt; * Provides additional block entropy; greater security since there is=
 less probability of predicting the next block.<br>&gt;<br>&gt; ## Cons:<br=
>&gt; * Could initially lower total transaction fees per block.<br>&gt; * M=
ust be first be programmed.<br>&gt;<br>&gt; ## Solution operation:<br>&gt; =
This is a simplistic view of the operation. The actual operation will need =
to be determined in a spec for the programmer.<br>&gt;<br>&gt; 1. Determine=
 the target block size for the current block.<br>&gt; 2. Assign a transacti=
on priority to each transaction in the pool.<br>&gt; 3. Select transactions=
 to include in the current block using probability in transaction priority =
order until the target block size is met.<br>&gt; 5. Solve block.<br>&gt; 6=
. Broadcast the next target block size with the current block when it is so=
lved.<br>&gt; 7. Block is received.<br>&gt; 8. Block verification process.<=
br>&gt; 9. Accept/reject block based on verification result.<br>&gt; 10. Re=
peat.<br>&gt;<br>&gt; ## Closing comments:<br>&gt; It may be possible to ve=
rify blocks conform to the proposal by showing that the probability for all=
 transactions included in the block statistically conforms to a probability=
 distribution curve, *if* the individual transaction priority can be recrea=
ted. I am not that deep into the mathematics; however, it may also be possi=
ble to use a similar method to do this just based on the fee, that statisti=
cally, the blocks conform to a fee distribution. Any zero fee transactions =
would have to be ignored. This solution needs a clever mathematician.<br>&g=
t;<br>&gt; I implore, at the very least, that we use some method that valid=
ates full transaction reliability and enables scalability of block sizes. I=
f not this proposal, an alternative.<br>&gt;<br>&gt; Regards,<br>&gt; Damia=
n Williamson<br>&gt;<br>&gt;<br>&gt;<br>&gt;<br>&gt; ______________________=
_________________________<br>&gt; bitcoin-dev mailing list<br>&gt; <a href=
=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfo=
undation.org</a><br>&gt; <a href=3D"https://lists.linuxfoundation.org/mailm=
an/listinfo/bitcoin-dev">https://lists.linuxfoundation.org/mailman/listinfo=
/bitcoin-dev</a><br>&gt;<br></div></div>

--001a113f6514a5a01505609c3b1f--