summaryrefslogtreecommitdiff
path: root/db/a3d431adcf3f2cffc07e572e91b76f6ce09725
blob: 922d11d2a61e62662d00948506817893cd8144a3 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
Return-Path: <willtech@live.com.au>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id 4327372A
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu,  4 Jan 2018 09:01:17 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from APC01-PU1-obe.outbound.protection.outlook.com
	(mail-oln040092254052.outbound.protection.outlook.com [40.92.254.52])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F215814D
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu,  4 Jan 2018 09:01:12 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; 
	h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
	bh=TnugCOSLlKjldm3dkY50j/rS0pTuRgVMjGdqykF4b0Y=;
	b=JRloUSaJWUp2Tbj/qDPUSBdvJbe4onZ414rYWheXvHfN7IoMrVtI5JiEi6fc4BCaLVHjYoI1Y+HEl/O2zfo6SGp3QGtkBLQHquGqD9uFyefi8ISIpXiaBII3UzSoWlhyPtzwv/BYlk8Aye9ras4XTsBBpM7cO6TXlL/rVQEKYB2L+L8Wj60wi5g2rCz5Tj8D/JUXf7Ufn8emfr0SlglAp2m3+YUzDlRvCQqW5Darl6xfr6QkKKN0w0MMBIUdyEw849qd7h8YohPQ9WV2+Koqtxm3EsUWBu62ONHJUHSYtR2bM0Im5ZwW8gpONR4jV0cJRgI7QCLEa4zhoqPyBC+ZBw==
Received: from SG2APC01FT045.eop-APC01.prod.protection.outlook.com
	(10.152.250.55) by SG2APC01HT112.eop-APC01.prod.protection.outlook.com
	(10.152.251.226) with Microsoft SMTP Server (version=TLS1_2,
	cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.302.6;
	Thu, 4 Jan 2018 09:01:09 +0000
Received: from PS2P216MB0179.KORP216.PROD.OUTLOOK.COM (10.152.250.57) by
	SG2APC01FT045.mail.protection.outlook.com (10.152.251.135) with
	Microsoft SMTP Server (version=TLS1_2,
	cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.345.12 via
	Frontend Transport; Thu, 4 Jan 2018 09:01:10 +0000
Received: from PS2P216MB0179.KORP216.PROD.OUTLOOK.COM ([10.171.225.19]) by
	PS2P216MB0179.KORP216.PROD.OUTLOOK.COM ([10.171.225.19]) with mapi id
	15.20.0366.009; Thu, 4 Jan 2018 09:01:10 +0000
From: Damian Williamson <willtech@live.com.au>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Thread-Topic: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
	Priority For Ordering Transactions In Blocks
Thread-Index: AQHTgvBb8N0JH3cL9Uqi71Bvfy6OVqNja4nV
Date: Thu, 4 Jan 2018 09:01:10 +0000
Message-ID: <PS2P216MB0179F13FF364666D92E6B2F89D1F0@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
References: <PS2P216MB0179D103B5D93555D166E74F9D180@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
In-Reply-To: <PS2P216MB0179D103B5D93555D166E74F9D180@PS2P216MB0179.KORP216.PROD.OUTLOOK.COM>
Accept-Language: en-AU, en-US
Content-Language: en-AU
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-incomingtopheadermarker: OriginalChecksum:36CDFB29954F18AE5110379C4359AEECE4E4BD2F493442D16BFB811DA4E134A9;
	UpperCasedChecksum:CF62FAB4A97ADDFAD5AB2380FD1C30BCFE352BB93FDD6CD94D7EBA00099D2459;
	SizeAsReceived:7190; Count:46
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [KV9OVIvY9eAjP00nC3Oilt7iXj6mZK6Z]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; SG2APC01HT112;
	6:yiCtBblOMu6sPT/p9Edrl2OuCQ3iVYrE7b1KD5JrUHU9eBHHqGV3zrsy+q98kvBhXak2BSaVPbNTiTReq9+yF0JVIJc1IGLPIKn5kIxJfh4jGApO/lJyJpe+O9/OoS696BlhbVRMCd2A29oMwBXmm3vIsAg1IR12yY4coMM8prur6RKmCpa97WgQ/uefU32/sPnTseZBB6XuyEcgbjCajDRgO7WMY9WPYQqm17RuZ4xcz9jRRVYWgiyS9oYRk4qL5iH37J86GBlJoLxKHIYM7NlQfHZtc/sTCamhfI7sXe09Lj8TvbdtTwRcp745Ww7JxvteAOsNDTLSIc2ykeueHPsvgCCkKKjm5U16D6IEqys=;
	5:Te7hVTzVObMOdMLPLnjeCdbgjsU7nIslpuVOXL+OiI0q8iGtFXa3oqztj1P2An4a7YfRXViE+OswfiGTUQjX/J2Ou9j2CJOqH5OHre5ULeNT62e9d7AxrerwPfK+ai2+8QUB+BiAK7oKzICf6Pn4yk6gJiyWH+moWDelM5eKyHg=;
	24:aHQIrx0UCZt5DV+mn1LE1TTRSX58XgnPNe6hdAOvlY7VoTOMMAgdoSgOSHXa/WNK0xZtPisbtNxECd07NhFEqPuGEg9gD68SpbotlHS4JUA=;
	7:677zJlWEQxcGiF3JkDViLJ0eURWue+IWEE3UbO//SZWb15vRS5Z2xKgBGW8ICudciqUjEW+y25LOIY/Now0A/1oFps2ivSHqmFpz7d2sVOQ3tNaMkN9llnhHtK2DQNyu13AfxXe5iPlemBu9r7UFJ/KN9wKxitGu6YNmvlrtbfJ54dNeegUgAC/bWYcQfTdGJnZpR7u0Ji5yXwy7GSHAooienCHxauAzPVWCk7sUhA+VS6ASYn9WUrsL2iHmHa3E
x-incomingheadercount: 46
x-eopattributedmessage: 0
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
	RULEID:(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125374)(1701031045);
	SRVR:SG2APC01HT112; 
x-ms-traffictypediagnostic: SG2APC01HT112:
x-ms-office365-filtering-correlation-id: 75136429-d6a7-44e0-6b19-08d55351b2e5
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031);
	SRVR:SG2APC01HT112; BCL:0; PCL:0;
	RULEID:(100000803101)(100110400095); SRVR:SG2APC01HT112; 
x-forefront-prvs: 054231DC40
x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT;
	SFP:1901; SCL:1; SRVR:SG2APC01HT112;
	H:PS2P216MB0179.KORP216.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:;
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_PS2P216MB0179F13FF364666D92E6B2F89D1F0PS2P216MB0179KORP_"
MIME-Version: 1.0
X-OriginatorOrg: outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 75136429-d6a7-44e0-6b19-08d55351b2e5
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jan 2018 09:01:10.1004 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT112
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Thu, 04 Jan 2018 13:36:08 +0000
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction
 Priority For Ordering Transactions In Blocks
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Jan 2018 09:01:17 -0000

--_000_PS2P216MB0179F13FF364666D92E6B2F89D1F0PS2P216MB0179KORP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

This proposal has a new update, mostly minor edits. Additionally, I had a l=
ogic flaw in the hard fork / soft fork declaration statement. The specific =
terms of the CC-BY-SA-4.0 licence the document is published under have now =
been updated to include additional permissions available under the MIT lice=
nce.


Recently, on Twitter:

I am looking for a capable analyst/programmer to work on a BIP proposal as =
co-author. Will need to format several Full BIP's per these BIP process req=
uirements: ( https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki=
 ) from a BIP Proposal, being two initially for non-consensus full-interope=
rable pre-rollout on peer service layer & API/RPC layer and, a reference im=
plementation for Bitcoin Core per: ( https://github.com/bitcoin/bitcoin/blo=
b/master/CONTRIBUTING.md ). Interested parties please reply via this list t=
hread: ( https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-Decem=
ber/015485.html ) #Bitcoin #BIP


Regards,

Damian Williamson


________________________________
From: bitcoin-dev-bounces@lists.linuxfoundation.org <bitcoin-dev-bounces@li=
sts.linuxfoundation.org> on behalf of Damian Williamson via bitcoin-dev <bi=
tcoin-dev@lists.linuxfoundation.org>
Sent: Monday, 1 January 2018 10:04 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Pr=
iority For Ordering Transactions In Blocks

Happy New Year all.

This proposal has been further amended with several minor changes and a
few additions.

I believe that all known issues raised so far have been sufficiently
addressed. Either that or, I still have more work to do.

## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks

Schema:
##########
Document: BIP Proposal
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks
Published: 26-12-2017
Revised: 01-01-2018
Author: Damian Williamson <willtech@live.com.au>
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order
##########

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transaction bandwidth limit.
2.  The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.

It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.

It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.

The previous threads for this proposal are available here:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html

In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.00001000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.

In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.

It is generally assumed that miners currently prefer to include
transactions with higher fees.

### 2. The need for this proposal

We all must learn to admit that transaction bandwidth is still lurking
as a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has
still not confirmed after now more than six days - even waiting twice
as long seems quite reasonable to me (note for accuracy: it did
eventually confirm). That transaction is a valid transaction; it is not
rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely
it is ever to confirm due to rising transaction numbers and being
pushed back by transactions with rising fees.

I argue that no transactions with fees above the dust level are rubbish
or junk, only some zero fee transactions might be spam. Having an ever-
increasing number of valid transactions that do not confirm as more new
transactions with higher fees are created is the opposite of operating
a robust, reliable transaction system.

While the miners have discovered a gold mine, it is the service they
provide that is valuable. If the service is unreliable they are not
worth the gold that they mine. This is reflected in the value of
Bitcoin.

Business cannot operate with a model where transactions may or may not
confirm. Even a business choosing a modest fee has no guarantee that
their valid transaction will not be shuffled down by new transactions
to the realm of never confirming after it is created. Consumers also
will not accept this model as Bitcoin expands. If Bitcoin cannot be a
reliable payment system for confirmed transactions then consumers, by
and large, will simply not accept the model once they understand.
Bitcoin will be a dirty payment system, and this will kill the value of
Bitcoin.

Under the current system, a minority of transactions will eventually be
the lucky few who have fees high enough to escape being pushed down the
list.

Once there are more than x transactions (transaction bandwidth limit)
every ten minutes, only those choosing twenty-minute confirmation (2
blocks) from the fee recommendations will have initially at most a
fifty percent chance of ever having their payment confirm by the time
2x transactions is reached. Presently, not even using fee
recommendations can ensure a sufficiently high fee is paid to ensure
transaction confirmation.

I also argue that the current auction model for limited transaction
bandwidth is wrong, is not suitable for a reliable transaction system
and, is wrong for Bitcoin. All transactions with valid fees must
confirm in due time. Currently, Bitcoin is not a safe way to send
payments.

I do not believe that consumers and business are against paying fees,
even high fees. What is required is operational reliability.

This great issue needs to be resolved for the safety and reliability of
Bitcoin. The time to resolve issues in commerce is before they become
great big issues. The time to resolve this issue is now. We must have
the foresight to identify and resolve problems before they trip us
over.  Simply doubling block sizes every so often is reactionary and is
not a reliable permanent solution.

I have written this proposal for a technical solution but, need your
help to write it up to an acceptable standard to be a full BIP.

### 3. The problem

Everybody wants value. Miners want to maximise revenue from fees (and
we presume, to minimise block size). Consumers need transaction
reliability and, (we presume) want low fees.

The current transaction bandwidth limit is a limiting factor for both.
As the operational safety of transactions is limited, so is consumer
confidence as they realise the issue and, accordingly, uptake is
limited. Fees are artificially inflated due to bandwidth limitations
while failing to provide a full confirmation service for all valid
transactions.

Current fee recommendations provide no satisfaction for transaction
reliability and, as Bitcoin scales, this will worsen.

Transactions are included in blocks by miners using whatever basis they
prefer. We expect that this is usually a fee-based priority. However,
even transactions with a valid fee may be left in the transaction pool
for some time. As transaction bandwidth becomes an issue, not even
extreme fees can ensure a transaction is processed in a timely manner
or at all.

Bitcoin must be a fully scalable and reliable service, providing full
transaction confirmation for every valid transaction.

The possibility to send a transaction with a fee lower than one that is
acceptable to allow eventual transaction confirmation should be removed
from the protocol and also from the user interface.

Bitcoin should be capable of reliably and inexpensively processing
casual transactions, and also priority processing of fee paying at
auction for priority transactions in the shortest possible timeframe.

### 4. Solution summary

#### Main solution

Provide each valid transaction in the mempool with an individual
transaction priority each time before choosing transactions to include
in the current block. The priority being a function of the fee (on a
curve), and the time waiting in the transaction pool (also on a curve)
out to n days (n =3D 60 days ?), and extending past n days. The value for
fee on a curve may need an upper limit. The transaction priority to
serve as the likelihood of a transaction being included in the current
block, and for determining the order in which transactions are tried to
see if they will be included.

Nodes will need to keep track of when a transaction is first seen. It
is satisfactory for each node to do this independently provided the
full mempool and information survives node restart. If there is a more
reliable way to determine when a transaction was first seen on the
network then it should be utilised.

> My current default installation of Bitcoin Core v0.15.1 does not
currently seem to save and load the mempool on restart, despite the
notes in the command line options panel that the default for
persistmempool is 1. In the debug panel, some 90,000 transactions
before restart, some 200 odd shortly after. Manually setting
persistmempool=3D1 in the conf file does not seem to make any difference.
Perhaps it is operating as expected and I am not sure what to observe,
but does not seem to be observably saving and loading the mempool on
restart. This will need to be resolved.

Use a dynamic target block size to make the current block. This marks a
shift from using block size or weight to a count of transactions.
Determine the target block size using; pre-rollout(current average
valid transaction pool size) x ( 1 / (144 x n days ) ) =3D number of
transactions to be included in the current block. The block created
should be a minimum 1MB in size regardless if the target block size is
lower.

If the created block size consistently contains too few transactions
and the number of new transactions created is continuously greater than
the block size will accommodate then I expect eventually ageing
transactions will be over-represented as a portion of the block
contents. Once another new node conforming to the proposal makes a
block, the block size will be proportionately larger as the transaction
pool has grown.  If block size is too large on average then this will
shrink the transaction pool.

Miners will likely want to conform to the proposal, since making blocks
larger than necessary makes more room in each block potentially
lowering the highest fees paid for priority service. Always making
blocks smaller than the proposal requires will in time lower the
utility value of Bitcoin, a different situation but akin to the
current. Transactions will still always confirm but with longer and
longer wait periods. The auction at the front of the queue for priority
will be destroyed as there will be eventually no room in blocks besides
ageing transations and, there will be little value paying higher than
the minimum fee. Obviously, neither of these scenarios are in a miner's
interests.

Without a consensus as to what size dynamic block to create,
enforcement of dynamic block size is not currently possible. It may be
possible for a consensus to be formed in the future but here I cannot
speculate. I can only suggest that it is in the interest of Bitcoin as
a whole and, in the interest of each node to conform to the proposal.
Some nodes failing to conform to the proposed requirements of dynamic
size or transaction priority in this proposal will not be destructive
to the operation of the proposal.

If necessary, nodes that have not yet adopted the proposal will just
continue to create standard fixed size unordered blocks, although, if
the current mechanisms of block validation include the fixed block size
then it is unlikely that these nodes will be able to validate the
blockchain going forward. In this case a hard fork and a full transfer
to the new method should be required. If dynamic blocks with ordered
transactions will be valid to existing nodes then only a soft fork is
required. There is no proposed change to the internal construction of
blocks, only to the block size and using an ordered method of
transaction selection.

> The default value for mempoolexpiry in Bitcoin Core may in future
need to be adjusted to match something more than n days or, perhaps
using less than n =3D 14 days may be a more sensible approach?

All block created with dynamic size should be verified to ensure
conformity to a probability distribution curve resulting from the
priority method. Since the input is a probability, the output should
conform to a probability distribution.

The curves used for the priority of transactions would have to be
appropriate. Perhaps a mathematician with experience in probability can
develop the right formulae. My thinking is a steep curve. I suppose
that the probability of all transactions should probably account for a
sufficient number of inclusions that the target block size is met on
average although, it may not always be. As a suggestion, consider
including some dust or zero-fee transactions to pad if each valid
transaction is tried and the target block size is not yet met, highest
BTC transaction value first?

**Explanation of the operation of priority:**

> If transaction priority is, for example, a number between one (low)
and one-hundred (high) it can be directly understood as the percentage
chance in one-hundred of a transaction being included in the block.
Using probability or likelihood infers that there is some function of
random. Try the transactions in priority order from highest to lowest,
if random (100) < transaction priority then the transaction is included
until the target block size is met.

> To break it down further, if both the fee on a curve value and the
time waiting on a curve value are each a number between one and one-
hundred, a rudimentary method may be to simply multiply those two
numbers, to find the priority number. For example, a middle fee
transaction waiting thirty days (if n =3D 60 days) may have a value of
five for each part  (yes, just five, the values are on a curve). When
multiplied that will give a priority value of twenty-five, or, a
twenty-five percent chance at that moment of being included in the
block; it will likely be included in one of the next four blocks,
getting more likely each chance. If it is still not included then the
value of time waiting will be higher, making for more probability. A
very low fee transaction would have a value for the fee of one. It
would not be until near sixty-days that the particular low fee
transaction has a high likelihood of being included in the block.

In practice it may be more useful to use numbers representative of one-
hundred for the highest fee priority curve down to a small fraction of
one for the lowest fee and, from one for a newly seen transaction up to
a proportionately high number above one-hundred for the time waiting
curve. It is truely beyond my level of math to resolve probability
curves accurately without much trial and error.

The primary reason for addressing the issue is to ensure transactional
reliability and scalability while having each valid transaction confirm
in due time.

#### Pros

*   Maximizes transaction reliability.
*   Overcomes transaction bandwidth limit.
*   Fully scalable.
*   Maximizes possibility for consumer and business uptake.
*   Maximizes total fees paid per block without reducing reliability;
because of reliability, in time confidence and overall uptake are
greater; therefore, more transactions.
*   Market determines fee paid for transaction priority.
*   Fee recommendations work all the way out to 30 days or greater.
*   Provides additional block entropy; greater security since there is
less probability of predicting the next block. _Although this is not
necessary it is a product of the operation of this proposal._

#### Cons

*   Could initially lower total transaction fees per block.
*   Must be first be programmed.

#### Pre-rollout

Nodes need to have at a minimum a loose understanding of the average
(since there is no consensus) size of the transaction pool as a
requirement to enable future changes to the way blocks are constructed.

A new network service should be constructed to meet this need. This
service makes no changes to any existing operation or function of the
node. Initially, Bitcoin Core is a suitable candidate.

For all operations we count only valid transactions.

**The service must:**

*   Have an individual temporary (runtime permanent only) Serial Node
ID.
*   Accept communication of the number of valid transactions in the
mempool of another valid Bitcoin node along with the Serial Node ID of
the node whose value is provided.
*   Disconnect the service from any non-Bitcoin node. Bitcoin Core may
handle this already?
*   Expire any value not updated for k minutes (k =3D 30 minutes?).
*   Broadcast all mempool information the node has every m minutes (m =3D
10 minutes?), including its own.
*   Nodes own mempool information should not be broadcast or used in
calculation until the node has been up long enough for the mempool to
normalise for at least o minutes (o =3D 300 minutes ?)
*   Alternatively, if loading nodes own full mempool from disk on node
restart (o =3D 30 minutes ?)
*   Only new or updated mempool values should be transmitted to the
same node. Updated includes updated with no change.
*   All known mempool information must survive node restart.
*   If the nodes own mempool is not normalised and network information
is not available to calculate an average just display zero.
*   Internally, the average transaction pool size must return the
calculated average if an average is available or, if none is available
just the number of valid transactions in the node's own mempool
regardless if it is normalised.

Bitcoin Core must use all collated information on mempool size to
calculate a figure for the average mempool size.

The calculated figure should be displayed in the appropriate place in
the Debug window alongside the text Network average transactions.

Consideration must be given before development of the network bandwidth
this would require. All programming must be consistent with the current
operation and conventions of Bitcoin Core. Methods must work on all
platforms.

As this new service does not affect any existing service or feature of
Bitcoin or Bitcoin Core, this can technically be programmed now and
included in Bitcoin Core at any time.

### 5. Solution operation

This is a simplistic view of the operation. The actual operation will
need to be determined accurately in a spec for the programmer.

1.  Determine the target block size for the current block.
2.  Assign a transaction priority to each valid transaction in the
mempool.
3.  Select transactions to include in the current block using
probability in transaction priority order until the target block size
is met. If target block size is not met, include dust and zero-fee
transactions to pad.
4.  Solve block.
5.  Broadcast the current block when it is solved.
6.  Block is received.
7.  Block verification process.
8.  Accept/reject block based on verification result.
9.  Repeat.

### 6. Closing comments

It may be possible to verify blocks conform to the proposal by showing
that the probability for all transactions included in the block
statistically conforms to a probability distribution curve, *if* the
individual transaction priority can be recreated. I am not that deep
into the mathematics; however, it may also be possible to use a similar
method to do this just based on the fee, that statistically, the block
conforms to a fee distribution. Any dust and zero-fee transactions
would have to be ignored. This solution needs a competent mathematician
with experience in probability and statistical distribution.

It is trivial to this proposal to offer that a node provides the next
block size with a block when it is solved. I am not sure that this
creates any actual benefit since the provided next block size is only
one node's view, as it is the node may seemingly just as well use its
own view and create the block. Providing a next block size only adds
additional complexity to the required operation, however, perhaps
providing the next block size is not trivial in what is accomplished
and the feature can be included in the operation.

Instead of the pre-rollout network service providing data as to valid
transactions in mempool, it could directly provide data as to the
suggested next block size if that is preferred, using a similar
operation as is suggested now and averaging all received suggested next
block sizes.

It may be foreseeable in the future for Bitcoin to operate with a
network of dedicated full blockchain & mempool servers. This would not
be without challenges to overcome but would offer several benefits,
including to the operation of this proposal, and especially as the RAM
and storage requirements of a full node grows. It is easy to foresee
that in just another seven years of operation a Bitcoin Full Node will
require at least 300GB of storage and, if the mempool only doubles in
size, over 1GB of RAM.

There has been some concern expressed over spam and very low fee
transactions, and an infinite block size resulting. I hope that for
those concerned using the dust level addresses the issue, especially as
the value of Bitcoin grows.

Notwithstanding this proposal, all blocks including those with dynamic
size each have limited transaction space per block. This proposal
results in a fee for priority service auction, where the probability of
a transaction to be included in limited space in the next available
block is auctioned to the highest bidders and all other transactions
must wait until they reach priority by ageing to gain significant
probability. Under this proposal the mempool can grow quite large while
the confirmation service continues in a stable and reliable manner.
Several incentives for attackers are removed, where there is no longer
multiple potential incentives for unnecessarily filling blocks or
flooding the mempool with transactions, whether such transactions are
fraudulent, valid or, otherwise. Adoption of this proposal and
adherence results in a reliable, stable fee paying transaction
confirmation service and a beneficial auction.

This proposal is necessary. I implore, at the very least, that we use
some method that validates full transaction reliability and enables
scalability of Bitcoin. If not this proposal, an alternative.

I have done as much with this proposal as I feel that I am able so far
but continue to take your feedback.

Regards,
Damian Williamson

[![Creative Commons License](https://i.creativecommons.org/l/by-sa/4.0/
88x31.png)](http://creativecommons.org/licenses/by-sa/4.0/)
<span xmlns:dct=3D"http://purl.org/dc/terms/"
href=3D"http://purl.org/dc/dcmitype/Text" property=3D"dct:title"
rel=3D"dct:type">BIP Proposal: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks</span> by [Damian Williamson
&lt;willtech@live.com.au&gt;](http://thekingjameshrmh.tumblr.com/post/1
68948530950/bip-proposal-utpfotib-use-transaction-priority-for-order)
is licensed under a [Creative Commons Attribution-ShareAlike 4.0
International License](http://creativecommons.org/licenses/by-sa/4.0/).
Based on a work at https://lists.linuxfoundation.org/pipermail/bitcoin-
dev/2017-
December/015371.html](https://lists.linuxfoundation.org/pipermail/bitco
in-dev/2017-December/015371.html).
Permissions beyond the scope of this license may be available at [https
://opensource.org/licenses/BSD-3-
Clause](https://opensource.org/licenses/BSD-3-Clause).


_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

--_000_PS2P216MB0179F13FF364666D92E6B2F89D1F0PS2P216MB0179KORP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"><!-- P {margin-top:0;margi=
n-bottom:0;} --></style>
</head>
<body dir=3D"ltr">
<div id=3D"divtagdefaultwrapper" style=3D"font-size: 12pt; color: rgb(0, 0,=
 0); font-family: Calibri,Helvetica,sans-serif,&quot;EmojiFont&quot;,&quot;=
Apple Color Emoji&quot;,&quot;Segoe UI Emoji&quot;,NotoColorEmoji,&quot;Seg=
oe UI Symbol&quot;,&quot;Android Emoji&quot;,EmojiSymbols;" dir=3D"ltr">
<p style=3D"margin-top:0;margin-bottom:0">This proposal has a new update, m=
ostly minor edits. Additionally, I had a logic flaw in the hard fork / soft=
 fork declaration statement. The specific terms of the
<span>CC-BY-SA-4.0</span> licence the document is published under have now =
been updated to include additional permissions available under the MIT lice=
nce.</p>
<p style=3D"margin-top:0;margin-bottom:0"><br>
</p>
<p style=3D"margin-top:0;margin-bottom:0">Recently, on Twitter:</p>
<p style=3D"margin-top:0;margin-bottom:0">I am looking for a capable analys=
t/programmer to work on a BIP proposal as co-author. Will need to format se=
veral Full BIP's per these BIP process requirements: ( https://github.com/b=
itcoin/bips/blob/master/bip-0002.mediawiki
 ) from a BIP Proposal, being two initially for non-consensus full-interope=
rable pre-rollout on peer service layer &amp; API/RPC layer and, a referenc=
e implementation for Bitcoin Core per: ( https://github.com/bitcoin/bitcoin=
/blob/master/CONTRIBUTING.md ). Interested
 parties please reply via this list thread: ( https://lists.linuxfoundation=
.org/pipermail/bitcoin-dev/2017-December/015485.html ) #Bitcoin #BIP</p>
<p style=3D"margin-top:0;margin-bottom:0"><br>
</p>
<p style=3D"margin-top:0;margin-bottom:0">Regards,</p>
<p style=3D"margin-top:0;margin-bottom:0">Damian Williamson<br>
</p>
<br>
<br>
<div style=3D"color: rgb(0, 0, 0);">
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font style=3D"font-size:11pt" face=
=3D"Calibri, sans-serif" color=3D"#000000"><b>From:</b> bitcoin-dev-bounces=
@lists.linuxfoundation.org &lt;bitcoin-dev-bounces@lists.linuxfoundation.or=
g&gt; on behalf of Damian Williamson via bitcoin-dev
 &lt;bitcoin-dev@lists.linuxfoundation.org&gt;<br>
<b>Sent:</b> Monday, 1 January 2018 10:04 PM<br>
<b>To:</b> bitcoin-dev@lists.linuxfoundation.org<br>
<b>Subject:</b> [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transac=
tion Priority For Ordering Transactions In Blocks</font>
<div>&nbsp;</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">Happy New Year all.<br>
<br>
This proposal has been further amended with several minor changes and a<br>
few additions.<br>
<br>
I believe that all known issues raised so far have been sufficiently<br>
addressed. Either that or, I still have more work to do.<br>
<br>
## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For<br>
Ordering Transactions In Blocks<br>
<br>
Schema: &nbsp;<br>
########## &nbsp;<br>
Document: BIP Proposal &nbsp;<br>
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In<br>
Blocks &nbsp;<br>
Published: 26-12-2017 &nbsp;<br>
Revised: 01-01-2018 &nbsp;<br>
Author: Damian Williamson &lt;willtech@live.com.au&gt;&nbsp;&nbsp;<br>
Licence: Creative Commons Attribution-ShareAlike 4.0 International<br>
License. &nbsp;<br>
URL: <a href=3D"http://thekingjameshrmh.tumblr.com/post/168948530950/bip-pr=
oposal-" id=3D"LPlnk655165" previewremoved=3D"true">
http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-</a><br>
utpfotib-use-transaction-priority-for-order&nbsp;&nbsp;<br>
##########<br>
<br>
### 1. Abstract<br>
<br>
This document proposes to address the issue of transactional<br>
reliability in Bitcoin, where valid transactions may be stuck in the<br>
transaction pool for extended periods or never confirm.<br>
<br>
There are two key issues to be resolved to achieve this:<br>
<br>
1.&nbsp;&nbsp;The current transaction bandwidth limit.<br>
2.&nbsp;&nbsp;The current ad-hoc methods of including transactions in block=
s<br>
resulting in variable and confusing confirmation times for valid<br>
transactions, including transactions with a valid fee that may never<br>
confirm.<br>
<br>
It is important with any change to protect the value of fees as these<br>
will eventually be the only payment that miners receive. Rather than an<br>
auction model for limited bandwidth, the proposal results in a fee for<br>
priority service auction model.<br>
<br>
It would not be true to suggest that all feedback received so far has<br>
been entirely positive although, most of it has been constructive.<br>
<br>
The previous threads for this proposal are available here: &nbsp;<br>
<a href=3D"https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-Dec=
ember/s" id=3D"LPlnk453598" previewremoved=3D"true">https://lists.linuxfoun=
dation.org/pipermail/bitcoin-dev/2017-December/s</a><br>
ubject.html<br>
<br>
In all parts of this proposal, references to a transaction, a valid<br>
transaction, a transaction with a valid fee, a valid fee, etc. is<br>
defined as any transaction that is otherwise valid with a fee of at<br>
least 0.00001000 BTC/KB as defined as the dust level, interpreting from<br>
Bitcoin Core GUI. Transactions with a fee lower than this rate are<br>
considered dust.<br>
<br>
In all parts of this proposal, dust and zero-fee transactions are<br>
always ignored and/or excluded unless specifically mentioned.<br>
<br>
It is generally assumed that miners currently prefer to include<br>
transactions with higher fees.<br>
<br>
### 2. The need for this proposal<br>
<br>
We all must learn to admit that transaction bandwidth is still lurking<br>
as a serious issue for the operation, reliability, safety, consumer<br>
acceptance, uptake and, for the value of Bitcoin.<br>
<br>
I recently sent a payment which was not urgent so; I chose three-day<br>
target confirmation from the fee recommendation. That transaction has<br>
still not confirmed after now more than six days - even waiting twice<br>
as long seems quite reasonable to me (note for accuracy: it did<br>
eventually confirm). That transaction is a valid transaction; it is not<br>
rubbish, junk or, spam. Under the current model with transaction<br>
bandwidth limitation, the longer a transaction waits, the less likely<br>
it is ever to confirm due to rising transaction numbers and being<br>
pushed back by transactions with rising fees.<br>
<br>
I argue that no transactions with fees above the dust level are rubbish<br>
or junk, only some zero fee transactions might be spam. Having an ever-<br>
increasing number of valid transactions that do not confirm as more new<br>
transactions with higher fees are created is the opposite of operating<br>
a robust, reliable transaction system.<br>
<br>
While the miners have discovered a gold mine, it is the service they<br>
provide that is valuable. If the service is unreliable they are not<br>
worth the gold that they mine. This is reflected in the value of<br>
Bitcoin.<br>
<br>
Business cannot operate with a model where transactions may or may not<br>
confirm. Even a business choosing a modest fee has no guarantee that<br>
their valid transaction will not be shuffled down by new transactions<br>
to the realm of never confirming after it is created. Consumers also<br>
will not accept this model as Bitcoin expands. If Bitcoin cannot be a<br>
reliable payment system for confirmed transactions then consumers, by<br>
and large, will simply not accept the model once they understand.<br>
Bitcoin will be a dirty payment system, and this will kill the value of<br>
Bitcoin.<br>
<br>
Under the current system, a minority of transactions will eventually be<br>
the lucky few who have fees high enough to escape being pushed down the<br>
list.<br>
<br>
Once there are more than x transactions (transaction bandwidth limit)<br>
every ten minutes, only those choosing twenty-minute confirmation (2<br>
blocks) from the fee recommendations will have initially at most a<br>
fifty percent chance of ever having their payment confirm by the time<br>
2x transactions is reached. Presently, not even using fee<br>
recommendations can ensure a sufficiently high fee is paid to ensure<br>
transaction confirmation.<br>
<br>
I also argue that the current auction model for limited transaction<br>
bandwidth is wrong, is not suitable for a reliable transaction system<br>
and, is wrong for Bitcoin. All transactions with valid fees must<br>
confirm in due time. Currently, Bitcoin is not a safe way to send<br>
payments.<br>
<br>
I do not believe that consumers and business are against paying fees,<br>
even high fees. What is required is operational reliability.<br>
<br>
This great issue needs to be resolved for the safety and reliability of<br>
Bitcoin. The time to resolve issues in commerce is before they become<br>
great big issues. The time to resolve this issue is now. We must have<br>
the foresight to identify and resolve problems before they trip us<br>
over.&nbsp;&nbsp;Simply doubling block sizes every so often is reactionary =
and is<br>
not a reliable permanent solution.<br>
<br>
I have written this proposal for a technical solution but, need your<br>
help to write it up to an acceptable standard to be a full BIP.<br>
<br>
### 3. The problem<br>
<br>
Everybody wants value. Miners want to maximise revenue from fees (and<br>
we presume, to minimise block size). Consumers need transaction<br>
reliability and, (we presume) want low fees.<br>
<br>
The current transaction bandwidth limit is a limiting factor for both.<br>
As the operational safety of transactions is limited, so is consumer<br>
confidence as they realise the issue and, accordingly, uptake is<br>
limited. Fees are artificially inflated due to bandwidth limitations<br>
while failing to provide a full confirmation service for all valid<br>
transactions.<br>
<br>
Current fee recommendations provide no satisfaction for transaction<br>
reliability and, as Bitcoin scales, this will worsen.<br>
<br>
Transactions are included in blocks by miners using whatever basis they<br>
prefer. We expect that this is usually a fee-based priority. However,<br>
even transactions with a valid fee may be left in the transaction pool<br>
for some time. As transaction bandwidth becomes an issue, not even<br>
extreme fees can ensure a transaction is processed in a timely manner<br>
or at all.<br>
<br>
Bitcoin must be a fully scalable and reliable service, providing full<br>
transaction confirmation for every valid transaction.<br>
<br>
The possibility to send a transaction with a fee lower than one that is<br>
acceptable to allow eventual transaction confirmation should be removed<br>
from the protocol and also from the user interface.<br>
<br>
Bitcoin should be capable of reliably and inexpensively processing<br>
casual transactions, and also priority processing of fee paying at<br>
auction for priority transactions in the shortest possible timeframe.<br>
<br>
### 4. Solution summary<br>
<br>
#### Main solution<br>
<br>
Provide each valid transaction in the mempool with an individual<br>
transaction priority each time before choosing transactions to include<br>
in the current block. The priority being a function of the fee (on a<br>
curve), and the time waiting in the transaction pool (also on a curve)<br>
out to n days (n =3D 60 days ?), and extending past n days. The value for<b=
r>
fee on a curve may need an upper limit. The transaction priority to<br>
serve as the likelihood of a transaction being included in the current<br>
block, and for determining the order in which transactions are tried to<br>
see if they will be included.<br>
<br>
Nodes will need to keep track of when a transaction is first seen. It<br>
is satisfactory for each node to do this independently provided the<br>
full mempool and information survives node restart. If there is a more<br>
reliable way to determine when a transaction was first seen on the<br>
network then it should be utilised.<br>
<br>
&gt; My current default installation of Bitcoin Core v0.15.1 does not<br>
currently seem to save and load the mempool on restart, despite the<br>
notes in the command line options panel that the default for<br>
persistmempool is 1. In the debug panel, some 90,000 transactions<br>
before restart, some 200 odd shortly after. Manually setting<br>
persistmempool=3D1 in the conf file does not seem to make any difference.<b=
r>
Perhaps it is operating as expected and I am not sure what to observe,<br>
but does not seem to be observably saving and loading the mempool on<br>
restart. This will need to be resolved.<br>
<br>
Use a dynamic target block size to make the current block. This marks a<br>
shift from using block size or weight to a count of transactions.<br>
Determine the target block size using; pre-rollout(current average<br>
valid transaction pool size) x ( 1 / (144 x n days ) ) =3D number of<br>
transactions to be included in the current block. The block created<br>
should be a minimum 1MB in size regardless if the target block size is<br>
lower.<br>
<br>
If the created block size consistently contains too few transactions<br>
and the number of new transactions created is continuously greater than<br>
the block size will accommodate then I expect eventually ageing<br>
transactions will be over-represented as a portion of the block<br>
contents. Once another new node conforming to the proposal makes a<br>
block, the block size will be proportionately larger as the transaction<br>
pool has grown.&nbsp;&nbsp;If block size is too large on average then this =
will<br>
shrink the transaction pool.<br>
<br>
Miners will likely want to conform to the proposal, since making blocks<br>
larger than necessary makes more room in each block potentially<br>
lowering the highest fees paid for priority service. Always making<br>
blocks smaller than the proposal requires will in time lower the<br>
utility value of Bitcoin, a different situation but akin to the<br>
current. Transactions will still always confirm but with longer and<br>
longer wait periods. The auction at the front of the queue for priority<br>
will be destroyed as there will be eventually no room in blocks besides<br>
ageing transations and, there will be little value paying higher than<br>
the minimum fee. Obviously, neither of these scenarios are in a miner's<br>
interests.<br>
<br>
Without a consensus as to what size dynamic block to create,<br>
enforcement of dynamic block size is not currently possible. It may be<br>
possible for a consensus to be formed in the future but here I cannot<br>
speculate. I can only suggest that it is in the interest of Bitcoin as<br>
a whole and, in the interest of each node to conform to the proposal.<br>
Some nodes failing to conform to the proposed requirements of dynamic<br>
size or transaction priority in this proposal will not be destructive<br>
to the operation of the proposal.<br>
<br>
If necessary, nodes that have not yet adopted the proposal will just<br>
continue to create standard fixed size unordered blocks, although, if<br>
the current mechanisms of block validation include the fixed block size<br>
then it is unlikely that these nodes will be able to validate the<br>
blockchain going forward. In this case a hard fork and a full transfer<br>
to the new method should be required. If dynamic blocks with ordered<br>
transactions will be valid to existing nodes then only a soft fork is<br>
required. There is no proposed change to the internal construction of<br>
blocks, only to the block size and using an ordered method of<br>
transaction selection.<br>
<br>
&gt; The default value for mempoolexpiry in Bitcoin Core may in future<br>
need to be adjusted to match something more than n days or, perhaps<br>
using less than n =3D 14 days may be a more sensible approach?<br>
<br>
All block created with dynamic size should be verified to ensure<br>
conformity to a probability distribution curve resulting from the<br>
priority method. Since the input is a probability, the output should<br>
conform to a probability distribution.<br>
<br>
The curves used for the priority of transactions would have to be<br>
appropriate. Perhaps a mathematician with experience in probability can<br>
develop the right formulae. My thinking is a steep curve. I suppose<br>
that the probability of all transactions should probably account for a<br>
sufficient number of inclusions that the target block size is met on<br>
average although, it may not always be. As a suggestion, consider<br>
including some dust or zero-fee transactions to pad if each valid<br>
transaction is tried and the target block size is not yet met, highest<br>
BTC transaction value first?<br>
<br>
**Explanation of the operation of priority:**<br>
<br>
&gt; If transaction priority is, for example, a number between one (low)<br=
>
and one-hundred (high) it can be directly understood as the percentage<br>
chance in one-hundred of a transaction being included in the block.<br>
Using probability or likelihood infers that there is some function of<br>
random. Try the transactions in priority order from highest to lowest,<br>
if random (100) &lt; transaction priority then the transaction is included<=
br>
until the target block size is met.&nbsp;<br>
<br>
&gt; To break it down further, if both the fee on a curve value and the<br>
time waiting on a curve value are each a number between one and one-<br>
hundred, a rudimentary method may be to simply multiply those two<br>
numbers, to find the priority number. For example, a middle fee<br>
transaction waiting thirty days (if n =3D 60 days) may have a value of<br>
five for each part&nbsp;&nbsp;(yes, just five, the values are on a curve). =
When<br>
multiplied that will give a priority value of twenty-five, or, a<br>
twenty-five percent chance at that moment of being included in the<br>
block; it will likely be included in one of the next four blocks,<br>
getting more likely each chance. If it is still not included then the<br>
value of time waiting will be higher, making for more probability. A<br>
very low fee transaction would have a value for the fee of one. It<br>
would not be until near sixty-days that the particular low fee<br>
transaction has a high likelihood of being included in the block.<br>
<br>
In practice it may be more useful to use numbers representative of one-<br>
hundred for the highest fee priority curve down to a small fraction of<br>
one for the lowest fee and, from one for a newly seen transaction up to<br>
a proportionately high number above one-hundred for the time waiting<br>
curve. It is truely beyond my level of math to resolve probability<br>
curves accurately without much trial and error.<br>
<br>
The primary reason for addressing the issue is to ensure transactional<br>
reliability and scalability while having each valid transaction confirm<br>
in due time.<br>
<br>
#### Pros<br>
<br>
*&nbsp;&nbsp;&nbsp;Maximizes transaction reliability.<br>
*&nbsp;&nbsp;&nbsp;Overcomes transaction bandwidth limit.<br>
*&nbsp;&nbsp;&nbsp;Fully scalable.<br>
*&nbsp;&nbsp;&nbsp;Maximizes possibility for consumer and business uptake.<=
br>
*&nbsp;&nbsp;&nbsp;Maximizes total fees paid per block without reducing rel=
iability;<br>
because of reliability, in time confidence and overall uptake are<br>
greater; therefore, more transactions.<br>
*&nbsp;&nbsp;&nbsp;Market determines fee paid for transaction priority.<br>
*&nbsp;&nbsp;&nbsp;Fee recommendations work all the way out to 30 days or g=
reater.<br>
*&nbsp;&nbsp;&nbsp;Provides additional block entropy; greater security sinc=
e there is<br>
less probability of predicting the next block. _Although this is not<br>
necessary it is a product of the operation of this proposal._<br>
<br>
#### Cons<br>
<br>
*&nbsp;&nbsp;&nbsp;Could initially lower total transaction fees per block.<=
br>
*&nbsp;&nbsp;&nbsp;Must be first be programmed.<br>
<br>
#### Pre-rollout<br>
<br>
Nodes need to have at a minimum a loose understanding of the average<br>
(since there is no consensus) size of the transaction pool as a<br>
requirement to enable future changes to the way blocks are constructed.<br>
<br>
A new network service should be constructed to meet this need. This<br>
service makes no changes to any existing operation or function of the<br>
node. Initially, Bitcoin Core is a suitable candidate.<br>
<br>
For all operations we count only valid transactions.<br>
<br>
**The service must:**<br>
<br>
*&nbsp;&nbsp;&nbsp;Have an individual temporary (runtime permanent only) Se=
rial Node<br>
ID.<br>
*&nbsp;&nbsp;&nbsp;Accept communication of the number of valid transactions=
 in the<br>
mempool of another valid Bitcoin node along with the Serial Node ID of<br>
the node whose value is provided.<br>
*&nbsp;&nbsp;&nbsp;Disconnect the service from any non-Bitcoin node. Bitcoi=
n Core may<br>
handle this already?<br>
*&nbsp;&nbsp;&nbsp;Expire any value not updated for k minutes (k =3D 30 min=
utes?).<br>
*&nbsp;&nbsp;&nbsp;Broadcast all mempool information the node has every m m=
inutes (m =3D<br>
10 minutes?), including its own.<br>
*&nbsp;&nbsp;&nbsp;Nodes own mempool information should not be broadcast or=
 used in<br>
calculation until the node has been up long enough for the mempool to<br>
normalise for at least o minutes (o =3D 300 minutes ?)<br>
*&nbsp;&nbsp;&nbsp;Alternatively, if loading nodes own full mempool from di=
sk on node<br>
restart (o =3D 30 minutes ?)<br>
*&nbsp;&nbsp;&nbsp;Only new or updated mempool values should be transmitted=
 to the<br>
same node. Updated includes updated with no change.<br>
*&nbsp;&nbsp;&nbsp;All known mempool information must survive node restart.=
<br>
*&nbsp;&nbsp;&nbsp;If the nodes own mempool is not normalised and network i=
nformation<br>
is not available to calculate an average just display zero.<br>
*&nbsp;&nbsp;&nbsp;Internally, the average transaction pool size must retur=
n the<br>
calculated average if an average is available or, if none is available<br>
just the number of valid transactions in the node's own mempool<br>
regardless if it is normalised.<br>
<br>
Bitcoin Core must use all collated information on mempool size to<br>
calculate a figure for the average mempool size.<br>
<br>
The calculated figure should be displayed in the appropriate place in<br>
the Debug window alongside the text Network average transactions.<br>
<br>
Consideration must be given before development of the network bandwidth<br>
this would require. All programming must be consistent with the current<br>
operation and conventions of Bitcoin Core. Methods must work on all<br>
platforms.<br>
<br>
As this new service does not affect any existing service or feature of<br>
Bitcoin or Bitcoin Core, this can technically be programmed now and<br>
included in Bitcoin Core at any time.<br>
<br>
### 5. Solution operation<br>
<br>
This is a simplistic view of the operation. The actual operation will<br>
need to be determined accurately in a spec for the programmer.<br>
<br>
1.&nbsp;&nbsp;Determine the target block size for the current block.<br>
2.&nbsp;&nbsp;Assign a transaction priority to each valid transaction in th=
e<br>
mempool.<br>
3.&nbsp;&nbsp;Select transactions to include in the current block using<br>
probability in transaction priority order until the target block size<br>
is met. If target block size is not met, include dust and zero-fee<br>
transactions to pad.<br>
4.&nbsp;&nbsp;Solve block.<br>
5.&nbsp;&nbsp;Broadcast the current block when it is solved.<br>
6.&nbsp;&nbsp;Block is received.<br>
7.&nbsp;&nbsp;Block verification process.<br>
8.&nbsp;&nbsp;Accept/reject block based on verification result.<br>
9.&nbsp;&nbsp;Repeat.<br>
<br>
### 6. Closing comments<br>
<br>
It may be possible to verify blocks conform to the proposal by showing<br>
that the probability for all transactions included in the block<br>
statistically conforms to a probability distribution curve, *if* the<br>
individual transaction priority can be recreated. I am not that deep<br>
into the mathematics; however, it may also be possible to use a similar<br>
method to do this just based on the fee, that statistically, the block<br>
conforms to a fee distribution. Any dust and zero-fee transactions<br>
would have to be ignored. This solution needs a competent mathematician<br>
with experience in probability and statistical distribution.<br>
<br>
It is trivial to this proposal to offer that a node provides the next<br>
block size with a block when it is solved. I am not sure that this<br>
creates any actual benefit since the provided next block size is only<br>
one node's view, as it is the node may seemingly just as well use its<br>
own view and create the block. Providing a next block size only adds<br>
additional complexity to the required operation, however, perhaps<br>
providing the next block size is not trivial in what is accomplished<br>
and the feature can be included in the operation.<br>
<br>
Instead of the pre-rollout network service providing data as to valid<br>
transactions in mempool, it could directly provide data as to the<br>
suggested next block size if that is preferred, using a similar<br>
operation as is suggested now and averaging all received suggested next<br>
block sizes.<br>
<br>
It may be foreseeable in the future for Bitcoin to operate with a<br>
network of dedicated full blockchain &amp; mempool servers. This would not<=
br>
be without challenges to overcome but would offer several benefits,<br>
including to the operation of this proposal, and especially as the RAM<br>
and storage requirements of a full node grows. It is easy to foresee<br>
that in just another seven years of operation a Bitcoin Full Node will<br>
require at least 300GB of storage and, if the mempool only doubles in<br>
size, over 1GB of RAM.<br>
<br>
There has been some concern expressed over spam and very low fee<br>
transactions, and an infinite block size resulting. I hope that for<br>
those concerned using the dust level addresses the issue, especially as<br>
the value of Bitcoin grows.<br>
<br>
Notwithstanding this proposal, all blocks including those with dynamic<br>
size each have limited transaction space per block. This proposal<br>
results in a fee for priority service auction, where the probability of<br>
a transaction to be included in limited space in the next available<br>
block is auctioned to the highest bidders and all other transactions<br>
must wait until they reach priority by ageing to gain significant<br>
probability. Under this proposal the mempool can grow quite large while<br>
the confirmation service continues in a stable and reliable manner.<br>
Several incentives for attackers are removed, where there is no longer<br>
multiple potential incentives for unnecessarily filling blocks or<br>
flooding the mempool with transactions, whether such transactions are<br>
fraudulent, valid or, otherwise. Adoption of this proposal and<br>
adherence results in a reliable, stable fee paying transaction<br>
confirmation service and a beneficial auction.<br>
<br>
This proposal is necessary. I implore, at the very least, that we use<br>
some method that validates full transaction reliability and enables<br>
scalability of Bitcoin. If not this proposal, an alternative.<br>
<br>
I have done as much with this proposal as I feel that I am able so far<br>
but continue to take your feedback.<br>
<br>
Regards, &nbsp;<br>
Damian Williamson<br>
<br>
[![Creative Commons License](<a href=3D""></a>https://i.creativecommons.org=
/l/by-sa/4.0/<br>
88x31.png)](<a href=3D"http://creativecommons.org/licenses/by-sa/4.0/">http=
://creativecommons.org/licenses/by-sa/4.0/</a>) &nbsp;<br>
&lt;span xmlns:dct=3D&quot;<a href=3D"http://purl.org/dc/terms/">http://pur=
l.org/dc/terms/</a>&quot;<br>
href=3D&quot;<a href=3D"http://purl.org/dc/dcmitype/Text">http://purl.org/d=
c/dcmitype/Text</a>&quot; property=3D&quot;dct:title&quot;<br>
rel=3D&quot;dct:type&quot;&gt;BIP Proposal: UTPFOTIB - Use Transaction Prio=
rity For<br>
Ordering Transactions In Blocks&lt;/span&gt; by [Damian Williamson<br>
&amp;lt;willtech@live.com.au&amp;gt;](http://thekingjameshrmh.tumblr.com/po=
st/1<br>
68948530950/bip-proposal-utpfotib-use-transaction-priority-for-order)<br>
is licensed under a [Creative Commons Attribution-ShareAlike 4.0<br>
International License](<a href=3D"http://creativecommons.org/licenses/by-sa=
/4.0/">http://creativecommons.org/licenses/by-sa/4.0/</a>).<br>
Based on a work at <a href=3D"https://lists.linuxfoundation.org/pipermail/b=
itcoin-">
https://lists.linuxfoundation.org/pipermail/bitcoin-</a><br>
dev/2017-<br>
December/015371.html](https://lists.linuxfoundation.org/pipermail/bitco<br>
in-dev/2017-December/015371.html).<br>
Permissions beyond the scope of this license may be available at [https<br>
://opensource.org/licenses/BSD-3-<br>
Clause](<a href=3D"https://opensource.org/licenses/BSD-3-Clause">https://op=
ensource.org/licenses/BSD-3-Clause</a>).<br>
<br>
<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
bitcoin-dev@lists.linuxfoundation.org<br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev">=
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev</a><br>
</div>
</span></font></div>
</div>
</div>
</body>
</html>

--_000_PS2P216MB0179F13FF364666D92E6B2F89D1F0PS2P216MB0179KORP_--