From damian at willtech.com.au Mon Feb 14 05:19:18 2022 From: damian at willtech.com.au (damian at willtech.com.au) Date: Sun, 13 Feb 2022 21:19:18 -0800 Subject: [Lightning-dev] [bitcoin-dev] A suggestion to periodically destroy (or remove to secondary storage for Archiving reasons) dust, Non-standard UTXOs, and also detected burn In-Reply-To: References: <382073c28af1ec54827093003cbec2cc@willtech.com.au> Message-ID: <70f0577110da5cf2d3b4db6b5a6ac91c@willtech.com.au> Good Afternoon, I am briefly reading the suggestion subject titled this email message. The problem this idea addresses is not new, it is as old as computer science with complexity and security varying from unused products in a supermarket database to unused bank accounts and records of transactions for archival. In the former, it is of no consequence to remove unused products from the database if the itemised sales history is not necessary and the report data is maintained ie. where the reporting is stored generated. Once the products are removed it is impossible to regenerate the detailed reports that called on the product-specific and sales information. Archival works in a manner to remove data from commonly used tables and to optimise them and if the data is later required it can be recalled from larger possibly slower storage, or simply kept in a larger less optimised table, in either case, all of the data is still available. In the latter, it is a matter of consequence and although archival is possible it is still necessary to ensure that all archives are backed up, and the data can never be removed. This is because if in an example transaction data is deleted after seven years then it is no longer possible to see how an account has its balance and an empty account with no transactions may be removed while actually still holding a significant balance. If a mistake is made the result is the same. This is because we allowed deletion of accounting records. Actually, the recommendation from Computer Science all along is the same as our case with Bitcoin - nothing should be deleted from the Blockchain forever. For one substantiative reason, this is because it is necessary for any client to be able to validate the blockchain in its entirety and some clients may only be receiving information as to the state of the blockchain from you. When you keep the entire blockchain you validate to everybody else that you have the correct records. I arbitrarily object to the use of pruned nodes but find them useful since the process of validation is completed in its entirety when a new node comes online even if it is pruned, but if only pruned nodes are available then everybody has to believe the other nodes and this is unacceptable for Bitcoin. It should be if some node is archiving UTXO's then it should count as a pruned node, but my suggestion is, instead of making a programming change just set your node with the parameter `prune=1` if you wish to allow manually pruning from the Blockchain to a specific height when you wish, or `prune= {>550} automatically prune blocks to stay under target size in MiB`. My chain state database after all is only 4.9GB and is hardly a concern for any operation of the standard Bitcoin Core client. The answer, again, is you should just leave it and get used to dealing with the bigger database. KING JAMES HRMH Great British Empire Regards, The Australian LORD HIS EXCELLENCY JAMES HRMH (& HMRH) of Hougun Manor & Glencoe & British Empire MR. Damian A. James Williamson Wills et al. Willtech www.willtech.com.au www.go-overt.com duigco.org DUIGCO API and other projects m. 0487135719 f. +61261470192 This email does not constitute a general advice. Please disregard this email if misdelivered.