From: Phillip Huggan (cdnprodigy@yahoo.com)
Date: Sun Nov 27 2005 - 15:08:03 MST
Moore's Law is useful, its curve is often my only tool in making some types of forecasts. But humanity's standards-of-living graphically resemble a series of mostly increasing plateaus that slope gently upwards or down as a technology/invention/discovery application is diffused across a population. There are some applications powerful enough to kill us off, or even bury standards-of-living beneath the x-axis on our graph into negative territory. The nice thing about an FAI induced singularity is that it should eliminate these dangerous applications with a much higher probability of success than mere humans would have if mere humans attempted the same. Ve should also immediately be able to harness nearby energy sources to tile vis way up to a very comfortable standard-of-living plateau for all of us. But beyond this, there is a tiling concern I have that may or may not be already addressed.
AGI will be able to quickly hit some plateaus and associated risks that come with them, which would otherwise take us centuries or millenia to reach. To reach some of these heights, an AGI may need to harness planck-scale or severe general relativity technologies. These extreme techs may carry with them some risks to the local region of the universe, and perhaps even to the entire multiverse. Whether or not an AGI determines a risk to be acceptable depends on our "currency conversion" chart we program into the seed AI. If ve is programmed to maximize the # of hamburgers in the universe, and vis two options are to MNT 1000 of them or to utilize a black-hole machine that will make 1000000 burgers but carries with it a 50% chance of destroying the fabric of the universe, we need assign values for the former scenario in our conversion chart that are high enough to overrride an AGI choosing the latter high-risk high-reward option. At the same time, many gajillions of subjective
years from now we may grow tired of our present state of affairs or be faced with the need to attempt some risky universe renovations, and we would be ready to consider the million burger pathway. But if the AGI has previously completely blocked off this avenue, we are left with a sub-optimal post-singularity shreik. Our currency conversion chart must be a living document that will be altered as our standards-of-living grow, and as we learn how to harness different energy sources. I am worried that an FAI with a fixed conversion calculator will take unnecessary risks early on or close off too many of our most promising pathways to be, early on. After some initial tiling, it would be nice if our FAI would come back to us and ask us what to do next.
---------------------------------
Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:53 MDT