From: Ben Goertzel (ben@goertzel.org)
Date: Fri Jun 04 2004 - 14:03:43 MDT
Eliezer,
If you trust human judgment so little -- even your own -- then how can
you trust yourself and the other SIAI insiders to make correct judgments
about when the collective-volition-embodying uber-AI is operating
correctly and when it isn't?
-- Ben G
> But do you, Ben, let alone the majority of modern-day
> Earthlings, have the
> ability to correctly weigh the risks and benefits of direct democracy
> controlling a superintelligence (EEEK!) versus collective
> volition? I
> don't. I could be wrong, and the collective volition could
> turn over the
> power to the UN General Assembly (EEEEEK!).
>
> It's people's lack of moral caution or fear of screwing up, their
> horrifying readiness to trust themselves with power, that
> causes me to turn
> the choice of meta-dynamic over to a collective volition
> (which is a form
> of humane transhuman intelligence, and one that structurally
> cares very
> strongly about our expected first-person perspectives on the
> issue). I
> don't trust myself. Nor do I trust other humans. From my studies of
> cognitive psychology I know in too much detail how little
> grasp the human
> mind has upon reality, how ready people are to make decisions
> on the basis
> of what sounds like a good idea at the time and how unrelated
> that is to
> their actual future experience and reactions. Just the
> prospect of humans
> deciding on the basis of their half-a-paragraph *description* of the
> outcomes, let alone their description of a human *prediction* of the
> outcomes, makes me want to flee screaming into the night.
>
> --
> Eliezer S. Yudkowsky http://intelligence.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT