From: Peter Voss (peter@optimal.org)
Date: Mon Sep 25 2006 - 09:12:06 MDT
A very good point.
However, Friendly AI, probably should not (have to) figure out what people
want, but perhaps to only help create an environment to will allow people to
flourish. Having said that, I have strong reservation about the possibility
of Friendly AI (some old notes:
http://www.optimal.org/peter/siai_guidelines.htm )
Peter
_____
From: owner-sl4@sl4.org [mailto:owner-sl4@sl4.org] On Behalf Of Norman Noman
Sent: Monday, September 25, 2006 5:46 AM
To: sl4@sl4.org
Subject: Questions/Problems regarding CEV
A fourth, and major, source of spread:
If you ask people what they want, they'll tell you, but they give totally
different answers depending on how they're asked. This is not because they
don't know their own minds, but because they're making it up as they go
along. Their desires are real; they're just generated in response to the
question. CEV does not define how people would be asked, and there is no
easy solution to this. To figure out how people would want to be asked, you
have to ask them how they'd want to be asked, etc.
Secretly probing their brains doesn't work either, because the information
simply isn't there. You can determine that they want sex, they want a raise,
they want to go home, and they want a cookie, but to try to extrapolate
these desires into a universal optimization process is insanity. It's like
interpreting a kindergarten scribble as the blueprint for a house. The only
way to get the necessary level of detail from the kindergartner is to ask
them to invent it, and what they invent depends on how you ask them.
There are innumerable other details of the extrapolation which are
undefined, and the resulting dynamic hinges on their definition. Do you plan
to define them? If so, how, and what outcome are you trying to achieve? If
not, how do you expect a scribble of a procedure to extrapolate a scribble
of human desires into anything approaching a coherent plan?
My strong suspicion is that you do plan to define them, and to define them
such as to aim for a specific target, but that you are making that target up
as you go along, as you are operating from a scribble yourself.
(I've donated $800 some dollars, so theoretically I'm allowed to talk about
this.)
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT