From: Eugene Leitl (eugene.leitl@lrz.uni-muenchen.de)
Date: Sun Oct 01 2000 - 03:10:17 MDT
Eliezer S. Yudkowsky writes:
> You need curiosity in order to think, learn, and discover, and you need to
> think, learn, and discover in order to be more generally efficient at
> manipulating reality, and being more generally efficient at manipulating
> reality means you can be more efficiently Friendly.
Being more efficient at manipulating reality also means the exact
opposite of above. A totally impotent devil is not much of a devil.
And please stop using that "Friendly" thing, because it means
different things to different people. (The child molester thinks he's
being very friendly to the child, the child probably sees it in a
somewhat different light. The Nazis were obviously only trying to
cleanse the world of an ancient evil. Is not whipping a pain pig
Friendly? Koran is Good/Evil, The Bible is Good/Evil, The Kapital is
Good/Evil, "The Road to Serfdom" is Good/Evil, parents telling a child
not to do A are being Friendly/Unfriendly <insert 10^8 similiar
examples>).
Friendly is not observer-invariant, goddamit. If you want to impose
your particular concept on the rest of us via (thankfully
hypothetical) AI proxy, then please call a spade a spade. Call it
Eliezer-Friendly, or something.
> "I'm not rightly sure what kind of thinking could lead to this confusion."
> Goals and subgoals are thoughts, not source code. Subgoals, if they exist at
That's not very helpful conceptry. Have you defined both terms, as you
use them? Since you don't seem to use them as most people.
I would rather use established concepts as "system state" and "system
Hamiltonian" when dealing with this. Have you read Ashby's classical
"Design for a Brain"? You can get it from Dover for cheap. It's old
(1952, 1960), but very modern.
To me (and most of AI people), a goal is a reasoning target. You
attempt to reach it in a series of steps, using a progress
metric. Because you frequently get blocked, or trapped in local
minima, you have to keep score of past milestones, and backtrack, if
you haven't been able to make progress for a while. In reality, this
is being done by an inference engine, sequential or parallel. It tries
to find a traversible path from A (say, a set of axioms) to B (Is blah
blah blah true, i.e. can it be inferred from A?), both being
specifyable in some formal language.
To me, this is a rather obsolete (and remarkably sterile) view of
navigation in (potentially very rugged) energy landscapes. Not that I
assume you're trapped in any such old AI trash, but it is sure
politically smart to avoid using words like "goals" in context of new
AI (if there is such a thing). People might misunderstand.
> all, exist as regularities in plans - that is, certain states of the Universe
> are reliably useful in achieving the real, desired states. Since "subgoals"
> have no independent desirability - they are only useful way-stations along the
> road to the final state - they can be hierarchical, or networked, or strange
> loops, or arranged in whatever order you like; or rather, whatever order best
> mirrors reality.
Huh. Whatever. And, of course, it is clearly obvious what is best
mirroring reality and what not, without being given a chance of being
evaluated by reality, since that wouldn't truncate the
particular-metric-Unfriendly behaviour space region.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:31:19 MST