From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sat Apr 19 2003 - 19:25:49 MDT
Thomas R Mazanec wrote:
> Interesting. An explanation of an apparent "anomaly" may be rational
> but "unscientific" because our science has not advanced to the point
> that we understand what caused the anomaly. A couple centuries ago,
> Thomas Jefferson allegedly said that he would rather believe two Yankee
> professors would lie than that a stone would fall from the sky. Perhaps
> in the year 3003 someone will invent a "time tunnel" that reaches back
> into your closet. I very much doubt it, but I am not *POSITIVE* that it
> cannot happen. If so, this portal will not be "magical" (except in the
> Arthur C. Clarke sense, of course). How would you respond to such a
> "portal"?
(Note: This is in reference to the author's afterword from "Non-Player
Character".)
By selectively considering your responses to situations that, while
"theoretically" possible, are only likely in fiction, you can end up
distorting your mentally precomputed reflexes away from the real world.
There are real anomalies - but not the ones from fiction, and considering
fiction can subtly train you to mishandle real anomalies. Now it is
perhaps better to be an SF fan who knows how to handle every kind of
fictional situation from magical closets to temporal paradoxes, than to
panic and have your mind shut down when anomalies are encountered. In
that sense SF training may be useful. But the real world is fundamentally
different from fiction; it runs on different rules. A real temporal
paradox need not send you on wacky time-travel adventures; the very first
prototype time device could tear a hole in space and wipe out human
civilization instantly. It'd make for a boring story, but that's no
reason to assume it can't happen. It's that kind of subtly biased
precomputation, precomputation based on imagining the wrong worlds, that
can lead you to train yourself into the wrong reaction.
Another problem: if you imagine possible "rational" explanations for
finding a portal in your closet, you may start thinking that it's more
likely to happen - after all, you thought of all these possible rational
ways it could happen, right? That is the flaw in reasoning by abduction
from false hypotheses. You have all these links in your mind leading to
explaining the anomaly of a portal in your closet, but those links are
likely to cut across the grain of real anomalies because you will not
*actually* find a portal in your closet.
Try keeping track of how many times people casually reason from fictional
evidence - "Brave New World" cited in discussions on cloning, "The Matrix"
in discussions of AI, "the Borg" in discussions of human-machine
interfaces. Humans don't have strong safeguards to prevent reasoning from
fictional evidence; we know the particular story isn't true, but we easily
pick up our concepts, our categories, the way we chunk the world, by
generalizing concepts from fictional experiences. Generalizing your
categories from fictional evidence is just as much a mistake as treating
"Brave New World" as a history book.
http://www.unlikeminerva.com/arc/64-4.php3
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT