From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sat Apr 03 1999 - 22:58:18 MST
Nick Bostrom wrote:
>
> We presumably want to say
> that only the former gives rise to a consciousness. And the relevant
> difference seems to be that in the case of the process, the various
> states are causally connected, whereas with the spatial pattern that
> is not so.
This is almost exactly the point where I threw up my hands and declared
for Penrose. The problem is that "causal connections", insofar as far
as I can understand them, are intrinsically counterfactual; the problem
with counterfactuals is that they are entirely subjective. Although I
continue to try, I have not been able to devise a theory of
instantiation eliminating either of these problems.
Let's suppose that we record the tape, states, position, and actions of
a conscious Turing machine, and play it back. Is the playback
conscious? Probably not, because there's no causality, right? And if
the playback was generated by quantum randomness, at some extreme level
of improbability, then the conscious being "recorded" doesn't exist yet, right?
Now let's say we run the playback, and, at each step, consult the
original state transition diagram to find out what the results *would*
have been, but then discard that result and load in the tape. In other
words, we compute each step, at each point along the recording, but we
don't connect them causally to each other; we discard the result and
load the recorded step, even though the two happen to be identical. Is
that Turing machine conscious? Does it make a difference whether the
record was generated by quantum randomness, by a methodical generation
of all possible records, or by actually recording an identical Turing machine?
Let's then say that we generate each step, compare it to the record,
and, if they differ, replace the step with the data from the record. If
the record is different in the slightest, it will prevail over the
Turing machine. But, since data and process are identical, the Turing
process continues untouched. Is that Turing machine conscious? What is
the precise difference between that and the previous Turing machine?
How about if the data is loaded from the record, compared to the
intermediate step, and the intermediate step used if it's different?
(Call this "machine Q".) That's definitely conscious, right? But that
would make use-the-record-if-different unconscious, right?
Since the intermediate step is never actually used, isn't that the same
as discarding it? No? Yes? Why?
When we say that the Turing machine enters state X because it
encountered a 1, we mean that it would have entered state Y if it had
encountered a 0. If a 0 would also have led to state X, we would say
that it moved Left because of the 1, because a 0 would have made it move
right. If both a 0 and a 1 lead to the same state, we say it's being
insensitive to the data.
Let's consider a Turing machine in which the state transition diagram is
a series of double boxes; each 1 box and 0 box contains a little scrap
of paper instructing the Turing machine what to do next. There's a
Chinese restaurant owner that runs around following the instructions
inside the box, moving counters on an infinitely long strip, keeping
track of which double box is the current state, and generally
implementing the Turing machine.
There's also a demon scurrying around that occasionally takes out one
strip, temporarily replaces it with a duplicate of a strip from the
other box (in the double-box set), and then switches it back after a
random time. Just for fun, we'll say the demon is quantum-random.
If the demon ever tampers with a box at the time the CRO (Chinese
restaurant owner) is looking in it, and the instructions are different,
then obviously the Turing machine will malfunction. And if the demon
deliberately watches the CRO and only replaces boxes that he never uses,
it seems intuitively obvious that the Turing machine is instantiated.
But what if, through pure quantum randomness, the demon replaces only
unused boxes while the CRO is reading the other one? Has the causal
connection been broken or not?
It seems to me that a theory of instantiation would have to dispense
with such counterfactuals entirely and rely on pure continuity of
isolated data, but I don't see any way to create such a theory and bind
it to physical reality. And note in particular that such a theory looks
like it would make machine Q unconscious, which seems unintuitive in the extreme.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/singul_arity.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:28 MST