Re: Informed consent and the exoself

From: Stirling Westrup (sti@cam.org)
Date: Tue Feb 22 2000 - 08:15:00 MST


Eliezer S. Yudkowsky wrote:

> Since I'm one of the people who may wind up actually deciding the seed AI's
> default motivations (in the event there isn't a forced solution), I've
> given some thought to the issue. I have two possible solutions:

> 2) Overlay informed consent. We may find it very difficult to conceive of
> simultaneously "knowing" and "not knowing" something, but I can imagine a
> cognitive architecture which would "protect" the core Buffy processes while
> maintaining the awareness and processing of the external self. Any given
> sequence of cognitive events, including emotional bindings dependent on the
> belief that Sunnydale is real, would proceed as if the knowledge that the
> world is a simulation did not exist, and memories of that experience would
> be formed; however, a smooth blend between that untouched core and the
> external awareness would be maintained. Thus you could remain "you" while
> being someone else.

This is the mental model that I've always used when thinking about sub-
selves that may need to operate in an environment in a manner consistent
with them being autonomous (such as doing anthropological research on
humans...)

-- 
 Stirling Westrup  |  Use of the Internet by this poster
 sti@cam.org       |  is not to be construed as a tacit
                   |  endorsement of Western Technological
                   |  Civilization or its appurtenances.


This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:26:57 MST