From: Billy Brown (bbrown@conemsco.com)
Date: Thu Feb 25 1999 - 15:35:12 MST
Nick Bostrom wrote:
> What Billy calls "mind control system" could
> in my opinion better be called "motivation system". Every agent needs
> a motivation system, and just because its motivation system was
> deliberately designed by another being rather than happened
> accidentally (due to genetical and environmental factors) doesn't
> make it any less part of what that agent is.
A proposed standard for distinguishing between the two cases:
Motivation System - A combination of goals and knowledge that leads an
entity to prefer some actions, and avoid others. The entity is free to
examine the contents of its own moral system, and to weigh its merits
against those of any other moral system it may encounter. If it decides
that some aspect of its moral system needs to be modified, it can do so.
Mind Control System - A moral system that incorporates special software
measures intended to ensure that it is never altered. The entity is either
unable to think about its moral system, or unable to compare it to other
systems, or perhaps simply unable to make changes to it.
As I understand it, your contention is that an entity with a strong
motivation system might be capable of changing it, but would never actually
choose to do so. Is that substantially correct?
Billy Brown, MCSE+I
bbrown@conemsco.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:09 MST