From: Michael Roy Ames (michaelroyames@yahoo.com)
Date: Sat Mar 04 2006 - 11:50:56 MST
Philip,
IRT "It seems to me everyone is making a mistake in thinking that checking
for friendliness of a program is like checking whether the program
halts."
--- Actually, I don't think that this is a mistake. We are attempting to
define Friendliness as a thing-that-can-be-verified - or at least exploring
the idea as a possible way of maintaining goal stability through recursive
self improvement.
IRT "I think there is absolutely no hope of being able to formally verify
anything about the results of a proposed course of action in the world."
--- As a bare statement, I would have to agree with you. However, it is not
the verification of the results of a proposed course of action that we
intend to verify. Rather it is the whether the *intended* results of the
action are Friendly, and provably so based on the definition of Friendliness
as reified in the AI system and its goals. Where the intended and actual
results differ the AI has failed to accurately predict outcomes, and this is
salient for learning.
Michael Roy Ames
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT