From: Stuart Armstrong (dragondreaming@googlemail.com)
Date: Wed Jul 16 2008 - 09:13:38 MDT
>> Actually, the difficulty I had in mind was the seeming impossibility of
>> *proving* one's source code to another. Sure, one SI can just send her
>> source code to another, but how does the other SI know that it's not just
>> some source code that she wants him to think she's running?
>
> A trusted third party with appropriate access (not necessarily a mind, maybe
> just the operating system) could confirm this.
Still don't see how that could overcome an SI deliberately intended to
deceive. Commands such as "do this for ten thousand years, then change
your code to this" would not be detectable. You'd need to give it a
very thourough check, be in control of the hardware while doing so,
and have an idea of the history of the source code.
Though a simple, goal driven code would be easier to check, a messy
evolved code might be more reliable if you are not allowed to check.
Stuart
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT