From: Samantha Atkins (samantha@objectent.com)
Date: Tue Apr 24 2001 - 00:59:11 MDT
Brian Atkins wrote:
>
> Yes evolving stuff in general can do some cool stuff, but it's a real
> pain to debug. The scientist in this article still hasn't figured out
> how his new chip layout even works... I would not want to be flying in
> a spaceship or some other kind of critical system that relied on such
> opaque chip designs. If it has evolved to work best in the lab conditions,
> what happens when it gets hit with some external radiation?
>
> In other words, I think using evolutionary techniques to discover new
> possibilities in technology is a great thing, but the new possibilities
> should be thoroughly understood and then incorporated into a human-designed
> final product. Using a raw evolution-derived piece of technology without
> completely understanding how it works is a recipe for disaster IMO.
In the end you can only test as much as you can in lieu of understanding
exactly how it all works. This is true for Friendly AI / Sysop plans
also. The AI must self-improve beyond what any human or set of humans
can design or understand if it is to meet its objectives. If we can
test a budding SI sufficiently to be reasonably confident of its final
dependability and benevolence then we should be able to do the same with
an SI that has had some components created by GAs. Or am I missing
some critical difference?
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT