From: Brent Thomas (bthomas@avatar-intl.com)
Date: Wed Jun 16 2004 - 17:04:08 MDT
Hmmm...i think Elizer and I may be speaking past each other to some
extent because of a terminology problem...
Let me review:
1) we (the human race in general and siai in particular) are heading
towards a singularity where technology will
create a F(hopefully)AI.
2) If the attempts to create a FAI are not successful - no problem
(except that we make in our own ignorance)
3) if a AI is created which is not F then (bad thing happens)
4) Once the FAI is created it will quickly gain the ability to modify
and control the physical meat-space. (most likely
through nanotech)
At this point is where my considerations of a personal 'right of
withdrawal' come into play.
Theoretically the Fai is able to manipulate and communicate with all
people on earth (sentients) at this point
and can do so with minimal effort.
5) acting according to the collective volition principals and plans (see
recent posts and the wiki) the
Fai will attempt to determine our collective volition and enable it.
The Fai here is functionally a very wise and powerful entity - it knows
a lot and can do a lot. Looking around itself
it will try to do what the people around it want done (collective
volition) if they were as smart/fast/better as they could be.
In terms of the environment the Fai should react to our desires as a
species (enabling our collective volition) but do so in a safe manner
(ie no paperclip worlds)
Interacting with the Fai is in my mind no different than interacting
with another person. It can express needs, desires, and plans of
action...sure talk all you like and lets discuss things!
It will (most likely) work behind the scenes to modify the environment
according to the collective volition...thats fine, clean up the water,
remove the pollution, whatever works out...no problem there...
Whoops...it started to transform the world into diamond....when the
change wave of transformation encountered the first sentient the wave
stops and the fai interacts with the sentient...hey I'd like to change
you over to diamond, let me explain why and tell me if its ok?
<sentient> No! I don't want to become diamond...keep me as I was [fai
proceeds to transform area around sentient into diamond but maintains an
enclave of normal environment around the sentient and does not transform
him. Change wave sweeps off into the distance and other encounters
proceed similarly (until and unless the collective volition realizes
that this is a silly example and we don't as a species want to become
diamond) - anyway this is a pretty silly example but it illustrates the
experience/interaction where when the FAI needs to modify a sentient it
asks, explains and only proceeds if ok'd
The point here is that the sentient is protected from unwanted
modifications (even or especially!) if the collective decides that is
what the sentient would want (if better/faster/smarter) and there is the
ability for recovery without mass extinction...collective realizes that
its algorithms were wrong after querying 10,000 (arbitrary number for
example) sentients (its projection does not match actual will) - no harm
was done fai reverts landscape from diamond to our normal world (whew!)
- option of personal last judgement for conditions of physical
modification allowed fai to realize its projections did not match
reality.
6) the actual point is just to perserve a personal 'last judgement' for
any external action (unsolicited!) that the
system attempts which would modify the person (physically!- I don't care
if it wants to talk to me...attempt to sway my opinion...interaction
here is just like another person..if someone wants to preach at me fine,
I reserve the right to walk away but I don't mind discussion (with
person or Fai))
The system can basically do whatever it wants to (based on extraplated
collective volition) environmentally preserving all the 'good' things
you're trying to accomplish with your scope and rules and transforms
(all the fiddly details that I expect you (siai, you personally,
whomever is actually coding the fai etc) to be working out ). There is a
lot of complexity here and I understand that you're still early days
(tick tick tick ;-) on this and have lots to work out. There will be
many special cases and rules to determine (either directly or via
optimization or some other method...however it works out is not
especially meaningful to me in this instant for this discussion).
But, once its working, it simply should not modify me (or any sentient)
without getting permission. Bottom line, full stop.
(and again this is physical modification...if it wants to talk that's
fine - just like interacting with any other wise and powerful person) I
don't claim to know how you'll code this or arrange for it to happen,
but that is the bottom line.
Just as I don't expect the policeman on the corner to slap me in
handcuffs if I'm minding my own business I don't want the fai to modify
me with out asking (and if its bothering to ask there is probably a good
reason and I'll probably go along with it after sufficient
explanation...but still that element of choice)
I hope this intent is clear, I'm sure you and others on the list can
follow this expectation (and I'm pretty sure that anyone (who hasn't
been brainwashed) would agree that 1) informed choice with option to
withdraw versus 2) modifications made 'in my best interest' (however its
been determined) --- hmmm 1 is prefered.
And in fact if the collective volition works as it should this rule of
'don't modify without asking' should almost be a given in the
dynamic...but still I wanted to 'spread the meme' and toss in my $0.02
(silly primate social dynamics) and I hope that even if you don't agree
that this fundamental invariant should be at the root of the system that
somewhere in your mind this idea will take root and all will be well -
theres really not a lot more I can do or say about this, so feel free to
reply or comment but otherwise I'll drop this for now.
Hmmm...what was that country that had a snake on its flag and the 'don't
tread on me motto?'
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT