From: Thomas McCabe (pphysics141@gmail.com)
Date: Mon Oct 29 2007 - 16:17:26 MDT
The FAI, basically, extrapolates what would happen if *we*
reprogrammed our motivational systems. If someone has a mental
illness, and wants to get rid of it, the FAI will extrapolate their
desires without the illness interfering. This is also the process used
to get rid of "stupid stuff we did when we were young"-type things
that have been mooching around in our heads- the FAI tries to simulate
the natural self-debugging process we all seem to go through. As for
people who are mentally ill enough to not be capable of understanding
what we mean by "mental illness" and why it is a bad thing, quite
frankly, I'm not sure how to handle them.
- Tom
On Oct 29, 2007 5:39 PM, Matt Mahoney <matmahoney@yahoo.com> wrote:
>
> --- Thomas McCabe <pphysics141@gmail.com> wrote:
>
> > CEV is a metamorality system. It doesn't say XYZ is good or bad: it
> > defines a procedure for how to determine if XYZ is good or bad. Apples
> > and oranges.
>
> From the paper: "...our coherent extrapolated volition is our wish if we knew
> more, thought faster, were more the people we wished we were, had grown up
> farther together; where the extrapolation converges rather than diverges,
> where our wishes cohere rather than interfere; extrapolated as we wish that
> extrapolated, interpreted as we wish that interpreted." (I hope this isn't
> taken out of context).
>
> My objection is that "were more the people we wished we were" makes CEV
> undefined if we allow the AI to reprogram our motivational systems. The AI
> could make us want to be the kind of person that allows the AI to tell us what
> we want. But if we disallow reprogramming the motivational system then we
> could not treat many mental illnesses. I gave examples where neither the
> person's wish before or after the change would be a reliable indicator of what
> a rational person would wish for. The only other alternative would be a
> complicated rule that we would probably get wrong.
>
>
> -- Matt Mahoney, matmahoney@yahoo.com
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT