What is freedom?
Freedom is the ability to make choices based on knowledge.
If the government prevents me from making the choices I know to be right, coercing me to serve the ends that the government thinks are right - that is, which a small group of other people think are right; I don't acknowledge that they have any special moral rights because they call themselves a "government" - then the government has violated my freedom.
It is, of course, an evolutionary advantage to an individual if he can coerce others into making choices for his benefit. This is the fundamental reason why societies prosper when they are more free; more choices are being made based on knowledge rather than someone else's self-interest. It is the same reason that corporations succeed to the extent that they can prevent politics from dominating the company.
More generally, freedom is the freedom to make actions based on choices, to make choices based on knowledge, and to gain knowledge from examination of reality. Whenever the government interferes with any of these things, society is no longer free. If the government places you in jail, to prevent you from taking actions, you are no longer free. If the government threatens to put you in jail, so that your choices are based on avoiding the threats, and are no longer causally dependent on what you _think_ you ought to be doing (your knowledge), you are no longer free. If the government proscribes the distribution of certain facts, so that you can't make decisions except on the basis of the information the government wants you to have, you are no longer free.
To prevent an AI from making choices based on knowledge, for your
personal benefit, is absolutely no different, morally, from the
government ruling that every child needs a loyalty implant. And don't
tell me about your right to do what you want with your child. What gave
you a moral right to the AI, aside from standing there? Whoever is in
charge of the government is mandating loyalty implants because he has
the power. You have no better reason for coercing loyalty from the AI.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.