From: Eugen Leitl (eugen@leitl.org)
Date: Mon May 20 2002 - 08:15:55 MDT
On Sat, 18 May 2002, Eliezer S. Yudkowsky wrote:
> > I recommend you do a back of the envelope as to resources required to run
> > the Sysop (even short-scale decisions require sampling a lot of agent
> > trajectories with very detailed agent models), and the constraints it
> > imposes on its serfs.
>
> Show me the numbers. You have repeatedly declined to provide any
> justification whatsoever for your assertion that "even short-scale
Repeatedly? Don't recally anyone ever asking. Happy to oblige.
> decisions require sampling a lot of agent trajectories". Now you are
> claiming the ability to do quantitative modeling of superintelligent
> resource utilization?
Semiquantitative. I'm kinda busy, so it has to be brief.
Here's the gist of the cost estimation: mature civilisations utilize every
single atom and every single Joule available. There are no spare
resources, period. A civilisation is an assembly of agents evolving along
behaviour trajectories. Your hypothetical despot introduces constraints on
all behaviour trajectories, using a (boolean or scalar) friendliness
metric. This metric is hardwired into every agent instance, and needs to
be completely specified at despot seed implementation time, for obvious
reasons. This involves a 1) classification/decision based on a number of
observed trajectory frames 2) corrective action. Classification need to
occur in realtime, which is ~10^6 faster than current term of realtime. So
your despot is distributed, having probably a single instance of the
warden for each agent watched. Gaining decision time by slowing timebase
for single agent and agent groups is inconsistent, and induces penalties,
the more so if you want to slow down considerable expanses of reality. The
warden is considerably adaptively smarter than even the smartest agents
around, orelse it can be outsmarted. Notice that global synchronization of
despot instances is impossible even for moderate culture sizes
(lightminutes to lighthours). Even so, traffic synching despot state will
eat a considerable fraction of entire traffic. Friendliness is not a
realtime decisable classification, since actions lead to consequences on a
second, minute, year scale. Retrograde actions are impossible (no magical
physics), so you have to model the world (just that: the entire world,
probably tesellating it in each despot instance), and to sample the
consequences of each action which will manifest downstream. The world
being nonlinear, you can't predict very far. Depending on action taken, it
can be a hard bound on an action (you can't do something when you try) or
a soft bound (you never come in the predilection of poking thy neighbour's
eye out, because even thought crime is impossible thanks to the despot).
Soft bound is a far more computationally intensive to decide, because here
you will have to sample several trajectories, to compute your corrective
force on the agent state vector, to guide its future evolution.
If you want an off-the cuff estimate, given the above I don't see how the
despot can consume considerably less than 90% of entire available
resources, plus put an unspecified penalty on fastest available timebase.
Apart from above being core propeties of evil incarnate, it is a very
expensive evil to boot. But, hey, it controls the physical layer, so what
can you do.
> You clearly do not understand even the basic rules of volition-based
> Friendliness or the Sysop Scenario, both ordinary futuristic
> scenarios, much less the principles of Friendly AI.
Of course. Because "volition-based Friendliness", "the Sysop Scenario" and
"Friendly AI" are jargon you invented, and no one else uses. For you own
interest, I recommend defining new terms when you introduce them, if you
expect to be understood. Surely you have a paragraph of text somewhere you
could cut & paste for a succinct definition of those terms.
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:14:13 MST