From: Anders Sandberg (asa@nada.kth.se)
Date: Sat Nov 01 1997 - 17:49:16 MST
Holger Wagner <Holger.Wagner@lrz-muenchen.de> writes:
> > Holger Wagner <Holger.Wagner@lrz.uni-muenchen.de> writes(that's me :-)):
> >
> > > 1) Today, humans are by no means perfect. They have a certain idea of
> > > what they do and what the consequences are, but it happens quite often
> > > that something "impredictable" happens. If you apply changes to the
> > > ecologic system, can you really predict what consequences this will have
> > > in the long run - and would you take the responsibility?
>
> Anders Sandberg <asa@nada.kth.se> replied:
> > We cannot predict the long-term consequences of our
> > actions. Period.
>
> Is that a proven fact? I mean: will it NEVER be possible, even with
> superintelligence?
I don't have the time to answer at length, but I think one can make a
very good case for the existence of actions whose outcome even Jupiter
brains cannot predict accurately. There are a lot of processes that
are simply chaotic and will microscopic perturbations (or lack of
information) will cause divergence from our prediction of them (the
canonical example is the weather). Then there are the processes that
are strictly undecidable, such as the halting problem in computer
science: it is impossible to create a general algorithm to determine
whether a given computer program halts or not. Since these two classes
of problems could well be hidden inside the consequence prediction
problem, it is most likely in general impossible to predict the long
term consequences of our actions even for SI.
> Maybe short-term predictions will be sufficient as we gain more control
> over nature. What about simulating ecologic, economic, social etc.
> systems? Today, we may not have enough computing power (and probably not
> the right theories and algorithms, either), but wouldn't this be a very
> valuable field for research?
It certainly would. I think we can at least avoid some of the most
obvious in retrospect disasters this way. With evolutionary modelling
we might even notice some of the more nonlinear dangers.
> My idea was to give the computer a program that can deal with as much
> data as available, start a simulation and see how similar the results in
> the simulation are to those in reality. That'll give you a basis to
> improve.
A good start, but simulations can fool you. It is always possible to
make a simulation conform to reality, but it may become brittle and
overtrained: when confronted with *real* data instead of the data it
has been tested on it simply doesn't work. This is why we neural
network people always try to validate our models a lot by giving them
previously unknown data.
> I've been thinking about the "education" problem today. I believe that
> education is the ONLY thing I'd be willing to pay taxes for. Good
> education should be freely available to all human beings. I don't care
> if a "government" or another company takes care of that, but it has to
> be done much better than today!
I couldn't agree more!
-- ----------------------------------------------------------------------- Anders Sandberg Towards Ascension! asa@nada.kth.se http://www.nada.kth.se/~asa/ GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:45:05 MST