From: Philip Sutton (Philip.Sutton@green-innovations.asn.au)
Date: Sun Mar 14 2004 - 11:40:56 MST
Hi Ben,
> BG: Basically, what this reply says to me is that you don't believe
> there will be a Singularity in the same sense that many of us do. Your
> Singularity is more in the spirit of the more mild-mannered of
> Kurzweil's statements. In my view, the Singularity will be an
> incomparably larger change than any of these previous changes that
> you're describing.
> BG: You don't seem to accept the possibility of truly fundamental
> change in the nature of reality and/or mind. ........ your vision of
> the future seems SL3 not SL4
I went back to Eliezer's reference that you quoted and read some of the
links from that document to get a better feel for whether I'm thinking
more in the SL3 or the SL4 mode.
(http://yudkowsky.net/sing/shocklevels.html).
I agree with you that, post-singularity, we cannot know what things will
be like in terms of the nature of reality and/or mind. So in that sense I
think it's fair to say that, at least, intellectually I can glimpse the SL4
discontiuity. But it's hard to come to terms with it (a) because it's
fundamentally unknowable and yet (b) what can be imagined vaguely
could shake up the status quo in every direction so strongly.
Maybe what I've been trying to say can be better understood as being
relevant to the *lead up* to the singularity proper. I think the lead up
will not be 'just' the time between pre- and post-singularity but will be a
significant time in its own right (perhaps with the power to send
significant reverberations into what follows the singularity).
Whether or not there is ever more than one AGI I think the introduction
of one or more AGIs into the universe will add massively to the
complexity of the universe (or the parts of the universe where AGI/AGIs
have not been before). AGIs will be capable of faster and faster
thought for any given level of complexity of thinking. But it may well be
that AGIs choose sometimes or often to expand the complexity of their
thought rather than simply shrinking the absolute amount of time taken
to reach conclusions. So it's not a foregone conclusion that subjective
time will shrink for all issues, and at all times, in the dramatic way that
some people have speculated. AGIs when first emerging with have a
prodigeous amount to learn and assimilate. And developing new deep
knowledge and insights will, I imagine, continue to be, at times, a non-
trivial task. Maybe a measure of wisdom of new self-improving AGIs
will be the extent to which they take advantage of the acceleration of
subjective time to enable better-thought-out actions in absolute time.
I still feel that a super AGI or populations of AGIs will still be faced with
key questions like:
- what to learn and what not to learn?
- what to invent and what not to?
- what to change and what not to change?
- when to change things and when not to?
- what to build on and what to abandon?
- what to keep from the present/past and what not to keep?
Ben, do you think these questions will become obsolete? Surely
evolution is unlikely to become obsolete and if that is the case wouldn't
these questions remain relevant?
I can't see that raising issues like these is a sign of a failure to move
from shock level 3 to shock level 4 thinking/mind sets??
The model of the approach to the singularity where we have life more
or less as we have known it and then in a flash we have the post
singularity world seems to me to be almost designed to wish away the
hard thinking about how to move up to or make the transition to the
singularity.
It reminds me of the early Bolsheviks who refused to speculate on the
future post-revolution because the world would be remade. And look
where that got us! :)
If we can assume that evolution is not abolished as a meta-process by
the singularity then we can assume that history still counts as a shaper
of the future. If that's true then the lead up to the singularity is very
important - so I think we need to expand our thinking about this phase.
And as workable AGIs develop (not super AGIs initially) they can be
recruited to work on the issue of understanding options for the lead-up
to the singularity. This joint working process automatically then makes
the lead up to the singularity even more complex and interesting.
Cheers, Philip
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT