(WAS: Re: IA vs. AI was: longevity vs singularity)
den Otter wrote:
>
> ----------
> > From: Eliezer S. Yudkowsky <sentience@pobox.com>
>
> > I think we have a serious technological disagreement on the costs and
> > sophistication of uploading. My uploading-in-2040 estimate is based on
> > the document "Large Scale Analysis of Neural Structures" by Ralph Merkle
> > (http://www.merkle.com/merkleDir/brainAnalysis.html) which says
> > "Manhattan Project, one person, 2040" - and, I believe, that's for
> > destructive uploading.
>
> Then somewhere else you wrote:
>
> > I would
> > expect virtually everything Drexler ever wrote about to be developed
> > within a year of the first assembler, after which, if the planet is
> > still around, we'll start to get the really *interesting* technologies.
> > Nanotechnology is ten times as powerful and versatile as electricity.
> > What we forsee is only the tip of the iceberg, the immediate
> > possibilities. Don't be fooled by their awesome raw power into
> > categorizing drextechs as "high" nanotechnology. Drextechs are the
> > obvious stuff. Like I said, I would expect almost all of it to follow
> > almost immediately.
>
> So you've said yourself that nanotech is to be expected around 2015,
> and that even today's most advanced Drextech designs would soon be
> obsolete. How does this jive with a 25(!!!) year gap between the first
> assemblers and uploading? I bet that if nanotech would be as potent as
> you assume, full uploading could be feasible in 2020, about the same
> time as your AI. If time is no longer a factor, uploading becomes more
> than ever the superior option.
The key phrase is CRNS. As you point out, Merkle's article does not include nanotech. In the event that nanotechnology was developed, then we have a new measure of time: NRNS. I would expect spaceship technology at about 6m NRNS, a working lunar colony at 2y NRNS, uploading at 4y NRNS, active shields against gray goo at 1y NRNS, and military attack goo at 3m NRNS. There will never be shields against military goo - see later post.
> Merkle's article is a conservative extrapolation of current technology,
> *it does not include nanotech*. I didn't see that quote "Manhattan
> project, one person, 2040" either. He simply concludes that: "If we
> use the [conventional] technology that will be available in 10 to
> 20 years, if we increase the budget to about one billion dollars,
> and if we use specially designed special purpose hardware --
> then we can determine the structure of an organ that has long
> been of the greatest interest to all humanity, the human brain".
You're right. Interesting. Maybe I'm using the wrong source, but that's the only one I could find, and I really thought that was the one. Looking into it, now, I see that what's being proposed isn't actually uploading, it's just getting a detailed wiring diagram from a series of brain sections over the course of three years. I don't think that would preserve sentience or even basic computational ability - it would just be a useful hint for AIers, which is what the article is about. I really seem to recall a technological estimate for actual non-nanotech uploading that said 2040.
> > You're talking about off-the-shelf uploading.
> > You're talking about nondestructive-uploading kiosks at the local
> > supermarket.
>
> No, though these could very well be feasible with 2020 nanotech.
Yeah, about 6y NRNS.
> > That's 2060 CRNS
>
> Yeah, right, 45 years after the first functional assembler. Wow,
> progress will actually be *slowing down* in the next century.
> Would this result in an anti-singularity or something?
[snip]
> > Then they'd damage uploading more. It's a lot harder to run an
> > uploading project using PGP.
>
> Why would this be harder for IA than for AI?
Because one could at least try to run an AI on a network of home computers using secure IP, while IA neurosurgery requires a complete hospital, neuroimaging equipment, and a lab capable of adapting (admittedly off-the-shelf) technology originally developed for epilepsy for brain stimulation.
Uploading requires either a huge lab or nanotechnology. You can't hide the lab, and nanotechnology is a whole 'nother issue.
> > Do I think nanotechnology is going to blow up the world? Yes.
>
> .......but probably not before we (most likely in a substantially
> augmented form) can escape to space.
Well, that's where I disagree. In the event that nanotechnology is developed, say by Zyvex, either the U.S. government will confiscate all of it, or Zyvex will have to keep it secret until it can fight the government and win - and it'll have to develop all the applications on its own, which makes that possibility fairly unlikely, especially since the phrase "military applications" doesn't seem to have passed through their mind.
So the US confiscates it, and either conquers the world, or keeps it a secret, or the secret gets out and somebody tries to launch a preemptive nuclear strike. I'm not really sure how this would play out, but it seems to me to end either in a dictatorship or in a nanowar. I don't see where the lunar colonies would come from.
> > Do I
> > lift the smallest finger against it? No.
>
> Unless nanotech is absolutely crucial to build your AI, this statement
> doesn't make any sense. This race is way too important to worry
> about good sportsmanship.
Maybe it's too important *not* to worry about good sportsmanship. If I were to assassinate my own role model, not only would I tick off every single person who could conceivably have helped me, and not only would I kill the only person with enough wariness and influence to keep things from getting worse if something *does* go wrong, but I'd also be inviting retaliatory strikes on AI researchers. I really don't think anyone's chances of survival would be helped by a deathmatch among transhumanists.
I can't keep nanotechnology from being developed - only keep it from it being developed by the people I can influence, who are probably the best of all evils.
> > I really don't see that much of a difference between vaporizing the
> > Earth and just toasting it to charcoal. Considered as weapons, AIs and
> > nanotech have equal destructive power; the difference is that an AI can
> > have a conscience.
>
> It has a *will*, an intelligence (but not necessarily a conscience in
> the sense that it feels "guilt"). An ASI is an infinitely more
> formidable weapon than nanotech because it can come up with new ways to
> crush your defences and kill you at a truly astronomical speed.
> Like the Borg, who adjust their shielding after you've shot a couple
> of them, only *a lot* more efficient. Nanotech is just stupid goo
> that will try to disassemble anything it comes into contact with it
> (unless it's another goo nanite -- hey, you could base your defenses
> on that). So...avoid contact. Unless the goo is controlled by a
> SI (not that it would bother with such hideously primitive
> technology), it can be tricked, avoided and destroyed. Try that
> with a SI...
It's not goo I'm worried about, it's deliberately developed weapons. Yudkowsky's Fourth Threat: "Technologies with military potential are *always* used." I don't think there's much more of a defense against nanotechnological weaponry than there is against SIs. I mean, yes, an SI is unimaginably large overkill while a nanowar is just a little overkill, but, like I said, what's the difference between vaporizing the planet and toasting it to charcoal? Is there that much of a difference between being shot and being nuked? Either way, you're dead.
> These possibilities are non-trivial, certainly when the military
> start fooling around with AI, in which case they're likely to
> be the first to have one up and running. Hell, those guys might
> even try to stuff the thing into a cruise missile. So no, I don't
> see why I should trust an AI more than myself.
I do worry about the possibilities of "AI abuse". Maybe I don't worry enough. Maybe I'm dissing Zyvex for being naive about military applications while being just as stupid myself. Maybe *any* technology capable of saving the world will be capable of blowing it up two years earlier. One of the things that makes navigating the future so interesting is that the problems are not guaranteed solvable. Sometimes you're just doomed. But...
I guesstimate that, in self-enhancing AI, there's a sharp snap from prehuman to posthuman intelligence. With any luck, none of the seed stages will be intelligent enough to be a major threat, none of the end stages will be dumb enough to be controllable, and none of the intervening stages will last long enough to be a problem.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way