From: Al Billings (memoria@memoria.com)
Date: Sat Feb 17 2001 - 22:46:23 MST
The article in question is at:
http://www0.mercurycenter.com/svtech/news/top/docs/billjy021701.htm
Here it is in full:
Guiding Science
Technologist Bill Joy leads a debate over how far we should go with new
machines
BY RICHARD SCHEININ
Mercury News
Bill Joy is once more trumpeting the dangers of technology run amok.
Speaking in San Francisco this week, there was a moment when the co-inventor
of Java brought home the deep impact of his argument: Perhaps science should
stop manipulating genes, he suggested, even if new gene therapies might save
a child from incurable cancer.
``No, we don't have to fix it,'' said Joy, the co-founder and chief
scientist of Sun Microsystems, if the cost to society is dire.
He's said it before, he continued, and ``people have left the room'' over
his suggestion ``that I could imagine letting someone suffer to protect the
group.''
There it was: Joy had called for a new ethical consideration of research
into new technologies that, he contends, could lead to the extinction of the
human species in the 21st century. The panel of experts who responded to
Joy -- a geneticist, a robotics researcher, a biochemist, a philosopher, an
ethicist -- didn't necessarily buy his arguments. Yet Joy had put one of the
great questions of the age right out there in the open: In the face of
technologies that might kill us, are we willing to curtail the rights of the
individual to protect society?
The Wednesday night forum at Grace Cathedral was called to consider how the
swiftly emerging fields of genomics, robotics and nanotechnology might
converge and affect humanity's future. Now there is a broad discussion on
this subject, a dialogue across the boundaries of science and religion.
Wednesday's event was a sign of this faith-science convergence: Its sponsors
included the American Association for the Advancement of Science's Program
of Dialogue on Science, Ethics and Religion; the Center for Theology and the
Natural Sciences in Berkeley; the Episcopal Working Group on Science,
Technology and Faith; Grace Cathedral; and the Episcopal Diocese of
California.
The jump-off point for the forum was a long, widely discussed and decidedly
apocalyptic article that Joy wrote in the April 2000 issue of Wired magazine
called ``Why the Future Doesn't Need Us.'' (It's available at
www.wired.com/wired/archive/8.04/joy.html)
In a nutshell, Joy forecast doom: ``I think it is no exaggeration to say we
are on the cusp of the further perfection of extreme evil,'' he wrote, ``an
evil whose possibility spreads well beyond that which weapons of mass
destruction bequeathed to nation-states, on to a surprising and terrible
empowerment of extreme individuals.''
Joy began his talk Wednesday by reviewing portions of what he wrote: He
believes that commercial forces -- consumed by profit, unconcerned with
risk -- are propelling us toward calamity. By 2030, he says, computers will
be a million times more powerful than they are now and there will be no time
left to curb their impact:
a.. Biotechnology will allow for routine cut-and-pasting of genetic
material -- and the creation, by accident or intent, of virulent pathogens
that could decimate whole societies.
b.. Self-replicating, intelligent robots will emerge in the laboratory,
then merge with people, then, conceivably, subjugate and even replace the
human species.
c.. Nanotechnology, through which scientists attempt to build machines the
size of molecules, may lead to enormous benefits. For instance,
micro-machines with the intelligence of supercomputers might conduct
search-and-destroy missions against cancer cells inside the body. But the
technology might also give rise to self-replicating micro-robots, or
``nanobots,'' created in the laboratory to, say, kill weeds. There's always
the chance, though, that the micro-robots will instead mutate, escape and
proliferate like mad, eradicating all plant life on the planet.
Turning on technology
There was more than a little irony in hearing Joy, a celebrated creator of
technology, turning on it.
``Come on, Bill!'' exclaimed panelist Manuela Veloso, a leading builder of
robots and professor at Carnegie Mellon University's Robotics Institute.
``You shut Sun and I will close my laboratory,'' she challenged him, adding
that advances in robotics could not happen without ``extremely fast and
beautiful computers. Who made them? You did. Not me.''
Certainly, humanity has lived with the specter of technological destruction
for some time. Since July 16, 1945, when the first atomic test was conducted
in the New Mexico desert, a fiery sword of Damocles has hung over humanity's
head. Eyewitnesses were so filled with awe that they resorted to religious
imagery to describe the spectacle: ``I am become death, the destroyer of
worlds,'' physicist J. Robert Oppenheimer, the father of the bomb, quoted
from the Bhagavad-Gita.
Yet governments have thus far succeeded in stepping away from the nuclear
abyss. And despite the much-talked-about threat of bio-terrorism, it hasn't
happened, argued Ken Culver, executive director of the department of
pharmacogenetics for the Novartis Pharmaceuticals Corp. in New Jersey.
``What's most important?'' asked Culver, another panelist, who says
experimental gene therapies are already demonstrating curative powers.
``Each of us has people in our families who encounter the health-care system
only to be told there's nothing that can be done to help loved ones who are
sick. Do we really want to relinquish our understanding of the human
genome'' -- the newly discovered sequence in which human genes are strung
together -- ``because of speculative theory . . . that sinister behavior
will occur?''
James Heath, a biochemist and nanotechnology researcher at the University of
California-Los Angeles, told Joy that ``nanobots'' are ``science fiction.''
Workaday scientists in his field are trying to create incredibly
energy-efficient micro-machines to alleviate the developing world's energy
shortage -- and California's, too.
More than once, Veloso scolded Joy: ``The future is not so dark.''
But Joy, rail-skinny, a little disheveled and still looking like the
Berkeley graduate student he once was, said that, if anything, it's getting
darker.
Seeing bleak
In the past year, it was reported that Russian scientists in 1990 combined
anthrax with a soil bacterium to make a new anthrax resistant to vaccines.
Last month, Australian scientists announced that, while trying to engineer a
contraceptive vaccine for laboratory mice, they accidentally created a
disease that killed the mice.
That scientists might accidentally kill people, or that deranged people
might set out to do just that, is not science fiction, Joy told the audience
of 200 who filled most of the assembly hall in the cathedral's basement.
So what exactly is coming out of all this besides argument and depression?
A surprising lot.
In an interview before the forum, Joy noted that the emerging wave of
technologies is forcing discussion of fundamental questions. What's the
right way to live? What does it mean to be in the world? What is the value
and danger of knowledge? What are our highest values?
He doesn't describe himself as a religious person. But more and more, he
finds himself thrust into religious communities. Last year, while camping in
the desert, a friend offered to introduce him to Rabbi Zalman
Schachter-Shalomi, a leader in what's known as the ``Jewish Renewal''
movement who lives, like Joy, in Colorado.
``Zalman said, `Maybe we should declare that nanotechnology isn't kosher,
and maybe the pope should declare it a mortal sin,''' Joy said. ``I said,
`That's an interesting perspective. Most of the people in my company don't
think like that.'
``I'm not trying to be apocalyptic. I'm just observing facts,'' he
continued. ``You can be in denial or you can talk about it. Or you can look
at it sort of mystically and say, `It'll all work out OK. Just have faith.'
But religion doesn't say that, does it? Religion says if it doesn't work
out, the big flood is going to come and we'll have to start all over
again.''
Wednesday night, he told the audience that he wants to be an optimist. He
floated some ideas for reducing the risk of technological calamity:
Establish a Hippocratic oath of responsibility for scientists; ``monetize''
risk, by factoring it into corporate spreadsheets as a hedge against
recklessness in technological development; and, most controversially, set
limits on free speech to stop the spread of dangerous technological
knowledge. Not all information, not all computer coding, should be shared
with just anyone anymore, he said.
This last suggestion has caused critics to describe Joy as an enemy of the
democratic, open society. His response: ``What are we getting from society
if we can't protect it from these kooks?''
No one engaged Joy on the free-speech issue.
But some of his warnings were being heard.
Philip Clayton, who chairs the Sonoma State University philosophy
department, compared advancing technology to a ``supertanker'' that can be
slowly steered but not stopped. ``What if the Brave New World has already
arrived?'' he asked.
Clayton asked the other panelists not to dismiss Joy's predictions out of
hand. Joy had mentioned an apocryphal story of Native American elders who
never make a tribal decision until they consider its impact on the next
seven generations.
``Do we vote for the good of the seventh generation?'' Clayton asked. ``Or
do we generally vote for what we take to be our personal good? How
enlightened are we?''
At this moment, not very. Yet the ``multi-generational impact'' of
technology must be assessed, said ethicist Audrey Chapman, who heads the
Dialogue on Science, Ethics and Religion of the American Association for the
Advancement of Science. The largest association of scientists in the
country, it is holding its annual conference in San Francisco this weekend.
``We live in a permissive society . . .'' Chapman said. ``There's this idea
that individuals should have the right to adopt and use any technologies
that they consider advantageous.''
Debating rights
As the evening wound down, conversation was returning to individual rights
vs. collective rights.
The matter of the hypothetical child, sick with cancer, returned.
Joy repeated that he might abstain from seeking new treatments if it
involved risky genetic research.
``Bill, you are afflicted with fixing all the bugs in your software,'' said
Veloso, the robotics researcher. ``But you don't want other scientists to be
afflicted with fixing people.''
Culver, the geneticist, was perturbed by Joy's position: ``It isn't about
individual health,'' he told Joy. ``It's about all our health, which
includes individuals.''
Joy considered that statement for a moment. Then he said that not knowing
everything there is to know ``may have group benefit as well.''
----------------------------------------------------------------------------
---- Richard Scheinin can be contacted at (408) 920-5069 or by e-mail at rscheinin@sjmercury.com.
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:05:55 MST