WWW.HERESY.COM : A computer pioneer's crisis of faith.

Women, married men, and active gays should all be ordained, Cardinal Joseph Ratzinger told L' Osservatore Romano today. "'The clergy must be representative of the faithful.'" One can imagine the reactions such a statement would elicit. Similarly, when Bill Joy, computer guru extraordinaire, wrote in the April issue of Wired magazine that, if unchecked, technology would bring human society to extinction, the scientific-technological community was astounded. Since its inception, Wired has been the cheerleader for the new information era, exulting in an Internet-fueled transformation of human life and an era of unparalleled economic growth. It shares a libertarian belief that innovation and skill are goods in themselves, and that the Internet should remain a totally self-regulating community. Hackers are its natural heroes.

Styled by Fortune "the Edison of the Internet," Bill Joy is a founder and now chief scientist for Sun Microsystems, and former cochairman of a presidential commission on the future of information technology. In his Wired article, "Why the Future Doesn't Need Us," Joy argues that we are making humans an "endangered species," and that it is time to "rethink our utopian choices."

To understand Joy's new-found fears, one must look at their origin. His new dystopian vision is rooted in his utopianism. Joy traces his unease to a meeting with Ray Kurzweil, inventor of the first reading machine for the blind and author of The Age of Intelligent Machines. Kurzweil foresees--and welcomes--an era in which humans will fuse with robots. Joy notes that Kurzweil quotes extensively from the Unabomber Manifesto of Ted Kaczynski, which presents two alternatives: either humans allow machines to take over completely (since they will make better decisions), or we act to maintain control over them. Still, in the latter case, large systems will continue to be under the sway of elites, and once machines can do the work the elites require, the masses will no longer be needed.

Now enter thinkers like Hans Moravec, founder of the world's largest robotics program at Carnegie Mellon University and the author of Robot: Mere Machine to Dependent Mind. According to Moravec, robots are another species, one in competition--necessarily successful--with humanity. As humans are squeezed toward extinction, we will be forced to merge with robots, sacrificing our identity for the promise of virtual immortality.

Joy believes rising computer power will create an intelligent robot by 2030, and that eventually robots will reproduce themselves, without requiring human intervention. Whereas older methods of mass destruction, such as the atom bomb, required huge operations that were relatively easy to detect, twenty-first-century technologies--genetics, nanotechnology (extremely small devices operating at the atomic level), and robotics (these three collectively known as GNR)--will not require large facilities or rare raw materials. Instead, knowledge alone will enable their production and use. With unprecedented knowledge at the command of robots, humans will become expendable. Thus, writes Joy, we face "knowledge-enabled mass destruction," a destructiveness "amplified by the power of self-replication."

A second lurking danger is that humans, wittingly or not, will one day download their consciousness into robots, just as we now download computer data, which will lead to the eventual loss of our humanity. (The chemical interactions of neurons in the brian, Kurzweil has speculated, can be replicated electronically.) Genetic engineering will present multiple problems, not only the prospect of human cloning--still far off--but of unanticipated consequences, such as those already associated with altered crops, migration across species, etc. Nanotechnology is further along than we once realized, says Joy, and "as with nuclear technology, it is far easier to create destructive uses for nanotechnology than constructive ones." What most alarms Joy, however, is GNR's "power of destructive self-replication." In the twentieth century, nuclear, chemical, and biological weapons were military in nature, and were developed primarily in government laboratories. But in GNR, we are aggressively pursuing powerful new technologies "within the now unchallenged system of global capitalism and its manifold financial incentives and competitive pressures."

What can be done to protect against these dangers? Joy believes we must abandon our present course: "The only realistic alternative I see is relinquishment: to limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge." Is this possible? While pessimistic, Joy believes there is some basis for hope. He notes that the world's leaders have virtually abandoned the drive to develop new weapons of mass destruction. Still, to relinquish GNR will be far more complicated. It will require not only an unprecedented system of verification but also a powerful code of ethical conduct capable of forcing scientists and engineers to eschew--and even report--work on "knowledge-enabled mass destruction." Such a prospect flies in the face of the age-old quest of Western science for knowledge and domination.

Are Joy's fears--let alone his hopes--justified? The key question concerns robotic self-replication. Someday, mobile robots, nano or full-sized, may be able to acquire the physical materials necessary for their own reproduction. Will they be aware of it? Whether machines can attain self-consciousness is the greatest controversy in the artificial intelligence community. Joy notes that Berkeley philosopher John Searle--himself not a theist--argues, on philosophical grounds, that machines cannot become conscious, that all they can do is computation [see Bernard Prusak, page 28]. If Searle is right--despite the perils created by genetic engineering and human-robot hybrids--humanity has a future. Human choice--expressed through the workings of organized society--will be capable of controlling unbridled science-based change. Whether we will so choose is an entirely different matter.

Joy's discussion does not consider the vast proportion of human beings who are, and will continue to be, dirt poor, and those in the thrall of deadly ethnic and religious passions. He fails to touch on how people in these groups will react to the technological perils of the twenty-first century. Yet his salient essay has a great deal to say about our present impasse. Religion has been replaced by science as the source of apocalyptic dread, and the age-old temptation to grasp at immortality has yielded to the scientific quest for that goal through technology. As Carl Jung pointed out long ago, flying saucers have replaced angels in the popular imagination, and space travel has become the cornerstone of our new mythology. Whether, in order to have a future, we will need to revive the old categories and myths that have traditionally inspired civilization may, in the final analysis, be the deeper question underpinning Joy's amazing apostasy.

Victor Ferkiss, a long-time contributor, is the author of The Future of Technological Civilization.