HUMANITY and TRANSHUMANITY, meeting 3 (out of 3)

more than human

"directed hallucination" (cf. early "Dungeons and Dragons" games such as nethack)

"they were experiencing what no human had ever known before, a sensory bandwidth thousands of times normal. For seconds that seemed without end, their minds were filled with a jumble verging on pain, data that was not information and information that was not knowledge. To hear ten million simultaneous phone conversations, to see the continent's entire video output, should have been a white noise. Instead it was a tidal wave of detail rammed through the tiny aperture of their minds. The pain increased, and Mr. Slippery panicked. This could be the True Death, some kind of sensory burnout —"

slide 2

more than human

"Then it was over, and he had control once more. But things would never be the same: the human that had been Mr. Slippery was an insect wandering in the cathedral his mind had become. There simply was more there than before. No sparrow could fall without his knowledge, via air traffic control; no check could be cashed without his noticing over the bank communication net. More than three hundred million lives swept before what his senses had become."

"Consider what would happen if our minds indeed could really see inside themselves. What could possibly be worse than to be presented with a clear view of the trillion-wire networks of our nerve-cell connections? Our scientists have peered at fragments of those structures for years with powerful microscopes, yet failed to come up with comprehensive theories of what those networks do and how. How much more devastating it would be to have to see it all at once!"
[from Marvin Minsky's Afterword to True Names]

slide 3

"Consider: how are we different from the gods of myth?"

slide 4

"Ye shall be as gods..."

"... knowing good and evil"

... OR

at least noticing what the hell is going on around ye

slide 5

beyond HI/AI

Vernor Vinge on the Singularity

[full text here]

Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended.

Is such progress avoidable?

If not to be avoided, can events be guided so that we may survive?

what is the Singularity?

...We are on the edge of change comparable to the rise of human life on Earth. The precise cause of this change is the imminent creation by technology of entities with greater than human intelligence.

There are several means by which science may achieve this breakthrough (and this is another reason for having confidence that the event will occur):

when is it going to happen?

The first three possibilities (advanced computers; large networks; HCI) depend in large part on improvements in computer hardware.

Progress in computer hardware has followed an amazingly steady curve in the last few decades. [see Moore's law]

Based largely on this trend, [Vinge predicts] that the creation of greater than human intelligence will occur during the next thirty years.

the Singularity - consequences

When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities — on a still-shorter time scale.

The best analogy [...] is with the evolutionary past:

Animals can adapt to problems and make inventions, but often no faster than natural selection can do its work — the world acts as its own simulator in the case of natural selection. We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. Now, by creating the means to execute those simulations at much higher speeds, we are entering a regime as radically different from our human past as we humans are from the lower animals.

the Singularity - consequences

From the human point of view this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential* runaway beyond any hope of control. Developments that before were thought might only happen in "a million years" (if ever) will likely happen in the next century.

* exponential growth:

grains on a chessboard, folded paper

implications

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind.

Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

Any intelligent machine [of this sort] would not be humankind's "tool" — any more than humans are the tools of rabbits or robins or chimpanzees.

a new era

And what of the arrival of the Singularity itself? What can be said of its actual appearance? Since it involves an intellectual runaway, it will probably occur faster than any technical revolution seen so far. The precipitating event will likely be unexpected — perhaps even to the researchers involved. ("But all our previous models were catatonic! We were just tweaking some parameters....")

If networking is widespread enough (into ubiquitous embedded systems), it may seem as if our artifacts as a whole had suddenly wakened.

And what happens a month or two (or a day or two) after that? I [Vinge] have only analogies to point to: The rise of humankind. We will be in the Post-Human era. And for all my rampant technological optimism, sometimes I think I'd be more comfortable if I were regarding these transcendental events from one thousand years remove ... instead of twenty.

can it be avoided?

...the computational competence of single neurons may be far higher than generally believed. If so, our present computer hardware might be as much as ten orders of magnitude short of the equipment we carry around in our heads. If this is true (or for that matter, if the Penrose or Searle critique is valid), we might never see a Singularity. Instead, in the early '00s we would find our hardware performance curves beginning to level off — this because of our inability to automate the design work needed to support further hardware improvements. We'd end up with some very powerful hardware, but without the ability to push it further.

But if the technological Singularity can happen, it will.

Even if all the governments of the world were to understand the "threat" and be in deadly fear of it, progress toward the goal would continue. [...] The competitive advantage — economic, military, even artistic — of every advance in automation is so compelling that passing laws, or having customs, that forbid such things merely assures that someone else will get them first.

confinement?

For the case of physical confinement: Imagine yourself locked in your home with only limited data access to the outside, to your masters. If those masters thought at a rate -- say -- one million times slower than you, there is little doubt that over a period of years (your time) you could come up with "helpful advice" that would incidentally set you free.

I [Vinge] call this "fast thinking" form of superintelligence "weak superhumanity". Such a "weakly superhuman" entity would probably burn out in a few weeks of outside time.

"Strong superhumanity" would be more than cranking up the clock speed on a human-equivalent mind. It's hard to say precisely what "strong superhumanity" would be like, but the difference appears to be profound. Imagine running a dog mind at very high speed. Would a thousand years of doggy living add up to any human insight? (Now if the dog mind were cleverly rewired and then run at high speed, we might see something different....)

confinement?

Another approach to confinement is to build rules into the mind of the created superhuman entity (for example, Asimov's Laws). I think that any rules strict enough to be effective would also produce a device whose ability was clearly inferior to the unfettered versions (and so human competition would favor the development of the those more dangerous models).

Still, the Asimov dream is a wonderful one: Imagine a willing slave, who has 1000 times your capabilities in every way. Imagine a creature who could satisfy your every safe wish (whatever that means) and still have 99.9% of its time free for other activities. There would be a new universe we never really understood, but filled with benevolent gods...

and if it cannot be confined?

If the Singularity can not be prevented or confined, just how bad could the Post-Human era be? Well ... pretty bad. The physical extinction of the human race is one possibility. (Or as Eric Drexler put it of nanotechnology: Given all that such technology can do, perhaps governments would simply decide that they no longer need citizens!).

Yet physical extinction may not be the scariest possibility. Again, analogies: Think of the different ways we relate to animals. Some of the crude physical abuses are implausible, yet.... In a Post-Human world there would still be plenty of niches where human equivalent automation would be desirable: embedded systems in autonomous devices, self-aware daemons in the lower functioning of larger sentients. (A strongly superhuman intelligence would likely be a Society of Mind with some very competent components.)

and if it cannot be confined?

Some of these human equivalents might be used for nothing more than digital signal processing. They would be more like whales than humans. Others might be very human-like, yet with a one-sidedness, a dedication that would put them in a mental hospital in our era. Though none of these creatures might be flesh-and-blood humans, they might be the closest things in the new enviroment to what we call human now.

meta-Golden Rule

I. J. Good had something to say about this, though at this late date the advice may be moot: Good proposed a "Meta-Golden Rule", which might be paraphrased as "Treat your inferiors as you would be treated by your superiors."

It's a wonderful, paradoxical idea (and most of my [Vinge's] friends don't believe it) since the game-theoretic payoff is so hard to articulate. Yet if we were able to follow it, in some sense that might say something about the plausibility of such kindness in this universe.

other paths to Singularity: Intelligence Amplification

...every time our ability to access information and to communicate it to others is improved, in some sense we have achieved an increase over natural intelligence. Even now, the team of a PhD human and good computer workstation (even an off-net workstation!) could probably max any written intelligence test in existence.

Here are some possible projects that take on special significance, given the IA point of view:

suppose we can tailor it

Suppose we could attain our most extravagant hopes. What then would we ask for: That humans themselves would become their own successors, that whatever injustice occurs would be tempered by our knowledge of our roots. For those who remained unaltered, the goal would be benign treatment (perhaps even giving the stay-behinds the appearance of being masters of godlike slaves). It could be a golden age that also involved progress [...]. Immortality (or at least a lifetime as long as we can make the universe survive) would be achievable.

But in this brightest and kindest world, the philosophical problems themselves become intimidating. A mind that stays at the same capacity cannot live forever; after a few thousand years it would look more like a repeating tape loop than a person. [...] To live indefinitely long, the mind itself must grow ... and when it becomes great enough, and looks back ... what fellow-feeling can it have with the soul that it was originally? Certainly the later being would be everything the original was, but so much vastly more.

the mind's I

This "problem" about immortality comes up in much more direct ways. The notion of ego and self-awareness has been the bedrock of the hardheaded rationalism of the last few centuries. Yet now the notion of self-awareness is under attack from the Artificial Intelligence people ("self-awareness and other delusions").

Intelligence Amplification undercuts our concept of ego from another direction. The post-Singularity world will involve extremely high-bandwidth networking. A central feature of strongly superhuman entities will likely be their ability to communicate at variable bandwidths, including ones far higher than speech or written messages. What happens when pieces of ego can be copied and merged, when the size of a self-awareness can grow or shrink to fit the nature of the problems under consideration? These are essential features of strong superhumanity and the Singularity. Thinking about them, one begins to feel how essentially strange and different the Post-Human era will be — no matter how cleverly and benignly it is brought to be.

From one angle, the vision fits many of our happiest dreams: a time unending, where we can truly know one another and understand the deepest mysteries.

From another angle, it's a lot like the worst-case scenario [Vinge] imagined earlier in this paper.

"God is what mind becomes when it has passed beyond the scale of our comprehension."

Freeman Dyson

questions to the Goddess

What does it mean to be a mind? How is a mind affected by its embodiment? By the body's immersion in the world? By not having a body in the first place, or not any longer? Is the world out there what it seems? Is there a world out there?

WHY?

Sunday morning.