CHAPTER SIX
“A lot of people are starting to say that TITAN could go just that way. What do you think, Ray? Could it evolve the capacity for feeling emotions? Could it develop an awareness of its own existence?” Dr. Jacob Manning, one of the three who had arrived from Princeton earlier in the day, put the question while they were summing up in Dyer’s office after seeing Hector in action. The subject of the discussion was the notion that TITAN might integrate its capabilities on a global scale sufficiently to emerge as an intelligence in its own right.
“Obviously we can expect to see its behavior becoming more coordinated worldwide as time goes by,” Dyer replied from where he was sprawled in leisurely fashion behind his desk. “If you take as a working definition of intelligence: ‘A measure of a system’s ability to learn from experience and to modify its own behavior appropriately to what it has learned,’ then we’d have to concede that TITAN has taken a rudimentary step in that direction already. So yes—it could become intelligent. But I think it would be a mistake to draw too many inferences based on the human model.”
“How do you mean—emotions and that kind of thing?” Sally Baird, also from Princeton, spoke from the far corner.
Dyer nodded. “What are emotions?” he asked. Without waiting for an answer, he continued. “I’d submit that an emotion is a stereotyped behavioral pattern that’s been reinforced through natural selection because it has demonstrated a survival value. Obviously an animal that gets mad and fights or gets scared and runs stands more chance of staying healthy than one that feels nothing, does nothing and gets eaten. You buy that?”
“I’ll buy that,” Sally agreed.
“Good. So if you take a more general view of it, an emotion is a behavioral tendency that a self-modifying system evolves because it is beneficial in helping the system to accomplish whatever its basic programming compels it to want to accomplish. In the case of organic systems that have arrived via the mechanics of organic evolution, the ‘whatever’ happens to be survival.”
“Yes, I think I see what you’re driving at,” Steve Conran came in from beside Manning. “You’re saying that an inorganic intelligence might well evolve its own compulsive traits but there’s no reason why they should bear any resemblance to human emotions. Our emotions derive from the survival needs and wouldn’t have any inherent value to a system that came from origins that were totally different.”
“That’s what makes the most sense to me,” Dyer affirmed.
“Interesting,” Manning mused, half to himself. “I wonder what traits they’d be. As Ray says, they’d probably reflect whatever basic function the system had been designed to perform.”
“Well, in the case of something like TITAN, it might turn out to be insatiably curious . . . or compulsively rational, or efficient or something,” Conran suggested. “Certainly, if you look at it that way, there’s no reason why it should become a threat to anybody, is there?”
“So how about awareness?” Sally asked. “What’s the likelihood of it getting to the point where it not only knows, but it knows that it knows?” Dyer was nodding even before she had finished asking the question.
“Again, I think that if it ever came to possess anything like that, it would be radically different from a human understanding of awareness.” He paused to collect his thoughts for a second while the three visitors waited expectantly. “A man is aware of himself as existing in the localized region of space that’s defined by the focal point of his senses. He has evolved the ability to construct mental models of extensions to that space, which he and other objects move around in. But something like TITAN will perceive the universe through billions of sensory channels distributed all over the surface of the Earth and beyond. On top of that, its ‘senses’ cover the whole spectrum from high-power proton microscopes in research labs to the big orbiting astronomic telescopes . . . from galactic gravity-wave detectors to the infrared sensors lowered into the ocean trenches.” Dyer swept his eyes across the three faces in front of him and spread his hands expressively. “TITAN moves pieces of itself around in millions of places at once—the ISA probes that are nosing around Jupiter . . . robot freighters under the Arctic ice caps . . . all kinds of things. How can we even begin to imagine how an awareness as totally alien as that would perceive itself and the universe around it?”
They went on to debate this issue at some length.
Eventually Jacob Manning raised the question of Man being able to devise a means of communicating with an intelligence as alien as this.
Dyer thought about it and then replied: “Well, we already do communicate with it all the time, of course . . . at least we do with parts of it. But if you mean what I think you mean—will we ever get to talk to it as a total entity?—then I’m not so sure.”
“How come?” Sally asked, sounding mildly disappointed.
“You can see the problem if you take a human organism, say, as an analogy,” Dyer replied. “All the cells and organs in the body communicate among themselves in their own specialized languages—chemical messengers, neural codes and that kind of thing. But the individual as a totality doesn’t comprehend those languages. The processes that they relate to are controlled unconsciously.
“I think that the same kind of thing would apply if TITAN ever evolved the kind of self-aware intelligence we’ve been talking about. The economic, commercial, industrial and all of the other processes it’s built to take care of are all parts of a vaster organism. It wouldn’t be cognizant of the languages in which the transactions going on inside itself were being conducted . . . What language it would be equipped to speak, I don’t think any of us can imagine at this stage.
“So to answer your question—no, I don’t think we’d be able to talk to it . . . not all of it. That’d be like a cell somewhere in your big toe wanting to talk to whoever runs the whole body. The two just don’t think or talk at the same level.”
When the Princeton delegation eventually departed, having declared themselves more than satisfied by the day’s accomplishments, Dyer escorted them back to the main entrance. Just as he was turning away from the door, a tubby olive-green-suited figure with a pink face and equally pink bald head bounded through.
“Ah, Ray! Just the person. I was meaning to talk to you about a couple of things.” It was Professor Edward Richter, head of the Shannon School and Hoestler’s immediate boss.
“Hello, Ted. How are things?” Dyer halted momentarily to allow Richter to fall in step beside him.
“I only got back from California earlier today,” Richter said. “I was talking to Vince Lewis over lunch . . . Hello, Peggy . . . He told me about yesterday. It’s tough about your unit, huh?”
“Well, nothing’s final yet,” Dyer answered, with a philosophic shrug.
“I hope you’re right. You and your people were just starting to make progress, too. It came at a bad time.”
“You should see the latest,” Dyer told him with feeling.
“Looking good, eh?” Richter shot an approving sideways look as he bounced jerkily along. “I called in on Frank Wescott’s outfit while I was at CIT. They’re still way behind what you’re doing. You’re out in front in a league of your own if CIM doesn’t go and pull the funding rug out and foul it up.”
Dyer agreed automatically. At least, he reflected, it was something to know that his feelings weren’t unshared. It was no secret that Hoestler was angling for an early retirement and that he was more interested in making waves with his yacht than inside the Department. Lewis was guaranteed to drift whichever way the tide of political expediency happened to flow. But Richter had the energy, the commitment and the clout to emerge as a valuable ally if a move to lobby CIM turned up on the cards. Richter’s vigorous campaigning had brought a lot of CIM money to CUNY in years gone by, and had probably proved as important a factor as any in the Shannon School becoming the focus of HESPER development in the United States.
“Is Frank affected by the latest from CIM, too?” Dyer asked.
“I don’t know. They didn’t mention anything about it while I was there, but maybe it hadn’t gotten down the line yet. What they are getting is a fair amount of flak from different places about TITAN going its own way someday and turning into some kind of alien intelligence that decides to do its own thing. That seems to be the latest scare story going around. I don’t know . . . maybe something about the Maskelyne screw-up has leaked out somewhere.”
“Interesting,” Dyer mused. “We just had some people here from Princeton who were probing more or less the same kind of thing. I hope the media don’t get involved and start a public hue and cry about it.”
They halted at the intersection of the main thoroughfare of the Self-Adaptive Programming Department and the corridor that led to Dyer’s unit. Richter drew a pace nearer to Dyer and regarded him quizzically for a moment.
“Tell me something, Ray,” he said. His voice had dropped a decibel from its normal public-address mode. “What’s your opinion on all this? Could TITAN turn into a real threat one day? If there’s a risk that it could, what things ought we to be doing about it now?” Richter was obviously referring to the questions raised within CIM that Lewis had outlined in Hoestler’s office the previous day. Dyer had been thinking about little else ever since. His reply was slow and thoughtful.
“We have to accept that it could develop behavior patterns that we’d be forced to describe as intelligent. It might become self-aware, and if it did I’m certain that its awareness would be completely alien to anything that we know. But I think a lot of people are trying to project human qualities onto it without any real justification. That I can’t see.”
Richter nodded his head vigorously as if his own feelings on the matter had just been confirmed.
“That’s what I hoped you’d say,” he agreed. “So we’re in agreement regarding where we ought to go from here, eh?”
“What’s that?” Dyer asked.
“Go flat-out on developing FISE,” Richter replied, sounding slightly surprised. “The answer to avoiding more Maskelyne-type problems is to get something like FISE in, surely, not to back off and pull HESPER out.”
Dyer chewed his lip dubiously. “I’m not a hundred percent convinced it is,” he said. “I’ve been doing a lot of thinking about it. It wasn’t all that long ago when I said I was confident I understood HESPER, but look what’s happened since.”
“Sometimes it can be worse to stick with half-measures than to go the whole way,” Richter insisted. “I think this is one of them. The trouble with HESPER is that it’s halfway between machines that are totally predictable and totally dumb, and machines that are smart enough not to be dangerous. We’re in midair over the high jump and it’s just as easy to land one side as the other . . . except that if we choose backwards, we’ll only have to do the whole thing again later. I say let’s finish the job now while we’ve got the momentum to do it.”
“But what if it turns out we don’t know as much as we thought we knew again, Ted? Okay, we go ahead and put smarter FISES in everywhere and the system starts doing things we don’t like. What then?”
Richter shrugged.
“Then we take ’em out if we have to. We either fix it or pull the plug.”
“And take the risks in the meantime, huh?”
“What’s got into you, Ray.” Richter gave Dyer a puzzled glance. “We take risks every day of the week anyway. This one is no bigger than any and a lot smaller than most. When you buy a house, you know you may have to spend money to repair it one day. You don’t go live in a tree instead in order to avoid the risk. You accept the risk because the benefits outweigh it.”
“I don’t know so much . . .” Dyer studied the floor between their feet. “Building a machine that might not work out is one thing, Ted. Handing a whole planet over to something you’re not sure you understand is another. Two days ago I’d have agreed with you, but now I’ve seen what can happen even with HESPER . . . I don’t know . . .”
“Aw, come on, Ray,” Richter urged. “You’re just taking it to heart because you had a lot to do with designing HESPER in the first place. Try and get away from the personal angle.”
“We’ll see,” Dyer murmured.
“Well, I hope you’ve straightened it all out in your head by Thursday,” Richter said. “Has anybody told you yet that we’re going to Washington on Thursday?”
“Washington? No. I’ve been tied up with these Princeton people all day. Who’s going to Washington? What’s happening there?”
“There’s a meeting being held at CIM headquarters,” Richter informed him. “A lot of government and technical people are getting together to try and agree on some recommendations as to which way we go from here. Vince asked me to go because he’s tied up in the city on Thursday. He told me I ought to take you along too, since you’re the HESPER giant. Get Betty to call my secretary for the details. I think the meeting’s due to start around ten, so we could go down on the tube.” Richter frowned to himself and shook his head. “I was going to push for going ahead hard on FISE. It could mean another big contract for us, you know. I’d assumed you’d be backing me up. Hell, it is your unit that’s at stake here!”
“Oh, you don’t have to worry on that score,” Dyer said. “I’ll back you all the way on pushing for ongoing FISE research. What I’m uneasy about is the idea of putting something like FISE into the net prematurely. If we could only make CIM accept that those are two separate issues. From what I’ve heard, they seem to have gotten it into their heads that they both have to get scrapped or fly together.”
“It’s the accountants who are running the show as usual,” Richter declared. “They’re saying that if they’re not going to see any payoffs from FISE in the foreseeable future, they don’t want to put any money into it in the foreseeable future.”
“I know. Anyhow, we’ll just have to wait and see what happens on Thursday. It sounds like it could be interesting.”
“Fine.” Richter paused to rummage further through the pending section of his eternally overflowing mental file cabinet. When he spoke again his tone was guarded and conspiratorial.
“And don’t forget what we said about SAP coming up for grabs in the not too distant future. It’s pretty obvious that Sigmund doesn’t have any big plans on staying around for a long time. You’ve more than earned a move up and it would be a popular choice.”
“That’s okay. I haven’t forgotten about that,” Dyer murmured. “But I’d prefer to clear out things up first. Now that this HESPER thing’s boiled up I’d feel better in the front line until it’s over. Don’t forget I had a lot to do with putting it together the way it is.”
“I know that,” Richter said. “But you should think hard about it. Strictly between you and me, there’s a good chance that Vince might be open to persuasion to put internal funding in to keep FISE going even if CIM does pull out. It’s only a possibility though, you understand. With a SAP hat on your head you could end up doing FISE a lot more good in the long run than by hacking code in the front line. You think about it.”
Dyer nodded and indicated that he understood Richter’s meaning.
“Good!” Richter exclaimed in a hardy voice. “It’s all set for Thursday then. I gotta fly. You think about what I said. Oh, another thing . . .” Richter had started to walk away. He continued speaking as he swung into a U-turn and headed back again. “That movie company that Vince brought in—Zeegram or something—they’re involved in some award dinner in town tonight. They sent Vince a complimentary ticket. He can’t make it and neither can I, but he’d like the University to send somebody all the same. Have you got anything special fixed up?”
“Tonight? No. Want me to take it?”
“I’d appreciate it. You never know, it could turn out to be fun. I’ll call you in five minutes from my office with the details. You’ve got an escort provided by Zeegram thrown in. I think she’s been here a couple of times. Fanning . . . Fenning . . . something like that. I’ll let you know.”
Dyer stared after Richter’s once-again receding figure.
“Thanks a lot,” he said flatly.
He was still staring after Richter’s retreating figure when Kim appeared from the same direction. He waited for a moment and they began walking toward the lab together.
“What did you do to him?” she asked, giving Dyer a look of unconcealed astonishment. He returned it with a puzzled frown.
“Who, Ted?”
“No. Our mutual friend in Internal Services. Judy’s been on mural graphics all day and he can’t do enough to help. I asked you to kick his butt, not set fire to it.”
“Oh that.” Dyer couldn’t contain a wry grin. “It’s amazing what a few friendly words delivered with tact can do, isn’t it?”
“Really?” Kim sounded dubious.
“You know me,” he said in a voice that could have meant anything. Kim’s expression deepened to one of outright suspicion.
“And another thing,” she said. “Something’s happened to Al too. He’s got through a week’s work since this morning. You couldn’t possibly have had anything to do with that as well, could you?” Dyer tossed his open hands carelessly but said nothing. “Which reminds me,” Kim went on, “there are a few points he’s brought up that I think we should go over sometime. It’s getting a bit late now but I’m not in any particular hurry. How would you feel about staying on and getting it out of the way tonight?”
“ ’Fraid not tonight, Kim,” he said as they stopped in front of the lab door. “Any other time would be okay, but tonight I have a date.”
“Say . . . how about that!” Kim’s eyes widened ominously. “Who’s the lucky lady?”
Dyer sighed as they went in. “You wouldn’t believe me if I told you.”