CHAPTER FIVE
Dyer arrived at Sigmund Hoestler’s office a few minutes before 2 p.m. He was shown straight in and to his mild surprise found that Vincent Lewis, the Dean of the Faculty, was there too. Hoestler, a big man with sagging fleshy cheeks and a shock of uncontrollable wiry hair, motioned Dyer into an empty chair next to where Lewis was sitting, and leaned forward to come straight to the point.
“I’m afraid we have some very serious problems that are going to affect you directly, Ray,” he said in his usual throaty voice. “It looks as if we may be forced to close down your unit.”
Dyer was halfway through the process of sinking back into a characteristically relaxed posture. The bombshell made him sit up again as if the chair had suddenly acquired a few kilovolts. He knew that Hoestler was a man of few words, but even so, the bluntness of the statement had caught him totally unprepared. He had barely begun opening his mouth to frame a question when Hoestler spoke again.
“I only found out about it myself this morning. Vince was in Washington over the weekend with the Secretary for CIM and some of his people. So don’t get the idea that it’s just petty local politics or anything like that. Vince, you could probably tell Ray about it better than I could.”
Dyer turned expectantly toward Lewis, his features contorted into a frown of disbelief. Communications And Information Management was a comparatively new executive department of state, formed eighteen years previously in 2010. Originally it had been instituted in response to the need for a single authority to assume overall responsibility for operation of the integrated data communications and computing network that emerged when the military systems were declassified and merged into the already integrated commercial-industrial-scientific complex to form the EARTHCOM net. When HESPER nodes were later incorporated to transform EARTHCOM into the Totally Integrated Teleprocessing and Acquisition Network, TITAN, the Department of CIM automatically became the administrative authority for the NORAM Sector of the global system. As Hoestler had in effect said, the Department of CIM didn’t mess around with interdepartmental university politics.
Lewis was impossibly tall and impossibly thin. He sat splayed in his chair at all angles like a marionette whose limbs had come out of joint everywhere, leaving him held together only by his clothes. When he was standing up he never failed to cut a distinguished figure, with his elegant crown of white hair, deeply lined face and inevitable immaculate, dark three-piece suit. Dyer had always found him something of an aloof and remote kind of person, but right now Lewis was showing every sign of distress and genuine concern.
“Certain events have happened recently, Ray, that have caused CIM to reconsider the whole philosophy of adding HESPER capability into the net,” he said. “Some very senior people are pressing for TITAN to be reverted back to EARTHCOM until we get firm answers to some important questions. In a nutshell, they’re saying that the move to upgrade EARTHCOM was premature, that we didn’t know enough about HESPER at the time and we still don’t, and that HESPER ought to be pulled out until we do.”
Dyer looked from one to the other and spread his upturned palms.
“Events . . . ? What events?”
“About a week ago, TITAN came within a hair’s breadth of killing five people,” Lewis told him somberly. Dyer stared at him incredulously. Before he could say anything, Lewis went on. “It appears that HESPER program structures are capable of integrating to a far greater degree than anybody thought. They’re starting to link things together in ways they were never supposed to and the results in behavior are impossible to predict.”
Hoestler explained, in response to the still bemused look on Dyer’s face. “It used the Maskelyne mass-driver to bomb an ISA survey team on the Moon. Could have wiped them out.”
“What?” Dyer turned an incredulous face toward Lewis but the Dean nodded regretfully to confirm Hoestler’s words.
“One of the HESPER-controlled subsystems in the Tycho node was given the job of shifting a piece of terrain that was forming an obstruction,” he explained. “It was supposed to use normal earth-moving equipment to do it, but nobody bothered to tell it that. Somehow it managed to connect together information from several subsystems that shouldn’t have been connected, and came up with what it thought was a better shortcut to solving the problem. According to the people who analyzed the system dump afterward, it seemed quite proud of itself.”
Lewis went on to describe the incident on Luna in greater detail. As Dyer listened, his initial astonishment changed to growing concern. In 2020 he had moved out of neurological research in order to apply his knowledge of learning psychology to the field of self-adaptive programming and, after spending some time at M.I.T., had come to CUNY to set up the HESPER Unit, which had since gone on to spearhead development of the very techniques that were now being applied worldwide to transform EARTHCOM into TITAN. His knowledge of the technicalities of HESPER programming was shared by fewer than a handful of people. If it was anybody’s, it was his baby.
“Unfortunately there happened to be an ISA team sitting practically on top of the target,” Lewis continued. “But naturally, that didn’t mean very much to the computers.”
“Twenty sixty-pound packages of rock coming down at over a mile a second,” Hoestler commented. “Every one was roughly equivalent to a two-thousand-pound bomb.” He shrugged and made a face.
HESPER machines were learning machines, designed to be capable of identifying connections between previously nonrelated factors in order to solve new problems or to solve old ones in newer and better ways. But if what Lewis had said was correct, this capability was beginning to extend itself in ways that had never been intended, nor in fact even foreseen as possible. If the obstruction had been on the edge of Maskelyne Base itself instead of out on some remote construction site on Procellarum, there could easily have been a death toll of hundreds. And if this kind of thing could happen in the circumstances surrounding the events on Luna, what other kinds of things might happen anywhere, at any time?
They could easily instruct TITAN never to do that particular thing again, it was true, and TITAN wouldn’t, but that wasn’t the point. The point was that TITAN had demonstrated a capability to approach a perfectly reasonable objective from a totally unexpected direction, and in doing so come up with a solution that was inarguably rational from the machine’s point of view but which, for other reasons that could never with the present state-of-the-art be conveyed to the machine, was absolutely unacceptable. Its next such experiment might well result in worse than a mere narrow escape.
“Okay.” Dyer exhaled and nodded curtly. “I can see the problem. What I don’t see is how it affects the unit. What has all this got to do with closing the unit down?” A new expression of disbelief spread across his face as a possible answer struck him. “You’re not telling me they’re panicking and putting a total ban on further research are you? That’s ridiculous! They’re gonna need all the expertise and facilities they can get if they’re going to straighten TITAN out. We’ve got just the—”
Lewis interrupted with a wave of his arm and a shake of his head. “I didn’t mean we’re going to throw everybody out on the street,” he said. “But the projects that your unit is currently working on are probably going to be stopped. That line of research is being funded by CIM with the aim of producing the technology that’s supposed to replace HESPER one day. Now the guys at CIM are saying that they don’t even want to think about what comes after HESPER because it’s obvious we don’t understand HESPER yet as much as we thought we did. In fact a lot of people are saying we should tear HESPER out of the system completely and only think about putting it back in when we can prove it’s safe.”
“In other words the money being spent on FISE could better be spent on other things for the time being, so FISE goes down the tubes,” Hoestler summed up.
“I’m sorry, Ray, but it looks as if that’s the way it is,” Lewis said apologetically. “As you yourself more or less said a second ago, there’s going to have to be a big re-examination of the whole HESPER concept. We’ll probably be able to reassign your people to a new CIM contract in that area as soon as some specific objectives have been worked out. In the meantime, if I were you, I wouldn’t waste too many nights’ sleep hoping for any Nobel Prizes. You’ll probably have to wrap it all up pretty soon.”
Kim was just coming out of the lab when Dyer arrived back at the outer door to his office.
“Hi,” she greeted cheerfully. “Betty told me you’ve just been over to see Hoestler. Did you get a chance to mention the business with the graphics moms?” Dyer turned his head in her direction but his eyes were far away.
“Uh? Oh . . . er no,” he mumbled. “I’m sorry. I guess I must have forgotten about it.” With that he walked on in, leaving Kim wearing a puzzled frown.
He sat for a long time, staring at the papers on top of his desk. Lewis’s revelations had shocked him to the core in a way that he was only now beginning to appreciate. He had been as convinced about the potential benefits of HESPER as he had about anything in his life, and he had devoted more than a little effort to convincing others. TITAN had gone ahead on the basis of his recommendations as much as anybody’s. To be sure, the final decisions had not been his to make, but the people who had made them had relied on the facts that he and others like him had presented. And a whole world had relied on those people and their advisers.
His mind went back to some of the things that Laura had said over lunch and to the confident—almost arrogant—reassurances that he had voiced a little over an hour before. Suddenly he felt far from reassured himself. He didn’t feel arrogant at all.
He rose and went through the inner door into the lab. Betty greeted him with a couple of messages which he only half heard, one of which was a reminder that some members of a research team from Princeton were due the next morning and would be spending most of the day with the unit. Pattie tried beguiling him with a silent, innocent, wide-eyed look which he ignored. At that moment Judy Farlin came out of Kim’s office, rummaged around in a file drawer for a few moments and then went back in carrying a folder. Dyer turned abruptly and went back into his own office where he called the Superintendent of Internal Services, bawled him out at considerable length and secured a guaranteed reservation for Judy for the first thing the next morning. He came back out, gave the details to Betty and asked her to pass them on to Judy. Then, feeling a little better, he went on through to the lab bay to see how Ron and Chris were getting along.
Hector propelled himself across the floor of the kitchen, stopped in front of the broken window and paused while FISE considered the situation.
“What happened?” Dyer asked. Ron, who was standing with his elbows resting on the opposite side of the tank, raised his head.
“We told him that the garbage pail had to go out in the yard.” he explained. “So he threw it through the window.” Dyer grunted and returned his gaze to the tank.
Hector reached out and grabbed hold of one of the jagged fragments of glass that remained around where the window had been. PROPS immediately caused a vivid red line to appear across Hector’s hand. The gash proceeded to ooze blood profusely but Hector ignored it and continued tugging experimentally at the piece of glass in an effort to remove it.
“Hold it. Hold it there, Chris,” Ron called out. The figure in the kitchen froze. “Now FISE,” Ron said, adopting his stoic voice. “There are a few more things that you have to get straight about Hector. Glass cuts. Hector does not like being cut. You don’t cut bits off him or permit him to be cut by anything if you can avoid it. Okay? You have to find a way to fix the window without cutting Hector in the process.” A few seconds elapsed while Chris completed keying in the last addition to FISE’s growing store of information.
“Question,” FISE’s voice said from the grille.
“What?” Ron inquired.
“When Hector was shaving, his hair got cut. Why was that okay?”
“Oh yeah, I forgot that,” Ron agreed. “When any part of Hector’s body starts to get cut it hurts, just like you already know for things that are too hot. When he feels that, he’ll respond with a reflex that overrides everything else he’s doing. Hair is an exception. It doesn’t hurt when it’s cut. An unshaven face is not a nice thing. Shaving in the morning is okay.”
“Thanks,” FISE acknowledged.
“Before you go any further, let’s just try something,” Dyer suggested. “I’d like to see how well it understood what Ron just said. Chris, could you reset to the point just before where Hector grabbed at the glass, and force the same action.” Chris took a while to compose the commands. Dyer and Ron watched intently as Hector flashed back to his previous starting posture and approached the window once more. He grabbed at the glass as before but this time his hand jerked back again instantly. PROPS could justify no more than a slight scratch.
“Not bad,” Ron conceded, sounding impressed.
“I’m almost tempted to suggest that we might be safe in upgrading his IQ to one,” Chris murmured, leaning back in his chair to stretch his arms.
Dyer felt a sudden urge of excitement They were getting there! It was a slow and tedious business, certainly, but the first signs were there. It was all beginning to come together. To cut it off at this point would be tragic.
“Reset again, Chris, and let FISE handle it himself,” he said. “Let’s see if he can figure out a better way.”
Hector tried several approaches, including wrapping his hand in the tablecloth and then in a towel, but Ron vetoed all of them. Eventually FISE gave up and Ron supplied a hint by suggesting that if Hector looked in the utility closet, he might find something with which to knock out the pieces of glass.
“The hammer is used for knocking things, but it would break the glass,” FISE protested. “You told me that breaking glass isn’t okay. What am I supposed to do?” Ron got excited again and delivered a lengthy exposition on the profound insights required, after which Hector made a reasonable job of clearing and cleaning out the window frame. Chris told PROPS to materialize a pane of glass and Hector placed it squarely in the frame after first, on his own initiative, stopping to put on a pair of gloves that just happened to be in the utility closet.
“This is interesting,” Dyer commented. “Look. He didn’t just turn away. He’s waiting and watching the glass. FISE has connected it with something else he’s learned somewhere that’s telling him it might not be very stable.” Sure enough, PROPS weighed up the shape and angle of the pane, couldn’t make up its mind and flipped a random number. The pane began to fall inward. Hector stepped forward, caught it in one hand and repositioned it more carefully.
Three enthusiastic roars of approval greeted the performance. For once, Ron treated FISE to a jubilant stream of ungrudging congratulations. Chris reconsidered his earlier statement and suggested that the machine might qualify for whatever IQ category lay above one. Although the thought had been half in his mind, Dyer decided it was not the time to mention the things that had been said in Hoestler’s office earlier in the afternoon. After all, he told himself, Lewis had not gone further than using the word “probably” several times. Nothing firm had been decided yet.
“Chris and I are gonna go eat out while we’re deciding where to go later,” Ron called across the lab as he pulled on his coat. “Want to join us, Ray?” Dyer looked up from the console, where he was studying some of the new coding linkages that FISE had constructed in the course of the afternoon’s exercise.
“Huh? Oh, no . . . I think I’ll stay on a while. This looks interesting. I’ll have to take a rain check on it. Thanks all the same.”
“Okay. See ya tomorrow.” Ron moved away toward where Chris was standing waiting by the door. “Chris, great idea!” he said as they began moving out into the corridor. “Why don’t we catch a game later? There’s one on tonight that I promise ya is going to be terrific.”
“Netball or rounders?” Chris’s voice inquired disdainfully.
“Hey, you’ve said that before. What is all this stuff?”
“Oh, just a couple of English games,” Chris told him matter-of-factly.
“Really? Big-league stuff and all that, eh?”
“No. Actually they’re normally played by schoolgirls.”
“Schoolgirls?” Ron’s voice rose in sudden outrage. “SCHOOLGIRLS! Hey, exactly what the hell are you getting at? If you’re telling me what I think you’re tel—” Dyer grinned to himself as the exchange faded away. Judy left with Betty and a few minutes later Allan tossed across a sheepish goodnight and went out to join Pattie, who was waiting in the corridor. Dyer returned to the displays glowing on the console.
One of the basic objectives of FISE was to investigate ways of enabling the computer to make generalized inferences from a few specific experiences, in much the same way as a child learns. The incident with the window bothered Dyer despite its encouraging aspects. FISE hadn’t been able to generalize sufficiently. If he knew enough not to allow Hector to burn himself, he should have been capable of generalizing to the extent of not allowing Hector to harm himself at all. Dyer had an idea where the problem lay and he spent the next two hours building a new section of system code and modifying some key parameters. Then he switched on the tank again to set up some situations to test out what he had done.
He gave Hector a few simple tests to check that he had not inadvertently introduced any major bugs, and then left Hector to put the tools away while Dyer thought about the next part. He wanted to find out if Hector would deduce for himself that an object in contact with a hot object would itself get hot and should therefore be avoided.
At that moment the kitchen door was nudged open and Brutus ambled in. Brutus was Hector’s comical white dog, a mischievous scamp with a black patch around one eye. He tended to appear from time to time when PROPS was getting bored. Dyer smiled faintly as he watched. Brutus wandered on into the room and Hector’s head suddenly jerked around to look directly at him, which meant that FISE had just become aware of the dog’s presence.
Nothing unusual happened for a while. Brutus sniffed here and there as PROPS commanded and Hector looked around every now and again to keep track of his movements, giving an uncanny simulation of human behavior in the process. And then Brutus moved over to the window. At once Hector rushed over from the utility closet, scooped Brutus up in his arms, hauled him across to the door and threw him outside.
Dyer froze the image and blinked at it in amazement. Then he became excited. He hastily composed a message requesting an explanation for the action and played it into the console. FISE’s voice responded at once.
“There are pieces of broken glass on the floor by the window. Glass cuts. I must not allow Brutus to get cut. It’s not okay.”
Dyer was astonished. Nobody had ever said anything about Brutus getting cut; they had told FISE only about protecting Hector. Dyer queried the point accordingly.
FISE replied, “I am hurt if I get burned or if I get cut. Brutus is hurt if he gets burned. Therefore Brutus is probably hurt if he gets cut too. Brutus is like me. Things like me and Brutus do not like being hurt, I must stop us from getting hurt if I can. Allowing things like me and Brutus to get hurt is not okay.”
Dyer was stunned. FISE had made the gigantic step of generalizing Hector’s basic attributes to include other animate objects . . . and without being told to!
He stared across at the blank cubicle that housed FISE and shook his head in wonder. To kill it now would be madness, he thought to himself. Just twelve more months at this rate . . . He sat up with a start.
Kill it? Christ! He’d caught himself thinking about it as if it really were a living being. He clicked his tongue in self-reproach. Getting sentimental about a machine. That would never do. What’s the time? Hell! Eight already. Time to go home and fix something to eat. Sharon will probably call. Not really interested . . . Eat out and see if any of the guys are around tonight.
He shut down the system and walked over to the door to collect his windbreaker from the stand. At least, he thought as he doused the lights and turned to survey the deserted offices, he was now certain in his own mind of one thing: Ripping HESPER out of TITAN was not the way to go. Backward was never the way to go. Given some improvements, FISE would never make the kind of error in judgment that the HESPER machines at Tycho had made. The way to get things right was to go full speed ahead on perfecting FISE and getting it into the net, not shrinking back from it and running the other way.
When he was halfway down the corridor he remembered something else: Three years previously he had been just as certain about HESPER. Suddenly he didn’t feel so sure.
None of the guys were in town that night but he found that he really didn’t care very much. He had enough on his mind to occupy him.