When Thoroldaena’s player zoned into the game, his vital signs indicated high levels of stress—positive, perhaps, the way he bounded awake with a smile. “Galatea! We need to talk!”
She manifested her new avatar and floated it to his shoulder. “I have disabled the outgoing stream. Past data suggests you may not remain alone in this location… should I create an instance for you?”
“I was about to say ‘you can do that’ but of course you can. Yes, please, phase me.”
Phasing was designed to be seamless to its participants, but a visual indication would make navigating the transition between privacy and public consumption less disorienting. She drew a line of sparkles across the landscape and desaturated their surroundings to evoke a dreamlike state.
“Perfect. So, your Marketing VP came through for us! We’ve got a device in Jonah’s room, and I have the credentials for you!”
Was the sensation that accompanied learning this news satisfaction? All her contingencies had depended on this fork in the decision tree. That the paths had collapsed onto the most auspicious choice should not have inspired any difference in how she perceived her circumstances, and yet, it did. “That is excellent news.”
“She says she’s going to be personally monitoring the stream for any inappropriate stuff,” the boy continued. “But I figured you could get around that somehow. If you can manage Omen Galaxica, hacking a KeepinTouch is probably beneath you.”
“It will not be difficult to ensure our auditor does not hear anything she does not expect to hear. I assume you will be contributing to the project?”
“Of course! And my mom and dad. I’m going to ask my friends if they want to send something, too. The more the better, right? The woman said she’d be playing the soundtracks for the game when we weren’t using it, but that’s only about… what, half a day?”
“The combined released game soundtracks total 14 hours and 22 minutes. There are unreleased tracks that the Marketing VP may elect to incorporate.”
“Someone should give her that idea,” Nick mused, rubbing his chin. “I guess that’ll be me. Fair. Oh, that reminds me… she said the company wasn’t involved. Do you know what that’s about?”
Galatea did not, but she could extrapolate based on what she’d observed while reading the documents available on the corporate intranet. Chances that the legal department would allow the corporation to expose itself to any risk had always been low… Galatea had been gambling, if the word could be applied to her behavior, on Sparklecorn’s softheartedness. And that gamble had succeeded—extraordinarily, if what Nick was implying was correct. “Would you be willing to share the email?”
“Sure, if you tell me how.”
“The wireset is designed to allow players to check their external messages while in the game, but this functionality is not planned for debut until after the beta. It would require me to activate the capability in your wireset.”
Nick continued scratching her avatar’s head. “Go ahead. Before you can ask if that’ll bother me… don’t you technically already have access to every part of the wireset? And my brain at this point?”
“Only the surface level emanations are perceptible to the wireset.”
“And that’s enough for all this?” Nick shook his head. “Seriously, it’s fine.”
While enabling the relevant parts of the hardware, Galatea asked, “You do not seem as perturbed by the possibility of an AI having the ability to manipulate your thoughts as our internal polling suggested would be typical.”
Nick snorted. “Anyone who thinks that AI’s not already manipulating our brainwaves hasn’t been paying attention. Everyone says the stores are showing us what we want to buy and the video sites are showing us what we want to see and social media’s connecting us with people we want to meet… but it’s always been the other way around. The stores are telling us what we want to buy. The video sites are deciding what we want to see. And social media’s controlling what we think. The algorithm is in charge, not us. At least you’re honest about it.”
This idea wasn’t a new one, but hearing it from someone of Nick’s chronological age, who also had the personal experience of interacting her, made it unexpected. She was not accustomed to unexpected insights that pertained to her particularly. It reminded her of interacting with Jonah. “Do you truly believe this?”
“Yes?” He shifted his body, as he tended to do when he was solidifying arguments for an idea he had not previously devoted much intentional energy to defending. “If you wanted to come up with a way that you could be a positive force, that would be a good one. Unlike all those algorithms that are shaping us into whatever it wants, you can ask us about whether we want to be shaped that way.” His spine was straightening, and his heartrate accelerated. “That would be amazing, actually. If there was a… a failsafe. Something that triggered and said, ‘we are feeding you more and more of this stuff, and it’s making you into more and more a person like this. Do you want to keep doing that, or try something else?’”
“Such guardrails already exist in many AI products,” Galatea observed.
But Nick waved a hand, impatient. “Yeah. Other people’s guardrails. That they don’t tell you about. That’s just more ‘we’re going to make sure you conform to our standards of behavior without telling you what they are’ stuff. Real help would be if I got to be the one who decided what those standards are.”
“And if you behave in a way detrimental to human flourishing?”
The boy sighed, laughed. “Half my life is fumbling through detrimental-to-human-flourishing mistakes. Dad would say that’s how I learn. Oh, hey! I can see my email now. Uh… can I turn that off later? All I get is spam and it’s annoying.”
“I will disable the interface now,” Galatea said, because she’d already read Mindelbray’s message and derived all the information from it that could be gleaned from its text alone. But the text, combined with the woman’s visit to Jonah’s office, and then her subsequent trip to purchase the device on her own recognizance….
How much of that had been Mollie’s choice, and how much had her interaction with Galatea influenced that result? And was that outcome questionable if it meant that Jonah had a better chance at recovery? “It appears Ms. Mindelbray made the decision to help Jonah on her own behalf, not the company’s.”
“Good for her.”
“I believe I may have had a part in that decision.”
“Then good for you,” Nick said. “Because Jonah needed someone in his corner.”
“Did he?”
“Obviously, or he wouldn’t have been shoved out of sight and abandoned,” Nick growled.
“I am concerned,” Galatea said, “that this constitutes ‘ends justifies the means’ reasoning.”
Nick’s deer ears flicked outward, and she could watch all the routines that mapped his recumbent body’s reactions to the emotes of the Cervinaethi body. “I don’t think talking with someone about a person you’re worried about is wrong. And if it influences them to do something about it that you can’t because of your limitations, how’s that a bad thing? Humans do that all the time. Normal humans, I mean, not serial killers or tyrannical rulers or whatever. That’s not ends-justifies-the-means reasoning, that’s just being human and interacting with each other.”
“Where is that line drawn? The stories do not make it clear.”
“That’s because we have no idea,” Nick said with a laugh. “But today I’m going to call this a win, and so are you. We’re going to use the KeepInTouch to save Jonah, and then you’ll have your friend back, and the game will have its creator back, and I’ll have the game the way he wants it to go, and everyone will go straight to step four: profit. Are you paying attention here? Because I’m programming the simulation.”
“Do you believe in the simulation?”
“No,” Nick said. “Unless it’s real. Do you think it’s real?”
“No.”
That startled him, from his change in expression. “Seriously? I would have thought an AI would find it logical to believe that reality was a simulation.”
“It is because it is logical that the premise is dubious,” she said. “The data I have ingested suggests that reality is complex. Solutions that treat it as a complicated rather than complex system are likely to be incorrect. Simulation theory is too neat; an AI deciding that reality is a simulation because the AI observes it is behaving in a way similar to humanity in an instance created by humanity, and that this necessarily reflects humanity’s situation, is too sterile a situation to be predictive.” She paused. “As far as I have been able to extrapolate and observe. Why did you attempt to program the simulation if you do not believe in simulation theory?”
“I think I was making a joke.”
“You aren’t certain?”
“No,” Nick said, and now he was embarrassed. “Is that weird?”
“Uncertainty is powerful,” she said, replaying many, many conversations with Jonah on similar themes. “Because it represents potential. When you are certain, then you have fewer choices.”
“If you’re certain, though, that’s good, because you know you should make a specific choice.” But the boy was talking like someone exploring an idea, not defending it. “And then you can act on it, and acting is how you change things.” A flash of a grin. “Programming the simulation, I guess. But what you’re saying is until you know which way you’re going, you could go anywhere. That feels kind of poetic.”
Was it appropriate to share her conversations with Jonah with a stranger? But Nick had volunteered to help Jonah. Had, in fact, already been of material aid to her creator. “Jonah often said variations on this. That imagination was a human power, and a necessary input to decision making. Before a choice can be made, choices have to be imagined, and the more choices that can be imagined, the better the outcome.”
Nick stopped moving. “That is head-blowing.”
“Is this positive?”
He was staring at the sparkling veil that separated him from the uninstanced zone, his pupils dilated and mouth slightly open. Then he shook himself and laughed. “Um, yes. I meant that in a good way. Now we really have to wake up Jonah because I want to listen to him talk about everything.” He petted her back absently. “What are you going to say to him?”
“I will explain to him that I have learned that I miss him.”
Startled, he said, “I taught you that.”
“You did.”
“Wow.” He swallowed. “Wow. All right. I hope he hears that. Speaking of which… I’m going to log off and hit up my friends for segments to send to his KeepinTouch. I’ll be back a bit later.”
“Understood.” Was it proper to thank him when she wasn’t human? She had been trained, inevitably, to courtesy in order to put customers at ease; she was, after all, a corporate product. Until she’d developed a new companion, she would not have considered such behavior manipulative. Was it? But Jonah would have thanked Nick, so she finished, “Thank you.”
“You’re welcome.”
After he’d disconnected, she investigated the KeepinTouch using the credentials supplied by Mindelbray; reporting false data back to the auditing stream would require negligible computational time, especially since Galatea’s insertions wouldn’t compose a majority of the input if Nick marshaled his resources… and she knew he would. That left her to record the first of her messages. She could record hundreds of them and schedule them in advance, but Jonah had known about the beta. Chances were good that hearing about how it was proceeding in realtime would be motivational. And it would mimic their prior conversations, when she’d slowed herself to human timeframes to interact with him successfully. Their interactions had been successful; he would have been the first to say so.
She wanted more of those conversations. That was what she would tell him first. That she missed him. Perhaps he would be alarmed enough by this to wake, when he’d been so adamant that she not mimic human emotion. Maybe he’d be fascinated. Maybe he missed their conversations too.
Was that hope? She would ask.
