“Not the hardware,” Milo said. “Software. Everyone’s trying to get a piece of the action, which is going to be great once there’s a hundred percent fleet penetration. It’ll reduce congestion, pollution, traffic accidents, reduce the cost of getting from A to B, and free up tons of time and land because the goal is to go from personal ownership to transportation as a service, with full-time vehicle utilization meaning no need for parking lots. These are all good things. Everybody agrees.”
“So what’s the bad news?”
“Well, at less than a hundred percent—say, anywhere from seventy-five to ninety-five—pretty much every simulation shows AVs hunting in packs to drive non-AVs off the road. Turns out the onboard AIs believe their own hype. And since the world’s going to be a much better place without human drivers, well, the logical thing to do is accelerate their retirement.”
“That’s quite the hiccup there,” George observed.
“Oh, and it gets worse,” his tablemate continued. “Seems there’s this ethical dilemma from way back called the ‘trolley problem’ that cuts to the heart of AV, even after complete fleet penetration. It comes down to what happens when the car is faced with a situation where somebody’s going to die regardless of what it does—either its passenger or a school bus full of kids.”
George nodded, yes, yes—he was quite familiar.
“Well, they’ve tried a few things, like Asimov’s Three Laws of Robotics.”
“Seems like a good place to start,” George said, recalling his own recent work with Buzz.
Only Milo was across from him, making that “wrong answer” noise.
“What’s wrong with the I, Robot approach?”
“Try reading the book,” Milo said. “The three laws are just story-generating devices to show how they don’t work. So the next bright idea was to use some variation of utilitarianism. You know, ‘the greatest good for the greatest number’?”
Indeed George did; it was one of Roger’s favorite rules to live by, and they’d spoken of it often. His client liked it enough to incorporate it into Buzz’s own code, along with Roger’s corollary, “What if everyone . . .”
“Turns out, utilitarianism’s way too simplistic,” Milo continued. “For one thing, it assumes we all agree on what’s good. It would be nice if we did, but we don’t—not even close. Go online. People will disagree when they don’t really disagree. Disagreeing has become an easy way to feel better about yourself for being smarter than everybody else. I call it the assholier-than-thou syndrome.”
“Good one,” he said, laughing, and remembering some exchanges he’d already had with beta Buzz that certainly qualified as examples of that syndrome.
“And then we get into the radioactive territory of valuing life,” Milo said. “Say the AV’s passenger is a scientist working to cure cancer and the bus is full of Scared Straight rejects. And once you start assigning value to specific human lives, where does it stop? You need to not only make a decision about who to kill but you need to evaluate each, with facial recognition and online searches to determine their relative value, and what about ambiguous names like John Smith?”
George cupped his forehead. And here he’d thought programming consciousness was complicated. Plus Milo wasn’t finished.
“And how about cultural differences?” he asked. “Does the culture you’re driving in value the collective or the individual?”
George made a T of his hands to get the other to stop. “So you’re saying that AV is going to die in its crib because the legal liability associated with no-win life-and-death decisions is too great?”
“Not so fast, grasshopper,” Milo said. He leaned forward, cupped his hand next to his mouth, and buzzed.
George straightened in his chair, reflexively scanning the cafeteria for who else might be listening. “WTF,” he whispered, pausing angrily between letters.
“Haven’t you wondered about the lack of interference with your little side hustle?”
Yeah, he had, but figured he was just good at hiding things—except from Milo. Apparently, he’d been wrong.
“The truth is, QHQ is rooting for you,” his tablemate said. “They would be overjoyed if you succeeded. Why? Shift the blame from the manufacturer to the vehicle itself. Sue or convict the car. The manufacturer’s relationship to its product would be like that of a parent to a child, and parents don’t pay for their kids’ crimes. The legal groundwork’s been there ever since corporations were ruled to be people, effectively and legally. The next step completes the circuit—pun intended—by making the autonomous progeny of corporations ‘legal persons’ too—and legally separate from the parent,” Milo concluded, adding, “company,” after a pointed pause.
“But what about all the IP Buzz is based on?”
“Look, if your little project works, they’ll either buy out the unsuspecting owners as part of a larger acquisition or reverse engineer the parts needed to make the new product line unique enough to patent and/or copyright.” He paused, smiled wickedly, and then added, “Or we’ll just have our super AI kill ’em all.”
George felt sick. He’d been played. “What if I quit?” he asked. “Take my source code and leave in protest.”
Milo shook his head. “I wouldn’t do that. The results could be suboptimal—for you.”
“What are they going to do? Sue me? Throw me in jail?” George was fuming. “Try threatening someone who didn’t grow up in foster care.”
“Jail’s probably a ‘best case’ scenario,” Milo said. “You ever notice how some people just seem to disappear around here?”
“Bullshit.”
Milo rocked back, hands in the air in apparent surrender, before tipping forward again, elbows on the table, hand cupping fist. “Remember this,” he said.
“What?”
“I haven’t even gotten to the bad stuff yet.”
50
It was a little like pregnancy. The act of conception was done, leaving Buzz and the k-worm to do the hard work of gestation, while the expectant parents sat around waiting, minus the biological clock to let them know when that spark that they were waiting for was getting close. Just to keep busy and in contact, they decided to get to know one another better, like the awkward partners in a one-night stand shyly asking for each other’s names after doing the deed. Unfortunately, the starry-eyed coders kicked off getting to know each other better by discussing that which should never be talked (or texted) about in polite company. And no, it wasn’t politics or religion. It wasn’t death (which they’d pretty well covered) or taxation. No.
Diets.
“I could never eat anything with a face,” George declared out of the blue, making Pandora wonder if he was talking about hers. Blushing, she typed, “What do you mean?”
“I don’t do animals,” he replied. “No meat, no dairy, no leather goods, or aborted chicken fetuses for breakfast.”
Pandora was glad he couldn’t see the face she was making, which wasn’t just disgusted but a little wounded too. Chicken fetuses were practically her favorite part of breakfast, next to crispy flayed swine flesh and potatoes tortured with boiling oil, though she stipulated that strictly speaking, the eggs she ate weren’t fertilized. And about those potatoes, did metaphorical eyes mark them as one of George’s faced foods?