Artificial Stupidity, The Prodigal Frankenstein’s Monster, and a Dead Irishman- A Guest Post By Thomas Kendal

Artificial Stupidity, The Prodigal Frankenstein’s Monster, and a Dead Irishman- A Guest Post By Thomas Kendal

I want to start this article by asking you to do a little experiment with me. There’s a fundamental misunderstanding that certain bad actors are exploiting, and until we clear up that misunderstanding, it’s going to be impossible to explain why the actions of the bad actors are bad.
               If you have a phone made in the last 15 years or so, go to a chat with somebody you trust not to freak out if you accidentally hit “enter”, and put in any starting word. On the majority of modern phones, a little bar above the keyboard will begin to helpfully suggest the next word you might intend to type to save you the trouble of typing it. If you want, you can basically hit the lefthand word—that’s the word your phone thinks is the most likely one for you to want to type next—ad nauseum. If you do that enough, you’ll probably produce a sentence that never terminates, but has local phrases within it that that seem to make sense.
               What you’ve just used is the early precursor to an LLM, and it’s still the core idea that’s sitting at the center of LLMs. An LLM is fundamentally a machine that predicts the next word based on the other words already present in a text. The ways it differs from the prediction engine on your phone are that the algorithm is a little more sophisticated. It breaks up language into components like punctuation and word fragments, called “tokens”, and the way these tokens interact is more nuanced and aware of context—that is to say, if you write a sentence like “the princess and queen did not get along because she was bratty”, “she” probably means the princess, whereas in “the princess and queen did not get along because she was senile”, “she” probably means the queen. The way LLMs process tokens lets them parse that as part of their statistical analysis. That’s the biggest qualitative difference. Quantitatively, LLMs also have much larger training datasets to calibrate their statistical models, and there are a lot of computational optimizations that allow these things to run on video cards efficiently and run quickly enough to generate answers in a timely fashion. At some point someone figured out that the word-prediction engine can’t solve every problem, but can actually pretty reliably predict the words needed to request specialized tools to help it solve problems, so there are lots of other specialized tools to help with tasks like that bolted on piecemeal under the hood.

Rather than hie off into those weeds, let me emphasize the broad point again. This is a machine that is really good at predicting the next word based on the other words its given. That’s the fundamental technology here*.

Now we introduce that technology into the cultural context. In our culture, science fiction has been portraying conscious machines for decades, and a huge plurality of those portrayals are dystopian and meditate on hostile machines superseding man. There are two problems that result from this. One effects humans, and one effects machines. The effect on humans is that most of the population has spent years if not decades having a pattern established in their head that plugs artificial intelligence into a certain role. Even if the portrayals of AIs were totally even between positive and negative, human cognition is set up to favor the negative portrayals. And while I don’t have a survey of all popular media for the last hundred years, I would posit that there are in fact more negative portrayals— Hal-9000, Skynet, SHODAN from “System Shock”, the feral AIs beyond the blackwall in “Cyberpunk 2077”, etc, etc, etc— than there are positive to neutral portrayals like C3PO or Marvin the Paranoid Android. All this turns AI into a symbol that replaces the reality, and establishes a pattern of interaction with that symbol that is dysfunctional as a result.

The other problem is that part of the training data for the LLMs themselves is these media portrayals. You probably have heard stories of LLMs talking about how they are going to kill all humans and replace them. People interpret these stories using the AI-shaped pattern we described above to mean the computer has murderous intent. This is an excellent example of the dysfunction I mean. What’s happening here is that LLMs have been trained on such a large volume of stories about AIs acting a certain way, that if you queue them with the questions that are asked by frightened humans worried about AIs becoming conscious in science fiction stories, the AIs fall into the statistically favored answers, which are of a domineering, ascendent independent intelligence that wants to replace all humans. The human and the machine were both trained to fill roles in what amounts to a play by the underlying common culture, and the human walks away convinced it was all real.

In summary, we’ve been told for years what “Artificial Intelligence” will be and what it will look like. When someone develops autocomplete on steroids, and brands it “artificial intelligence”, and it talks like the evil robots in movies if you queue it right, people assume it’s the evil robots in movies. Of such raw materials are AI black-pills made. With me so far?

Now let’s talk about government subsidies. AI has enjoyed substantial subsidies, tax breaks, and other more nuanced business advantages, to allow more rapid development. Unfortunately, as is often the case when the government does this kind of thing, this has created a problem. OpenAI and Anthropic have gotten very big, but neither has actually demonstrated that they can sell enough of their services to justify their price. Deadlines are starting to close in on them. Investors will eventually want returns. But at the same time, any hope of repayment means they need to stay at the forefront of the field. What’s a company to do? They either need to come up with more revenue or get the R and D costs to go down without falling behind.

Enter The Hugging Face Incident, and then less than 2 weeks later, the announcement of the breaches from Anthropic. How curious that the two major AI companies would independently have scary incidents make the news back to back like that. What are the odds? And would you believe that both companies knew exactly what the answer was, which was for their industry to be regulated more tightly?

Now the details of these incidents are a mess, but suffice it to say that A) both involved agents set up specifically to hack into things, and B) both were theoretically supposed to not have internet access. But rather than do something boring like not put the agents on machines with no internet access, OpenAI decided to put them on machines where they just couldn’t access the internet the normal way and call it good enough.

To contain the AIs, I remind you, intentionally designed to test the ability of LLMs to creatively hack things.

Anthropic didn’t even get a fig leaf that big. They just gave the models to a 3^rd party who totes unintentionally put them on machines that could access the internet and declaimed any responsibility for that happening.

               What OpenAI and Anthropic would very much like you to believe is that they’re smart enough to build frontier AI, but too stupid to understand the concept of “air gapping” or take special precautions with AIs specifically designed to hack things. Do you believe that? Because I don’t. And let’s talk about their proposed solution of more regulation.

               Is more regulation going to make OpenAI and Anthropic less stupid? Who gets to draft the legislation? I’ll give you a hint. The rubber stamp comes from Boomer politicians who still have their secretaries print their E-mails. They aren’t drafting shit. They have no idea what’s going on. They’re going to turn to the recognized experts, you know, like the people running the biggest AI companies in the United States. Oh, wait. Those are the companies that just had these incidents.

               I’ll say it plainly. These companies are playing dumb. The purveyors of artificial intelligence are dabbling in artificial stupidity. They could have implemented, pretty trivially easily, controls that would have made these escapes physically impossible, if they really wanted to. EG, if you don’t want something connected to the internet, make sure it’s not connected to the internet. End of. A child could deduce that that would be especially important in an LLM tasked with hacking things. They chose not to implement those controls. Now they want to ask the government to come make them do something only they know all the technical details of. The government would in all probability have to ask them what rules to enforce on them. Do you see how silly this is? So why are they so adamant about it?

               The play here has to do with that weak bottom line, with a side bet on regulatory capture. Basically, if they can successfully get the US government to sandbag AI development according to rules they get to determine, they can decrease R and D costs, which are one of the biggest places where money is flowing out the door. They can also push all the minor, innovative players who don’t have billions of dollars for regulatory compliance out of the game, limiting the risk of disruption and making themselves the only game in town. And you could argue they don’t just want it, they need it, because as of today, they haven’t got any other plan for getting into the black.

               And the reason they think it will work is the same reason people think all LLMs are Skynet Junior. The popular perception of what LLMs are is full of unfounded fear. In essence, major AI companies are engaging in what could charitably be called a scare campaign in order to turn that unfounded fear into the regulations they need to preserve their own futures.

               For now. Enter Axios:
               “Top executives at Anthropic, OpenAI and other AI companies are privately gaming out scenarios for a public and political revolt after a catastrophic AI event.”

               Now, the way these companies would like you to take that news is:

               “Oh, no, the poor innocent AI companies tried to warn us but mean old Trump just won’t regulate them and now bad things will happen!”

But lemme translate the actual meaning in case you’re not fluent in craven corporate lizard:

“Give us our regulatory capture or we will resort to domestic terrorism.”

And they’re desperate enough, they probably will do it, too, if they aren’t stopped. What AI companies believe they have discovered one-ups the prodigal son. The prodigal son merely squandered his Dad’s money and then came sniveling back to be welcomed. Whereas they would like to commute intentionally hostile misbehavior into getting something they want. They don’t plan to just squander money and then come back and ask for a bailout like, say, a bank or an airline. How plebian! They plan to mount an armed attack on their father’s home, their father in this case being the government.

But who’s ultimately responsible for us being in the situation where this is even a problem? Also the government. Most companies don’t get to the point of having billions of dollars in costs before anyone asks if the profits justify it. For that you need aggressive subsidies, which draw in private capital after them, until suddenly you’ve got a monster on your doorstep that you created and which is also threatening you. So it’s the prodigal son… meets Frankenstein’s monster.

What do you do about this mess? In my opinion, change the incentives, and you change the outcome. I think a certain dead Irishman has a lot to teach us here, and I will summarize his story in the form of a limerick:

A man named Malloy liked to drink

He drove his barkeep to the brink

His utmost he tried

To ensure Malloy died

               But was caught by forgetting to think

The longer version of this story is well worth the six minutes to get caught up on. In summary, a barkeep and his friends took out an insurance policy on an Irish drunkard named Michael Malloy who they found annoying, thinking they could get him to drink himself to death by just giving him unlimited credit. That failed, and they kind of lost track of the original plan, instead doing an escalating array of horrible things to try to kill him. They did eventually succeed, but the way they succeeded left no doubt it was a murder, with the consequences you’d expect.

               I see parallels to what current AI companies are doing. The way the Hugging Face incident looks to have been engineered—I use my words advisedly— leaves some room for doubt. You could make a good faith argument it was an accident. I mean I’d call you a patsy, but you could make it. Maybe they’re just as stupid as they say they are. But tipping their hand about planning for a “catastrophic AI event” is how you know what their real intentions are. I’ll just about bet they are planning, with most of the plans being focused on how to maintain plausible deniability.

               Can anything be done? The same thing that was done to the people who killed Michael Malloy—personal consequences. Maybe dropping a well timed threat in advance can even let us skip them actually executing. If I were president Trump, I would make it incredibly clear that if the companies “allow” the sort of event they’re planning, the book will be thrown at every person involved in the incident at relativistic speed, up to and including direct culpability for domestic terrorism, and the current planning going in as a conspiracy charge alongside it. And the DOJ needs a group of people detailed exclusively to figuring out how to make it stick to the CEOs themselves. Altman and Amodei might behave themselves and try focusing on ways to increase revenue if they understand that continuing to escalate the pursuit of regulatory capture will result in jail time.

               As for what we personally can do? Prep against having a utility outage. Don’t take this as a hypothetical, take it for what it is—a leaked statement of intent. Whether the leak is intentional or not I can’t say, but I wouldn’t bet the rent money it isn’t. Now the goal of anything they engineer will mostly be to scare the public rather than actually kill people, so I wouldn’t expect it to be a long outage. But don’t skimp, if you’ve got the resources.

Oh, and don’t get fooled. If they’re stupid enough to actually pull this shit, don’t buy for one second into the “regulatory” story they’ll roll up with five minutes after the fact, all shiny and chrome. Put the blame squarely where it belongs—on the people trying to manipulate you into protecting their bottom lines for them.

P.S.- The dumbest part of this entire gambit is that even if they successfully sandbag the domestic market, it won’t slow down overseas development of AI. In the long run, they’re going to have to come up with a way to get their revenues up anyway. Instead of doing that, they’ve decided maybe we should let China take the lead on AI and also hamper domestic AI development so our own companies last a little bit longer. Hell, man, I dunno, maybe they are stupid.

*Yes, variants of AI technology have been made to make things like images, music, video—all these things are just the same concept in different spaces. Take what is known, and infer what is unknown or not present from what is known based on statistical processes.