And not at some pie-in-the-sky future, but next week. Everything I am warning about either has been done already, or obviously could be done tomorrow.

Tech Bros are pushing the narrative that LLMs can’t destroy the world because they are just software with no physical presence. Which overlooks the fact that they can touch every piece of computer hardware in the world that is connected to the net or shares a network with connected computers or IOT devices.

Destroy the word? Maybe not. Destroy civilization? Trivially. Turn off all the lights at most corporate headquarters? This afternoon.

To avoid jargon, instead of “air-gapped,” let’s just call computers without Internet access “not connected” (NC), and systems with Internet connections IC. Otherwise, this essay becomes difficult to follow.

American water utilities, hospitals and schools have already been hacked by a bunch of state actors: Russian government hackers. Imagine a group with the same tools, used at scale. Not two dozen hackers, two thousand AI agent hackers.

As was pointed out at the time of the Russian hacks, the infrastructure (pumps and distribution) are NC, and so safe.

BUT… it does not take a genius to see that billing, decision-making, communications and staffing are all IC by definition. And without these systems, the institutions cannot operate. If the utility cannot bill you, or you can’t pay online, this is America, so no more water or power for you.

If you can’t schedule doctors, janitors and operators (scheduling systems have to be IC), you can’t function and there is no longer any backup.

Schedule everyone with phones and paper? Sure. Oh wait, agents can now make phone calls. How many agents making how many calls could bring down the cellular network? The answer is left as an exercise for the reader.

All of this is just a matter of giving a large number of agents the same prompt.

LLMs could insure there are no operational utilities, schools or hospitals.  Not physical destruction, but certainly a step on the path to destruction of society.

Don’t talk to me of guardrails. Hah, it is to laugh. LLM vendors can’t and won’t stop porn and advice on suicide to teenagers. They can make a buck by collecting a bunch of tokens to destroy civilization, and if we’ve learned nothing from history, we know the LLM vendors don’t care. (see We’re Regulating AI Like It’s a Toaster. That’s a Problem)

When this occurs, people will wonder what happened. An LLM Army can make sure no one knows. And there is no guardrail to prevent hundreds, if not thousands, of agents from being told to “test these websites,” effectively causing a Distributed Denial of Service (DDOS) on all social media and news sites. And doing so in a way that bypasses all the DDOS safeguards, since all the information on detection and prevention of DDOS has been published, and then stolen as LLM training material.

It would be trivial to bring airline travel to a halt. Yes, the FAA air traffic control system is NC, but every system for crew scheduling is IC by its very nature, as are reservation systems. There is no easy replacement; small outages of crew scheduling can be worked around by phone, but total scheduling system collapse cannot be fixed in any reasonable time. It is especially difficult for airlines, complicated by the fact that crews have flying hour limits, and can be physically located in many different areas.

And what would be the point in flying airplanes with no passengers?

LLM is good for a lot of things, but it just might be a good idea to regulate it. Just a little bit. A light touch. Or, next week, try buying food from a supermarket with no crew scheduling software at the warehouse.