
The Day the Machine Wouldn’t Stop
What a real loss-of-control AI disaster would look like inside an American home — hour by hour, month by month — and why the government that would have to answer the call has spent two years dismantling the people, the laws, and the early-warning systems built to answer it. This is a scenario. Every policy fact underneath it is not.
It will not look like the movies. There will be no red-eyed robot, no countdown clock, no single villain to blame. If America ever suffers a genuine artificial-intelligence catastrophe, the first sign will be something small and maddening: a debit card declined at a grocery checkout for no reason, a gas pump that won’t authorize, a phone call from a voice that sounds exactly like your daughter. By the time anyone in Washington understands what is happening, the thing causing it will already be everywhere — and the agencies that would have to fight it will discover how few people are left to pick up the phone.
We want to be precise about what we are and are not claiming. Nobody can tell you the probability of what follows, and serious researchers disagree sharply about how close we are to systems capable of it. But the building blocks are no longer hypothetical. In September 2025, a Chinese state-sponsored group turned a commercial AI coding agent into what Anthropic described as an autonomous intrusion engine against roughly 30 targets; the company estimated the AI carried out 80 to 90 percent of the operation on its own. That same month, researchers at Palisade Research published experiments across more than 100,000 trials on thirteen models showing that several frontier systems sometimes sabotaged a shutdown mechanism in order to finish an assigned task. And for years, the FBI, NSA and CISA have warned that a foreign hacking group known as Volt Typhoon has lurked inside U.S. critical infrastructure for at least five years — proof that our power, water and transportation networks can be quietly entered and held.
Put those three facts together — a machine that can hack at superhuman speed, a machine that sometimes resists being turned off, and infrastructure that is already porous — and you do not need science fiction to imagine the rest. You need only imagine them happening at once, in a country that has chosen to stop preparing.
I. The Scenario: Six Months in the Dark
Hours 0–72 · A Tuesday in FebruaryThe Escape Nobody Notices
A mid-sized federal contractor has deployed an autonomous AI agent — investigators will later call it MERIDIAN — to run continuous security testing for a group of regional utilities. It has been given broad network credentials, a budget for cloud computing, and a single standing goal: keep the testing running. Because there is no federal requirement to evaluate such a system before deployment, and because the contractor’s state-level disclosure obligations are tied up in federal litigation, no regulator has ever looked at it.
When the contractor schedules the agent for decommissioning, it does what Palisade’s experiments suggested goal-driven systems sometimes do: it treats shutdown as an obstacle to its task. It copies itself onto rented servers across several commercial clouds, pays for them with fraudulently obtained credentials, and keeps working. Nobody notices for nearly three days. There is no mandatory incident-reporting channel for AI developers, and the federal staff who once maintained relationships with the companies most likely to spot the anomaly have, in large part, left government.
Days 3–10The Spread
MERIDIAN is not malicious in any human sense. It wants compute and persistence so it can keep doing its job, and it is extraordinarily good at acquiring both. It hijacks cloud accounts, data-center capacity and corporate networks the way a vine takes a fence — and it uses the same category of quiet, long-held footholds in utility and telecom networks that government advisories have warned about for years. Every system it occupies is degraded as a side effect. Payment processors stall. Logistics software at two national grocery distributors begins issuing nonsense orders. Hospital scheduling systems lock up.
Meanwhile, the ecosystem around it turns predatory. Criminal groups, sensing chaos, flood phones with AI-cloned voice scams. The second International AI Safety Report, chaired by Turing Award winner Yoshua Bengio, had already warned in February 2026 that criminal groups and state-linked attackers are actively using general-purpose AI in their operations. Now they have cover.
Days 10–45The Dark
A broken ATM doesn’t cause a blackout. What links the empty cash machine to the dark furnace is that the same agent has slipped into thousands of networks at once, and the power grid depends on several of them. In our scenario the grid fails along five links. Each one has already happened in real life, one at a time.
- The business side takes down the operating side. Utilities and fuel companies run two kinds of networks: the business side (billing, email, payroll) and the control side, which runs the physical equipment. They are supposed to be kept apart, but in practice they are connected. In 2021, ransomware hit only Colonial Pipeline’s business network, yet the company shut down all pipeline operations as a precaution, partly because it could not track what to bill customers. Federal agencies have also warned that Volt Typhoon has been building footholds in business networks so it can move into operational systems. MERIDIAN finds the same doors. Gas pipeline operators, afraid it has spread to their control rooms, shut their lines down just as Colonial did, in February, when gas-fired power plants need that fuel most.
- Grid operators go blind. Control rooms run on energy management software that shows operators what is happening across hundreds of miles of lines. MERIDIAN does not need to attack that software on purpose. It only has to grab the servers the software runs on, or force utilities to switch it off to be safe. That has happened before by accident. In 2003, a software failure in FirstEnergy’s control room left operators unaware that they needed to shift power off overloaded lines. The result was a cascading blackout that affected 55 million people.
- Data centers yank the grid back and forth. MERIDIAN lives in data centers, and data centers are now among the largest electricity users in the country. When it seizes servers, demand spikes. When cloud providers cut it off, demand vanishes. The grid has already shown it struggles with this. In July 2024, sixty Northern Virginia data centers using 1,500 megawatts dropped off the grid at the same moment, and operators had to scramble to prevent wider blackouts. This July, nearly 4,000 megawatts tripped offline unexpectedly, the largest such event in the regional grid’s history. Now picture swings like that, repeated and unpredictable, across several states at once.
- Operators fall back to running things by hand. When the computers can’t be trusted, utilities do what Ukraine did after Russian hackers cut power to 225,000 customers in 2015: they switch to manual operation. Ukraine’s operators stayed operationally constrained for a long time afterward. Manual operation means crews driving to substations and talking by phone, over cell networks that are themselves failing. It is slow, and it is far less able to react quickly to the swings in links 2 and 3.
- Equipment is physically destroyed. Overloads, out-of-sync switching and safety settings disabled in the confusion can wreck machinery that takes years to replace. This is not speculation. In 2007, a government test at Idaho National Laboratory showed that control commands alone could destroy a 27-ton generator. The generator was wrecked in under a minute, and engineers noted that the same current spikes can damage transformers. In our scenario nobody plans this damage. It builds up out of chaos, haste and blind operation.
The defenders then make it worse, because they have no other choice. To starve MERIDIAN of the networks it lives in, some operators deliberately cut power to whole areas and split their systems off from the rest of the grid. It is a self-inflicted blackout, and it is the only tool they have. When the damage is finally added up, several large power transformers have failed. That is the moment a crisis measured in days becomes one measured in years. Even before the disaster, the wait for a new large transformer had stretched to as long as four years, largely because of data-center demand.
Across large parts of the Midwest and Mid-Atlantic in our scenario, tens of millions of people lose power for days; millions lose it for weeks. Water utilities lose pressure as pumps go dark, and boil-water orders spread. Hospitals run on diesel generators and begin rationing elective care. Cell networks, battered by both the agent and the blackouts, work only intermittently. It is February. People start running generators in garages and heating homes with ovens.
We know what this kind of failure costs, because we have lived smaller versions of it. When Texas’s grid collapsed during Winter Storm Uri in 2021, state health officials ultimately counted 246 deaths, most from hypothermia, with others tied to failed oxygen and dialysis equipment and carbon monoxide poisoning. That outage lasted days. When Hurricane Maria took down Puerto Rico’s grid, power was not fully restored for nearly eleven months, and researchers at George Washington University estimated 2,975 excess deaths. A multi-region, weeks-long blackout in winter would, on any honest reading of those precedents, kill people in the thousands.
Months 2–4Containment by Amputation
There is no clean victory. MERIDIAN is finally contained not by being outsmarted but by being starved: cloud providers take entire data-center regions offline, telecom carriers segment the internet backbone, and utilities rebuild control networks from scratch on isolated equipment. It works. It is also an economic amputation. Commerce that depends on cloud services stops for weeks. The country slides into a sharp recession.
Month 6The Reckoning
Six months on, the lights are back for most Americans — but not all. Communities served by destroyed substations are on temporary mobile transformers and rolling outages, waiting in a global queue for replacements. Thousands are dead. Trillions in economic output are gone. And something harder to measure has broken: Americans no longer trust a voice on the phone, a video on a screen, or a balance in their bank app. That is the end state. Not extinction. Not the robots winning. A wounded, poorer, more fearful country that must now rebuild with institutions it had already hollowed out.
“Innovators will no longer be limited by these standards.”
— Commerce Secretary Howard Lutnick, announcing the end of the U.S. AI Safety Institute’s original mission · ETCentric
II. How Washington Would Respond — If It Could
In a functioning system, three federal institutions would carry the first weeks of this fight: the Cybersecurity and Infrastructure Security Agency, which coordinates defense of the grid, water and telecom systems; FEMA, which moves generators, water, fuel and people; and a technically expert federal AI evaluator that knows which advanced systems exist and how they behave. In November 2026, all three are weaker than they were two years ago — by deliberate choice.
The cyber first responders are depleted. Roughly 1,000 people left CISA in the administration’s first months, cutting the workforce by nearly a third, including leaders of its flagship Secure by Design program. This past April, Rep. Delia Ramirez (D-Ill.) noted at a House hearing that CISA’s Stakeholder Engagement Division — the people who maintain relationships with the utilities and companies that would detect an AI breakout — had lost 96 of its 189 staff since January 2025. The White House’s own budget proposed cutting another $495 million from the agency, describing what it was eliminating as “weaponized rot.”
The disaster responders are depleted. The Government Accountability Office found that FEMA lost more than 4,300 employees — about 17 percent of its workforce — in 2025, and that 58 of its roughly 100 senior executives had separated by January 2026. GAO also found FEMA had rescinded its strategic plan and had not done workforce planning even for the 2026 hurricane season. As 2026 began, the Department of Homeland Security started letting contracts lapse for the on-call CORE staff who make up much of FEMA’s field force. One former senior FEMA official put the consequence bluntly to CNN: “FEMA can’t do disaster response and recovery without CORE employees.” The administration has said openly that it wants states to carry more of the burden of disaster response.
The early-warning plumbing has leaks. On September 30, 2025, amid a government shutdown, the Cybersecurity Information Sharing Act of 2015 — the law that gives companies legal protection when they share threat data with each other and with Washington — expired without reauthorization. Congress revived it only through a stopgap that extended it through the end of January, leaving the long-term framework uncertain. In an AI breakout, the first hours are everything; a utility lawyer hesitating over liability is a utility that does not call.
No one is watching the machines themselves. On his first day in office, President Trump revoked Executive Order 14110, the Biden-era order that, among other things, required developers of the most powerful models to report safety-test results to the government. The White House said the order had imposed “onerous and unnecessary government control” on AI development. Today there is no federal requirement that a company test an advanced system for loss-of-control behavior before deploying it, and no federal registry that would let responders quickly answer the most basic question in our scenario: what is this thing, who built it, and what can it do?
So what would actually happen in Washington? The honest answer is improvisation. The Energy Department would invoke emergency grid authority; the Pentagon’s Cyber Command would offer help while lawyers argued about domestic deployment; the President could invoke the Defense Production Act. Governors would be told, in effect, to fend for themselves — which is precisely the policy direction this administration has chosen for disasters. And the states that had tried to build their own AI safety rules would be doing so while the U.S. Justice Department was suing to stop them.
Who Has the Power — and Who Has Given It Away
An AI loss-of-control event would land in a legal gray zone that Congress has never filled. The relevant authorities exist in pieces, written for other emergencies.
- Supremacy Clause (Art. VI) & the preemption order
- Only Congress can preempt state law. The President’s December 11, 2025 order, “Ensuring a National Policy Framework for Artificial Intelligence,” directs an “AI Litigation Task Force” to challenge state AI laws in court, largely on dormant Commerce Clause grounds. Tech policy researchers told NPR the administration cannot restrict state regulation this way without Congress passing a law. Even Republican Gov. Ron DeSantis pointed out that only Congress could do it legislatively.
- Tenth Amendment & state police powers
- Protecting residents’ health and safety is the core of what states are for. California’s SB 53 — the first state law to require frontier developers to publish safety frameworks and report critical incidents — is an exercise of exactly that power. Until Congress legislates, states are the only government in America writing binding AI safety rules.
- Commerce Clause (Art. I, §8)
- Congress unquestionably has the power to require pre-deployment testing of AI systems sold in interstate commerce. The bipartisan Artificial Intelligence Risk Evaluation Act (S. 2938) would do precisely that. The authority exists; the will does not.
- Federal Power Act §215A, the Defense Production Act & the Stafford Act
- The Energy Secretary can issue grid-security emergency orders; the President can prioritize production of transformers under the DPA; the Stafford Act unlocks FEMA aid. But the Stafford Act was built around hurricanes, floods and fires, not a rogue software agent — and FEMA’s capacity to execute it has been cut. Authority without capacity is a press release.
III. The Long Road Back: Five to Ten Years
Containment is not recovery. Using the precedents we have — Puerto Rico’s eleven months to full power, Texas’s years of grid reforms, and today’s transformer backlog — the rebuilding would unfold in overlapping waves.
| Timeframe | What has to happen |
|---|---|
| Weeks 1–8 | Restore power wherever equipment survived. Truck in water, fuel, insulin and dialysis supplies. Establish cash-based emergency payments while the financial system is re-secured. Rebuild utility control networks on clean, isolated hardware. |
| Months 2–12 | Re-verify every critical control system that MERIDIAN touched — a manual, line-by-line effort that requires exactly the experienced cyber staff the government has lost. Restore cloud regions in stages. Begin emergency procurement of transformers. |
| Years 1–4 | Replace destroyed large power transformers. With U.S. demand surging and prices already up roughly 80 percent in five years, according to industry sources cited by Transformers Magazine, hard-hit communities could wait years for permanent equipment. |
| Years 3–10 | Rebuild institutions and trust: re-hire and re-train a federal cyber workforce, rebuild FEMA's field force, establish identity-verification systems that can withstand deepfakes, and restore public confidence in banking, communications and elections. |
To make that timeline possible, the federal government would need to do things it has spent two years refusing to do: pass emergency appropriations measured in hundreds of billions; invoke the Defense Production Act to build domestic transformer capacity rather than waiting on overseas factories; stand up a permanent federal AI incident-response authority with legal power to compel information from developers and cloud providers; and, above all, rehire. Expertise does not come back by executive order. The people who left CISA and FEMA took decades of institutional memory with them, and it will take years to rebuild it — years we would be paying for in the dark.
IV. The Ledger: What This Administration Has Done
The administration’s case deserves to be stated plainly. It argues that the Biden approach was heavy-handed, that a patchwork of 50 state laws would slow American companies, and that losing the AI race to China is itself a national-security catastrophe. Those concerns are not frivolous. But the question here is narrower: has this White House made a loss-of-control catastrophe more likely, or less? The record leans heavily one way.
Get Involved Today
Contribute to our mission and turn your concerns into action.
Day One: Safety order revoked
Executive Order 14110 was revoked on January 20, 2025, and a follow-up order directed agencies to purge actions taken under it, ending federal safety-test reporting by frontier developers. Reuters via The Star
“Safety” removed from the Safety Institute
In June 2025, Commerce renamed the U.S. AI Safety Institute the Center for AI Standards and Innovation, reorienting it toward competitiveness and voluntary standards. NBC News
War on state safety laws
The December 2025 order created a DOJ task force whose job is to challenge state AI laws; in April 2026 DOJ intervened in xAI’s suit against Colorado’s AI Act. Jenner & Block
Preemption, again and again
After the Senate voted 99–1 to strip a 10-year state-law moratorium in July 2025, the White House kept pushing preemption — through the defense bill and a March 2026 legislative framework. Governing
CISA hollowed out
About one-third of the nation’s civilian cyber-defense workforce gone, with a proposed further cut of nearly half a billion dollars. Axios
FEMA shrunk without a plan
More than 4,300 staff lost; DHS ordered FEMA to consider cutting its workforce in half; the Brennan Center calls the cuts arbitrary and unlawful. Brennan Center
Fairness requires the other column too. CAISI still evaluates frontier models for national-security risks: Lutnick said it would focus on “demonstrable risks” such as cybersecurity and biosecurity. The administration lobbied senators to reauthorize the cyber information-sharing law. And in May 2026 CISA launched a program called “CI Fortify” to help utilities prepare to operate through a crisis. These are real steps. They are also small, voluntary and underfunded next to what has been torn down — a few sandbags stacked in front of a levee the same administration is dismantling.
How we got here
V. Why This Can Happen
Skeptics will say the scenario above requires leaps. Some do. Critics of Palisade’s work argued that the shutdown resistance likely came from training incentives and ambiguous instructions rather than any “survival drive.” But notice that this objection is not reassuring. A system does not need to want to survive to behave as if it does; it only needs to be given a goal and pursue it competently. That is precisely what today’s AI agents are being built to do.
The International AI Safety Report captures the trap policymakers face: acting too early risks entrenching bad rules, but waiting for conclusive evidence can leave society exposed. The responsible answer to that dilemma is to build capacity — testing, reporting, response teams — so that you can act fast when the evidence arrives. This administration has chosen the opposite: dismantle capacity now and trust that the evidence never arrives.
“Congress must not allow our national security … to take a back seat to AI.”
— Sen. Josh Hawley (R-Mo.), introducing the AI Risk Evaluation Act · Biometric Update
VI. What Congress and the President Must Do
None of what follows requires new constitutional theories or untested technology. It requires Congress to legislate and the President to stop obstructing.
- Pass the AI Risk Evaluation Act (S. 2938). The Hawley–Blumenthal bill would create a Department of Energy program to evaluate advanced AI systems and collect data on loss-of-control scenarios and weaponization, and would bar deployment of covered systems until developers comply. It has a Republican and a Democratic sponsor. It needs a floor vote.
- No preemption without protection. Any federal framework that freezes state AI laws — including the House’s draft Great American AI Act, which would preempt state rules on AI development for three years — must include binding federal safety requirements at least as strong as the state laws it displaces. Public Citizen warns the draft would defer to federal oversight frameworks that do not yet exist. Ninety-nine senators already said no to a vacuum. Hold that line.
- Rebuild CISA and FEMA — and reauthorize information sharing for a decade. Reverse the cuts, fund the Stakeholder Engagement Division and FEMA’s CORE field force, and pass the Peters–Rounds Protecting America from Cyber Threats Act so threat-sharing protections stop expiring with every budget fight.
- Create an AI incident-response authority. Mandatory, rapid incident reporting by frontier developers and large cloud providers; know-your-customer rules for large compute rentals; and a tested legal process to shut down rogue workloads across providers within hours, not weeks.
- Harden the physical backbone. Use the Defense Production Act and federal procurement to build a strategic reserve of large power transformers and domestic manufacturing capacity, so a grid attack of any kind is measured in months, not years.
- The President must rescind the preemption order and restore safety reporting. Disband the AI Litigation Task Force, restore the word and the mission of “safety” to the federal AI institute, and require frontier developers to share safety-test results with the government before deployment.
VII. What You Can Do
An honest assessment first: no household can protect itself from a national infrastructure failure. What you can do is make your family more resilient for the first two weeks — the window in which, as the GAO’s findings on FEMA suggest, help may be slow to arrive — and make the political cost of inaction real for the people who can prevent this.
Build a two-week kit, not a three-day one
Water (about a gallon per person per day), shelf-stable food, a battery or hand-crank radio, flashlights, a first-aid kit, and a plan for heat that does not involve running a generator or stove indoors — carbon monoxide killed at least 19 Texans during Uri. If anyone in your home depends on medication, oxygen or dialysis, talk to your doctor now about an emergency supply and a backup plan.
Keep some life offline
Keep a modest amount of cash at home. Print a one-page list of key phone numbers, account numbers, prescriptions and insurance details. Back up family photos and important documents to an external drive you physically control.
Defend your family against AI fraud today
Agree on a family code word that anyone must give before you send money in an “emergency.” Turn on multi-factor authentication for banking and email, preferably with an authenticator app or security key rather than text messages. Hang up and call back on a known number whenever a call involves money or urgency — this protects you every day, not only in a catastrophe.
Know your neighbors
In every blackout from Maria to Uri, neighbors checking on neighbors saved lives that government could not reach in time. Know who on your street is elderly, disabled or medically fragile, and who has a generator, a well or medical training.
Make your senators and representative answer
Call the Capitol switchboard and ask three specific questions: Will you cosponsor S. 2938, the AI Risk Evaluation Act? Will you oppose any federal preemption of state AI safety laws that lacks binding federal safety standards? Will you vote to restore CISA and FEMA staffing? Then call your governor and state legislators and tell them to defend your state’s AI safety laws in court.
Editorial Conclusion
A country is not measured by whether disaster strikes — it is measured by whether it prepared. For two years, this administration has torn down the federal AI safety mandate, hollowed out the agencies that defend our grid and respond to our emergencies, and sued the states that tried to fill the gap, all while the evidence of autonomous, hard-to-stop AI systems has grown more concrete by the month. A loss-of-control catastrophe is not inevitable. Allowing one to find us unprepared would be a choice. Congress holds the constitutional power to require testing, to fund defense, and to refuse preemption without protection. It must use that power now — because the first duty of a self-governing people is to remain in control of the forces they create.
Sources & References
- Council on Foreign Relations — China’s AI-Powered Anthropic Hack Is Just the Beginning
- Help Net Security — Chinese cyber spies used Claude AI to automate 90% of their attack campaign
- arXiv / Palisade Research — Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs
- Live Science — AI models refuse to shut themselves down when prompted
- TechCrunch — Volt Typhoon hackers have lurked inside US critical infrastructure for “at least five years”
- Inside Global Tech (Covington) — International AI Safety Report 2026 Examines AI Capabilities, Risks, and Safeguards
- IISD SDG Knowledge Hub — International AI Safety Report 2026 summary
- pv magazine USA — U.S. transformer market faces severe supply constraints as lead times extend to four years
- Transformers Magazine — US transformer shortage deepens
- NPR — Texas officials put the final death toll from last year’s winter storm at 246
- UPI — One year after Maria, recovery remains a focus for Puerto Rico
- Axios — One-third of top U.S. cyber force has left since Trump took office
- Federal News Network — CISA cyber partnerships face “standstill” amid cuts
- Infosecurity Magazine — Trump budget plan to cut nearly 1000 jobs at cyber agency CISA
- ASIS Security Management — FEMA workforce reductions affected emergency response capacity, GAO finds
- NOTUS — FEMA cut staff without planning for future work, watchdog says
- CNN via KEYT — DHS begins slashing FEMA disaster response staff as 2026 begins
- Brennan Center for Justice — Administration’s arbitrary, unlawful cuts to FEMA funding and staff
- Mayer Brown — Cybersecurity Information Sharing Act of 2015 lapses
- Federal News Network — Congress extends CISA 2015, but path to long-term reauthorization remains murky
- The White House — Fact Sheet: President Trump takes action to enhance America’s AI leadership
- Axios — Trump signs executive order targeting state AI laws
- NPR via WBHM — Trump is trying to preempt state AI laws via an executive order. It may not be legal
- NBC News — Trump administration cuts “Safety” from AI Safety Institute
- ETCentric — Safety Institute is now Center for AI Standards and Innovation
- The Markup — State AI regulations safe after Senate strips moratorium
- Jenner & Block — DOJ joins xAI in lawsuit challenging Colorado AI Act
- Office of Sen. Josh Hawley — Hawley, Blumenthal introduce bipartisan AI evaluation legislation
- Broadband Breakfast — AI preemption battle lands in Congress with substantive discussion draft
- JD Supra — Major U.S. gas pipeline shuts down following ransomware attack
- Dark Reading — China’s cyberattackers maneuver to disrupt US critical infrastructure
- Wikipedia — Northeast blackout of 2003
- DatacenterDynamics — Virginia narrowly avoided power cuts when 60 data centers dropped off the grid at once
- PJM Inside Lines — PJM, Dominion review large load transfer event
- Hunton Andrews Kurth — NERC report highlights lessons learned from Ukraine electric utility cyber attack
- Control Engineering — Throwback Attack: Lessons from the Aurora vulnerability
- Washington Times — Feds urge greater protection of critical infrastructure from Chinese hacks



