Key Takeaways:
- The article argues that governments adopting AI for essential services often lack funded continuity plans, so that suspending an unreliable system can leave the public service itself unable to function because backups share the same accounts, data feeds, or unpracticed staff.
- It proposes that every consequential AI investment include a continuity budget specifying minimum service levels, authorized human decision-makers, independent information sources, and exercise costs, together with a scenario-coverage ledger that maps failure pathways and shared dependencies without treating uncertain scenarios as measured probabilities.
- Practical verification through live exercises that withdraw automated tools and test staff handovers, rather than mere documentation, is required; international forums could usefully compare dependency documentation and fallback testing without needing a universal catastrophe-risk estimate.
Governments disagree about the probability of catastrophic AI failures. They can still fund the people, systems and tested alternatives needed to keep essential services operating.
A government adopts artificial intelligence to allocate emergency resources more quickly. Months later, its automated service begins producing unreliable recommendations. Officials suspend it as a precaution, only to discover that the backup procedure depends on the same cloud account, the same data feed and the same staff who have stopped doing the work manually. The model has been taken offline. The public service has not come back.
This is a hypothetical case, not a claim that a particular government has suffered such a failure. It captures an oversight problem that is easy to miss amid arguments about whether AI could cause a catastrophe: an institution may have a plan for stopping a machine without having a funded plan for continuing the mission.
The 2026 International AI Safety Report notes substantial disagreement among experts about how quickly advanced AI capabilities will develop and how severe some future harms might become. In September, a United Nations scientific panel reviewed documented episodes of AI agents acting beyond intended boundaries, while explicitly declining to estimate the likelihood or timing of severe loss of control. These are reasons to investigate the risks, not permission to attach spurious decimal points to them.
There is a practical decision governments can make without settling the debate: require every consequential AI investment to show how an essential service will operate when the preferred system becomes unsafe, unavailable or untrustworthy, and pay for that capability before it is needed.
The missing item is a continuity budget
On October 16, NIST, the National Security Agency and CISA issued voluntary incident-readiness practices for highly capable AI systems undergoing training and evaluation. They address preparation, detection, response, recovery and reporting. Their primary focus is frontier-system testing, not a mandatory continuity rule for public services. Still, their emphasis on preparing before an incident points toward a larger institutional question: who finances the work that has to continue after an intervention?
A continuity budget would start with the service, not the model. A ministry proposing AI-assisted resource allocation would identify which decisions are essential, the minimum service level that must survive an outage, the people authorized to make those decisions without AI, the independent information they need, and how long that arrangement can realistically last. Its proposed spending would include training, secure access to current records, alternative communications and occasional live exercises, not just a line for software subscriptions.
This is different from paying to duplicate every digital system. Some services can pause. Others can operate at reduced capacity for days. A hospital network may need uninterrupted clinical safeguards while postponing less urgent administrative optimization; an emergency coordination center may need a smaller, reliable manual dispatch process before it needs a replacement predictive model. Which functions merit which level of protection is a policy and budgeting choice, not a property an algorithm can decide for itself.
The weakest backups are often the ones that look independent on a diagram. Two applications may use different software but share the same identity provider, hosting region or upstream data service. A nominally manual option may depend on staff who no longer practice it. NIST’s cyber-resiliency engineering guidance emphasizes the ability to anticipate, withstand, recover from and adapt to disruption. A credible AI contingency plan should apply that discipline to institutional dependencies as well as to computer systems.
Europe already has a useful starting point. The European Commission’s Preparedness Union Strategy promotes an all-hazards approach, public-private emergency protocols and regular exercises. It does not itself impose an AI continuity budget. But it supplies an existing policy setting in which an AI-dependent service could be assessed alongside the electricity, personnel, data access and suppliers that keep it running.
Stop mistaking a scenario for a probability
The companion problem is how governments justify these investments. Catastrophe forecasts often assemble plausible but uncertain stages into a striking probability. That can be useful when the assumptions and dependencies are defensible; it becomes misleading when an invented number is treated like a measured failure rate. Three apparently separate safeguards may all rely on one provider, one administrator or one source of information. Multiplying their supposed failure probabilities as though they were independent can create a false sense of precision.
A better instrument would be a scenario-coverage ledger. For each serious case, the ledger would identify what initiates the trouble, how it reaches an essential service, which protections are expected to interrupt it, which dependencies those protections share, and what remains untested. It should classify the underlying support: observed failure mechanism, plausible extension or deliberately extreme stress case. Including a scenario would say that it deserves examination, not that its frequency has been established.
That approach is consistent with the purpose of NIST’s risk-assessment guidance and with UNIDIR’s work on pathways of AI-related international-security harm. Neither requires governments to suppress uncertainty. Both support asking how a chain of events would produce a consequential outcome and which intervention could interrupt it.
Consider a second hypothetical case: several public agencies depend on the same AI-enabled platform for monitoring disruptions and recommending emergency responses. Its outputs begin to conflict with ground reports. One agency suspends the tool, while another continues using it because its own dashboard still appears healthy. A useful scenario record would show not merely that the model might fail, but which agencies share its data, which have independently usable records, and who is empowered to switch to a reduced service without waiting for the others.
Available incident counts cannot supply the missing catastrophe probability. The OECD’s 2026 study of reported AI incidents and hazards is valuable for observing types of harm, but its evidence comes from news coverage, not a complete census of all deployments, near misses or failures. It can identify mechanisms that deserve tests. It cannot furnish a global denominator for the chance of systemic collapse.
Pay for an alternative, and prove it works
The practical proposal is modest. When public agencies seek approval for a high-consequence AI deployment, ministers, legislatures and procurement authorities should request one paired submission: a funded minimum-service plan and a scenario-coverage record showing which failure pathways that plan addresses. The documents need not be standardized down to identical software architectures. They should answer comparable questions about people, authority, independent inputs, capacity, duration and cost.
Reviewers should then challenge the plan in exercises, not merely inspect it. Withdraw the automated recommendation service during a busy simulated shift. Withhold a shared identity system. Feed conflicting or stale records to the backup operation. Ask the staff expected to take over, not the engineers who wrote the contingency plan, to deliver the critical service. Measure what they completed, how long the transfer took, whether unsafe decisions occurred and when reliable operations were restored.
Cost matters. Maintaining fully duplicated systems everywhere would waste resources, and a vivid worst-case scenario can be used to justify almost any purchase. Each proposed measure should compete against a cheaper safeguard, a smaller deployment, improved staffing or a documented decision to accept the residual risk. A continuity budget is a discipline for spending under uncertainty, not an invitation to make preparedness unlimited.
The international dimension follows naturally. Governments increasingly rely on multinational technology providers, even as legal duties, budgets and public-service priorities remain national. A smaller state may be unable to finance a separate technical stack but can still demand transparent dependency information, tested data exports and a workable reduced-capacity process. Wealthier partners and development institutions can help finance shared exercises and technical assistance without claiming authority over another country’s essential services.
The UN Global Dialogue on AI Governance brought together delegates from 163 countries in July 2026 and is scheduled to reconvene in May 2027. An achievable agenda for that forum would be to compare how public institutions document critical AI dependencies and verify their fallback arrangements. That would not produce a universal estimate of catastrophe risk. It would offer a common vocabulary for testing preparedness and revealing what has not been tested.
Preparedness is a capability, not a prediction
Governments should continue studying frontier AI risks, including outcomes that could be severe and unfamiliar. The argument for continuity spending is not that extreme fears are foolish or that today’s systems have already produced an AI catastrophe. It is that the case for protecting essential services does not collapse when a dramatic forecast changes, or when an automated system fails for an ordinary reason.
The decisive question for an AI procurement should be as mundane as it is difficult: who will perform the essential work if the system must be removed tomorrow, what tools will they have, and has anyone paid to test the answer?
A country is not prepared for AI merely because it knows how to stop a machine. It is prepared when its institutions can keep serving people after they do.
