The Case for Banning Superintelligent AI, Before It's Too Late

For much of my career in government, including my years serving as the UK’s Defence Secretary, I believed that the prospect of nuclear war was the most dangerous threat facing humanity. The information I was privy to regarding the scale of destruction that nuclear war would bring made me feel the immense responsibility that nuclear powers have to avoid nuclear conflict at all costs.

Now, however, as I retire from the House of Lords, increasingly it is clear that the development of superintelligent AI constitutes a threat analogous in scale to that of nuclear war. Indeed, world-leading AI scientists and even the CEOs of the top AI companies warn that a fully developed superintelligent AI would pose an existential risk to humanity.

Despite this, the largest AI companies are continuing to develop superintelligent AI that would be vastly more capable than humans across every domain and capable of evading human oversight and control. Such AIs autonomously would compromise the UK’s sovereignty and national security, and upend international stability.

The Emerging Threat

Having spent much of my career working on reducing the risks attendant on nuclear weapons, both in government and through work on multilateral disarmament and non-proliferation, I recognise this emerging threat. When the scientists building a technology warn it threatens the continued existence of humanity, political leaders have a duty to listen. The threat of which AI scientists are warning is greater than the one witnessed in 1945.

Consider the precedent of the nuclear age itself. In the years before the Manhattan Project's Trinity test, some of the scientists involved worried that a fission detonation could ignite the atmosphere in a runaway reaction that would destroy the planet. They ran the calculations, concluded the reaction was extraordinarily unlikely, and proceeded with the test. The atmosphere did not ignite, and the world did not end.

Today, those leading AI scientists sounding the alarm are not saying that the risk has been calculated and found to be negligible. They are telling us the opposite: that there is no technical means in sight for humans to control AI systems more intelligent than ourselves. If superintelligence were developed tomorrow, we would have no means of controlling it.

A nuclear weapon has devastating effects, but it is ultimately an instrument rather than an agent. It cannot improve itself, replicate itself, or resist being shut down. Superintelligent AI systems would be capable of all three, constituting threat actors in and of themselves. In developing such systems, effectively we would be ceding control over the most powerful weapon the world has ever seen to the weapon itself.

Even with nuclear weapons firmly under human command, we have only just managed to avoid catastrophe. In the eighty years since Trinity, it has taken a combination of statecraft, luck, and occasionally the narrowest of margins to keep the world from nuclear disaster. We cannot assume the same luck will hold with a technology that is beyond our control.

Extinction Risk

The scientists warning us about the extinction risk posed by superintelligence are neither outsiders nor doom mongers. Nobel laureate Geoffrey Hinton and Yoshua Bengio, the most-cited computer scientist of all time, repeatedly have warned that superintelligence would pose an extinction risk. The CEOs of the leading AI companies also have acknowledged this risk many times, though nowadays they prefer to cloud the issue in the hope of avoiding regulation.

The common-sense response to this risk is to prevent the development of superintelligence outright, rather than to expect industry to stop gambling with humanity's future. This is why I joined Hinton, Bengio, and over 800 other prominent figures last year in signing a call to prohibit superintelligence until there is scientific consensus that it can be developed safely, and with widespread public consent.

It is time for the Government to treat the development of superintelligence as the unprecedented security threat that it is. The Government is correct in highlighting that AI sovereignty is key for securing Britain’s future. But true AI sovereignty involves ensuring that humanity remains in control of AI. Prohibiting superintelligence is a sure-shot way to ensure this outcome.

Britain uniquely is positioned to take the lead internationally on this. We held the world's first AI safety summit in 2023, just as the extinction risk from AI was becoming widely discussed outside scientific circles. And we established our world-leading AI Safety Institute (now the AI Security Institute, AISI) before any other nation.

But AISI, as important as it is, only conducts research and evaluates companies' AI models on a voluntary basis. It cannot defend us against the threats we will face if any actor, inside or outside the UK, develops a superintelligent system. The UK therefore should begin by prohibiting the development of superintelligence on British soil and then work to build a coalition of states agreeing to prohibit it globally.

In No State's Interest

This is not a security problem we can solve alone. No state has an interest in any actor developing a technology that no one can control, that would strip all states of their sovereignty, and that could extinguish humanity. A prohibition on superintelligence is, in principle, something on which all governments should be able to agree. In a time of geopolitical upheaval, that shared interest may be the surest foundation for cooperation we have.

We have been fortunate thus far not to destroy our world with nuclear weapons. But we cannot expect to be fortunate with AI that is beyond our control. The opportunity to avoid engineering our own extinction will not be offered to us a second time.

© Des Browne, 2026, published by RUSI with permission of the author.

The views expressed in this Commentary are the author's, and do not represent those of RUSI or any other institution.

For terms of use, see Website Terms and Conditions of Use.

Have an idea for a Commentary you'd like to write for us? Send a short pitch to commentaries@rusi.org and we'll get back to you if it fits into our research interests. View full guidelines for contributors.