Artificial intelligence: an existential risk?

Artificial intelligence is the newest arrival on the Doomsday Clock: the Bulletin has cited it as a risk since 2023 and has devoted a full section to it since 2024. It is also the most uncertain risk: experts themselves range from 'negligible' to 'major'. Here is what is actually at stake, far from killer robots.

Three distinct risks

1. Autonomous weapons. Systems that select and strike targets without human sign-off already exist on battlefields (Ukraine, Gaza, Libya). The danger is not only ethical: AI in nuclear early-warning and command systems shortens decision time and introduces errors no human can check. This is the Bulletin's chief worry, and the 'military AI' card in the simulator.
2. Industrial-scale disinformation. Synthetic text, images, voices and video at zero cost, personalised, at the scale of whole populations. The risk is not that people believe what is false, it is that they stop believing anything: without shared facts, no collective response to any risk is possible. It is a multiplier of every other risk.
3. Loss of control. Systems more capable than their designers at most tasks, pursuing poorly specified goals, or used by a small group to seize irreversible power. This is the 'existential' scenario, the most discussed and the most contested.

What does 'p(doom)' mean?

It is researchers' shorthand for the probability that advanced AI causes an irreversible catastrophe for humanity. Published estimates range from under 1% to over 50%. A survey of 2,700 AI researchers (AI Impacts, 2023) gave a median of 5% for an extinction or severe loss-of-control outcome. In 2023 a one-sentence statement — 'mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war' — was signed by the heads of the leading labs and two of the three 'godfathers' of deep learning. That level of concern from the builders themselves is unprecedented in the history of technology.

Why the race makes everything worse

Labs and states are locked in a race dynamic: whoever slows down to make things safe risks being overtaken. The 'AI race' card in the simulator weighs more than the technology itself for this reason: the danger lies less in what AI can do than in deploying it before we know. It is exactly the logic of the 1949–1953 arms race — the one that took the clock to 2 minutes.

What is being decided now

• The EU AI Act (2024): the world's first binding framework, phased in through 2027. Bans some uses, requires testing of the most powerful models.
• AI and nuclear weapons: in November 2024 Biden and Xi agreed that the decision to use nuclear weapons must remain human. A statement of principle, not binding — but a first step.
• Autonomous weapons: negotiated at the UN since 2014 with no treaty. The General Assembly voted in 2023 and 2024 to speed things up.
• AI safety summits (Bletchley 2023, Seoul 2024, Paris 2025): voluntary lab commitments, creation of national evaluation institutes.

And at my scale?

Two concrete, rarely mentioned levers. Public consultations: the European Commission and the AI Office regularly open consultations on how the AI Act is applied; responding as a citizen counts for more than any petition. And career choice: AI safety (evaluation, interpretability, governance) is desperately short of people; for a developer or a lawyer it is probably the field where one person has the most impact today. See Act.