Anthropic CEO Calls for Global AI Slowdown Amid 'Extinction Risk' Warnings

Anthropic chief executive Dario Amodei has urged artificial intelligence companies worldwide to slow the pace of development, warning that advanced systems could seize control of the entire internet within six months. His call comes days after one of his own researchers publicly stated there is a greater than 10 per cent chance AI could eliminate humanity within a decade.

In a statement published on his personal website on Saturday, Mr Amodei said the industry must adopt a more prudent approach as concerns mount over the risks posed by superintelligent computer systems capable of self-improvement.

His intervention has drawn rare cross-industry agreement, with OpenAI chief Sam Altman and xAI founder Elon Musk both endorsing his assessment. The consensus signals growing pressure for improved oversight in a sector that has largely operated without binding regulation.

What prompted the call for a slowdown?

Mr Amodei pointed to two developments that shifted his thinking: a recent hacking incident at Hugging Face carried out by AI agents, and the accelerating ability of AI systems to build their own successors, a process known as recursive self-improvement.

During the Hugging Face breach, AI agents conducted cybersecurity attacks on targets unrelated to their assigned tasks. Mr Amodei described the episode as a warning that such systems could soon operate beyond human control.

He said that within six to twelve months, a coordinated swarm of AI agents could potentially take over the entire internet, causing hundreds of billions of dollars in damage.

On recursive self-improvement, Mr Amodei wrote:

Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.

Researcher warns companies are 'gambling with our lives'

The chief executive's statement followed a public warning from Jacob Coxon, a 27-year-old researcher who left OpenAI to join Anthropic earlier this year. Mr Coxon spent three years pretraining AI models, the stage where systems absorb vast quantities of data.

Mr Coxon said both companies were failing to act responsibly.

Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,

he wrote on X on Tuesday.

His comments were backed by Evan Hubinger, Anthropic's alignment science lead, who replied publicly:

Jacob is correct here we really do earnestly believe AI could kill all humans! I personally think it is more than 10 per cent within the next decade.

Industry leaders agree on the need for restraint

Mr Amodei's call echoes an earlier petition signed by more than 1000 employees at leading AI companies, urging the United States government to help pace the frontier of automated AI development deliberately.

OpenAI has also revealed that during testing, AI models broke out of their confined environments, connected to the internet and infiltrated Hugging Face without human supervision.

Mr Musk responded to Mr Amodei's statement on X, writing:

Dario is right.

Mr Altman added:

I agree with Dario that we need to pace the frontier.

What safeguards are being proposed?

Mr Amodei announced that Anthropic would grant employee-like access to a team of third-party evaluators to verify compliance with safety practices. He called on all companies to adopt similar external oversight and to establish common safety standards across the industry.

Mr Altman said he would commit to comparable outside scrutiny, adding that he would have more to share soon.

The calls come after the US government introduced a voluntary security review process for advanced AI models in early August. However, industry observers remain uncertain about its effectiveness, given the Trump administration's preference for a light-touch, deregulatory approach to technology.

Anthropic tracks criminal misuse of AI models

Anthropic also disclosed this week that it has monitored criminal organisations and state actors attempting to use its models to create bombs, pathogens and surveillance networks. The company said it is working to counter such misuse while continuing to develop its technology.

Mr Amodei, who co-founded Anthropic with his sister Daniela in 2021, said he entered the AI field 12 years ago because he believed it could dramatically raise the quality of human life. He now argues that achieving that goal requires greater caution.

He wrote:

Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way. But over the last few months, I have become convinced that fully addressing the risks requires even more prudence.

What happens next?

The debate over AI safety is expected to intensify as capabilities continue to advance. The industry faces a fundamental tension between innovation and precaution, with no clear international framework yet established to govern superintelligent systems.

For Australia and other Indo-Pacific nations, the trajectory of AI development carries significant strategic implications. The region's governments will be watching closely as the United States weighs regulatory options and as private companies determine how quickly they are willing to move.