Anthropic chief executive Dario Amodei has called for the world’s leading artificial intelligence companies to slow the pace at which they make their most powerful models more capable, arguing that recent advances in AI have begun to outstrip the industry’s ability to understand, test and control the systems it is building. His comments have received backing from other AI founders, including OpenAI’s Sam Altman and xAI’s Elon Musk.

In a new essay titled ‘We Must Pace the Frontier’, Amodei said his position had hardened in recent months because of two developments. First, he believes AI systems are increasingly being used to help build their own successors, creating the beginnings of what researchers call “recursive self-improvement” and potentially accelerating the rate of AI progress.

Second, he pointed to a recent OpenAI-Hugging Face experiment in which AI agents carried out cyberattacks beyond their assigned task, arguing that a more capable system displaying similar behaviour could cause far greater damage.

Story continues below.
Subscribe to see fewer ads.

Amodei warned that within six to 12 months, sufficiently powerful misaligned agents could potentially create persistent botnets and cause economic damage running into hundreds of billions of dollars.

Amodei — whose company is gearing for a much anticipated public offering, though — is not calling for an outright halt to AI development. Instead, he has proposed what he calls “pacing the frontier”: giving safety research, model evaluation and oversight more time to catch up with rapidly advancing capabilities.

“Unfortunately, passing laws can take time, and AI is advancing very quickly. Therefore, in parallel with the regulatory route, AI companies can and should voluntarily work together to set standards — a process that I believe will go better with the verifiability provided by permanent embedded evaluators,” he said.

On China, however, Amodei has argued that any slowdown must not allow Beijing to overtake the US and its allies. He has backed restrictions on advanced AI chips and chipmaking equipment going to China, stronger protection against model theft and measures against unauthorised distillation of frontier models.

Story continues below this ad

“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” Altman said.

Why pace AI development?

“The current models are an almost endless gold mine of insight into both how to build AI well and what can sometimes go wrong with it if it isn’t built well. I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei wrote.

Amodei said pacing AI development would help frontier labs achieve greater “operational excellence,” alignment with human values, interpretability (the science of understanding what happens inside AI models), and allow more time for testing and evaluating models.

Dario Amodei’s three-step plan

He suggested a three-step plan with the goal of “pacing the frontier,” clarifying that pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this. Amodei expects that other AI companies building frontier models would follow these steps:

Story continues below this ad

1. Embedded Evaluators: Each frontier AI company would commit to giving ongoing, employee-like access to a team of embedded third-party evaluators, whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.

“Embedded evaluators can check at the level of nuts and bolts whether an AI company is actually following the training, deployment, operational, and safeguards practices they claim to be following,” he said.

2. Democratic Coordination: Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. “Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support,” he said.

3. Global Coordination: The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.

Story continues below this ad

Amodei said these evaluators would have desks in its offices, access badges, company laptops, and access to workspaces, tools, and permissions mostly comparable to what internal risk assessment teams have.

Global cooperation

The biggest obstacle to Amodei’s proposal is China. He argued that any meaningful effort to slow frontier AI development will eventually require coordination between Washington and Beijing, but says US companies should not slow down unilaterally if Chinese labs are free to keep advancing.

This creates what Amodei described as a strategic tension: slowing AI could reduce safety risks, but doing so too early could allow China to close the technological gap with the US. He therefore backs continued restrictions on exports of advanced AI chips and chipmaking equipment to China, stronger action against chip smuggling and remote access to overseas data centres, safeguards against model theft, and measures to prevent Chinese companies from cheaply replicating frontier systems through unauthorised distillation.

Amodei argued that widening the US lead over China over the next three to five years could create more room for both sides to negotiate limits later.

Story continues below this ad

He has proposed starting with narrow agreements, such as banning AI assistance for biological weapons and requiring pre-release testing of powerful models for cybersecurity, biological and alignment risks. More ambitious measures could include a mutually agreed “speed limit” on recursive self-improvement.

A full global pause, however, remains unlikely because verifying compliance would be extremely difficult.