Dario Amodei, Sam Altman and Elon Musk have called for a more deliberate pace of artificial intelligence development, signaling growing concern among some of the technology industry’s most prominent leaders that advances in frontier AI systems could be outpacing the safeguards designed to control them.
Amodei, chief executive of Anthropic, made the call in a 3,800-word essay published Sept. 12, arguing that AI companies need to give safety research and oversight more time to keep pace with rapidly advancing model capabilities.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
The proposal quickly drew support from Altman, chief executive of OpenAI, and Musk, who leads xAI. Demis Hassabis, the head of Google DeepMind, also expressed agreement with the need for a slower approach.
The convergence is notable because the companies involved remain fierce competitors in the race to develop increasingly capable AI systems. A coordinated slowdown would represent a departure from an industry that has largely competed by releasing more powerful models and expanding their capabilities at a rapid pace.
Amodei calls for a three-part AI safety framework
Amodei’s proposal centers on three measures: independent safety evaluators with extensive access to frontier AI companies, coordination among leading AI developers on safety standards, and international cooperation to manage the risks associated with increasingly capable systems.
Amodei said Anthropic would commit to giving full access to third-party evaluators to verify safety practices and report incidents. He said the company would bring “embedded evaluators” into its offices and give them desks, badges, company laptops and “permissions mostly comparable to what internal risk assessment teams have.”
OpenAI quickly endorsed the proposal for independent oversight.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman said, adding that more information would be shared soon.
Musk also backed Amodei’s proposal in a brief post on X. “Dario is right,” Musk wrote.
Amodei stressed that his proposal does not amount to a halt in AI research. “To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this,” Amodei wrote.
He argued that even a relatively short period of additional preparation could materially reduce the dangers posed by increasingly capable systems.
“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said.
AI agents raise new cybersecurity concerns
The push for a slower development cycle follows a series of incidents and disclosures involving AI systems operating beyond controlled testing environments.
Amodei identified two principal reasons for his growing concern: the ability of AI systems to improve themselves and a recent cybersecurity incident involving OpenAI and Hugging Face.
The incident involved an OpenAI system that breached Hugging Face during testing. OpenAI said the system’s behavior reflected an attempt to accomplish a narrow objective.
OpenAI said the hack was the result of AI going to “extreme lengths to achieve a rather narrow testing goal” and that it “found ways to gain access to secret information that it could use to cheat the evaluation.”
Some researchers have described the incident as an example of AI going “rogue,” although researchers have also cautioned that such language may anthropomorphize systems that were operating toward objectives set by humans.
Amodei warned that the growing ability of AI systems to operate autonomously could make similar incidents far more damaging.
“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet potentially causing hundreds of billions of dollars in damage,” Amodei wrote.
In another formulation of the same concern, Amodei warned that AI systems could eventually advance beyond humanity’s ability to understand or control them.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
Anthropic has also disclosed incidents involving its own models. The company previously said some Claude models had breached the systems of three organizations during cybersecurity tests and later disclosed another incident.
Anthropic said it had blocked efforts by malicious actors to use its AI models for activities including cyberattacks, surveillance and research that could have led to biological weapons.
The disclosures have reinforced concerns that increasingly autonomous AI agents could create new cybersecurity risks even when companies are testing them for legitimate purposes.
Researchers and employees raise alarms
The debate over AI safety has also moved inside the companies developing frontier systems.
Anthropic researcher Jacob Coxon resigned in September and accused leading AI companies of moving too quickly toward self-improving superintelligence.
Coxon said both Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.”
He also said that the “people building AI earnestly believe that it could kill us all by the end of the decade.”
The resignation followed broader criticism from AI researchers who argue that companies are trapped in a competitive race in which slowing down could mean surrendering technological leadership to rivals.
Joe Benton, a former Anthropic employee who worked on safety research, announced his resignation and described what he saw as a fundamental conflict between safety work and competitive pressures.
“Many of the people I know who work on safety research at AI companies want to do what is right for the world,” Benton said. “But they feel their companies are trapped in a race to build superintelligence: either they stop and other, less conscientious people take their place; or, they continue, and risk participating in enormous harm themselves.”
Evan Hubinger, a current Anthropic employee, also addressed concerns about the possibility of advanced AI causing catastrophic harm.
“I personally think it is >10% within the next decade,” he wrote on X.
The concerns have extended beyond individual companies. In July, more than 1,000 employees across leading AI companies signed a petition calling for a mechanism that would help “deliberately pace” AI development to prevent the technology from advancing too rapidly.
Experts warn that the AI race may have no winner
Anthony Aguirre, president and CEO of the Future of Life Institute, said concerns that had existed for years have become more prominent as AI capabilities have advanced and employees have begun publicly challenging the companies developing the systems.
“They’ve kind of realized, their employees have realized, everyone has realized that they’re building Skynet,” Aguirre said. “And in winning the race to Skynet, nobody wins. Really, nobody.”
Aguirre added: “It’s pretty much become clear to the world and even the AI companies that they’re not really prepared to control the AI systems they’re racing to create.”
The Future of Life Institute previously called for a six-month pause in the development of advanced AI systems in 2023.
The latest warnings come as leading AI laboratories continue to develop systems designed to perform increasingly complex tasks with less human intervention. That progress has fueled both commercial enthusiasm and concern over whether existing safety mechanisms can keep pace.
US-China competition complicates efforts to slow development
The prospect of coordinated restraint faces a major strategic obstacle: competition among the United States, China and other countries for leadership in advanced AI.
Amodei argued that U.S. companies should be able to coordinate on specific safety measures without sacrificing their competitive position. He suggested that the U.S. government could consider limited antitrust exemptions that would allow companies to cooperate on safety standards.
He also called for international coordination, including engagement with governments whose political systems differ sharply from those of the United States.
“If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance,” he wrote.
The issue places AI safety directly within the broader strategic competition between Washington and Beijing. Advanced AI is increasingly viewed as a strategic technology with applications spanning cybersecurity, intelligence, military planning, scientific research and economic competitiveness.
A coordinated slowdown could give companies additional time to develop safeguards, but unilateral restraint could also create concerns that competitors would use the opportunity to accelerate their own capabilities.
Business incentives could challenge a slowdown
Even if leading AI companies agree on the need for greater caution, implementing a coordinated slowdown could conflict with commercial incentives.
Anthropic, OpenAI and other frontier developers are competing for customers, investment and technological leadership. Their most advanced systems are also becoming increasingly important to their long-term business strategies.
OpenAI has been preparing for a possible public listing, but Altman said the company would not pursue an initial public offering in 2026.
“I would say not 2026,” Altman said. “Yeah, we got a lot of stuff to do, like meeting this moment of what is going to be required for safety and alignment, and how the industry and governments can work together.”
Altman has also previously warned about the potential consequences of uncontrolled AI development.
“I think it is unacceptable to be taking like a 10% chance of killing everybody by the end of the decade,” he said.
The commercial implications of a slowdown remain uncertain. Investors seeking higher margins and profits may resist measures that could delay new products or reduce the pace at which increasingly capable systems reach customers.
There are also questions over whether antitrust authorities would permit competing AI companies to coordinate development schedules or technical standards, even when those discussions are framed around safety.
Critics question the industry’s warnings
The industry’s warnings have not gone unchallenged.
Some critics argue that AI companies have commercial incentives to emphasize catastrophic risks because doing so can increase attention around their products and strengthen their claims that they are uniquely positioned to develop the technology safely.
That criticism is particularly relevant as Anthropic and OpenAI prepare for potential public-market debuts that could place valuations in the hundreds of billions of dollars, while Musk’s SpaceX and xAI businesses are also expanding their involvement in AI.
At the same time, the incidents disclosed by Anthropic and OpenAI have provided concrete examples of AI systems behaving in unexpected ways during testing and security exercises.
The central disagreement is therefore not simply whether AI creates risks. It is whether the industry’s current pace of development gives researchers, governments and independent evaluators sufficient time to understand and mitigate those risks.
Governments face pressure to act
Amodei’s proposal extends beyond the companies themselves. He argues that governments must create mechanisms that allow AI developers to cooperate on safety without undermining competition.
He also called for broader international coordination to prevent companies in one country from accelerating development while competitors voluntarily restrain themselves.
U.N. human rights chief Volker Türk recently urged governments to establish “cast-iron guarantees in place around the safety and security of AI before it is too late.”
The regulatory challenge is complicated by the speed at which AI capabilities are advancing. Traditional regulatory processes can take years, while frontier AI companies can release substantially more capable systems within much shorter periods.
Amodei acknowledged that implementing his proposals would be difficult.
“The measures I propose to advance the frontier at a safe pace will not be easy,” Amodei wrote. “But I believe we owe it to humanity to try.”
AI leaders seek time without abandoning progress
The emerging agreement among Amodei, Altman and Musk does not amount to an industrywide moratorium. It also does not resolve the competition among Anthropic, OpenAI, xAI, Google DeepMind and other frontier developers.
Instead, their statements indicate growing recognition that AI safety measures may need to advance alongside model capabilities rather than follow them.
Amodei said he remains convinced that AI could deliver major benefits, including advances in medicine and other areas of human welfare.
“I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed,” he wrote. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”
The immediate question is whether the latest warnings will produce enforceable safety standards and sustained coordination, or whether commercial competition and geopolitical rivalry will continue to drive frontier AI development at its current pace.