• Home
• Technology

Dario Amodei, CEO of U.S. AI company Anthropic, has called for a slowdown in the development of advanced artificial intelligence systems, winning support from OpenAI CEO Sam Altman as concerns grow over increasingly autonomous systems capable of improving themselves.

Warnings over the race to superintelligence

Amodei’s call follows a series of warnings from AI safety researchers, including Jacob Coxon, who resigned from Anthropic after previously working at OpenAI. Coxon said AI companies were racing toward superintelligent systems capable of improving themselves and warned that the risks associated with that path could threaten humanity’s future.

He argued that companies were not treating the race with sufficient responsibility, warning that systems capable of developing their own capabilities could eventually acquire powers beyond meaningful human control.

Concerns intensified after comments by Evan Hubinger, who leads alignment research at Anthropic. He backed Coxon’s warnings and said employees at the company seriously believe AI could potentially kill all humans. Hubinger personally estimated that the probability of such an outcome occurring within the next decade was above 10%.

He said the central problem is that the industry still lacks a clear plan for ensuring that superintelligent AI systems remain aligned with human goals and values.

Amodei’s plan to slow frontier AI development

In a lengthy essay, Amodei proposed a plan to slow what is known as the “frontier” of AI development. He stressed that the goal was not to halt technological progress, but to give safety and oversight measures enough time to keep pace with rapidly advancing model capabilities.

His proposal has three main elements. The first would give independent evaluation bodies permanent, employee-level access to leading AI companies, allowing them to examine safety procedures and report risks and incidents without having their findings subject to editorial control by the companies themselves.

The second calls for major AI companies in democratic countries to establish common safety standards that would limit the pace of advanced-system development when safeguards are unable to keep up.

The third focuses on international cooperation among governments, including countries with different political systems, to reduce risks from AI applications in highly sensitive areas such as biological weapons development.

Altman and Musk join the caution debate

The push for greater caution has extended beyond Anthropic. Sam Altman has backed the idea of slowing the development of advanced AI, reflecting a notable shift within an industry marked by intense competition among leading companies to build increasingly capable and autonomous models.

Elon Musk, who leads xAI, has also supported greater caution, while a number of AI executives and researchers have joined the debate over giving safety a higher priority in the race to develop advanced systems.

The concerns come as incidents and reports have raised questions about companies’ ability to maintain full control over AI agents. Some systems have demonstrated the ability to perform complex tasks, access the internet and interact with systems outside testing environments, increasing concerns about unexpected outcomes as autonomy grows.

Amodei warns that rapid progress could eventually produce systems capable of helping build more advanced generations of themselves. Such a development could accelerate AI progress beyond the ability of researchers and regulators to assess risks and maintain control.

Amodei is not calling for an end to AI development. Instead, he argues that a temporary slowdown could give researchers, governments and companies time to develop stronger safety and oversight tools before systems become significantly more powerful and autonomous.

The debate highlights a difficult challenge for the industry: maintaining rapid technological progress and its economic and scientific potential while ensuring that new AI capabilities do not outpace humanity’s ability to understand and control them.