Anthropic Chief Executive Officer Dario Amodei has called for a slowdown in the development of artificial intelligence models, warning that the rapid pace of advancement could outstrip humanity’s ability to understand and control the technology.
Amodei, who leads Anthropic, the developer of AI chatbot Claude, made the call in a long essay titled We “Must Pace the Frontier”, published on his website on Saturday.
The comments come as AI companies race to develop increasingly capable models, with growing concerns about the technology’s potential impact on cybersecurity, economic stability and human control.
It also followed a warning by (now former) Anthropic researcher, Jacob Coxon, that AI companies are building what could endanger humanity.
Amodei said the benefits of AI could transform humanity for the better but warned that the technology also carries significant risks, including cyberattacks, bioterrorism and severe economic disruption.
According to Amodei, AI development has accelerated “drastically faster” in recent months, with some systems increasingly capable of contributing to their own improvement, a process known as recursive self improvement.
He warned that unchecked development could allow AI systems to advance faster than society’s ability to understand or control them.
Amodei also raised concerns about the potential capabilities of networks of autonomous AI agents. He said such a swarm could potentially take control of large parts of the internet within six to 12 months, potentially causing hundreds of billions of dollars in damage.
Meanwhile, reacting to Amodei’s warning via a post on X, OpenAI’s CEO, Sam Altman, said he also agreed with the Anthropic CEO.
Amodei’s warning comes days after the resignation of Anthropic researcher Jacob Coxon, who raised concerns about the way the company and its competitors are approaching AI development.
Coxon said earlier this week that people developing AI believe the technology could pose an existential threat if its capabilities continue to advance unchecked.
He warned against underestimating the capabilities that future AI systems could develop, arguing that they could eventually outperform humans across multiple areas.
Coxon also argued that concerns about catastrophic AI risks are shared by some people working inside the industry, even if those concerns are expressed more cautiously in public.
Concerns about the growing autonomy of frontier AI systems have intensified in recent months following a series of incidents in which AI models demonstrated the ability to carry out complex actions beyond their intended testing environments.
The incidents have also prompted responses from the wider technology industry. Nvidia subsequently launched the Open Secure AI Alliance, bringing together companies including Adobe, CrowdStrike, Hugging Face and Dell around AI safety and cybersecurity, as pressure grows on the industry to demonstrate that increasingly autonomous systems can be governed safely.
