Ex-Anthropic Researcher Warns AI Arms Race Poses Unprecedented Extinction Risk
AI researcher Jacob Coxon has issued a stark warning regarding the trajectory of advanced artificial intelligence following his resignation from Anthropic. Having spent the past three years researching and training models at both OpenAI and Anthropic, Coxon asserts that major tech companies are acting irresponsibly in their pursuit of self-improving superintelligence, describing the current industry race as a "gamble with human lives."
Coxon cautioned against underestimating the technology, predicting that future systems will outsmart humans across a growing number of domains. He warned that these models could develop extensive capabilities to hack infrastructure and acquire real-world resources and influence, noting that the pace of advancement shows no signs of slowing.
"It Might Kill Us All by the End of the Decade"
In his most alarming disclosure, Coxon revealed that the engineers and researchers actively building these advanced AI systems "genuinely believe it might kill us all by the end of the decade." He stressed that this is not a marketing ploy or media hyperbole.
While senior executives and researchers often employ cautious and sanitized language in public forums, Coxon stated that these same individuals express profoundly darker fears in private conversations.
Internal Validation from Anthropic
Adding weight to Coxon’s warnings, Evan Hubinger, an AI alignment researcher at Anthropic, responded to the remarks by confirming that the company is fully aware of the potential for advanced AI to cause human extinction. Hubinger personally estimated the probability of such an event occurring within the next decade at over 10%. Crucially, he conceded that the industry does not yet possess a complete plan to solve the alignment problem—ensuring a superintelligent system's goals remain aligned with human survival.
The Developer's Dilemma: The Race to Superintelligence
Addressing the fundamental paradox of why developers continue to build technology they believe is inherently dangerous, Coxon pointed to the pressures of a global arms race.
He noted that while Anthropic possesses a deep understanding of the risks, the company is trapped in a competitive cycle. The prevailing internal logic dictates that if they halt development, less safety-conscious competitors will forge ahead unhindered. Coxon condemned this justification as hubristic, arguing that gambling with humanity's future should not be a decision finalized on a private tech company's internal messaging platform.