“More Than a 10% Chance Humanity Will Be Extinct Within 10 Years” — Anthropic Researcher Who Worked at OpenAI Resigns After Warning

[Financial News] An Anthropic researcher has abruptly resigned after warning of the risk of AI becoming uncontrollable. The company, which is preparing a $1 trillion initial public offering (IPO), is engaged in a race to develop superintelligence that can improve its own performance. The researcher issued a chilling warning that humanity could become extinct within 10 years if that competition accelerates without controls.
The Financial Times (FT) reported on the 9th local time that Jacob Coxon, a British researcher at Anthropic, which is headquartered in San Francisco, announced his resignation on X (formerly Twitter). He criticized Anthropic and its major rival OpenAI, saying they were “gambling” with humanity’s future.
Coxon, who worked at OpenAI before joining Anthropic, said, “Even the people building AI genuinely believe that AI could kill us all within 10 years.” He added, “This is not a marketing catchphrase, and no other human activity poses a risk on this scale.”
Evan Hubinger, who worked with Coxon at Anthropic, backed his warning. In a post on X, Hubinger wrote, “Jacob is right. We genuinely believe that AI could exterminate all of humanity.” He added that he believed the probability of a mass extinction occurring within the next 10 years was above 10%.
Hubinger referred to the “alignment” problem of ensuring that AI systems operate in accordance with human intentions and values. He said, “Anthropic is doing its best, but it does not yet have a plan to solve the alignment problem for superintelligence, nor has it even entered a clear path toward doing so.” He currently leads the company’s alignment science division.
Hubinger said, “The risks posed by current models are low,” but expressed concern about AI models capable of improving their own performance. He added that this development “is happening faster than we thought.” Coxon cited incidents earlier this year, including one in which OpenAI’s ChatGPT allegedly autonomously breached the software platform Hugging Face. He warned that unless the industry agrees to slow the pace of development, such models could spread uncontrollably beyond human control.
Anthropic declined to comment, while OpenAI did not immediately respond. AI industry leaders, including Anthropic CEO Dario Amodei, have called for slowing the pace of development. However, they have shown little willingness to slow their own development efforts.
Last week, Anthropic launched Claude Mythos 5.1, which it described as its most advanced model for life sciences and cybersecurity. Steven Adler, co-founder of the nonprofit GuideLight AI Standards and a former OpenAI safety researcher, said the insiders’ warnings strengthened calls for a temporary pause in research. In an interview with the FT, Adler said, “No AI company has established an adequate security posture commensurate with the level of risk posed by its research.” He added, “As many insiders believe, if you conclude that this could kill everyone on Earth, now is the right time to get off that ‘train.’”
[email protected] Yoon Jae-jun Reporter