"AI's Endgame Could Be Human Extinction"—Former Anthropic Researcher Issues Another Warning

[Financial News] A researcher who left Anthropic warned at a New York City Council hearing that "humanity is more likely to lose control over AI." He drew attention after leaving Anthropic while warning about the risks of the race to develop artificial intelligence (AI).

On the 5th (local time), according to multiple foreign media outlets, including CNBC, former Anthropic researcher Jacob Coxon appeared at a New York City Council hearing on AI and said, "We still don't know how to control any AI system."
He warned, "On our current trajectory, humanity is more likely to lose control over AI than not, and that could end in human extinction."
At the request of New York City Council Speaker Julie Menin, who is pursuing a package of AI regulatory bills, Coxon took the stand and called the industry's approach "extremely reckless." He criticized it, saying, "The long-standing Silicon Valley maxim, 'move fast and break things,' may work for a photo-sharing app, but it does not work when creating the most powerful technology in history."
He also said, "There needs to be some form of speed control in the development of frontier models." Regarding the AI "kill switch," or emergency shutdown mechanism, under consideration by the New York City Council, he noted that it could help in the short term but would not be sufficient to ensure safety over the long term.
Coxon drew attention last month when he left Anthropic and criticized Anthropic and OpenAI, saying they were "gambling with our lives." A British national, Coxon is known to have researched methods for training new AI models using massive datasets. He moved from OpenAI to Anthropic earlier this year before leaving Anthropic.
He also criticized both OpenAI and Anthropic, saying, "Neither is acting responsibly," and adding, "They are racing toward self-improving superintelligence while betting our lives." He also joined a statement by AI industry researchers urging governments around the world to establish an international framework for regulating the pace of AI development.
 
Alexander Turner, a former Google DeepMind researcher, also gave testimony by video at the hearing. "China is not the only potential adversary. With a fairly high probability, we are competing domestically to create our own enemy—unaligned AI," he said. "It could one day become more powerful than China."
Representatives from OpenAI, Anthropic, Google, and Meta Platforms also appeared by video and testified under oath. OpenAI, Anthropic, and Google agreed to appear after the council warned that it would issue subpoenas. Speaker Menin said she would ask a court to enforce a subpoena against SpaceX's AI division, which had not responded to the request to appear.
[email protected] Kim Hee-sun Reporter