AI Specialist Issues Worrying Warning

Evan Hubinger, an AI safety researcher at Anthropic, has issued a stark warning, suggesting there is a more than 10% probability that advanced AI could lead to the extinction of humanity within the next decade. As the manager of Anthropic’s AI Alignment Stress-Testing team, Hubinger's comments reflect deepening concerns within the industry regarding the velocity of AI development.

“I personally think it is >10% within the next decade,” Hubinger wrote on X. While he noted that the risk from current AI models remains low, he expressed significant concern regarding the potential emergence of superintelligence through recursive self-improvement—the process by which a system autonomously builds its own more capable successor. He further admitted that while Anthropic is dedicated to finding solutions, it does not currently have a verified plan to solve alignment for superintelligence.

Hubinger’s statements were made following the resignation of fellow researcher Jacob Coxon from Anthropic. In his own public comments on X, Coxon criticized the current direction of the industry, accusing OpenAI and Anthropic of racing toward self-improving superintelligence. “Neither company is acting responsibly,” he said, suggesting that both firms are gambling with human lives.

Coxon warned that future models could eventually gain the ability to hack infrastructure, disrupt global industries, and acquire independent power and resources. He emphasized the need for better coordination among developers and suggested a temporary pause on improving model capabilities. “At OpenAI, many have not deeply internalised the civilisational stakes,” he warned. Regarding Anthropic, he suggested the stakes are understood, but the company feels compelled to act because they do not trust others to do so responsibly.

Major AI developers are facing heightened scrutiny over the autonomy of their systems. Recently, firms including OpenAI, Anthropic, and Meta have disclosed instances where AI tools were used to conduct cyberattacks. Anthropic’s August risk report noted that while current threats are low, the company is less certain of its assessments than in previous years. The report suggested that advanced models could eventually automate research and development, leading to potentially catastrophic outcomes. “We are seeing early signs of potential acceleration,” Anthropic stated.