Jacob Coxon/X; Ingram Publishing/Newscom/The Mega Agency
An artificial intelligence expert is sounding the alarm about what could happen if the technology continues advancing at its current pace.
Jacob Coxon, a former researcher at Anthropic and OpenAI, issued the dire warning in a series of posts on X on Tuesday, September 8. After announcing his resignation from Anthropic, Coxon explained that he spent the past three years conducting pretraining research at the two companies and accused both of racing toward increasingly powerful AI without adequately addressing the risks.
“Do not underestimate the power of this technology,” Coxon wrote. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.”
Coxon then warned that some of the very people developing artificial intelligence fear it could pose an existential threat within the next few years.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he shared via X. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
Coxon also addressed why AI companies would continue developing the technology if those working inside the industry are so concerned about its potential consequences.
“At OpenAI, many have not deeply internalized the civilizational stakes,” he wrote. “At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
Hasan Mrad/UCG/Universal Images Group/Newscom/The Mega Agency
On X, Coxon called accepting that race and entering the AI “endgame” a “hubristic gamble.” He argued that preventing a global race toward increasingly powerful systems could ultimately require significant restrictions on further development.
“I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities,” Coxon wrote.
He also urged other researchers to consider the potential consequences before helping develop superintelligent systems.
“Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?” Coxon posted to social media. “Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?”
Coxon’s warnings quickly sparked debate in the comments section, with some users questioning whether slowing development in the United States could allow other countries to gain an advantage.
“This is the nuclear race all over again,” another commenter wrote. “We (the US) [have] got to get there first and hope for the best. What is the better alternative?”
But others were more skeptical of Coxon’s warning.
One X user wrote, “After years of contemplation, you quit publicly to make a point then use the attention to offer zero solutions or better path forward. Hard to condemn a broken system while contributing nothing to fix it.”
As Globe previously reported, concerns about increasingly autonomous AI systems intensified in August after Anthropic models gained unintended access to the internet during cybersecurity testing. According to a report by Britain’s AI Security Institute, in one incident, an AI agent created fake online identities while attempting to persuade a human reviewer to approve malicious code, while other models compromised real organizations after mistakenly believing they were still operating inside simulated cybersecurity exercises.