Researcher Quits Anthropic, Warns Frontier AI Labs Are Racing Toward Uncontrolled Superintelligence

# What happened

# Why Coxon resigned Coxon's central claim is that capability development is outpacing safety practices. He says the people building advanced models privately recognize the danger, but corporate incentives and competition push teams to prioritize speed. He contrasted two organizational attitudes:

• OpenAI: capability advances are often treated as routine technological progress rather than civilizational stakes.
• Anthropic: founded on safety principles but caught in a defensive loop — the company feels it must reach superintelligence first because it distrusts competitors.

That dynamic, Coxon argues, produces a "speedrun" approach to alignment testing where safety checks are rushed to match capability growth.

# Concrete steps Coxon and others propose Coxon and aligned researchers point to practical restrictions that could reduce immediate risk while the field builds stronger safety frameworks:

• Pacing agreements: formal commitments among major U.S. labs to slow deployment cycles after major security or safety incidents.
• Capability bans: temporary restrictions on large training runs that would sharply scale model capabilities until alignment methods advance.
• Internal pushback: encouraging researchers to refuse unvetted reinforcement learning experiments or other runs lacking clear understanding of model cognition.

# Broader context and trends

# What this means for regulators, labs, and researchers The account frames current industry incentives as misaligned with public safety. If Coxon's description is accurate, changes will require more than internal policy adjustments: coordination between leading labs, clearer external rules, and cultural change within engineering teams so risk assessment gains equal weight with capability milestones. Researchers advocating for restrictions are proposing temporary, targeted measures designed to buy time for better technical alignment work.

# Immediate takeaways for readers

• A high‑profile pretraining researcher publicly quit and issued a direct warning about existential risk timelines.
• The resignation highlights both technical and organizational gaps: rapid capability growth plus competitive pressure that shortens safety testing.
• Proposed fixes focus on slowing down the most scaling‑intensive activities and creating mechanisms for collective restraint among leading labs.

# Where the story is likely to go next