The Resignation That Sparked a Debate
Jacob Coxon’s departure from Anthropic in late 2024 ignited fresh scrutiny over the pace of safety research inside leading AI labs. In a public post, Coxon warned that current trajectories toward superintelligence lack adequate guardrails, arguing that an AI alignment slowdown is necessary to prevent catastrophic outcomes. His resignation letter cited fundamental disagreements with leadership over resource allocation for alignment work versus capabilities advancement.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
Evan Hubinger, a senior research scientist at Anthropic, subsequently corroborated Coxon’s concerns in a series of technical comments. Hubinger acknowledged that his own probability of doom — p(doom) — remains elevated, citing unresolved challenges in scalable oversight and interpretability. The exchange has amplified calls from the research community for mandatory third‑party audits before frontier models cross critical compute thresholds.
The debate quickly extended beyond Anthropic, prompting other labs to reconsider their development cadence.
AI alignment slowdown: OpenAI’s voluntary pace
OpenAI has deliberately throttled the pace of its largest model training runs, a move confirmed by chief scientist Jakub Pachocki who urged peers to adopt voluntary slowdowns until safety research catches up.
The decision follows internal assessments that alignment work was lagging behind capability gains, prompting a reallocation of compute and personnel toward interpretability and scalable oversight, according to a TIME report.
Industry observers note that the self‑imposed brake could set a precedent for coordinated pacing across frontier labs, though critics argue that without external enforcement the slowdown may be uneven.
Legislators, observing these industry‑level adjustments, have begun drafting policies aimed at formalizing such safeguards.
Legislative Responses: FRONTIER Act and Ban Superintelligence Act
The FRONTIER Act, introduced by Senator Maria Cantwell and Representative Ro Khanna, would require frontier AI developers to submit detailed safety cases and undergo independent audits before models exceed 10²⁵ floating‑point operations. Cantwell said the bill “creates a clear regulatory floor so that safety research cannot be outpaced by capability gains,” while Khanna emphasized that “mandatory transparency prevents a race to the bottom among labs.”
The Ban Artificial Superintelligence Act, sponsored by Senator Josh Hawley and Representative Anna Eshoo, proposes a statutory prohibition on deploying any system that meets a defined superintelligence threshold. Hawley warned that “without a hard stop, the incentives to push past safety limits are overwhelming,” and Eshoo added that “the legislation reflects a growing consensus that voluntary pauses are insufficient.” Critics of the alignment community’s cautious stance argue that such restraint may delay needed safeguards, a point highlighted in a LessWrong critique that questions whether excessive prudence slows progress on robust alignment solutions.
Frequently Asked Questions
How could an AI alignment slowdown change the expected deployment schedule for next‑generation large language models?
A slowdown typically means reallocating compute resources from training to safety work, which can add months or even years to a model’s release timeline. Labs may prioritize interpretability and scalable oversight experiments, delaying large‑scale training runs until those tools mature. The net effect is a slower cadence of capability gains, potentially reducing the risk of deploying under‑aligned systems.
What thresholds does the proposed FRONTIER Act set for requiring independent safety audits of frontier AI models?
The bill specifies that any model exceeding 10²⁵ floating‑point operations must submit a detailed safety case and undergo an independent audit before further scaling. It also calls for documentation of alignment research progress and risk assessments tied to the model’s compute budget. These criteria aim to create a regulatory floor that triggers oversight once a model reaches a defined frontier level.
What concrete actions can an organization take to implement OpenAI’s voluntary pause while keeping alignment research on track?
Companies can shift a portion of their compute allocation to dedicated alignment projects, such as interpretability tooling and scalable oversight experiments. They may also establish internal checkpoints that pause training if safety metrics fall behind predefined milestones. Hiring additional alignment researchers and conducting third‑party audits of progress are further steps to ensure safety keeps pace with capability development.
Last Updated on September 10, 2026 7:08 pm by Laszlo Szabo / NowadAIs | Published on September 10, 2026 by Laszlo Szabo / NowadAIs
