Anthropic has circulated a draft bill that would allow regulators to block the distribution of AI models if they pose a “foreseeable, imminent” catastrophic risk. The proposal sets quantitative thresholds: scenarios that could put more than 50 people at risk of death or cause a billion dollars in property damage would meet the bill’s definition of catastrophic risk.
Who assesses the risk
Under the draft, analysis of a model’s potential for catastrophic harm would rely on “independent evaluators” registered with the U.S. Commerce Department. Those evaluators could be paid by AI developers provided that payments are not contingent on evaluation outcomes. The draft does not specify mechanisms to prevent payment from influencing results.
If an evaluator determines an imminent catastrophic risk, they would have 72 hours to notify the Commerce Department. The Commerce Department would then be required to seek a halt to distribution from a D.C. judge. The bill would explicitly bar any federal agency from unilaterally stopping a model’s release.
Context within industry debate
The draft ties the catastrophic thresholds to a narrow set of origins: creation or release of a biological weapon, a model evading its developer’s controls, or a cyberattack. Those numeric thresholds echo California’s recently enacted Senate Bill 53, which uses similar figures to define catastrophic risk.
The proposal sits amid broader industry debates over oversight. Demis Hassabis, chairman of DeepMind, previously suggested a self-regulatory body modeled on the Financial Industry Regulatory Authority (FINRA). By contrast, Jensen Huang, CEO of Nvidia, has argued that the AI industry does not need new laws. Companies have also floated voluntary approaches such as embedded or third‑party evaluators.
Anthropic confirmed the draft and a spokesperson said: “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.” The draft has been presented to lawmakers, and its registration and judicial-review provisions define the concrete steps it would impose if adopted.
Original source: TechCrunch AI