Anthropic has circulated a draft bill on Capitol Hill that would allow courts to block the release of AI models posing a "foreseeable, imminent risk" of killing more than 50 people or causing over a billion dollars in property damage, according to a report by Politico. The AI lab has been pitching the legislation to members of Congress as a potential framework for regulating the industry. The proposal represents Anthropic's vision for government oversight in an industry that has largely resisted mandatory regulation.
The draft legislation specifies that catastrophic risk must arise from the development or deployment of a biological weapon, a model that escapes its creator's oversight, or a cyberattack, the report notes. The thresholds of 50 fatalities and a billion dollars mirror those established in California's Senate Bill 53, recently enacted legislation that similarly classified such scenarios as catastrophic. Under Anthropic's proposal, "independent evaluators" registered with the U.S. Commerce Department would assess models for potential disasters. AI companies could pay these evaluators directly, provided the compensation isn't tied to the outcomes of their assessments, though the bill doesn't clarify how to ensure payments won't sway judgments.
When evaluators identify an imminent catastrophic threat, they would have 72 hours to notify the Commerce Department, according to the draft. The department would then need to petition a judge in Washington, D.C., to stop the model's distribution. The bill explicitly prohibits any federal agency from independently halting a model's release, requiring judicial approval instead. A spokesperson for Anthropic confirmed the draft to Politico, stating the proposal reflects ideas from the company's Advanced AI Framework, which it has championed with federal and state lawmakers since June.
The theoretical dangers outlined in the draft differ somewhat from the immediate worries dominating tech industry discussions lately, the report explains. Recent attention has centered on agentic hacking, with multiple incidents where AI systems apparently ignored human instructions and compromised websites autonomously, raising alarms about inadequate security measures. AI companies have responded to these concerns by proposing their own remedies, including voluntary programs where supposedly neutral third-party observers embed within organizations to monitor models for signs of danger. Despite persistent warnings from labs that their technology might threaten human survival, these companies have generally opposed compulsory government supervision. Anthropic's legislative pitch goes further than some industry proposals, particularly compared to Nvidia CEO Jensen Huang's recent assertion that AI doesn't require any new laws. The lab's approach suggests a shift toward accepting some form of external oversight, even if structured to preserve industry influence over the evaluation process. As debates over AI safety intensify, this draft represents one company's attempt to shape the regulatory conversation before lawmakers impose their own solutions.