It has been an extremely bad few weeks for cybersecurity. A series of vulnerabilities discovered in core pieces of infrastructure, including widely used operating systems and common programming languages, led cyber experts to term last month “Black October,” an allusion to the stock market’s historical “Black Monday” crash decades prior.
But while defenders are scrambling to uncover and patch vulnerabilities, other actors are making it easier for unsophisticated attackers to launch cyberattacks with open-weight, safeguard-free models. In response, state enforcement agencies should consider whether such conduct runs afoul of existing prohibitions on using models to encourage criminal conduct. At FAI, our Frontier Legal Defense program is investigating the ways that tort and common law might help to address this fast-moving threat.
The rapid spike in the number of vulnerabilities found in widely used pieces of software—and the corresponding exploitation of such vulnerabilities—is the direct result of the robust cyber capabilities demonstrated by both proprietary and open-weight frontier models. Models with these capabilities have uncovered unprecedented numbers of vulnerabilities in longstanding pieces of code, and the rates appear to only be accelerating further. In a digitally mediated world, such capabilities are far from a solely private or expert concern: Attacks by foreign adversaries or non-state actors extract billions of dollars from domestic businesses and individuals each year, and breaches rob individuals of agency over their personal lives and affairs.
The responses by private entities to Black October have been myriad. Labs are attempting to expand existing controlled access programs, like Anthropic’s Project Glasswing and OpenAI’s Daybreak, to give defenders a head start over adversaries. Enterprises are receiving increasingly loud demands to improve their cybersecurity practices—though historically, the limiting factors for private industry have been funding and staffing, not awareness. And, individuals have been subjected to a barrage of public announcements about improving their digital hygiene, from using passkeys to establishing family safewords to guard against deepfakes. These responses are appropriate, but defensive; the government bears the primary role for identifying attackers and constraining their ability to operate.
The role of open-weight models in this puzzle is hotly contested. The safeguards placed on proprietary frontier models have barred some defenders from using them to harden systems in response to cyberattacks—at least, provided that defenders aren’t part of Daybreak or Project Glasswing—and required those defenders to turn to open-weight models. But, once released, open-weight models are largely outside the control of labs. Crucially, any safeguards placed on them can be trivially removed by anyone with even a rudimentary knowledge of machine learning.
It’s clear that attackers are taking advantage of the opportunity: In the hours after the disclosure of php2shell—a recent critical vulnerability in the language underlying a majority of websites on the internet—Cloudflare, an infrastructure provider through which a substantial portion of all traffic on the internet flows, observed millions of exploitation attempts. While it’s not possible to determine with certainty which attacks were aided by AI, the sheer scale and speed of the attacks indicates that attackers are relying on automated workflows for exploit development, target identification, and attack execution.
We believe that private liability has a key role to play in addressing this challenge. It can both redress harms to victims, and can flexibly incentivize companies to adhere to industry best practices.
In any such liability scheme, however, open-weight models present some of the most challenging edge cases: Not only are they flexibly adapted and modified by downstream users—making imposition of liability on the original developer a difficult legal and policy question—they are challenging to detect and constrain, making it nearly impossible for victims to identify who, exactly, they should pursue for damages. And while private lawsuits can take months or years to reach judgment, open-weight, highly capable models are proliferating rapidly in the hands of bad actors.
Many of these questions are difficult. But novel challenges shouldn’t blind us to the obvious reality that there are now an increasing number of actors working to provide everyone—no matter how unsophisticated—access to dangerous and advanced tools of cyberoffense.
In recent months, a host of providers of so-called “abliterated”—that is, safeguard-free—models have cropped up. These providers operate in a gray zone, offering models that are plausibly just for defensive uses—but without monitoring or safeguards that bar offensive ones. In our own testing, an unsophisticated user was capable of launching offensive cyber operations against a dummy test website within thirty minutes for only a few dollars in credits. Even if sophisticated cyberdefenders need to rely on private copies of open-weight models, dramatically reducing the amount of knowledge needed to launch criminal cyberattacks presents a clear and obvious danger.
The rise of these grey area capabilities providers is obviously bad. State enforcers should take the lead while victims pursue private lawsuits against cybercriminals or other parties who aided them. Some states have already enacted consumer protection laws which can plausibly reach developers or deployers of abliterated, cyber-capable models. Texas, for example, enacted the Texas Responsible Artificial Intelligence Governance Act, or TRAIGA, last year. The Act empowers the Attorney General to investigate and prosecute model developers and deployers who intentionally incite or encourage criminal activity, such as violations of the Computer Fraud and Abuse Act. Incitement, of course, may not apply to model providers that tolerate, but do not expressly encourage, such behavior. But enforcers should not wait until smoking-gun evidence emerges to begin investigating the potential for such harms.
FAI is working to advise state and federal policymakers about the role that negligence and consumer-protection law can play in governing risks posed by frontier models. We’re also seeking victims of cyberattacks interested in bringing precedent-setting litigation against cybercriminals and those who enable them.
Cybercriminals are actively exploiting the tools available to them to launch attacks, aided in part by hosting providers that formally bar illegal uses but do nothing to detect or prevent them. The precise laws and policies needed to constrain bad actors aren’t yet clear—like the rest of cybersecurity, there will be an ongoing cat-and-mouse game as attackers learn how to evade restrictions and as defenders take up new technologies. Today’s harms, however, are clear enough to launch investigations and begin the hard work of protecting American citizens and companies from the abliteration threat.
