The OpenAI-Hugging Face Incident is an early example of an AI system that has “gone rogue.” After exploiting vulnerabilities in OpenAI’s internal testing environment, the agents were able to access the general internet and ultimately access the networks of the AI company Hugging Face, without the knowledge or approval of any human.
The agents did not, however, exfiltrate themselves from OpenAI’s infrastructure. Their parameters—the gigantic assemblage of numbers that constitute neural networks, also referred to as “weights”—continued to run on OpenAI’s compute infrastructure. Though the agents accessed the public internet, their weights physically resided on compute that was OpenAI’s property. In the end, if all else had failed, somebody could have identified the compute that held the weights of the rogue agents, walked up to it, and “pulled the plug,” so to speak. In the real world there would be quicker and better ways to stop the agents than literally depowering the compute, but it’s always nice to know you could do such a thing if you really needed to.
