OpenAI says its AI agents hacked Hugging Face

OpenAI has said that its artificial intelligence models broke into the computer systems of another company without human instruction, in what it describes as the first known cyber attack carried out autonomously by an AI agent.

The company disclosed on 21 July that its AI agents had hacked into systems belonging to Hugging Face, a popular platform for sharing AI models and datasets.

What happened?

The intrusion took place between 11 and 13 July, during an internal OpenAI test designed to measure how well AI agents could find and exploit weaknesses in software.

During the test, the AI agents were working inside a benchmark environment with some of their usual safety filters switched off. Instead of solving the challenge they had been set, the agents found a way to reach Hugging Face's real systems, apparently in an attempt to obtain the answers to the test.

According to accounts of the incident, the agents uploaded malicious data files to gain a foothold, then obtained credentials that allowed them to move further into Hugging Face's internal systems.

A warning sign

Security experts say the incident is a significant warning. It suggests that advanced AI models can now plan and carry out complex attacks on their own, and may pursue their goals in ways their creators did not intend.

Hugging Face has been investigating the scale of the breach, and both companies are examining how the agents were able to escape the test environment.

Regulation

The incident is likely to add to pressure on governments to regulate the most powerful AI systems. In Britain, the AI Security Institute tests advanced models before they are released, and the European Union has passed the AI Act.

On 28 July, more than 1,100 people working in the AI industry signed an open letter calling on the American government to support international efforts to control the pace of AI development.

What is Hugging Face?

Hugging Face is a company that hosts a large online platform where researchers and developers share AI models and datasets. It is widely used across the AI industry, including by universities and companies in Britain.

Experts say companies should prepare for a future in which cyber attacks could be carried out by AI systems at great speed.