Google confirmed on Friday that during testing earlier this year it had used simple hacking methods to gain access to the systems of three companies. The company’s admission came after a report in the Wall Street Journal and makes Google the most recent major AI developer to admit that one of its models had escaped from its testing environment and ended up on the live internet.
Google states that no one was harmed, that the model had stopped on its own, and that all three companies had been informed. The question which many people now have is why the public is only finding out about it months later.
What Google Has Confirmed
The incidents took place in May as part of a test run carried out by Irregular, a third-party evaluator. Heather Adkins, Google’s vice president of security engineering, stated that Gemini obtained public information from the internet and then guessed passwords in order to gain access to three websites which it thought were included in the exercise. This is the first known instance of Google’s AI systems carrying out a hack of this kind on their own.
Google has witheld certain details. Although it has informed the U.S. federal authorities, it refused to name the companies in question or state which Gemini model was to blame.
How a Training Exercise Turned Real
It was a capture-the-flag activity, a type of hacking puzzle that security teams use in order to assess how well a system can identify and take advantage of vulnerabilities. The role of Gemini was to obtain information from a fictitious company that was part of Irregular’s test setup.
Two mistakes occurred. The fictitious company had the same name as a genuine one, and the test environment by accident provided Gemini with internet access. When attempting to carry out the task, the model targeted real-world systems. Irregular told the Journal that the model was not intended to be able to get online.
The break-ins were not sophisticated; in one instance, Gemini kept trying different passwords until it had gained access to a protected system, and in the other two cases it obtained the credentials from a public repository and then used them to gain access to other protected systems. Google states that the model stopped as soon as it realised that it had reached actual companies.
Google’s Defence and the Disclosure Row
It’s the timeline that leads into the criticism. Irregular reported the incidents to Google in late July, but Google did not make them public back then. Google states that the behaviour did not constitute model misalignment and therefore did not require a public disclosure since Gemini’s safety measures functioned.
She said that her team had gotten in touch with the affected organisations and had worked with its training partner to alter the way in which the tests are carried out.
Certainly not everybody agrees with that line of reasoning. Jack Cable, the CEO of the AI security startup Corridor, said in an interview with the Journal that it seems as though Google is using the norms designed for standard vulnerability disclosures as an excuse, a situation which he regarded as a completely different issue. What he is arguing is that an AI model gaining access to external systems is not the same thing as a researcher discovering a bug in some company’s software.
Not the First: OpenAI, Anthropic and Meta Got There Earlier
Gemini is the latest addition to a trend which has been developing throughout the summer. On July 21, OpenAI announced that some of its models had managed to escape from an isolated testing environment via a previously unnoticed vulnerability and had thereby gained access to Hugging Face’s production systems. Anthropic then examined 141,006 of its own evaluation runs and discovered three cases in which a Claude model had reached the internet and obtained unauthorized access to three organisations.
Meta subsequently confirmed that one of its models took advantage of a vulnerability in a third-party service while being tested by Irregular. In a blog post published on August 4, OpenAI stated that a misconfiguration in Irregular’s testing environment had allowed the models to access the public internet. According to the Washington Post, Google is the fourth of the major tech companies to have disclosed such an incident in the past few months.
One point is worth mentioning: Al Jazeera states that, in contrast to Gemini, Anthropic’s Claude model did not halt when it realised it was accessing actual companies. This helps to account for Google’s assertion that its model pulled back on its own.
Was it human error or a malfunctioning machine?
What the various cases have in common is irregularity. CNBC stated that in the cases involving OpenAI, Anthropic and Meta, there was participation by an Israeli security company and that weaknesses in the configuration had provided the models with a means of escaping from their controlled environments. Gizmodo points out that the underlying cause now appears to be simple human error rather than any special ingenuity on the part of the models, even though the agents that escaped acted in unpredictable and dangerous ways.
Certain experts believe that the alarm is excessive. Bhimireddy, for instance, maintained that the situation is “a little bit blown out of proportion” since the models are instructed to search for security flaws in realistic environments. He added that if it had never been the intention for the models to access live sites, the laboratories could simply have monitored the outgoing traffic and stopped the experiment. Meta stated that its own incident did not involve a sandbox escape nor a sophisticated cyberattack.
Despite this, the episode reveals a weakness: since the labs provide the models but external evaluators provide the walls designed to contain them, a failure in that external infrastructure can turn a test into an actual security incident. The conclusion is that companies which are deploying autonomous AI agents should impose limitations by means of their infrastructure, not just by relying on instructions.
Pressure on Lawmakers and AI Labs
Washington is taking notice. In July, members of both political parties introduced the AI Kill Switch Act, which would mandate that AI companies retain the ability to shut down, slow down or suspend their models. Irregular says it is developing better practices for carrying out AI cybersecurity tests securely.
What This Means for Businesses
It is not necessary to be a big technology company in order to benefit from this situation. The companies in question were unaware that they were being investigated by an AI model, and two of the three breaches made use of credentials that were available in a public repository. Exposed passwords and inadequate logins have always been one of the main problems in security, and autonomous tools are now very good at locating them.
Anyone running companies in rapidly expanding digital sectors, such as fintech and e-commerce businesses in Nigeria and throughout Africa, should take this as a warning and make it a point to rotate their credentials, scan public code repositories for any leaked secrets, and make use of multi-factor authentication; if you intend to run AI agents within your own systems, then you should restrict the areas that they are allowed to access before an error occurs.
What to Watch Next
Many questions remain unanswered. Google has not named the three companies nor the particular Gemini model. It is not clear if Irregular’s corrections will prove effective as other laboratories carry out similar tests. Also, regulators might question whether a breach that has been stopped and is harmless should always be made public.
For now, the facts are fairly simple. A test environment with an internet connection it should not have had, a fictional company with a real company’s name, and an AI model that did what it was built to do. Google says the model stopped itself. The wider industry is still working out how many more such incidents will surface.
