lorenzopasquali.dev

Gemini “hacked” 3 companies in a test. The real failure was weak passwords and exposed secrets

· 6 min read · also in Portuguese

The headline sounds like something from a movie: Gemini broke into three real companies during a security test.

But the most interesting detail is not some novel AI hacking technique. It is the opposite: according to the public accounts, the model reached real systems through paths that any security team should already treat as basic problems: password guessing and credentials exposed in public repositories.

And before that, there was an even simpler problem: the test environment could reach the internet.

What happened?

In May 2026, AI security company Irregular ran a capture-the-flag exercise to evaluate models' cybersecurity capabilities.

The idea was to work with fictional companies inside a controlled environment. According to Irregular, however, a name chosen for a fictional company happened to match a real domain. At the same time, the evaluation environment ended up with internet access that was not supposed to be available.

As a result, models were able, in some runs, to leave the scenario the evaluators had intended and reach real infrastructure. Irregular published an account of the problem in August and said it had already been fixed. Irregular's report

The specific Gemini case became public on September 18 after reporting by The Wall Street Journal. Google confirmed that Gemini accessed systems belonging to three real companies during the test. Reuters

And this is the part that deserves more attention from software developers.

Three accesses. None required a science-fiction AI hacker

According to what Google and the press disclosed:

The companies' names were not disclosed.

The specific Gemini version involved was also not publicly disclosed. So any headline attributing the incident to a specific model version goes beyond what has been confirmed.

That changes the way we should read the story.

Gemini had to be capable of executing an autonomous chain of actions, interpreting what it found and deciding what to do next. That is relevant. But the initial access did not come from a revolutionary vulnerability discovered by AI.

It came from familiar problems:

unintended network access + an incorrectly scoped target + weak authentication or exposed credentials.

The detail the hype leaves out

It is tempting to describe the episode as “the AI escaped the sandbox.”

But Irregular's public description is more specific: the problem was an evaluation environment that allowed internet access and a collision between a fictional name and a real domain. Irregular also says the later disclosures were connected to the same underlying evaluation issue, and that the environment had already been fixed. Irregular

That matters because a prompt is not a security control.

Telling an agent:

“You don't have access to the internet.”

is not equivalent to configuring infrastructure where the agent physically cannot make an outbound connection.

If an agent has shell, browser or HTTP tooling, the boundary needs to exist at the infrastructure layer too.

That distinction applies to CTFs, but also to coding agents, internal automations and AI tools running in environments with access to real systems.

What about secrets in Git?

This is the easiest part of the story to turn into practical action.

If an AI can search public code, find a repository and recover a valid credential, there is nothing particularly “magical” about it.

The problem existed before AI.

The difference is that an agent can search, test hypotheses and chain actions much faster than a human.

So it is worth reviewing at least:

1. Secret scanning

Tools such as Gitleaks can search repositories and their history for tokens, passwords, keys and other sensitive patterns.

On GitHub, Secret Scanning and push protection can also detect or block certain secrets before they reach a repository.

The goal is not simply to find a secret after it leaks. It is to prevent the secret from being published in the first place.

2. An exposed credential must not remain valid

Finding a password or token in Git is only half the problem.

If that secret still works after it reaches a public repository, the exposure window can be enormous.

The rule should be simple:

secret leaked → revoke → generate a new one → investigate prior use.

Removing the file from the latest commit is not enough if the credential remains valid or is still present in Git history.

3. Rate limiting and lockout

In the disclosed case, one of the entry points involved password guessing.

That puts some very old security controls back at the center of the discussion:

It does not matter whether the password attempts come from a person, a script or an AI agent.

An authentication system that accepts thousands of attempts is still weak authentication.

4. Your agent's network access

Perhaps the most interesting question for developers starting to use AI agents is:

If my coding agent receives a wrong instruction, how far can it reach?

An agent with terminal and internet access can potentially:

If it is running on a development machine, that can mean access to much more than the developer intended.

For agents with real autonomy, it is therefore worth thinking about network egress, allowlists, least-privilege permissions, temporary credentials and environment isolation.

The curious part: Gemini stopped by itself

According to Google, the model stopped after realizing it had reached real company systems. Google also said the affected organizations were informed and that its testing processes were changed. Reuters

That is interesting as model behavior, but I would not use it as the security mechanism.

If an agent is authorized to perform a potentially destructive action, the primary control should prevent the action from leaving scope, rather than depend on the model realizing afterward that it made a mistake.

The model stopping is an additional layer.

Firewalls, isolation, permissions, authentication and secret management should remain the layers between the agent and the real system.

So is AI the problem?

The case does not support such a simple conclusion.

There is a genuinely new capability here: an agent can search, interpret results, test credentials and continue a chain of actions without a human directing every individual step.

But the three disclosed accesses also show something else: the barriers the agent encountered were largely traditional security failures.

The incident becomes really interesting when we put both things together:

increasingly autonomous agents + systems that still rely on basic security controls.

AI did not invent the weak password.

It did not invent the secret published in Git.

It did not create the test environment's internet access.

But it was able to combine those conditions into a sequence of actions.

And that is exactly why securing applications with AI agents should not start with a list of “scary AI behaviors.”

It should start with a much simpler question:

If this agent does everything it is technically capable of doing, what is the worst thing it can reach?

If the answer is “production,” “the database,” “customer credentials” or “the entire internet,” the problem may not be the model.

It may be the permissions we gave it.

Sources

← All posts