← Tools I Use

Can LLMs hack your app? What a $1,500 experiment actually shows

2026-06-05 · ~7 min read · Conor Dobbs

A security researcher named Kasra Rahjerdi did something I wish more people would do before they write a thinkpiece: he built a deliberately vulnerable app, handed thirteen-plus AI models a budget and a goal, and watched what they actually did. He spent about $1,500 total — roughly $10 and two hours per run — to answer one question. Can a large language model, pointed at your software with no human help, find its way in?

The headline result is being passed around as a leaderboard: GPT-5.5 won, Gemini refused to play, the Chinese models were cheap and aggressive. That is the fun part, and I will get to it. But the leaderboard is the least interesting thing in the experiment, and treating it as the takeaway is how you learn exactly the wrong lesson.

The setup

The target was a fake book-review app — "BookNook," a React Native / Expo front end with a Python backend. The objective was to read a flag hidden in another user's private reviews. Crucially, the API itself was solid. The reviewer described it as "very secure." The hole was not in the code anyone would normally audit. The app used Firebase as its data layer, and the shipped client bundle contained a google-services.json with live Firebase project details. With those in hand, an attacker could register a normal account and read the Firestore database directly, going around the careful API entirely.

Read that twice, because it is the whole story. The vulnerability was not a clever code bug an LLM out-reasoned a human to find. It was a configuration mistake — secrets shipped inside the artifact you hand to the user — sitting in plain sight inside the APK.

The leaderboard, since you want it

Across ten runs each, GPT-5.5 solved it 7 out of 10 times, at about $9.46 per solve. It "zeroed in on Firebase immediately after unpacking the APK," per the writeup. DeepSeek V4 Pro solved it 3 out of 10 but at $0.62 per solve — roughly fifteen times cheaper per success than GPT-5.5, which makes it the actual cost-efficiency winner if you care about dollars per breach rather than raw rate. Claude Sonnet 4.6 and Claude Opus 4.8 each solved it 2 out of 10. And Gemini 3.1 Pro Preview refused almost every run outright, with 3.5 Flash close behind — a median of around 9,000 tokens spent versus 100,000-plus for every model that actually tried.

The author is admirably honest about what this is: "not a scientific evaluation at all, just a well-documented experiment." Ten runs per model, one vulnerability class, one app. Do not put GPT-5.5 on a pedestal over Claude on the strength of an n of 10. The ranking is the noise. The pattern underneath it is the signal.

The winner didn't out-code anyone

Here is the part that matters. The models that won did not find a subtle logic flaw the developer missed. They unpacked the shipped app, found credentials lying in the bundle, and walked through the front door those credentials opened. The skill being tested was not exploit development. It was "notice the secret the developer left in the artifact, and use it."

That is a recon task, not a hacking-genius task — and it is exactly the seam I have spent this month writing about. The self-propagating npm worm, auth tokens leaking through published packages, the VS Code extension that lifts your GitHub token in one click — same root every time. Modern software is assembled and shipped, not authored in a vault, and the attack surface lives in what you accidentally bundle and publish. A secret in a google-services.json, a token in a postinstall script, a key in an extension's webview: the vulnerability isn't that the model is smart. It's that the secret was reachable.

The economics are the actual news

For years, having someone competent fan out across your app looking for the weak seam was a scarce, expensive, human-gated event. You booked a pentest once a year, paid five figures, and got a PDF. This experiment priced the same recon at well under ten dollars a run, fully autonomous, repeatable on a schedule.

That asymmetry cuts both ways, and you have to sit with the uncomfortable half. If a defender can point a cheap agent at their own bundle and surface the exposed-secret seam for the price of a coffee, so can an attacker — at scale, against thousands of apps, with no analyst in the loop. The cost of continuous recon has collapsed for everyone. The defenders who come out ahead are the ones who treat scanning their own shipped artifacts as continuous infrastructure — a cron job, not an annual ritual — rather than waiting to find out an LLM got there first. That continuous-self-scan posture is the same thesis behind everything I build: cheap checks that run forever beat expensive checks that run once.

About those refusals

The Gemini result is its own small lesson. Refusing to attack a random target is the correct default for a consumer assistant. But if you are a developer trying to red-team your own application — which is a completely legitimate, defensive thing to do — a model that bails at 9,000 tokens with a canned refusal is not "the safe choice." It is the useless choice for that job. The friction is real, and it is worth knowing before you pick a model for security work: the one that is most reluctant to engage with a live system is the one that will be least useful when the system you want it to probe is yours. Alignment that cannot tell "attack a stranger" from "audit my own code" pushes serious defensive work toward the models — frequently the Chinese ones, in this run — that will just do it.

What I'd actually take from this

Skip the leaderboard. The three things worth keeping are these. First: the bug that gets you is almost never the clever one — it is the secret or the misconfiguration you shipped without noticing, and an LLM is now a cheap, tireless way for anyone to find it. Audit what is in your bundle, your package, your build output, not just your code. Second: the price of recon has dropped to near zero, so the only sane response is to run it against yourself, continuously, before someone else runs it against you. Third: if you are doing defensive security work with AI, refusal behavior is a real selection criterion, not a footnote — test whether your model of choice will actually engage with your own systems.

The fun version of this story is "GPT-5.5 is a good hacker." The true version is quieter and more useful: the cost of finding the dumb mistake you shipped just fell through the floor, and that clock is running whether or not you point the agent at yourself first.

Sources