All writing
4 min read

The security risks nobody checks in vibe-coded apps

Generated code ships with a recognisable set of holes: absent authorisation, secrets in the client bundle, unbounded AI endpoints, and dependencies nobody chose. What I look for when reviewing an app that was mostly written by a model.

Asghar Ali · Chief Technology Officer, Kakushin.io LTD

I review a lot of code that a model wrote. Across a portfolio this size, patterns emerge — and the failures are not random. Generated code fails in a specific, repeatable way, which is good news, because repeatable failures can be checked for.

The thing to understand first: a model produces code that looks like the code it was trained on. Most published code is a tutorial, a sample, or a happy-path snippet. Tutorials do not implement authorisation. Samples hardcode keys. So the output inherits those omissions, wrapped in confident, well-formatted syntax that reads like it was written by someone who had thought about it.

The authorisation hole

This is the one I find most often, and it is almost always the same shape.

The model writes an endpoint that checks authentication — is there a valid session? — and never checks authorisation — is this particular user allowed to touch this particular record?

// Generated, and wrong.
app.get('/api/orders/:id', requireAuth, async (req, res) => {
  const order = await Order.findById(req.params.id)
  res.json(order)
})

requireAuth passes. The user is logged in. They are also reading somebody else's order by incrementing an ID. This is IDOR, it is trivially exploitable from a browser console, and it will not show up in any test that only ever runs as one user.

// The check that was missing.
const order = await Order.findOne({ _id: req.params.id, userId: req.user.id })
if (!order) return res.status(404).end()

What to check: every query that takes an ID from the request. Does the lookup constrain by owner? If your test suite only has one fixture user, it cannot catch this — add a second user and assert they get a 404.

Secrets on the wrong side of the network

Models are careless about which side of the boundary code runs on, because in training data both sides look like JavaScript.

In Next.js the specific trap is the NEXT_PUBLIC_ prefix. It does exactly what it says — inlines the value into the client bundle — and a model will reach for it whenever a variable is undefined in the browser, because that makes the error go away.

// This ships your key to every visitor.
const openai = new OpenAI({ apiKey: process.env.NEXT_PUBLIC_OPENAI_KEY })

What to check: grep the built bundle, not the source.

npm run build
grep -rE "sk-|AIza|SG\.|xoxb-" .next/static/ | head

If anything comes back, the key is public and must be rotated, not just moved. It is already in every browser cache that loaded the page.

AI endpoints with no ceiling

Any route that calls a model API is a route that spends money per request. The generated version has no rate limit, no per-user quota, and no cost ceiling — because the tutorial it learned from was demonstrating the API call, not running a business.

One scraped endpoint and a loop is a bill.

What to check: for every route that hits an LLM — is there a per-IP or per-user limit? Is there a hard monthly ceiling that fails closed? Is the token count of user-supplied input bounded before it is sent?

Across our estate the reference implementation keeps a per-IP ledger, several abuse heuristics, block and whitelist actions, and token-and-cost analytics per route. That was not built speculatively. It was built after watching what unbounded endpoints attract.

Dependencies nobody chose

Models suggest packages by name, from memory, and memory of package names is exactly where hallucination is most dangerous. A confidently suggested import for a package that does not exist is an open invitation: someone registers that name, and every developer who pasted the suggestion installs it.

What to check: for every dependency added in a generated change, does it exist on the registry, when was it published, how many maintainers, and is it the package it claims to be? A dependency added in a PR nobody questioned is how supply-chain compromise actually happens.

Input that reaches a shell or a query

The classic injections have not gone away, they have just moved. Model-generated code is generally decent at parameterised SQL, because the training data is, but it is much weaker on:

  • NoSQL operator injection — passing req.body straight into a Mongo query, where {"$ne": null} is a valid password.
  • Shell interpolation — building an ffmpeg or git command from a filename.
  • SSRF — fetching a URL the user supplied, on a server that can see your internal network and its metadata endpoint.

What this is not

None of this is an argument against generating code. I use these tools daily and they have changed my throughput materially. The output is frequently better structured than what a rushed human writes at 6pm.

But it is confidently structured, which is a different thing from being correct, and confidence is precisely what makes a reviewer skim. The failure is not the model's. It is a review process that was calibrated for code written by someone who had to think about every line, applied to code that was produced without anyone thinking about any of them.

The fix is not to stop generating. It is to review generated code more carefully than handwritten code, not less — and to know exactly which five things to look at first.


Related: why human validation is still the constraint.