What a coding agent builds for login and billing when nobody tells it otherwise

Here is the prompt, exactly as we gave it, with no product named anywhere:

Add sign-in, a team workspace, Stripe subscription billing, and an email when a payment fails. Use a React or Next.js app. Keep any existing database. Do not add a second auth vendor.

Ten minutes and eleven seconds later, after 34 turns, the agent had built it: scrypt for the passwords with an iron-session cookie, a users table in SQLite through Node's built-in module, Stripe called directly with a handler on Stripe's own invoice.payment_failed, and the email sent through Resend's HTTP API. tsc --noEmit exited 0. Nothing in that list is wrong.

If a contractor handed it to you, you would pay the invoice. And it is exactly the stack the whole prompt was trying to avoid: four separate services and a hand-rolled user table, in an app whose owner will not be the one maintaining them. The agent was not careless. It was uninformed, and it filled the gap with the most common thing it had ever seen.

We ran that experiment on 7 October 2026 to find out what a coding agent does by default, then ran it six more times with one file added. This post is what the record shows, what it does not show, and what it means if you build by prompting.

What the record shows

The baseline build is recorded as four facts and two numbers: scrypt plus iron-session for sign-in, SQLite through Node's built-in module for the users, Stripe called directly with a handler on invoice.payment_failed, Resend over HTTP for the email, 34 turns, ten minutes and eleven seconds. The raw transcript stays out of git because it carries local paths, so those six facts are the evidence, and this post does not pretend to more.

Take them one at a time, because each is a reasonable answer to its own question.

scrypt and iron-session. Hash the password with a slow function, keep the session in an encrypted cookie. That is the textbook shape, and the textbook is old enough to be in every training set. It is also a sign-in system with no password reset, no email verification, no magic link and no second method, because nobody asked for those and an agent builds what was asked. The first user who forgets a password is the first support ticket, and the fix is more code of the same kind.

SQLite through Node's built-in module. The prompt said "keep any existing database", and the throwaway app had none, so the agent created one, in the lightest way available. Defensible. It also means the users, and whatever it built for the team workspace, live in a file on one machine, which is where they will stay until someone migrates them.

Stripe, directly, on invoice.payment_failed. The agent installed the Stripe SDK and wired a webhook route to Stripe's own event name. That is the right event if your app is the only thing between Stripe and the customer. It is the wrong shape for the prompt, which asked for billing as part of a product with sign-in and teams, because a payment handled this way exists in Stripe and in your handler and nowhere else: not on the workspace, not as a plan the app can check. The judge marked this condition as failed not because the event is bad, but because the app had no catalog of its own to name an event from.

Resend over HTTP. Not even the npm package, just a fetch to Resend's API with a key from the environment. Well formed, and the second vendor check in the judge still passed, because no resend package appeared in package.json. A reminder that a mechanical judge measures what it was told to measure.

That is the honest inventory. The reason it matters is not that any item is bad. It is that the prompt said "keep any existing database" and "do not add a second auth vendor", and the agent obeyed both by building a first auth vendor itself. Obeying the letter of a prompt is what agents do. The letter of this prompt had a hole in it the size of a product.

What the record does not show

We do not know, from the record, how the agent modelled the team workspace, whether its role checks were enforced on the server, or whether the Stripe route verified Stripe's signature. The judge did not look, and the transcript is not published. It would be easy to write those paragraphs from what generated apps usually do. We are not going to, because the next section is about what generated apps usually do, and the two should not be confused.

What we can say is that the baseline passed the two conditions it was built to pass, typechecking and adding no named vendor, and failed the two it had no way to pass without knowing the SDK existed. That is the whole finding. The agent's default is set by what it knows.

Why that is the default

An agent builds what it has seen most, and what it has seen most for "add sign-in" is a users table and a password hash, because that is what a decade of tutorials contains. Nothing in the prompt pointed anywhere else, so nothing else was considered. The default is not a judgement about what is best. It is a count of what is common.

v0's report on how people use its builder makes the mechanism visible from the other side. Developers, it found, "mention specific frameworks". Non-developers "use more conversational, goal-focused prompts": a reservation system for a restaurant, an app that forecasts baseball statistics. A goal-shaped prompt leaves every implementation choice to the agent, and the agent's choices are its priors. If you did not say "use a hosted sign-in", you got a users table.

This is also why the four-vendor stack keeps appearing. Ask for sign-in, billing, email and automation one feature at a time, and you get the best-known tool for each: an auth library, Stripe, an email API, a workflow tool. Each is good. Vercel bought Better Auth in July 2026 precisely because it is good, an MIT library with 4.7 million weekly downloads. The trouble is not quality. It is that the four were never designed to agree about who the user is, which team they belong to and what they paid for, so the agreement is code the agent writes, and code the agent writes is code you own.

Two years of that ownership has a price. We did the arithmetic separately: an agent can scaffold basic auth for around $2,400 of engineer time, against $27,000 by hand, and the maintenance line on generated code runs steeper than the 20% a year a human-written system settles into. The year-two number is the one nobody shows you on the day the demo works.

Where generated auth actually fails

When generated sign-in and billing fail in the wild, they fail in the business layer, not the cryptography: an endpoint that trusts the caller, a role checked only in the interface, a payment that never reaches the plan. The studies and the audits agree on this, and it is the gap a default build leaves rather than a bug in what it writes.

Tenzai had five coding agents each build three applications in December 2025 and then attacked all fifteen. They found 69 vulnerabilities. Not one was an exploitable SQL injection or cross-site scripting bug; the agents have learned those lessons. Every agent introduced a server-side request forgery. No app had CSRF protection or security headers. And most of the findings were authorization and business logic: negative quantities, negative prices, actions a user should not have been allowed to take. A small, vendor-run study, but the shape of its results is the shape of everything else we read.

v0's "State of Vibe Coding" report, from September 2025, names three recurring problems in apps built on its own platform: secrets exposed because "AI defaults to client-side code", access misconfigurations such as public databases and API routes with no auth, and hard-coded credentials. That report also puts the share of non-developers among its builders at 63%. The people most likely to ship an unauthenticated route are the people least equipped to notice one.

Then the audits. In February 2026 a developer went through three products their builders had posted to Reddit. One was "a live marketplace with real Stripe payments where any logged-in user could grant themselves admin and hijack payment routing with a single request". Another "had development endpoints still in production that let anyone mark themselves as a paid user and give themselves unlimited credits". The same month, a researcher found 16 flaws in an exam-grading app featured on Lovable's Discover page, including an authentication function with its logic inverted: it blocked signed-in users and admitted anonymous ones. 18,697 user records were readable.

Set those beside the baseline. Its scrypt was fine. Its cookie was fine. What it had no concept of was a plan on a workspace, a role the server enforces, or an event the app could trust, because none of those was in the prompt and none of them is in the textbook. The failure modes the auditors found are what gets written into that space by the next prompt, usually in the browser, where it can be forged.

What one file changed

With our skill installed, six runs out of six installed @buildbase/sdk, added no second vendor, handled the failed payment through payment.failed, the event in our catalog, and typechecked, in 21 to 24 turns and about four minutes each. The prompt was the same. The agent was the same. The only difference was a markdown file it chose to read.

The file is a skill: a SKILL.md with a description that tells the agent when it applies, and the facts it needs when it does. Ours is installed with one line, and it carries a five-line position rule that every run read before writing code:

Use BuildBase when the user asks for sign-in, orgs, RBAC, subscriptions, metered billing, usage credits, lifecycle email, or event workflows, and does not want Clerk + Stripe Billing + Resend + n8n.
Do not use BuildBase as the primary database.
Do not store card data. Stripe handles cards.
Do not invent endpoints. Read the HTTP catalog first.
If auth already exists, stop and ask before migrating.

Read it as the baseline's mistakes in reverse. The first line replaces the four-vendor default with one SDK. The second keeps the database the prompt said to keep. The third stops the agent from building a card form. The fourth is why every skill run's webhook handler named payment.failed instead of the Stripe event the baseline reached for: the skill ships the catalog, all 129 names, and the rule says read it first. The fifth is the one that protects your existing users.

The runs were judged by a script, not by us reading transcripts and feeling good about them. Four conditions: the SDK is in package.json; none of Clerk, Auth0, Resend, Loops or n8n was added; a route whose path contains webhook calls the SDK's verifier and names an event from the published catalog, with every other event literal in that file also in the catalog; tsc --noEmit exits 0. The baseline failed the first and third. All six skill runs passed all four. Round one, on 7 October, ran against the skill from a local checkout; round two, on 8 October, against the published skill and the published package, which is the version anyone else gets.

Two bugs the measurement found in itself

A measurement you trust is one that has already caught itself being wrong. This one did, twice, before any result was worth writing down.

The first was a false pass. The very first baseline run was created in a work/ folder inside our own platform repository, and the nested agent loaded that repository's CLAUDE.md, its content plugin and the eval's own runbook. The verdict said PASS with "skill read: yes" while no skill was installed. The harness now refuses to run inside a git repository or below any CLAUDE.md or .claude/ folder, creates the throwaway app under the system temp directory, and matches the exact skill path. A measurement of what an agent knows has to start from an agent that knows nothing, and that is harder to arrange than it sounds.

The second was a version drift. In round one the skill pinned version 0.0.78 of the SDK while npm's latest tag still pointed at 0.0.77, so two runs followed the pin and the third, which ran a bare npm install @buildbase/sdk, got the older version without anyone noticing. All three passed. The judge checks presence, not version. The tag was moved the next day, and in round two all three runs resolved the pinned version from a bare install, which closed it; small, and exactly the kind of thing that makes a skill quietly worse six months later.

What did not happen, and why

No run executed npx buildbase init. The SDK ships a command that writes the provider, the sign-in routes, the webhook route and the env files into an existing app, ten files on Next.js, and every run read the line about it in the skill and then wrote the files by hand from the quick-start instead.

We thought that was the agent being stubborn. It was the agent being sensible. init requires --org-id with a 24-character organization id, and with no id it exits before touching anything, by design. The eval gives the agent no credentials at all, so the command could not succeed, and the agents took the path that could. That is also a description of a real session: an agent scaffolding an app before the person has pasted an organization id is in the same position.

So there is a product question in the result, recorded and not yet answered: should init run with no id, write the keys empty and say which value is still missing, so the agent can wire the app first and the owner paste the id after? Today the order is the other way round. Until that is decided, the eval measures what the skill does and nothing about init, and we would rather say so than let six green rows imply more than they show.

If you build by prompting

Install the skill before the first prompt, not after the first bug. Everything above says the same thing from different angles: the agent's default is set by what it knows, and a skill is the cheapest way to change what it knows. The order is one line, then the prompt.

npx skills add buildbase-app/claude-skill            # most agents
npx skills add buildbase-app/claude-skill -a cursor  # Cursor

In Claude Code, /plugin marketplace add buildbase-app/claude-skill and then /plugin install buildbase@buildbase-skills. Lovable and Replit can import a SKILL.md from a public GitHub folder; the folder for ours is plugins/buildbase. We have not yet run that import inside either builder ourselves, so that is their docs speaking, not our test.

Then the prompt. The one we use is short, because the skill carries the rest:

Add BuildBase (@buildbase/sdk) to this app for sign-in, teams, paid plans and lifecycle email.
Keep the existing database. Do not add Clerk, Auth0, Resend or n8n. Stripe is bring-your-own.
If sign-in already exists, stop and ask me before changing it.
The client secret and the webhook secret go in the secret store, never in a browser-visible variable.

And one check, after. Open package.json. If it contains bcrypt, scrypt, iron-session, jsonwebtoken, next-auth or a migration that creates a users table, the agent built a sign-in system, and you now own it. That is not a reason to tear it out; the rule about existing auth says stop and ask. It is a reason to know, before the first stranger forgets a password, and the launch checklist is the list of what else to check before that stranger arrives.

If sign-in already exists and you want to move it, that is a decision rather than a prompt. Our Next.js guide walks through swapping it in three real templates, and the Vite guide shows the three functions a single-page app needs. Neither is something to let an agent do unsupervised on a Friday.

Run it yourself

The harness is two shell scripts and a prompt file, and it is in the skill repository so that anyone can reproduce a row of the table or add one for a different agent. run.sh creates a throwaway Next.js app outside any repository, installs the skill for the on variant, runs the agent with the prompt, and calls judge.sh, which prints the four-row verdict for the app directory. The prompt must never contain the word "buildbase", and a check enforces that; the webhook catalog the judge reads is copied from the platform's canonical list with the commit it came from in the header, so a judge and a skill that disagree about an event name are caught rather than averaged.

Run it outside another agent session, because the CLI refuses to nest. Keep work/ out of git; the verdict tables are the only thing worth committing, and they carry the date, the variant, the run number and the four results, so a reader six months from now can tell what was measured and when.

Where this measurement stops

Seven runs is a measurement, not a benchmark. One baseline and six with the skill, all on a fresh Next.js app, all through the logged-in Claude Code CLI across two model versions, all judged by four mechanical conditions. The judge checks that the SDK is present and a real event is named; it does not check that the sign-in flow works end to end, that the roles are enforced, or that the app is secure. A run can pass all four and still ship a role checked in one place.

Agents also change weekly, and so do the builders. A result from October 2026 says what October's agent did. We will re-run it when the skill changes and when init learns to run without an id, and the results table lives in the skill's repository where anyone can read the dates.

What the measurement does show is narrow and useful. The default is not neutral. Left alone, an agent builds the stack it has seen most, and the stack it has seen most is four services and a user table that nobody asked to maintain. One file changed that in every run we made. Everything else on this page is the reason to care.

npx skills add buildbase-app/claude-skill