The Sentry That Never Fired

The app had Sentry. It was in package.json, it was in the config files, the wizard had clearly run. On any checklist that stops at “is error tracking installed?” this app passes. It was also, in the six weeks it had been live, completely silent — not one event, not one alert, not one issue in the dashboard. The owner believed production was watched. Production was not watched. That is a worse place to stand than knowing you have nothing.

This is a representative audit — the exact pattern I hit over and over on apps built with Claude Code, Cursor, Bolt, and v0. Names and paths are changed, but every finding is real-shaped and drawn straight from the kind of report the launchworthy skill produces. Call the app shipmate: a Next.js app on Vercel, built with Claude Code, wired to Sentry, about to onboard its first paying team.

It got a scorecard. Operations & Recovery did not pass.

Note the zero. There is no CRITICAL here — no leaked secret, no open table, nothing that ends the business in an afternoon. The worst finding on shipmate is a [HIGH], and I am going to call it a [HIGH], because inflating it to CRITICAL for drama would be exactly the kind of pattern-matching this whole exercise exists to replace. A [HIGH] that hides behind a green checkmark is interesting because it looks handled. That is the entire story.

The finding that ends the app: the Sentry that never fired

The dangerous finding is never the missing thing. Missing error tracking is honest — you know you’re flying blind, and you either fix it or you accept it. The dangerous finding is the one that looks handled and isn’t. shipmate’s Sentry was installed and inert, and here is the exact line that did it.

sentry.client.config.ts:6:

Sentry.init({
  dsn: process.env.SENTRY_DSN,
  tracesSampleRate: 1.0,
  enabled: process.env.NODE_ENV === "production",
});

Two problems stacked on top of each other, and both of them are invisible until you go looking. First, enabled only turns Sentry on in production — which is defensible in principle, except it means you can never see it work in development. You test locally, init never runs, nothing breaks, and you conclude it’s fine. Second, and fatally: SENTRY_DSN was nowhere. Not in .env.example, not in the Vercel project’s environment variables, not referenced in any deploy config. In production, process.env.SENTRY_DSN resolved to undefined, and Sentry silently no-ops on a falsy DSN. It does not throw. It does not warn. It just quietly captures nothing, forever.

So the SDK loaded, init ran in production exactly as intended, and every error since launch went into the void — while the config file, sitting there in the repo, read as proof the problem was solved.

The fix is not “install Sentry.” It’s already installed. The fix is to make the DSN real in the environment that actually runs, and then to prove — with a live event — that it fires.

The false alarm: “we have Sentry, so error tracking is done”

Here is the credibility move, and the reason to trust an audit that gets this right instead of one that just greps package.json.

Point a generic “is my app production-ready?” pass at shipmate and it sees @sentry/nextjs in the dependencies, sees three config files, and checks the box. Error tracking: handled. Move on.

The distinction is the whole game:

• Sentry in package.json → tells you the wizard ran. Nothing more.
• A live test error in the dashboard within a minute → the only proof error tracking exists.

Everything between those two — the config files, the DSN variable, the reassuring green checkmark — is just scaffolding that can be present and dead at the same time. This is the same discipline as never flagging a Supabase anon key as a leak: the surface signal (“there’s a key in the bundle,” “there’s an SDK installed”) is not the thing that matters. What matters is the layer behind it — whether RLS actually restricts that key, whether an event actually lands. Judge the behavior, not the artifact.

The rest of the punch list

The inert Sentry is the headline, but Operations & Recovery had more holes, and they compound. Ranked, so the scope stays honest:

Three findings, one theme: shipmate had the appearance of an observed system and none of the substance. Every piece was a step short of doing its job — the DSN a step short of being set, the server config a step short of being imported, the alerting a step short of reaching a person.

Why this keeps happening

None of this means Claude Code did anything wrong, or that shipmate’s owner was careless. The wizard ran correctly. The config was textbook. The tools are genuinely excellent at producing the shape of a hardened app — the files, the dependencies, the reassuring green. What they cannot do is the one thing that actually proves it: throw a real error at the live deployment and watch it land. That step lives outside the code, in the deployed environment, on a URL that isn’t your laptop — which is precisely where it’s invisible while you build.

That gap is the whole reason launchworthy exists. It plays bouncer at the door of production. It knows that an installed SDK is a claim, not a proof; it refuses to mark error tracking handled just because the dependency is there; and it hands you the live-test procedure, not just the finding. Installed but silent counts as no tracking — and the only thing that changes that verdict is an event you can see in the dashboard, right now, from the app your users actually hit.