What Tier C is for
Tier C runs Vitest suites against the real staging backend: the shared staging Convex deployment, the Clerk development instance, the production Hono API, and — for pipeline cases — Trigger.dev. Tiers A and B never touch a deployment. Tier C exists for exactly the things they cannot see:- scheduled side effects on a real deployment
- real argument validators, as deployed
- real Clerk JWT verification, end to end
- outbound
fetchfrom Convex actions - webhook round-trips and pipeline completion
Running it
No test files found and prints the filter it used. That is deliberate: once a
suite exists, a silently-empty run is exactly the failure mode a weekly job must
never hide.
The isolated-user pattern
Every suite creates a brand-new Clerk user per test and deletes it afterwards. Each identity is a real Clerk account that really signs in — the suite mints real session JWTs from it, so Convex and the Hono API verify the same tokens the app would present. This is the isolation mechanism, and it also gives every run a fresh API rate-limit bucket.cleanup() clears the test user’s rows, asserts that every remaining* counter
is zero — a leaked row fails the test rather than silently accumulating on
staging — then deletes the Clerk user. It runs always, even when the wipe
failed, so a failing test cannot leak identities.
The +clerk_test local-part suffix is mandatory: every fixture mutation is
gated on it server-side and throws without it.
The production guard
Three independent layers, all of which must agree before a single row is written:1
assertStagingConvexUrl() in the harness
Enforced by the suite’s
beforeAll. It rejects the production deployment with
a distinct message, then rejects every other host. It is a pure function and
is unit-tested in the smoke suite with no network.2
The pnpm scripts hardcode the staging URL
There is no secret and no env file that can be repointed at production — you
would have to edit
package.json.3
assertNotProdDeployment() server-side
Every fixture mutation calls it first, so even a hand-rolled client cannot run
fixture writes against production.
Required environment (names only)
When a required variable is missing, suites skip rather than fail, and the
harness names exactly what is absent. Do not add
TIER_C_CONVEX_URL to a
committed .env file — keeping it out is what makes running Tier C a deliberate
act.
The admin runner
Some assertions need Convexinternal* functions, which have no public
equivalent. createAdminRunner() has three modes:
Codegen and typecheck are disabled on every CLI call, so Tier C never rewrites
convex/_generated or your working tree.
Tags
Tags live in describe/test titles because Vitest-t matches the concatenated
title.
Cost and blast radius
- Trigger.dev runs land in the prod environment, sharing queues, concurrency, and cost with real user work. An abandoned run keeps executing and keeps billing after Vitest has moved on — which is why the harness cancels a run it abandons on timeout, and why the CI concurrency group never cancels a run in flight.
- The notebook pipeline is a deterministic builder with no model call in its path, so that suite’s AI spend is effectively nil (compute is a fraction of a cent per full run). A case that forces genuine strain research is the expensive exception.
- Gmail classification calls a real model for candidates that miss the fast path. Cost scales with the scan window, so the window stays pinned narrow (~30 days). If a Gmail case fails on volume, do not widen the window to “get more signal”.
- Dispensary menu scans are paid per cold scan (5–6 minutes each) and run only under the paid opt-in.
- The Hono API leg hits the deployment real users hit, at production rate limits. Keep API assertions read-only and user-scoped.
retry: 0is deliberate. Never raise it to paper over flakiness — that pays twice for the same signal. Fix the flake or quarantine it with a tracked reason.- Staging is shared by every non-production build plus a daily cron. Never assert on a global count, a “latest N” listing, or a cross-user aggregate.
Gmail: the one shared-account exception
The Gmail-connected identity is a specific, pre-existing Clerk user with a linked Google account; its OAuth grant lives in Clerk, not in any env file. A freshly created test user has no Google account and cannot scan. So the Gmail suite:- runs against the identity named by
GMAIL_DIAGNOSTIC_EMAIL, not an isolated user; - is tagged
@shared-accountand must be serialized — one Gmail Tier C case at a time, and never while the Revyl order-import workflow is running, since both drive the same mailbox and the same per-user queue; - must clean up after itself explicitly, because the isolated-user teardown does not apply;
- must pin a narrow scan window and assert on candidate shape, not exhaustiveness.
Writing a new suite
- Prefer the public fixtures — they run the real validators, the real identity guard, and the real scheduled side effects, which is the whole point. Reach for admin fixtures only when a case needs data no public mutation can create.
- Scope every assertion to the test user’s own Clerk user id.
- Never string-compare a Trigger.dev run status; use the classifier helper, which fails closed.
- No
any— parse boundaries asunknownand narrow. - Log anything derived from real receipts or orders through the privacy-safe diagnostics helper. Tier C output lands in CI logs.
Troubleshooting
Related
Test Platform Overview
The three tiers, what each proves, and how to add a test.
Subscription-State Testing
Free, active-Pro, and lapsed-Pro on a gate-enforcing deployment.
