Skip to main content

What Tier C is for

Tier C runs Vitest suites against the real staging backend: the shared staging Convex deployment, the Clerk development instance, the production Hono API, and — for pipeline cases — Trigger.dev. Tiers A and B never touch a deployment. Tier C exists for exactly the things they cannot see:
  • scheduled side effects on a real deployment
  • real argument validators, as deployed
  • real Clerk JWT verification, end to end
  • outbound fetch from Convex actions
  • webhook round-trips and pipeline completion
Limitation to state in every suite header: ConvexHttpClient has no subscriptions, so every convergence check is a poll. Tier C proves eventual correctness, not the reactive delivery useQuery gives the app.

Running it

Always run the smoke suite before a lane. It proves Clerk → Convex JWT verification, the fixture surface, the Hono API leg, the admin runner, and the production guard in under 30 seconds. A lane failure after a green smoke is a product signal rather than a harness signal. A per-area script whose suite file does not exist yet exits 1 with No test files found and prints the filter it used. That is deliberate: once a suite exists, a silently-empty run is exactly the failure mode a weekly job must never hide.

The isolated-user pattern

Every suite creates a brand-new Clerk user per test and deletes it afterwards. Each identity is a real Clerk account that really signs in — the suite mints real session JWTs from it, so Convex and the Hono API verify the same tokens the app would present. This is the isolation mechanism, and it also gives every run a fresh API rate-limit bucket.
cleanup() clears the test user’s rows, asserts that every remaining* counter is zero — a leaked row fails the test rather than silently accumulating on staging — then deletes the Clerk user. It runs always, even when the wipe failed, so a failing test cannot leak identities.
Never reuse the shared +clerk_test personas (power, testuser, onboarding, lapsed, ranked, …). They belong to Revyl, whose auth bypass wipes their stash on every launch — a Tier C test seeded on one can be cleared mid-run.
The +clerk_test local-part suffix is mandatory: every fixture mutation is gated on it server-side and throws without it.

The production guard

Three independent layers, all of which must agree before a single row is written:
1

assertStagingConvexUrl() in the harness

Enforced by the suite’s beforeAll. It rejects the production deployment with a distinct message, then rejects every other host. It is a pure function and is unit-tested in the smoke suite with no network.
2

The pnpm scripts hardcode the staging URL

There is no secret and no env file that can be repointed at production — you would have to edit package.json.
3

assertNotProdDeployment() server-side

Every fixture mutation calls it first, so even a hand-rolled client cannot run fixture writes against production.
Never turn TIER_C_CONVEX_URL into a repository secret. A secret would be a fourth, repointable layer and would defeat the other three.

Required environment (names only)

When a required variable is missing, suites skip rather than fail, and the harness names exactly what is absent. Do not add TIER_C_CONVEX_URL to a committed .env file — keeping it out is what makes running Tier C a deliberate act.

The admin runner

Some assertions need Convex internal* functions, which have no public equivalent. createAdminRunner() has three modes: Codegen and typecheck are disabled on every CLI call, so Tier C never rewrites convex/_generated or your working tree.

Tags

Tags live in describe/test titles because Vitest -t matches the concatenated title.

Cost and blast radius

Tier C spends real money and writes to a shared deployment. Budget before running.
  • Trigger.dev runs land in the prod environment, sharing queues, concurrency, and cost with real user work. An abandoned run keeps executing and keeps billing after Vitest has moved on — which is why the harness cancels a run it abandons on timeout, and why the CI concurrency group never cancels a run in flight.
  • The notebook pipeline is a deterministic builder with no model call in its path, so that suite’s AI spend is effectively nil (compute is a fraction of a cent per full run). A case that forces genuine strain research is the expensive exception.
  • Gmail classification calls a real model for candidates that miss the fast path. Cost scales with the scan window, so the window stays pinned narrow (~30 days). If a Gmail case fails on volume, do not widen the window to “get more signal”.
  • Dispensary menu scans are paid per cold scan (5–6 minutes each) and run only under the paid opt-in.
  • The Hono API leg hits the deployment real users hit, at production rate limits. Keep API assertions read-only and user-scoped.
  • retry: 0 is deliberate. Never raise it to paper over flakiness — that pays twice for the same signal. Fix the flake or quarantine it with a tracked reason.
  • Staging is shared by every non-production build plus a daily cron. Never assert on a global count, a “latest N” listing, or a cross-user aggregate.

Gmail: the one shared-account exception

The Gmail-connected identity is a specific, pre-existing Clerk user with a linked Google account; its OAuth grant lives in Clerk, not in any env file. A freshly created test user has no Google account and cannot scan. So the Gmail suite:
  • runs against the identity named by GMAIL_DIAGNOSTIC_EMAIL, not an isolated user;
  • is tagged @shared-account and must be serialized — one Gmail Tier C case at a time, and never while the Revyl order-import workflow is running, since both drive the same mailbox and the same per-user queue;
  • must clean up after itself explicitly, because the isolated-user teardown does not apply;
  • must pin a narrow scan window and assert on candidate shape, not exhaustiveness.

Writing a new suite

Rules:
  • Prefer the public fixtures — they run the real validators, the real identity guard, and the real scheduled side effects, which is the whole point. Reach for admin fixtures only when a case needs data no public mutation can create.
  • Scope every assertion to the test user’s own Clerk user id.
  • Never string-compare a Trigger.dev run status; use the classifier helper, which fails closed.
  • No any — parse boundaries as unknown and narrow.
  • Log anything derived from real receipts or orders through the privacy-safe diagnostics helper. Tier C output lands in CI logs.

Troubleshooting

Test Platform Overview

The three tiers, what each proves, and how to add a test.

Subscription-State Testing

Free, active-Pro, and lapsed-Pro on a gate-enforcing deployment.