Testing & Tracking Framework

​

The Premise

We do not know which of these shows will work. Anyone who claims to know which cannabis format goes viral on TikTok is guessing with confidence. What we can control is learning rate. If every video is logged with its script ID, hook ID, show, format, and runtime, then forty videos produce a real answer about what this audience wants. If they are posted without logging, forty videos produce forty opinions.

​

What Actually Predicts Reach On TikTok

Not all metrics are equally informative. Ranked by how much they should influence decisions: Follower conversion (follows per thousand views) is the one that matters for the business but it moves slowly. Read it monthly, not per-video.

​

Benchmarks

Targets for a new account in its first 90 days. The 3s, 15s, and 30s retention rows are now anchored to researched platform benchmarks rather than guesses — roughly 70% past 3 seconds, 60% at 15 seconds, and 50% at 30 seconds is the level that signals wider reach.

​

The AI-Content Variable

Two researched facts change how results here should be read:

• Properly labeled AI content carries no algorithmic penalty. Label every video. Unlabeled synthetic content runs a four-step path to a permanent ban.
• Low-quality AI loses roughly 30-45% of reach; high-quality AI performs within a few percent of anything else. So a weak result on a video with visible character drift or artifacts is a production failure, not a script failure — do not let it kill a script that was never fairly tested.

Add a quality_flag note to the log for any render with visible drift, and exclude those rows when comparing scripts. Set it at post time — the first analysis runs at 48 hours, so a flag added later cannot protect it.

​

The Test Design

Each batch isolates one variable. Everything else stays fixed. This is the part that is easy to skip and expensive to skip.

​

The First 40 Videos

A concrete plan for the ramp. Four weeks, ten videos a week, and at the end we know which shows to build a season around. At the end of week 4 there is a defensible answer to: which show, which hook family, which runtime, which register. That is the foundation of a real content calendar.

​

The Episode-1 Cliff

Numbered series decay hard. From a 57-video dataset of one animation channel: Sequels do not inherit the algorithmic bump. Combined with comedy having the lowest binge rate of any genre, the implication is direct: design for a permanent stream of pilots, not seasons. Practically, that means every episode must work cold for someone who has never seen the show, and a “part 2” should be a rare deliberate choice rather than a default. When comparing a series’ performance, compare episode 1 against episode 1 of other series — not episode 5 against episode 1, which will always look like failure.

​

Decision Rules

Applied at 48 hours for a first read and 7 days for the decision. TikTok’s long tail is real — a video can find an audience on day 5 — so no video is judged dead before day 7.

​

The Performance Log

The log lives at social-content/output/tracking/performance-log.csv in the repository. One row per posted video, filled at post time and updated at 48 hours and 7 days. The notes column is the most valuable one in the file and the easiest to leave blank. A number tells us what happened; the note is the only place the why survives long enough to be useful three months later.

​

Weekly Review

Thirty minutes, once a week. Four questions, in order:

​

Where This Connects

Individual episode drafts are tracked in the social_content_drafts table in Supabase, surfaced through the Social Content Drafts queue. That table tracks draft workflow state — generated, reviewed, approved. This CSV tracks published performance. They are deliberately separate: the draft queue answers “what is ready to post,” and the log answers “what should we make more of.” Joining them on script_id is the eventual path to an automated dashboard, once there are enough rows for that to be worth building.

​

Related