> ## Documentation Index
> Fetch the complete documentation index at: https://docs.highailabs.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Strain Music Videos
> AI-generated music videos featuring Professor High for every cannabis strain, synced to each strain's unique AI song.
## Overview
Strain Music Videos are fully produced AI music videos — one for every cannabis strain in the High IQ database. Each video stars **Professor High**, our cartoon cannabis scientist, as the main character living out the strain's story: how it changes his day, his vibe, his world. The visuals match the song's genre — G-Funk gets lowriders and palm trees, Dream Pop gets ethereal forests, Stoner Metal gets dark stages and smoke.
Every video is generated automatically from the strain's existing Lyria 3 AI song and Professor High's mascot image. The result is a genre-appropriate 3-minute music video that captures the character of each strain in a way that text and photos alone cannot.
Strain Music Videos are built on top of the [Strain Music Pipeline](/help/features/strain-music-videos#pipeline-architecture). A strain must have an AI-generated song before a video can be created.
## How It Works
The pipeline has 6 stages that transform a strain's existing song and mascot image into a complete music video:
The strain's timestamped lyrics are parsed into a structured timeline of sections (intro, verse, chorus, bridge, outro) with precise start and end times. Each section is split into individual clips of 10–20 seconds — matching how real music videos are edited, with multiple camera angles per verse.
An AI scene director (Gemini Flash Lite) reads the lyrics, music description, strain metadata, and genre to write a shot list. Each clip gets a scene description, camera direction, visual mood, and a shot type — either a **performance shot** (Professor High rapping or singing to camera) or a **story shot** (Professor High living the strain experience).
For each clip in the shot list, Gemini generates a 16:9 scene image (2048×1152) placing Professor High in the described setting. The Professor High reference image is included in every generation call to maintain character consistency. The first clip always starts from the strain's existing mascot image to anchor the video in that strain's visual identity.
Each scene image is animated into a video clip using fal.ai LTX-2.3 image-to-video. The clip's prompt includes the scene description and camera direction (tracking shot, slow zoom, pan, etc.). Clips are generated in parallel — a 13-clip video runs up to 5 at a time.
Performance shots — where Professor High is rapping or singing — get lip sync applied. The exact audio segment for that clip is extracted from the full song track and synced to the video clip using WaveSpeedAI. Story shots pass through untouched.
All clips are stitched together in order using ffmpeg with short crossfade transitions. The original Lyria 3 song is overlaid as the audio track, a thumbnail is extracted from the first chorus, and the final MP4 is uploaded to Supabase Storage. The strain's record is updated with the video URL.
## Pipeline Architecture
### Location
```
packages/trigger/src/tasks/strain-video/
├── 01-parse-lyrics.ts
├── 02-scene-director.ts
├── 03-scene-images.ts
├── 04-video-clips.ts
├── 05-lip-sync.ts
├── 06-compose-upload.ts
├── orchestrator.ts
└── batch.ts
```
Same structure as the Strain Music Pipeline in `packages/trigger/src/tasks/strain-music/`.
### Stage Summary
| Stage | Task | Model / Tool | Purpose |
| ------------------ | ----------------------------- | --------------------- | ---------------------------------------------- |
| 1 — Parse Lyrics | `strain-video-parse-lyrics` | Pure function | Parse `audio_lyrics` into timed clip timeline |
| 2 — Scene Director | `strain-video-scene-director` | Gemini 3.1 Flash Lite | Write shot list matching genre and lyrics |
| 3 — Scene Images | `strain-video-scene-images` | Gemini 3.1 Flash | Generate 16:9 scene images with Professor High |
| 4 — Video Clips | `strain-video-clips` | fal.ai LTX-2.3 | Animate scene images into video clips |
| 5 — Lip Sync | `strain-video-lip-sync` | WaveSpeedAI LipSync | Apply lip sync to performance shots |
| 6 — Compose | `strain-video-compose` | ffmpeg | Stitch clips, overlay audio, upload |
### Provider-Agnostic Design
Both video generation and lip sync use swappable provider interfaces. The pipeline does not depend on any specific model — providers can be changed without modifying the pipeline stages.
**Launch providers:**
* **Video:** fal.ai LTX-2.3 image-to-video
* **Lip Sync:** WaveSpeedAI LTX 2.3 LipSync (or LatentSync on Replicate)
**Future provider options:** Veo 3.1, Kling 3.0, Runway Gen-4, self-hosted LTX-2.3
### Character Consistency
Professor High's visual identity is maintained across all 6 stages by including the **Professor High reference image** (`mascot-images/reference/professor-high.png`) in every image and video generation call — the same pattern the music pipeline uses for multimodal context. The strain's mascot image serves as the opening frame so each video begins in the strain's established visual world.
## Running the Pipeline
### Single Strain (Dashboard)
Go to the [Trigger.dev dashboard](https://cloud.trigger.dev/projects/v3/proj_kmfmmftspsichqetwiom) and run the `strain-video-pipeline` task with the strain slug as input.
```json theme={null}
{ "slug": "blue-dream" }
```
Use **Preview Mode** during testing to store all intermediates for review:
```json theme={null}
{ "slug": "blue-dream", "preview": true }
```
### Batch Processing
Run `strain-video-batch` to process multiple strains. Pass specific slugs or a batch size:
```json theme={null}
{ "slugs": ["blue-dream", "og-kush", "wedding-cake"] }
```
```json theme={null}
{ "batchSize": 10 }
```
When no slugs are provided, the batch task draws from strains that have audio but no video, ordered by popularity rank.
### Environment Variables
| Variable | Required | Purpose |
| --------------------- | -------- | --------------------------------------------- |
| `FAL_KEY` | Yes | fal.ai API key for video generation |
| `WAVESPEED_API_KEY` | Yes | WaveSpeedAI key for lip sync |
| `REPLICATE_API_TOKEN` | Optional | Replicate key (alternative lip sync provider) |
### Mock Providers for Testing
Set these environment variables to use mock providers that skip external API calls and return placeholder files:
```bash theme={null}
VIDEO_PROVIDER=mock
LIP_SYNC_PROVIDER=mock
```
Mock mode runs the full pipeline logic and stores all intermediates in Supabase — useful for testing the compose stage or debugging scene direction without incurring API costs.
## Preview Mode
Preview mode stores every intermediate artifact for each strain so you can review and tune the output before committing to production:
| Path | Content |
| ----------------------------------------------- | -------------------------------------------------------------------- |
| `preview/{slug}/scenes.json` | Full scene breakdown from Stages 1–2 — shot list, timing, shot types |
| `preview/{slug}/images/scene_{index}.png` | Generated scene image for each clip |
| `preview/{slug}/clips/clip_{index}.mp4` | Individual video clip before lip sync |
| `preview/{slug}/clips/clip_{index}_lipsync.mp4` | Lip-synced version of performance clips |
Preview artifacts are accessible via the API:
```bash theme={null}
GET /api/v1/strains/slug/{slug}/video/preview
```
## Database and Storage
### Supabase Storage
Bucket: `strain-videos`
| Path Pattern | Content |
| ---------------------------------- | ------------------------------------------ |
| `{slug}_v{version}.mp4` | Final composed video |
| `thumbnails/{slug}_v{version}.jpg` | Thumbnail / poster frame |
| `temp/{slug}/clip_{index}.mp4` | Temporary clips (cleaned up after compose) |
| `preview/{slug}/...` | Preview mode intermediates |
### Database Columns (`strains_v2`)
| Column | Type | Description |
| ------------------------ | ----------- | --------------------------------------------------- |
| `video_url` | text | Final video URL in Supabase Storage |
| `video_duration_seconds` | numeric | Duration in seconds (\~180s, matching audio) |
| `video_version` | integer | Increments on each regeneration |
| `video_generated_at` | timestamptz | Timestamp of last generation |
| `video_thumbnail_url` | text | Poster frame URL for video embeds |
| `video_scenes_json` | jsonb | Full scene breakdown for debugging and regeneration |
### API
The strain complete endpoint includes `videoUrl` and `videoThumbnailUrl` when a video exists:
```bash theme={null}
GET /api/v1/strains/slug/{slug}/complete
```
All strains with audio (and optionally video) are available from the music endpoint:
```bash theme={null}
GET /api/v1/strains/music
```
The music endpoint supports optional query parameters:
| Parameter | Values | Default | Description |
| --------- | --------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sort` | `name`, `popular`, `newest` | `name` | Sort order for returned tracks |
| `minimal` | `1` or `true` | — | Returns a slim 7-field payload (`slug`, `name_display`, `audio_url`, `mascot_img_url`, `audio_genre`, `audio_duration_seconds`, `play_count`) instead of the full response (\~680 KB). Use for lightweight callers like the homepage player. |
| `limit` | `1`–`500` | all | Cap the number of returned rows |
Example — fetch 10 most popular tracks with minimal payload:
```bash theme={null}
GET /api/v1/strains/music?sort=popular&limit=10&minimal=1
```
All music responses are cached for 15 minutes (`s-maxage=900, stale-while-revalidate=1800`).
## Cost Model
Each video costs approximately **$3.50–$5.00** to generate using hosted APIs. Self-hosting LTX-2.3 is a planned future option to reduce costs significantly.
| Stage | Tool | Cost per Video |
| -------------- | ---------------------------- | ----------------- |
| Parse Lyrics | Pure function | \$0.00 |
| Scene Director | Gemini 3.1 Flash Lite | \~\$0.01 |
| Scene Images | Gemini 3.1 Flash (13 images) | \~\$0.13 |
| Video Clips | fal.ai LTX-2.3 (13 clips) | \~\$1.50–3.00 |
| Lip Sync | WaveSpeedAI (\~6 clips) | \~\$1.80 |
| Compose | ffmpeg (Trigger.dev compute) | \~\$0.02 |
| **Total** | | **\~\$3.50–5.00** |
Clip count varies by song structure — a 3-minute song typically produces 12–14 clips depending on section lengths and how many splits the parser applies. The cost range above assumes 13 clips with \~6 performance shots.
## Genre-to-Visual Style
The Scene Director maps each strain's audio genre to a visual world. Examples:
| Genre | Visual Style |
| ------------ | -------------------------------------------------------- |
| G-Funk | Lowriders, palm trees, 90s West Coast, warm amber light |
| Dream Pop | Ethereal forests, soft bokeh, pastels, floating elements |
| Stoner Metal | Dark stage, smoke machines, dramatic lighting |
| Trip-Hop | Urban nightscapes, neon, noir aesthetic |
| Neo-Soul | Warm interiors, golden hour, intimate vibes |
| Cloud Rap | Surreal landscapes, clouds, soft purple haze |
| Indie Pop | Bright colors, urban streets, playful energy |
## Mobile Music Hub
The High IQ mobile app includes a native **Music Hub** for streaming all AI-generated strain songs. It is distinct from the Strain Music Video pipeline (which generates full video productions) — the Music Hub plays the audio tracks that each video is built on.
Key features of the mobile Music Hub:
* **Personalized "Your Music"** section — strain songs cross-referenced with your stash and favorites, shown when ≥2 matches exist
* **Filters** — search by name, strain type, High Family, or genre
* **Now Playing screen** — gesture-driven seek slider, shuffle, repeat, BPM/key metadata, and collapsible lyrics panel
* **Lock Screen / Control Center integration** — playback continues and is controllable system-wide
## Future Phases
Replace the static mascot image on strain detail pages with the autoplaying music video as the hero visual.
Automatic upload to a dedicated strain music video YouTube channel after each video is generated.
Auto-generate 9:16 vertical crop edits from chorus sections for Instagram Reels and TikTok distribution.
Add the full music video to the strain detail carousel in the High IQ mobile app using expo-video. (Audio playback is already live via the Music Hub.)
Fire video generation automatically as a fire-and-forget follow-on after the music pipeline completes.
Run LTX-2.3 on dedicated GPU infrastructure to reduce per-video cost and improve generation speed.
## Related Features
* [Strain Music](/help/features/strain-music-videos) — The AI song pipeline that generates the audio each video is built on
* [Strain Discovery](/help/features/strain-discovery/searching) — Browse and search the full strain database
* [Label Scanner](/help/features/label-scanner) — Scan a product label to identify a strain and access its music video