> ## Documentation Index
> Fetch the complete documentation index at: https://docs.highailabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Research and Generation

> How orders, strain identity, research freshness, evidence, generation lifecycle, and immutable notebook publications fit together.

# Research and Generation

This is the canonical contract for moving from user-provided order data to a validated notebook. It separates five concerns that must never be collapsed into one boolean: recorded input, strain identity, research readiness, generation lifecycle, and publication state.

## End-to-end data flow

```mermaid theme={null}
flowchart TD
  A[Recorded order line] --> B{Identity resolution}
  B -->|Exact or approved alias| C[Canonical strain subject]
  B -->|Ambiguous| D[Preserve line; do not attach uncertain research]
  B -->|Unknown| E[Preserve line; create research-needed subject]
  B -->|Placeholder| F[Preserve line; not researchable without a real name]
  C --> G{Research readiness}
  E --> G
  G -->|Fresh| H[Freeze source snapshot]
  G -->|Stale or unresearched| I[Bounded research candidate]
  G -->|Refreshing| J[Wait or publish supported degraded content]
  G -->|No data or failed| K[Show explicit unavailable state]
  I --> H
  H --> L[Compose first-party Report V2]
  D --> L
  F --> L
  J --> L
  K --> L
  L --> M[Validate payload and evidence]
  M -->|Pass| N[Promote immutable publication]
  M -->|Fail| O[Keep current publication]
```

## 1. Preserve the recorded order

The order record is the user's source of truth. Save every line and retain stable line identity across manual text, image parsing, Gmail import, review edits, research callbacks, and notebook generation.

Required behavior:

* Missing price, dispensary, quantity detail, or strain type remains missing.
* Duplicate lines remain individually traceable.
* Mixed units are normalized only by an explicit deterministic rule.
* A valid order can contain zero matched strains.
* Queue acceptance must not change an order line to “researched” or “resolved.”
* A later exact research result may enrich the same line; it must not create a duplicate stash projection.

## 2. Resolve strain identity honestly

| Match state    | Behavior                                                      | Research attachment                           |
| -------------- | ------------------------------------------------------------- | --------------------------------------------- |
| Exact          | Attach canonical subject                                      | Allowed                                       |
| Approved alias | Record original name and attach canonical subject             | Allowed                                       |
| Ambiguous      | Preserve candidate context for review                         | Forbidden until resolved                      |
| Unknown        | Preserve the original name and create a research-needed state | No current research attached                  |
| Placeholder    | Preserve the menu label                                       | Not researchable until a real identity exists |
| Duplicate      | Preserve each order line; dedupe canonical research only      | Shared research, separate recorded facts      |

Do not use a weak “first search result” fallback. A wrong match is worse than an explicit unknown because it contaminates lessons, evidence, updates, and the long-term strain database.

## 3. Track research readiness and conflict separately

The readiness contract is:

| Readiness           | Meaning                                                                 |
| ------------------- | ----------------------------------------------------------------------- |
| `unresearched`      | No trusted attempt has completed                                        |
| `fresh`             | A successful immutable revision is within the approved freshness window |
| `stale`             | A successful revision exists but is eligible for refresh                |
| `refreshing`        | One fenced attempt currently owns the subject lease                     |
| `attempted-no-data` | A bounded attempt completed without publishable evidence                |
| `failed`            | The latest eligible attempt failed                                      |

Source conflict is separate metadata. A revision can be fresh and still contain a documented source disagreement.

<Warning>
  The new research-revision writes remain hard-off. Draft PR #1259 includes the schema, strict contracts, a loopback-only PostgreSQL 17 race lane, and lazy routing coordinators. It does not apply the hosted migration or wire live direct, Batch API, bulk, or scheduled engines.
</Warning>

### New strain behavior

When an order contains a strain that is not in the database:

1. Save the order line as unknown with its exact recorded name.
2. Create or reuse a stable research subject identity; do not bind by display-name search result.
3. Attempt a bounded claim only after the research engine is safely activated.
4. Record leased, cooldown, fresh-skip, no-data, failed, canceled, and success outcomes distinctly.
5. On success, atomically publish the immutable research revision and update the current research pointer.
6. Notify the exact waiting order item/run pair.
7. Create the stash projection exactly once if product policy allows it.
8. Never publish synthetic catalog stubs for a no-data result.

## 4. Freeze evidence for the notebook

Report V2 freezes a minimized canonical source snapshot for each attempt. The snapshot should contain:

* recorded order or selected-stash facts required by the notebook;
* stable order-item/source identifiers;
* the exact immutable research revision used for each subject;
* bounded canonical source URLs and retrieval checksums;
* explicit unavailable/conflict/quality metadata;
* no email body, account token, provider payload, address, phone number, or unrelated profile field.

Every generated claim must resolve to recorded evidence or research evidence in that snapshot. If it cannot, withhold it or mark the lesson unavailable.

## 5. Generate with recoverable lifecycle state

```mermaid theme={null}
stateDiagram-v2
  [*] --> Preparing
  Preparing --> Researching
  Researching --> Generating
  Generating --> Finalizing
  Finalizing --> Ready
  Preparing --> Canceling
  Researching --> Canceling
  Generating --> Canceling
  Finalizing --> Canceling
  Canceling --> Canceled
  Canceling --> Ready: Completion won race
  Researching --> Failed
  Generating --> Failed
  Finalizing --> Failed
  Failed --> Preparing: Retry new attempt
```

UI rules:

* **Minimize** or swipe-down closes the progress sheet but keeps the server task running.
* The tab-bar accessory can reopen active progress.
* Dismissal is scoped to one attempt and must not cancel it.
* Cancellation requires confirmation and shows requesting, pending, confirmed, failed, or too-late state.
* Missing numeric progress uses an indeterminate presentation; never invent a percentage or ETA.
* If a readable publication exists, keep it available throughout an update.

## 6. Publish immutable versions

A validated Report V2 promotion must atomically:

1. insert an immutable child publication;
2. bind it to the exact attempt, snapshot, artifact manifest, and payload checksum;
3. advance the notebook's current publication pointer;
4. update the denormalized current reader cache;
5. leave every prior publication unchanged and owner-readable.

Failed, canceled, abandoned, superseded, malformed, or ambiguous candidates never replace the current publication.

### Update journey

| State              | User experience                                                       |
| ------------------ | --------------------------------------------------------------------- |
| Current            | Read the validated current notebook                                   |
| Update available   | Compare reviewed source/evidence changes before acting                |
| Updating           | Keep reading current while the new candidate runs                     |
| Failed or canceled | Current remains unchanged; retry is explicit                          |
| Promoted           | New publication becomes current atomically                            |
| Reading history    | Open a validated safe projection; return to current at any time       |
| Invalid history    | Show unavailable; never label malformed content as a previous version |

Automatic freshness detection and the customer update request are still disabled. Studio models the entire journey so the contract can be certified before activation.

## Safe activation blockers

Before importing the lazy coordinators into live research engines, resolve and test:

* terminalize failures only after Trigger retry exhaustion;
* define behavior for `leased`, `cooldown`, `skipped-fresh`, and terminal duplicate dispositions without firing premature success callbacks;
* make the legacy “catalog row exists” shortcut revision-mode aware;
* preserve stable per-subject run identity across reordered and partial batches;
* settle or recover every claimed lease after partial begin failure;
* route direct, Batch API, bulk, and scheduled entry points through the same fenced contract;
* prove that no subject can publish another subject's evidence in a partial batch.

Then follow the exact order in [Notebook launch runbook](/planning/notebooks/launch-runbook).

## Verification map

| Change area                 | Minimum proof                                                                    |
| --------------------------- | -------------------------------------------------------------------------------- |
| Matching/order preservation | Mobile + Convex order-save and research-completion tests; Studio matching matrix |
| Report contract             | `@tiwih/report-contracts` tests, typecheck, and build                            |
| Engine composition          | `@tiwih/notebook-engine` tests, typecheck, and build                             |
| Research revision SQL       | Migration static audit plus approved PostgreSQL race/pgTAP lane                  |
| Trigger coordinator         | Disabled-construction tests, partial/reordered batch tests, full Trigger suite   |
| Publication/history         | Convex ownership, promotion, replay, export, deletion, and reader tests          |
| Mobile experience           | Unit/Convex/UI suites plus native progress/detail certification                  |
| Studio                      | Interaction/accessibility tests and static Storybook build                       |

## Related documentation

* [Notebook program overview](/planning/notebooks/overview)
* [Notebook Studio](/planning/notebooks/studio)
* [Notebook launch runbook](/planning/notebooks/launch-runbook)
* [Customer generation explanation](/help/features/notebooks/how-generation-works)