Visual evidence in creative intelligence reports
Competitor scans now point at the exact region of a screenshot that earns a pattern label — overlaid bounding boxes in both the full report and the chat card.
Show, don't summarize#
Until today the Creative Intelligence report told you which patterns competitors used and how often, but it never showed you where on the screenshot the pattern lived. You had to read the strategist's reasoning, open each thumbnail, and squint to find the coupon modal or the cashback stamp the report was citing. The visual proof was always one click away.
Today every creative intelligence report carries a typed visualEvidence array on the payload — one card per high-signal competitor screenshot, with normalized 0–1 bounding boxes anchoring each detected pattern to a specific region of the frame.
What changed#
Classifier returns bounding boxes#
The screenshot classifier (generateObject with a Zod schema, AI SDK v6) now asks the vision model to emit up to two boxes per detected pattern. Each box is a fraction of the image so the overlay scales perfectly without knowing the source resolution, and carries a short label ("Coupon modal", "Savings stamp", "Trust badge", "Price comparison chart"). Boxes that overflow the unit square or come back zero-sized are dropped at the boundary — the UI can never receive a misshapen rectangle.
When the model can label a pattern but can't visually anchor it (text-only captions, off-format compositions), the box list stays empty and the pattern still rolls up into the matrix as before. The fallback path that produced labels from metadata when vision failed continues to work; it just never produces boxes.
Visual evidence section in the full report#
The library detail view for a creative intelligence report has a new "Visual evidence" section sitting between the strategist thesis and the pattern matrix. It renders six to ten of the highest-signal annotated screenshots — competitor frames beat target frames at the same score because the report exists to surface what competitors do that we can borrow. Each card shows the overlay, the app + pattern label, and a one-sentence insight tying the screenshot back to the strategist's reasoning.
Cross-highlight is wired both ways. Clicking a pattern matrix row pulses the matching evidence cards (and dims the rest); clicking the "Show evidence" affordance on a recommendation does the same via inferred pattern match. There's also a "Clear filter" affordance so the workflow never feels stuck in a filtered view.
Visual takeaways in the chat card#
The Creative Intelligence inline card in chat now leads with a compact "Visual takeaways" carousel — the top three annotated thumbnails with hover-revealed labels — above the existing pattern pills. Reports without visualEvidence keep their original pill-only look, so older reports written before this shipped render exactly as they did yesterday.
Reusable annotated-screenshot component#
The overlay logic lives in a single AnnotatedScreenshot component used by both surfaces. It supports a full variant (labels always visible) for the report view and a compact variant (labels reveal on hover) for the chat carousel, plus a highlightedPatternId prop that lets the parent dim non-matching boxes during cross-highlight.
What this unlocks#
The pattern matrix used to ask the team to trust that the strategist read the screenshot correctly. Now the strategist has to point. Frames the model labels confidently get bright high-confidence outlines; ambiguous frames render in amber and tell you so. Disagreement with the model is now a one-click "this isn't where the savings stamp actually is" interaction instead of a multi-tab investigation.