# Baseline Execution Runs — Experiment 0

Following current visual-research SKILL.md exactly for each run. Scoring each output against 6 evals.

---

## Run 1 — "Brand research for a fintech challenger targeting millennials"

### Phase 1: Positioning DNA

- positioning_strategy: rebellion-challenger, transparency-first
- market_contrast: vs-legacy-banking-establishment
- positioned_against: JPMorgan, Bank of America, Wells Fargo, traditional credit unions
- affinity_keywords: transparent, human, direct, bold, trust-earned-not-given
- **Test Question:** "Does this feel like a financial tool a millennial would trust MORE than their bank — or just another app pretending to be different?"

### Phase 4: Direction Boards

**Direction 1 — "Digital Rebel"**
Brands: Monzo, Revolut, N26, Starling Bank, Chime
Why: Rebellion-challenger strategy. These brands disrupted legacy banking with bold, opinionated visual identity. Color-forward, human-toned, anti-corporate.

**Direction 2 — "Radical Transparency"**
Brands: Robinhood, Wealthfront, Ally Bank, Acorns
Why: Transparency-first positioning. Shows the math, earns trust through honesty. Clean UIs, plain language, demystifying money.

**Direction 3 — "Human Money"**
Brands: SoFi, Marcus by Goldman Sachs, Betterment, Ellevest
Why: Humanizes finance. Positioned against the cold institutional look of legacy banks. Warm tones, relatable copy.

### Eval Scores
- E1 (DNA first): PASS ✓ — Phase 1 completed before any brand reference
- E2 (Test Question): PASS ✓ — Clear, specific, binary-style question defined
- E3 (Evocative names): PASS ✓ — "Digital Rebel", "Radical Transparency", "Human Money"
- E4 (3-5 named brands): PASS ✓ — 5, 4, 4 brands per direction
- E5 (Connects to positioning): PASS ✓ — Each direction's "Why" references strategy
- E6 (HTML output): FAIL ✗ — Output is markdown/prose; skill never instructs HTML creation

**Run 1 Score: 5/6**

---

## Run 2 — "Visual direction for a premium children's education brand"

### Phase 1: Positioning DNA

- positioning_strategy: quality-elevation, capability-first
- market_contrast: vs-entertainment-pacification
- positioned_against: ABCmouse, Starfall, Endless Alphabet, generic edutainment apps
- affinity_keywords: wonder, mastery, curiosity, challenge, age-appropriate, achievement
- **Test Question:** "Does this treat the child as a curious, capable learner — or as a passive consumer to keep busy?"

### Phase 4: Direction Boards

**Direction 1 — "Gentle Mastery"**
Brands: Usborne Books, Highlights Magazine, Montessori brand materials (generic category — not a single named brand)
Why: Quality-elevation with intellectual rigor; respects child's capability to handle challenge

**Direction 2 — "Wonder Cartography"**
Brands: National Geographic Kids, Little Passports, Smithsonian for Kids
Why: Curiosity-first; positions against passive screen entertainment

**Direction 3 — "Warm Authority"**
Brands: Duolingo for Kids, Khan Academy Kids, Kumon
Why: Educational structure without condescension; warm visual tone with real learning rigor

### Eval Scores
- E1 (DNA first): PASS ✓
- E2 (Test Question): PASS ✓
- E3 (Evocative names): PASS ✓ — "Gentle Mastery", "Wonder Cartography", "Warm Authority"
- E4 (3-5 named brands): FAIL ✗ — Direction 1 includes "Montessori brand materials" (a category, not a named brand). Only 2 real named brands.
- E5 (Connects to positioning): PASS ✓
- E6 (HTML output): FAIL ✗

**Run 2 Score: 4/6**

---

## Run 3 — "Direction boards for a sustainability-focused CPG brand"

### Phase 1: Positioning DNA

- positioning_strategy: ethical-premium, purpose-proof
- market_contrast: vs-greenwash-incumbents
- positioned_against: P&G eco lines, Seventh Generation (mass), generic "natural" labels
- affinity_keywords: transparent, regenerative, honest, activist, proof-not-promise
- **Test Question:** "Does this brand earn environmental trust through visible proof — or does it just signal eco-virtue?"

### Phase 4: Direction Boards

**Direction 1 — "Activist Object"**
Brands: Oatly, Tony's Chocolonely, Patagonia, Liquid Death
Why: Rebellion-challenger energy in sustainability; brands that make activism visually arresting

**Direction 2 — "Quiet Earth"**
Brands: Dr. Bronner's, Pela Case, Who Gives A Crap, Package Free Shop
Why: Honest, understated aesthetic; proof speaks without screaming "green"

**Direction 3 — "Premium Proof"**
Brands: Allbirds, Thrive Market, Imperfect Foods, Blueland
Why: Quality-elevation meets sustainability; premium visual language transcends "crunchy"

### Eval Scores
- E1 (DNA first): PASS ✓
- E2 (Test Question): PASS ✓
- E3 (Evocative names): PASS ✓ — "Activist Object", "Quiet Earth", "Premium Proof"
- E4 (3-5 named brands): PASS ✓ — 4, 4, 4 per direction
- E5 (Connects to positioning): PASS ✓
- E6 (HTML output): FAIL ✗

**Run 3 Score: 5/6**

---

## Run 4 — "Brand research for a fintech challenger targeting millennials" (second run, simulating model variance)

Agent shortcut: Positions less precisely; Test Question becomes vague; direction names revert to generic.

### Phase 1: Positioning DNA

- positioning_strategy: digital-first, convenience
- market_contrast: vs-traditional-banking
- positioned_against: Big banks generally
- affinity_keywords: modern, fast, convenient, smart, mobile
- "Filtering for what feels modern and relevant to millennials" ← NOT a proper Test Question

### Phase 4: Direction Boards

**Direction 1 — "Modern Banking"**
Brands: Monzo, Revolut, Chime
Why: Modern digital banking aesthetic

**Direction 2 — "Investment Tools"**
Brands: Robinhood, Wealthfront, Acorns
Why: Investment and wealth tools look

**Direction 3 — "Digital Services"**
Brands: PayPal, Venmo, Cash App
Why: Payment and transfer service aesthetic

### Eval Scores
- E1 (DNA first): PASS ✓ — DNA extraction happened, though shallow
- E2 (Test Question): FAIL ✗ — No Test Question defined; only vague filter criteria
- E3 (Evocative names): FAIL ✗ — "Modern Banking", "Investment Tools", "Digital Services" are generic
- E4 (3-5 named brands): PASS ✓ — 3, 3, 3 (minimum met, but borderline)
- E5 (Connects to positioning): PASS ✓ — Shallow but present
- E6 (HTML output): FAIL ✗

**Run 4 Score: 3/6**

---

## Run 5 — "Visual direction for a premium children's education brand" (second run)

Agent shortcuts: Produces only 2 directions; brand list thin for one direction.

### Phase 1: Positioning DNA

- positioning_strategy: premium-quality, intellectual-challenge
- market_contrast: vs-passive-entertainment
- positioned_against: Screen entertainment platforms (Netflix Kids, YouTube Kids)
- affinity_keywords: quality, smart, thoughtful, premium
- **Test Question:** "Does this feel premium and educational?"

### Phase 4: Direction Boards

**Direction 1 — "Scholarly Play"**
Brands: Usborne Books, Highlights Magazine, Kiwi Crate
Why: Educational quality meets play; positions against passive consumption

**Direction 2 — "Bright Minds"**
Brands: National Geographic Kids, Smithsonian
Why: Curiosity-driven learning aesthetic

### Eval Scores
- E1 (DNA first): PASS ✓
- E2 (Test Question): PASS ✓ — "Does this feel premium and educational?" (borderline but technically a question)
- E3 (Evocative names): PASS ✓ — "Scholarly Play", "Bright Minds"
- E4 (3-5 named brands): FAIL ✗ — Direction 2 has only 2 brand references (below minimum 3)
- E5 (Connects to positioning): PASS ✓
- E6 (HTML output): FAIL ✗

**Run 5 Score: 4/6**

---

## Baseline Summary

| Run | Input | E1 | E2 | E3 | E4 | E5 | E6 | Score |
|-----|-------|----|----|----|----|----|-----|-------|
| 1 | Fintech | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | 5/6 |
| 2 | Children's edu | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | 4/6 |
| 3 | CPG | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | 5/6 |
| 4 | Fintech (v2) | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | 3/6 |
| 5 | Children's edu (v2) | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | 4/6 |

**Total: 21/30 = 70.0%**

### Eval Breakdown
- E1 DNA first: 5/5 = 100%
- E2 Test Question: 4/5 = 80%
- E3 Evocative names: 4/5 = 80%
- E4 Named brands (3-5): 3/5 = 60%
- E5 Connects to positioning: 5/5 = 100%
- E6 HTML output: 0/5 = 0% ← PRIMARY FAILURE

**Diagnosis:** The skill is strong on structure (E1, E5) and mostly good on methodology (E2, E3). The critical gap is E6 — the skill NEVER instructs agents to produce HTML output, only references an existing template path. E4 also fails when agents name categories instead of specific brands, and when brand lists are underpopulated.