Meal-Planner/docs/HANDOFF.md

# MealPlanner — Agent Handoff

You are taking over a project in mid-flight. Read `docs/ORIENTATION.md` first for the high-level. This file is the deep dive: what's real, what's stubbed, where the bodies are buried, and what to do next.

**Date of handoff: 2026-06-04. Last commits before handoff:**
```
8ad4ef6 feat(ui): bulk pantry add + plan-the-week button (Sprint 6 F3+F4)
f740f40 feat(ui): global keyboard shortcuts + shortcut help banner (Sprint 5 F2)
d78bd18 feat(ui): URL week selector + aisle-migration 0015 cast fix (Sprint 5 F5)
d71b67a feat(ui): global react-query error handler + plan-status a11y (Sprint 4 F7+F6)
427d8ac docs(review): add handoff document for UI audit work
e90a9d6 feat(ui): close 3 P2 audit findings + a11y sweep (Sprint 3)
f5fb755 fix(migration): simplify aisle migration + add persistent backup script
ccc70aa feat(ui): close 6 P1 audit findings + 1 bonus mobile fix (Sprint 2)
36038bb docs(review): mark Sprint 1 P0 fixes addressed in commit f3e4a44
f3e4a44 fix(ui): close 5 P0 audit findings (ingredients, cost, routing, mobile slots)
b522760 fix: cast qty/unit to str before html.escape in vote email shopping preview
```

**Focused UI/UX audit handoff** (Sprints 1, 2, 3 — 14 findings closed across 3 commits):
see `Review/handoff-ui-audit.md`. That doc is the right starting point for
anyone continuing the UI/UX work; the present file remains the project-wide
overview (backend, infra, family data, admin API, prior phases).

---

## TL;DR

The system is **fully operational end-to-end** on the Woolery family's home network. Vote emails now show recipe images, descriptions, ingredient lists, cooking steps, estimated costs, and a shopping list preview. Peter confirmed the email looks polished; Julia's feedback pending.

**Match accuracy:** 10,140 AUTO + 3 AUTO_LLM matches. 22 ingredients remain unmatched (genuine Lucky CA catalog gaps: olive oil, dried spices, chickpeas, etc.).

**Next tasks:**
1. **Spoonacular enrichment (5 remaining)** — run `scripts/enrich_recipes_spoonacular.py` again; 5 recipes still need images (daily quota was hit on 2026-05-11)
2. **Phase 8 Feedback UI** — `feedback` table exists; no UI reads/writes it yet

---

## Infrastructure — READ THIS FIRST

### Access
- App: `http://100.108.224.12:8081` (WireGuard `wt0` interface)
- Ports 80 and 443 are owned by `lifemanager-caddy-1` on this host — do NOT use them
- Always use `docker compose --env-file .env.test` (never bare `docker compose`)

### Stack up
```bash
cd /home/peter/Projects/MealPlanner
docker compose --env-file .env.test up -d
```

### Applying Python code changes
`docker cp` alone is NOT enough — the running process caches modules. Always:
```bash
docker cp backend/app/path/to/file.py mealplanner-backend-1:/app/app/path/to/file.py
docker compose --env-file .env.test restart backend
```

### DB connection
```bash
docker compose --env-file .env.test exec -T db psql -U mealplanner -d mealplanner
```
DB user is `mealplanner` (not `postgres` — that role does not exist).

### Admin API auth
```
Authorization: Bearer test-admin-token
```
(NOT `X-Admin-Token` — it's a standard Bearer header. See `backend/app/security.py`.)

### Key env vars (`.env.test`)
```
EMAIL_BACKEND=sendgrid
SENDGRID_API_KEY=<real key>
APP_BASE_URL=http://100.108.224.12:8081
SESSION_PASSWORD=test-family-password
ADMIN_TOKEN=test-admin-token
```

---

## Family data (live, seeded)

- Family: **Woolery**, 4 members
- Adults: **Peter** (peter@research.bike) + **Julia** (julia@research.bike)
- 2 kids without email addresses (voting not required from them)
- Family profile + members are in the DB. Seed script: `scripts/seed_family.py` (safe to re-inspect; will exit early if profile already exists)

---

## What changed in this session

### MVP login + auth gating (committed in `feature/mvp-login`, merged to master)
- `frontend/src/pages/Login.tsx` — password form, calls `auth.login()`, redirects to `/`
- `frontend/src/App.tsx` — added `/login` route, Sign out button in nav
- `frontend/src/api/index.ts` — 401 interceptor redirects unauthenticated users to `/login`
- The frontend is **built and deployed** inside the Docker frontend container

### nginx (committed in `59a15a2`)
- Rewritten with Docker DNS resolver (`127.0.0.11 valid=10s`) to prevent IP caching after container restarts
- Port mapped to `8081:80` (ports 80/443 conflict with `lifemanager-caddy-1`)
- Proxy pattern: `set $var` forces per-request DNS resolution — without this, a backend restart causes 502s until nginx restarts too

### Vote email enrichment (`aeed2a4`)
Each recipe card in the Friday proposal email now shows:
- Recipe name
- Ingredient list (resolved from `Ingredient` table via UUID lookup — the JSONB stores `ingredient_id`, not `name`)
- Collapsible `<details>` block with numbered cooking steps (`recipe.instructions` ARRAY)
- Estimated cost (sum of top-confidence grocery matches)
- Vote button

### Shopping list email improvements (`aeed2a4`)
- Ingredients grouped under each recipe heading (was flat deduplicated list)
- Fixed field name: `match.grocery_item.current_price` (was `.price` — column doesn't exist)

### Ingredient matcher — full rewrite (`ac2b575`, `d7a3f5c`, `2373883`)

**Root cause of old failures:** the old matcher iterated grocery items and matched them against ingredient names using `fuzz.WRatio`. Long branded product names containing an ingredient word incidentally scored very high — "Pampers Baby Fresh Scent Wipes" → "Ginger, **Fresh**".

**New algorithm in `backend/app/services/matcher.py`:**

```
For each ingredient:
  1. Exact-name fast path: lowercase-trimmed dict lookup against all grocery names
     → confidence 1.000, skip fuzzy entirely (handles "Lime" → "Lime")
  2. Fuzzy: partial_token_sort_ratio against all grocery names (limit=100)
  3. For each candidate above threshold (0.82):
     a. 100% recall: all ingredient sig-words must appear in grocery sig-words
     b. Category exclusion: grocery must not have disqualifying words absent from ingredient
        (bread, chips, pasta, margarita, butter, soda, juice, tuna, rotisserie, etc.)
     c. Precision floor (0.45): ingredient sig-words / grocery sig-words ≥ 0.45
     d. Combined score = partial_score × precision
  4. Store best combined score via ON CONFLICT DO NOTHING (preserves manual overrides)
```

**Stop words** (stripped from sig-word sets): fresh, organic, whole, large, small, medium, low, free, light, dark, raw, dried, frozen, canned, extra, virgin, pure, natural, classic, style, boneless, skinless, lean, grain, long, jarred, roasted, smoked, cooked, and, with, for, the.

**Benchmark on Lucky CA weekly ad + full produce catalog (11,044 items):**
- Before: ~25% correct (Pampers→Ginger, Red Wine→Bell Pepper, Garlic Bread→Garlic)
- After: ~90%+ correct

**Current match quality for recipe ingredients:**
```
Garlic                → Fresh Garlic ($4.99)           ✓ confidence 1.0
Lime                  → Lime (no price — sold by each) ✓ confidence 1.0
Cilantro              → Cilantro, Fresh ($1.99)         ✓ confidence 1.0
Bell Pepper, Red      → Organic Red Bell Pepper ($2.49) ✓ confidence 1.0
Ground Beef, 85/15    → 85% Lean Ground Beef ($5.99)   ✓ confidence 1.0
Cheddar Cheese, Sharp → Sharp Cheddar Cheese ($10.99)  ✓ confidence 1.0
Ginger, Fresh         → Ginger Root ($3.99)             ✓ confidence 0.5
Ground Turkey         → Butterball Ground Turkey ($6.99) ✓ confidence 0.67
Soy Sauce             → Kikkoman Soy Sauce ($3.99)      ✓ confidence 0.67
Salt, Kosher          → Coarse Kosher Salt ($2.99)      ✓ confidence 0.67
Olive Oil             → — (Lucky CA has none in catalog)
Tortilla, Corn        → — (not in catalog this week)
```

### Scraper fix — save priceless produce (`2373883`)
`backend/app/scraper/lucky_ca_scraper.py` `map_product()` previously returned `None` for items with no price, skipping them. Fresh produce (garlic, limes) is sold by the each with no catalog price. Removed the price guard — items with `current_price=NULL` are now saved and matched.

---

## What is real (verified)

Everything in the prior HANDOFF (Phase 4 thin slice, Phase 5 orchestration, Phase 6 SendGrid, Phase 9 generation) is still real. Key additions:

### Lucky CA API (Swiftly) — full catalog accessible
The Swiftly API is the same for ALL product categories, not just the weekly ad:
- **Taxonomy**: `GET https://luckysupermarkets.com/categories?_data=root` — works without user cookies, returns JSON with `taxonomies` list of 17 top-level category slugs
- **Products per category**: `GET https://prod.swiftlyapi.net/search/api/v1/products/categories?cat=Product%2F{slug}&limit=10000&store=757` with `Authorization: Bearer <swiftly_jwt>`
- **JWT**: auto-minted via `backend/app/services/swiftly_auth.py` — no manual token needed
- **17 categories**: produce (660 items), meat_seafood (265), pantry (1000), dairy_eggs_cheese (1000), frozen_foods, beverage, snacks, bread_bakery, deli_counter, etc.
- Current scraper scrapes all 17 categories; produce items now saved even without price

### Matcher runs automatically
`backend/app/services/scraper_service.py` calls `run_match_job(db, source_filter="lucky_california")` after every successful scrape. If you change matcher code, restart the backend before re-scraping so the new code is loaded.

---

## What is stubbed or missing

### Recipe images (Phase 10 — approved, not started)
- `recipe.image_url` is NULL for all 107 recipes → no photos in vote emails
- **Approved plan:** Spoonacular API enrichment script (107 recipes × 1 call = fits 150/day free quota)
- API returns image URL + description + improved instructions

### Recipe descriptions
- `recipe.description` is NULL for all recipes → no blurb in vote emails
- Spoonacular enrichment solves this alongside images

### Olive Oil + Corn Tortillas (Lucky catalog gap)
- Lucky CA's Swiftly catalog has no standalone olive oil or plain corn tortillas
- These show "—" in shopping list — correct behavior (better than wrong match)
- **Approved plan:** Ollama LLM matcher as a second pass using Lucky's product search API (`luckysupermarkets.com/search/products?q=<ingredient>`) to find items outside the Swiftly weekly ad

### Phase 8 — Feedback UI (done)
- `feedback` table now read/written via REST API
- Meal detail page shows star rating, never-suggest, reason dropdown, free-text comments

---

## Known caveats and traps

1. **Module caching.** `docker cp` without restart leaves old Python code running. Always restart backend after copying files.

2. **Bootstrap login hatch.** When no `family_profile` row exists, `auth.py` signs the literal string `"bootstrap"`. Woolery family is seeded so this is dormant. If DB is wiped, re-run `scripts/seed_family.py`.

3. **DB user is `mealplanner`.** `psql -U postgres` fails. Always use `psql -U mealplanner -d mealplanner`.

4. **Matcher ON CONFLICT DO NOTHING.** Manual matches (`source='manual'`) are never overwritten. If you set a manual match and want the auto-matcher to take over, delete the manual row first.

5. **`weekly_run` idempotency.** Each step sets its timestamp column on completion; re-firing is a no-op. To re-trigger a step, set its timestamp to NULL:
   ```sql
   UPDATE weekly_run SET finalized_at = NULL, status = 'running';
   ```

6. **Scraper `items_scraped` count appears stuck at 0 during run.** The count is only written on completion (26–60s). The status field stays `started` until then.

7. **`limit=10000` in Swiftly API.** Pantry and dairy categories return exactly 1000 items each — suspected server-side cap below our limit. Either multiple pages exist (no offset param observed) or those are genuine catalog sizes. Produce (660) and meat_seafood (265) look complete.

8. All prior caveats in the 2026-05-08 HANDOFF still apply (SQLEnum, transactional fixtures, `alembic downgrade base`, etc.).

---

## Admin API reference

```bash
# Trigger individual steps
curl -s -X POST http://localhost:8081/api/admin/orchestrate/{step} \
  -H 'Authorization: Bearer test-admin-token'
# Valid steps: scrape, generate, email, reminder, deadline, finalize

# Full week cycle (background)
curl -s -X POST http://localhost:8081/api/admin/orchestrate/run-week \
  -H 'Authorization: Bearer test-admin-token'

# Scrape status
curl -s http://localhost:8081/api/admin/logs/{scrape_log_id} \
  -H 'Authorization: Bearer test-admin-token'

# Weekly run status
curl -s http://localhost:8081/api/admin/orchestrate/status \
  -H 'Authorization: Bearer test-admin-token'

# Trigger fresh scrape + auto-match
curl -s -X POST http://localhost:8081/api/admin/scrape \
  -H 'Authorization: Bearer test-admin-token'
```

---

## Suggested next moves

### 1. Spoonacular recipe enrichment (images + descriptions)

Free tier: 150 req/day. 107 recipes = one run, one commit.

Plan:
- Write `scripts/enrich_recipes_spoonacular.py`
- For each recipe: `GET https://api.spoonacular.com/recipes/search?query={name}&apiKey=…` → pick best match → fetch details → update `recipe.image_url`, `recipe.description`
- `SPOONACULAR_API_KEY` needs to be added to `.env.test`
- Run once: `docker cp scripts/enrich_recipes_spoonacular.py mealplanner-backend-1:/app/ && docker compose --env-file .env.test exec backend python /app/enrich_recipes_spoonacular.py`

### 2. Ollama LLM matcher (Olive Oil, Corn Tortillas, etc.)

Approved architecture:
```
For each ingredient with no match OR confidence < 0.5:
  1. Query Lucky product search: GET https://luckysupermarkets.com/search/products?q={ingredient}
     (reverse-engineer the JSON API from that page)
  2. Extract top 5-10 results
  3. POST to Ollama: "I need {ingredient} for a recipe. Which is the best match?
     Options: [list]. Answer with just the product name or 'none'."
  4. Store result as source='auto_llm' in ingredient_grocery_match
```

Peter uses **Ollama Cloud** for LLM inference. Confirm the API endpoint + model to use.
A small model (llama3.2:3b or mistral:7b) handles "pick the right produce item" accurately.

### 3. Natural Friday cycle

Next Friday at 02:00 PT the scheduler runs automatically. No action needed. All fixes in this session are committed and the new matcher + scraper will run.

---

## File map (additions from this session)

```
backend/app/api/feedback.py                — new: GET/POST feedback endpoints
frontend/src/pages/MealDetail.tsx          — added Feedback section (rating, never-suggest, reasons)
frontend/src/api/index.ts                  — added feedback API methods
frontend/src/types/index.ts                — added Feedback interface
backend/app/schemas/__init__.py           — RecipeIngredient model_validator qty→quantity
```

---

## Final words

Trust the tests. Trust the live runs. Don't trust prose claims that something is "complete" without running the verification gate yourself.

**Current open proposals:**
- `docs/proposals/2026-05-23-feedback-driven-recipe-discovery.md` — pending user approval. No code yet (per the 2026-05-23 section below).

**Last updated: 2026-06-05** — UI/UX audit & fix cycle (Sprints 1, 2, 3, 4, 5, 6, 7, 8) complete. 20 findings closed (5 P0 + 6 P1 + 3 P2 + 6 §Future), code committed across 12 commits, build green. Sprint 1 deployed; Sprints 2-8 awaiting deploy. **Sprint 7 (`09c7525`, awaiting user deploy)** aligns "this week" to the upcoming Monday. **Sprint 8 (`efd1fc6`, awaiting user deploy)** implements the user's "Deny" semantics decision. **Sprint 9 (committed 2026-06-05, awaiting user deploy)** ships the F1 Onboarding Tour. See Sprint 7 + Sprint 8 + Sprint 9 sections below. Full UI-audit handoff at `Review/handoff-ui-audit.md`.

---

## New session: 2026-06-05 (continued)

### Sprint 9 — F1 Onboarding Tour (H10) — COMMITTED 2026-06-05

**User direction (2026-06-05):** "Proceed with the next phase in the redesign. Also add a phase to include a 'Deny Forever' button in the Recipes endpoint."

**Decision (this session):** F1 (Onboarding Tour) was selected as the next phase (the only §Future item with a clear UI scope; F8 Spoonacular + F9 Ollama are full backend proposals; the dead `Generate Meal Plan` CTA is a separate follow-up). The "Deny Forever" on Recipes was drafted as Sprint 10 and awaits explicit "proceed".

**What ships:**
- `frontend/src/components/OnboardingTour.tsx` (NEW, ~420 lines). Hand-rolled (no `react-joyride`) to keep the npm footprint flat.
- 4 steps: Dashboard / Pantry / Recipes / Shopping List. Each anchors to a `[data-tour="<id>"]` attribute on an existing element.
- `localStorage.getItem('mealplanner:onboarding-complete') === '1'` is the source of truth. Writes wrapped in try/catch.
- `?reset-tour=1` in any URL clears the key + strips the param via `navigate(..., { replace: true })` so a refresh doesn't re-trigger the reset.
- Keyboard: `1`–`4` jump to step, `←/→` step back/forward, `Esc` dismiss, `Tab` order is `Skip → Back → Next`.
- A11y: `role="dialog"`, `aria-modal="true"`, `aria-labelledby` → step title. Focus captured on open (primary action) and restored on close. Decorative scrim + anchor ring are `aria-hidden="true"`.
- Tooltip is a real `position: fixed` `<div>` (no portal). rAF loop reads anchor `getBoundingClientRect` while the tour is open; cancellable on close.
- Off-route fallback: if the user is on a different page than the current step's anchor, the tooltip renders as a centered card with an "Open <page>" CTA.

**Files modified:**
- NEW: `frontend/src/components/OnboardingTour.tsx`
- `frontend/src/App.tsx` (mount + flag at the App root)
- `frontend/src/pages/Dashboard.tsx:602` — `<Card data-tour="dashboard">`
- `frontend/src/pages/Pantry.tsx:185, 208` — header + add-form anchors
- `frontend/src/pages/Recipes.tsx:124` — Filters button anchor
- `frontend/src/pages/ShoppingList.tsx:231` — header anchor

**Build:** `npm run build` green (tsc 0 errors, vite 0 errors). One commit: `feat(ui): Sprint 9 — F1 onboarding tour (4-step welcome)`.

**Deploy:** `git pull` + `docker compose up -d --build frontend` (frontend-only, no migration, no backend rebuild). Verification: `Review/sprint9-verification.md` (8-step browser smoke + a11y check + reset-link test).

**No regression expected:** Sprint 9 does not touch Sprints 1-8. The anchor `data-tour` attributes are additive; the page components still render the same. The KeyboardShortcuts hook (Sprint 5) is mounted in `App.tsx` and unaffected. The react-query error handler (Sprint 4) is unaffected.

---

## New session: 2026-06-05 (early)

### Sprint 7 — Fix webui "empty meal plan" (date-semantics mismatch)

(Full section above.)

---

## New session: 2026-06-05 (continued)

### Sprint 7 — Fix webui "empty meal plan" (date-semantics mismatch) — COMMITTED `09c7525`

**User report (2026-06-05, 06:17 PT):** "Latest meal plans were emails to me this morning, but when I go to the webui, the Meal Planner page is empty."

**Root cause:** the orchestrator planned the **upcoming** Mon-Sun week (Fri 2026-06-05 → key 2026-06-08) but the frontend's `isoMonday()` returned the **current** Mon-Sun (Fri 2026-06-05 → 2026-06-01). 7-day mismatch on Fridays.

**Fix (Option C, proper cleanup):**

- `backend/app/services/orchestrator/runner.py:20-35` — `_current_week_start()` returns the **upcoming Monday** (today if Mon). Email subject (`f"Meal plan for week of {run.week_start_date}"` at `steps.py:305`) automatically picks up the new value.
- `frontend/src/lib/utils.ts:43-130` — `isoMonday` → `upcomingMonday` (deprecated alias kept). New `formatWeekRange(mondayIso)`. UTC-stable `formatIsoDate` (fixed a TZ bug where `toLocaleDateString` rendered the previous day for users in negative-UTC timezones).
- `frontend/src/components/WeekRangeNav.tsx` (NEW) — `[<]  Jun 8 — Jun 14  [>]` with clickable chevrons + clickable range label (jumps to upcoming week) + `This week` chip when off the upcoming week. Replaces the Sprint 5 inline segmented control on both pages.
- `backend/scripts/fix_2026_06_05_to_2026_06_08.sql` (NEW) — guarded `UPDATE meal_plan SET week_start_date='2026-06-08' WHERE week_start_date='2026-06-05';` (idempotent, transaction-wrapped). Optional commented block for 2026-05-29.

**Commit:** `09c7525`. Files: 13 changed, 679+/96-, 3 new. `npm run build` green.

**Sprint 7 deploy (user runs):**
```bash
cd ~/MealPlanner && git pull
docker compose exec -T db psql -U mealplanner -d mealplanner \
  -f /dev/stdin < backend/scripts/fix_2026_06_05_to_2026_06_08.sql
docker compose -f docker-compose.yml up -d --build backend frontend
```

**Verification log:** `Review/sprint7-verification.md` (12-step browser smoke + API curls + rollback procedure).

---

### Sprint 8 — "Deny" semantics (C + Z, hard-filter escalation) — IN PROGRESS

**User report (2026-06-05, follow-up):** "one of the meals was the meal that I rejected last week. After you fix the above, lets discuss what rejeccting means."

**Investigation:** the planner has no cross-week memory of denials. Denials live on the `meal_plan_item` row, are never consulted by the planner, and the `NeverSuggest` blocklist is empty for the user's family. The user's "Roasted Sweet Potato and Chickpea Bowl" was denied on 2026-05-15 but the recipe was still in the pool for the next 90+ days.

**User policy decision (2026-06-05, exact words):** "Hard filter. If it is denied this week twice, it should be considered denied for good."

**Policy (Sprint 8):**

| Action | Backend behavior | Decay |
|---|---|---|
| Approve | `item.approval_status = approved` | n/a |
| Deny this week (1st in 90d) | `denied` + `denial_expires_at = now() + 90d` | after 90d, eligible again |
| Deny this week (2nd in 90d) — **server-side auto-escalation** | `denied` + `denial_expires_at = NULL` + `NeverSuggest` row written | permanent |
| Never again (explicit) | same as 2nd-time auto-escalation | permanent |

**Scope (12 boxes):** see `.agent/plan.md` "Active sprint" section. Code changes are M-L.

**Key files (Sprint 8):**

- `backend/alembic/versions/0016_denial_decay_and_scope.py` (NEW) — adds `meal_plan_item.denial_expires_at` + `meal_plan_vote.denial_scope`. Partial index on `denial_expires_at` for fast lookup.
- `backend/app/api/meals.py:30-138` — 3 new helpers: `_apply_denial`, `_ensure_never_suggest_recipe`, `_has_prior_active_soft_denial`. `DENIAL_DECAY_DAYS = 90`.
- `backend/app/api/meals.py:510-552` — `deny_meal_item` accepts `?scope=this_week|never_again` (default `this_week`); returns `promoted_to_permanent`.
- `backend/app/api/meals.py:380-455` — `submit_vote` handles `vote: "approve" | "deny" | "never_again"`; returns `denial_scope` + `promoted_to_permanent`.
- `backend/app/api/meals.py:240-330` — `get_vote_page` HTML page renders 3 buttons; supports one-click `?scope=...` direct-vote for email.
- `backend/app/services/orchestrator/steps.py:283-300` — email template renders 3 direct-action links per recipe.
- `backend/app/services/planner/generate.py:59-99, 150-194` — `_load_blocklists` returns 3 sets; `soft_denied_recipes` is hard-filtered (per user decision).
- `frontend/src/api/index.ts:48-58` — `meals.denyItem(itemId, { scope })`.
- `frontend/src/pages/Dashboard.tsx:38-50, 385-410` — `MealCard` renders 3 buttons (Approve / Deny this week / Never again) for pending items. "Never again" is gated by `window.confirm`.

**Static checks (offline):** all imports + types + helper logic verified via Python AST + import-test against the venv. The 1 pre-existing test failure in `test_planner_filter.py::test_filter_blocks_by_cost` is **not** introduced by Sprint 8 (verified by `git stash` + re-run on a clean tree).

**Sprint 8 deploy (user runs):**
```bash
cd ~/MealPlanner && git pull
docker compose exec backend alembic upgrade head
docker compose -f docker-compose.yml up -d --build backend frontend
```

**Verification log:** `Review/sprint8-verification.md` (11-step browser smoke + API curls + email render + rollback).

---

## New session: 2026-06-05 (early)

### Sprint 7 — Fix webui "empty meal plan" (date-semantics mismatch)

(Full section above.)

---

## New session: 2026-06-03

**User report (2026-06-05, 06:17 PT):** "Latest meal plans were emails to me this morning, but when I go to the webui, the Meal Planner page is empty."

**Root cause (one-liner):** The orchestrator plans the **upcoming** Mon-Sun week (Fri 2026-06-05 → key 2026-06-08), but the frontend's `isoMonday()` returns the **current** Mon-Sun (Fri 2026-06-05 → 2026-06-01). Email subject, DB plan key, and the webui default URL are 7 days out of sync. The user opens the app, lands on the current Mon-Sun week which has no plan, and sees the "No plan yet" empty state.

**Specific evidence:**
- `_current_week_start()` in `backend/app/services/orchestrator/runner.py:20-24` returns the most recent Friday; on Fri 2026-06-05 it returns 2026-06-05. (Original code, untested in production under the new Sprint 5 frontend.)
- `isoMonday()` in `frontend/src/lib/utils.ts:44-50` returns the most recent Monday; on Fri 2026-06-05 it returns 2026-06-01.
- DB: the 2026-06-05 plan (`8be25c81-da0b-4944-8c06-919b0d616515`) has 3 pending items (Chicken Fajitas, Garlic Shrimp Scampi, Breakfast-for-Dinner Veggie Scramble). It is **not visible** in the webui default view.
- DB: there is **no** plan with `week_start_date=2026-06-01` (current Mon-Sun).
- Email was sent by `step_email` Friday 06:00 PT for `week_start_date=2026-06-05` (subject: "Meal plan for week of 2026-06-05"). After S7, the subject becomes "Meal plan for week of 2026-06-08" (the upcoming Monday).

**Fix scope (7 checkboxes — see `.agent/plan.md` for the full task list):**

1. **Backend `runner._current_week_start()`** — return the upcoming Monday (today if Mon, else next Mon). One-line body change.
2. **Frontend `isoMonday` → `upcomingMonday`** — same logic; rename for intent clarity. Add `formatWeekRange(mondayIso)` helper for the new nav.
3. **New `WeekRangeNav` component** (`frontend/src/components/WeekRangeNav.tsx`) — renders the user-requested `[<]  Jun 8 — Jun 14  [>]` pattern. Clickable chevrons; clickable range label (jumps home); `This week` chip when off the upcoming week. Replaces the Sprint 5 inline segmented control on both Dashboard and ShoppingList.
4. **SQL fix** (`backend/scripts/fix_2026_06_05_to_2026_06_08.sql`) — guarded `UPDATE meal_plan SET week_start_date='2026-06-08' WHERE week_start_date='2026-06-05';` so the user's just-voted-on plan moves to the new key. Optionally also migrates 2026-05-29 (operator opt-in via uncomment).
5. **Verification** — `npm run build` green; `Review/sprint7-verification.md` written with deploy + smoke checks.
6. **Docs** — Sprint 7 status blocks in `Review/ui-nielsen-audit.md`, `fix-ui-audit.md`, `Review/handoff-ui-audit.md` (this file), `docs/HANDOFF.md` (this section). Sprint 7 verification doc created.
7. **No new dependencies, no backend migration.** Frontend + backend rebuild only. The data fix is a SQL script the operator runs once.

**What "this week" means after Sprint 7:** the **upcoming** Mon-Sun week. The webui's default URL is `/` with no `?week=` param; the API is called with `week_start=upcomingMonday()`; the dashboard header shows `Week of Jun 8, 2026`; the clickable range label and chevrons let the user navigate.

**Thread 2 (cross-week "rejected" semantics) and Thread 3 (§Future backlog F1/F8/F9/dead-CTA) are deferred** until S7 is deployed + verified. See `Review/handoff-ui-audit.md` "Active sprint" callout.

---

## New session: 2026-06-03

### UI/UX audit & fix — 3 sprints, 14 findings closed

A full Nielsen-10-heuristics audit of the live deployment at `http://100.108.208.56:8082/` was performed using Playwright (NixOS-compatible Chromium at `/run/current-system/sw/bin/chromium --no-sandbox`; original screenshots in `/tmp/opencode/mp-review/screenshots/`). 14 findings (5 P0, 6 P1, 3 P2) plus 3 a11y items were addressed in three sprints, each ending in `npm run build` green.

**Audit & plan documents (all kept in sync, all in `Review/`):**
- `Review/ui-nielsen-audit.md` — the audit itself, with status blocks per sprint at the top
- `fix-ui-audit.md` — the implementation plan, with per-task implementation notes
- `Review/sprint2-verification.md` — Sprint 2 deploy + smoke-check checklist (includes the **backend migration** step)
- `Review/sprint3-verification.md` — Sprint 3 deploy + smoke-check checklist (frontend only)
- `Review/sprint4-verification.md` — Sprint 4 deploy + smoke-check checklist (F7 + F6, frontend only)
- `Review/sprint5-verification.md` — Sprint 5 deploy + smoke-check (F5 + F2 + 0015 fix; backend + frontend)
- `Review/sprint6-verification.md` — Sprint 6 deploy + smoke-check (F3 + F4; backend + frontend, no migration)
- `Review/handoff-ui-audit.md` — focused handoff for a fresh agent continuing UI-audit work

**Commits on `main` (ahead of `origin/main` by 9 prior WIP commits plus these 7):**

| Commit | Sprint | What |
|---|---|---|
| `f3e4a44` | 1 | 5 P0 blockers: recipe/meal ingredient field names, `$N/A` cost, `/recommended` 404, mobile empty slots |
| `36038bb` | 1 (docs) | Mark Sprint 1 P0 fixes in audit doc |
| `ccc70aa` | 2 | 6 P1s + S3.3: meal-card title clamp, MealDetail hero + SEO strip, pantry aisle select, shopping-list aisle map, mobile pantry scroll hint, recipes filters w/ Apply/Reset/active-count, mobile shopping-list 3-col grid |
| `f5fb755` | 2 (fix) | Migration 0015 simplification + persistent backup script (`persist_aisle_backup.sql`) + corrected container-based deploy commands |
| `e90a9d6` | 3 | 3 P2s + a11y: undo-toast (Dashboard refills slot; Pantry fully reversible), mobile nav nowrap, aria-current, `<main id="main-content">`, Badge `aria-label`/`icon` props |
| `427d8ac` | (docs) | Review/handoff-ui-audit.md |
| `d71b67a` | 4 | F7 global error handler (10 try/catch blocks deleted, QueryCache/MutationCache onError wired) + F6 plan-status aria-label |
| `d78bd18` | 5 | F5 URL week selector (backend `?week_start=`, frontend prev/next + `useSearchParams`) + **CRITICAL 0015 cast fix** (was blocking Sprint 2 deploy) |
| `f740f40` | 5 | F2 keyboard shortcuts (vim-style sequences, focus-search bus, help banner) + new `hooks/` and `components/ShortcutHelpBanner.tsx` |
| `8ad4ef6` | 6 | F3 bulk pantry add (POST /api/pantry/bulk + ShoppingList 'Add N to pantry' button) + F4 plan-the-week (POST /api/meals/{id}/fill-empty-slots + Dashboard dropdown) |

**Critical Sprint 2 deploy note:** the user must run on the deployment host *after* `git pull`:

```bash
# 1. Persistent backup BEFORE the migration (recommended)
docker compose exec -T db psql -U mealplanner -d mealplanner \
  -f /dev/stdin < backend/scripts/persist_aisle_backup.sql

# 2. Dry-run preview (no writes)
docker compose exec -T db psql -U mealplanner -d mealplanner \
  -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql

# 3. Apply the migration
docker compose exec backend alembic upgrade head

# 4. Frontend rebuild + restart
docker compose -f docker-compose.yml up -d --build frontend
```

The dev DB dry-run on this host shows 10,657 ingredient rows + 10,539 grocery_item rows = **21,196 rows** would change. The deployment-host DB will differ — operator judgment required. The `persist_aisle_backup.sql` creates two permanent `public.*_aisle_backup_0015` tables the operator can `DROP` after confidence is established.

**Sprint 3 deploy is frontend-only:**
```bash
git pull
docker compose -f docker-compose.yml up -d --build frontend
```

### Sprint 4 — F7 (global error handler) + F6 (plan-status a11y)

The first wave of `fix-ui-audit.md` §Future work. Two small items, no new deps, no backend changes.

**F7 — `lib/toast.tsx` + `App.tsx` + 3 page refactors:**
- New `extractErrorMessage(err, fallback)` and `showApiError(err, fallback)` helpers in `lib/toast.tsx`. The normalizer reads `err.response.data.detail` (string or Pydantic 422 array), then `err.message`, then the fallback. Closes the H9 "silent failure" finding for both queries (background refetches) and mutations.
- `QueryClient` now created with `QueryCache({ onError: showApiError })` and `MutationCache({ onError: showApiError })`. Default options: `queries: { retry: 1, refetchOnWindowFocus: false }`.
- 10 local try/catch toasts deleted across `Dashboard.tsx` (6: move/approve/deny/delete/generate + outer delete), `Pantry.tsx` (3: add/remove mutations + createIngredient), `MealDetail.tsx` (1: submitFeedback). 4 pre-flight client-side checks kept local (empty name, missing ingredient link, unresolved ingredient, "Failed to send vote emails" — that one is fire-and-forget via BackgroundTasks; see `Review/sprint4-verification.md` for the rationale).

**F6 — `Dashboard.tsx` plan-status Badge:**
- Added `aria-label={\`Plan status: ${mealPlan.status.replace(/_/g, ' ')}\`}` to the badge that shows draft / awaiting_approval / approved / rejected. Matches the per-item approval-status pattern from Sprint 3. Audit of all other `<Badge>` call sites confirmed no further aria-label work needed — every other badge is either a count or a self-describing tag.

**Verification:** `npm run build` green. Live smoke per `Review/sprint4-verification.md` (network-down is the easiest way to verify F7; DevTools + VoiceOver for F6).

**Sprint 4 deploy is also frontend-only:**
```bash
git pull
docker compose -f docker-compose.yml up -d --build frontend
```

### Sprint 5 — F5 (URL week selector) + F2 (keyboard shortcuts)

Second wave of §Future. F5 is the only §Future item needing backend support; F2 is fully frontend. **Plus a critical bug fix to Sprint 2's migration 0015** that was blocking the deploy.

**F5 — URL week selector (`?week=YYYY-MM-DD`):**
- Backend: `GET /api/meals` and `GET /api/shopping-list` now accept `?week_start=YYYY-MM-DD` (FastAPI `Optional[date] Query`). When set, the response is the MealPlan for that week (any status). When omitted, behaviour is unchanged.
- Frontend: new `isoMonday()`, `parseIsoDate()`, `shiftIsoDate()`, `formatIsoDate()` helpers in `lib/utils.ts`. `meals.getPlanned(weekStart?)` and `shoppingList.get(weekStart?)` take an optional ISO date.
- Dashboard + ShoppingList both: `useSearchParams('week')` reads the URL; `queryKey: [..., weekStart]` so navigating weeks fetches the right plan; segmented control (chevron-left | 'This week'/'Current' jump button | chevron-right) in the header. Mutations invalidate the week-aware key. Empty state branches on `isCurrentWeek` ('No plan for that week' vs 'No shopping list yet').

**F2 — Keyboard shortcuts (`g d/r/p/s` nav, `/` focus, `?` help):**
- New `hooks/useKeyboardShortcuts.ts`: vim-style sequence support (1.5s timeout), suppressed in inputs/textareas/contenteditable, ref-based so the listener is registered once.
- New `hooks/useFocusSearch.ts`: CustomEvent bus for cross-page focus. Pantry + Recipes subscribe.
- New `components/ShortcutHelpBanner.tsx`: dismissible help dialog (slide-down under nav) with `role=dialog` + `aria-label`. Auto-dismisses 6s; Escape dismisses.
- App.tsx mounts `<GlobalShortcuts />` (registers the shortcuts) and `<ShortcutHelpBanner />`.

**0015 cast fix (CRITICAL — blocks Sprint 2 deploy):**
- The CASE expression in `0015_normalize_pantry_aisles.py` failed with `operator does not exist: text = boolean` on the `varchar(100) aisle` column. Sprint 2's dry-run query used a different path so the bug was not caught.
- Fixed with explicit `::varchar(100)` cast on the whole CASE expression + simplified `WHEN '' THEN NULL` branch. Verified on local dev DB: migration now succeeds; the 21,196 rows the Sprint 2 dry-run predicted normalize correctly. The deployment host would have hit the same error.

**Sprint 5 deploy (backend + frontend):**
```bash
git pull
docker compose exec -T db psql -U mealplanner -d mealplanner \
  -f /dev/stdin < backend/scripts/persist_aisle_backup.sql
docker compose exec backend alembic upgrade head
docker compose -f docker-compose.yml up -d --build backend frontend
```

The order matters: backup → migration → rebuild. The migration will lock the `ingredient` and `grocery_item` tables for the duration; the persist script creates recoverable backups.

### Sprint 6 — F3 (bulk pantry add) + F4 (plan the whole week)

Third wave of §Future. Both M-L size, both with design decisions made by the user during planning.

**F3 — Bulk 'add checked to pantry' (ShoppingList):**
- Backend `POST /api/pantry/bulk` accepts `{items: HomePantryCreate[]}`; returns `HomePantryBulkResult { added, updated, skipped, results: [{ingredient_id, status, id, reason}] }`. Per-item failure model: unknown ingredient → `skipped` with reason, not a 4xx. Each item follows the same upsert semantics as the single-item endpoint.
- Frontend `mealPlannerApi.pantry.addBulk(items)`.
- ShoppingList gains a primary `Add N to pantry` button (next to the existing Reset button) that appears when `checked.size > 0`. Toast reports `added X, updated Y, skipped Z`. On success, only the items that landed in the pantry are removed from the checked Set; skipped items stay checked so the user can see what failed.
- **Scope decision:** ShoppingList only. Pantry does not have row-selection state, and adding multi-select to a 4-column table on mobile is a larger surface than Sprint 6's budget. The audit's F3 ticket can be re-scoped later.

**F4 — Plan the whole week (Dashboard):**
- Backend `POST /api/meals/{id}/fill-empty-slots` with body `{meal_types: [str, ...]}` returns `FillEmptySlotsResult { filled: [{day, meal_type, item}], failed: [{day, meal_type, reason}] }`. Iterates day 1..7 in order; skips already-occupied slots; picks a recipe (prefer un-used, fall back to any) and inserts as `pending`. Per-slot failure model — never aborts mid-batch. Invalid meal_type (e.g. `'brunch'`) returns immediately with a single FailedSlot explaining why.
- Frontend `mealPlannerApi.meals.fillEmptySlots(planId, mealTypes)`.
- Dashboard gets a primary `Plan the week` button (next to the Sprint 5 week-nav control) with a dropdown. Two options: `Dinners only` (sends `['dinner']`) and `All meals` (sends `['breakfast','lunch','dinner']`). Each option has a one-line secondary label.
- Toast reports partial-success precisely: `Planned 12 of 21 meal slots — 9 failed (e.g. <reason>)`. Query invalidated so new slots show up.
- **Scope decision:** dropdown (per design-call); partial-success (per design-call).
- **Out of scope:** the no-op `Generate Meal Plan` empty-state CTA at `Dashboard.tsx:415` (when the family has NO plan at all, distinct from the F4 case of "plan exists but slots are empty"). Routing that CTA needs a user-facing "create a new plan" path (orchestrator/admin flow), which is a different feature. Documented as a follow-up.

**Sprint 6 deploy (backend + frontend, no migration):**
```bash
git pull
docker compose -f docker-compose.yml up -d --build backend frontend
```

### Deployment-host vs dev-host (Tailscale gotcha)

This repo lives on a dev host (Tailscale `100.108.146.47`). The user's home server (Tailscale `100.108.224.12`) serves the live app at `100.108.208.56:8082`. The user's workflow is **commit locally, `git pull` on the deployment host, rebuild there**. Don't `docker compose up` on the local dev host expecting it to update the live site — it won't.

### Repo quirk: `.gitignore` blocks `frontend/src/lib/`

Pre-existing bug: `.gitignore` line 17 is `lib/` (the Python ignore), and it catches `frontend/src/lib/`. New files there need `git add -f` (the `toast.tsx` rename in Sprint 3 was force-added). The `lib/` ignore should arguably be `^lib/$` or `/lib/`, but that's a separate cleanup.

### Files added by this session

```
Review/handoff-ui-audit.md                       # Focused UI-audit handoff
Review/sprint2-verification.md                   # Deploy + smoke-check for Sprint 2
Review/sprint3-verification.md                   # Deploy + smoke-check for Sprint 3
Review/sprint4-verification.md                   # Deploy + smoke-check for Sprint 4 (F7+F6)
Review/ui-nielsen-audit.md                       # (rewritten) Audit with status blocks per sprint
fix-ui-audit.md                                  # The plan, with per-task implementation notes
frontend/src/pages/NotFound.tsx                  # 404 catch-all (B4)
backend/alembic/versions/0015_normalize_pantry_aisles.py  # Sprint 2 migration
backend/scripts/dry_run_aisle_migration.sql      # Read-only preview
backend/scripts/persist_aisle_backup.sql         # Persistent backup
```

### Files modified by this session

```
backend/app/api/meals.py                          # (pre-existing WIP + Sprint 5) added ?week_start= param
backend/app/api/shopping_list.py                  # (pre-existing WIP + Sprint 5) added ?week_start= param
backend/alembic/versions/0015_normalize_pantry_aisles.py  # (Sprint 2 + Sprint 5) cast fix
frontend/src/App.tsx                              # Sprint 4: QueryCache/MutationCache onError; Sprint 5: GlobalShortcuts + ShortcutHelpBanner
frontend/src/api/index.ts                         # (pre-existing WIP + Sprint 5) getPlanned/get take weekStart
frontend/src/components/ui/Badge.tsx              # icon + aria-label props
frontend/src/components/ui/EmptyState.tsx         # optional to prop
frontend/src/lib/toast.ts → toast.tsx             # renamed for JSX; showToast.undo() (B12), extractErrorMessage/showApiError (F7)
frontend/src/lib/utils.ts                         # cleanDescription() (B7), isoMonday/parseIsoDate/shiftIsoDate/formatIsoDate (F5)
frontend/src/pages/Dashboard.tsx                  # B5, B6, B12, F6 aria-label, F7 handler refactor, F5 useSearchParams + week nav
frontend/src/pages/MealDetail.tsx                 # B2, B3, B7, F7 submitFeedback onError
frontend/src/pages/Pantry.tsx                     # B8, B10, B12, F7 add/remove/createIngredient onError, F2 useFocusSearchOnShortcut
frontend/src/pages/RecipeDetail.tsx               # B1
frontend/src/pages/Recipes.tsx                    # B11, F2 useFocusSearchOnShortcut
frontend/src/pages/ShoppingList.tsx               # B9, S3.3, F5 useSearchParams + week nav
frontend/src/types/index.ts                       # PANTRY_AISLES, RecipeIngredient extensions
```

### Files added by this session (Sprint 5)

```
frontend/src/hooks/useKeyboardShortcuts.ts        # Sprint 5 F2: global keyboard handler
frontend/src/hooks/useFocusSearch.ts              # Sprint 5 F2: focus-search CustomEvent bus
frontend/src/components/ShortcutHelpBanner.tsx    # Sprint 5 F2: help dialog
```
```

---

## New session: 2026-05-24

### Unit conversion implementation
Completed implementation of recipe-to-grocery unit conversion to make cost estimates accurate.

**Files added:**
- `backend/app/utils/units.py` — `UnitConverter` class
  - Normalization: maps synonyms to canonical units (e.g. "TBS" → "tbsp", "pounds" → "lb")
  - Within-family linear conversion: lb↔oz↔g, cup↔tbsp↔tsp, dozen↔ea
  - Cross-family via density tables for ~30 canonical ingredients (e.g. rice cup→lb via 185g/cup / 453.592g/lb)
  - Fallback to dimensionless qty when conversion is impossible (preserves monotonic ranking signal)

**Files modified:**
- `backend/app/services/planner/cost.py` — multiplies `current_price` by `convert_qty(qty, recipe_unit, grocery_unit, ingredient_name)`
- `backend/app/services/planner/generate.py` — `_load_match_index` now joins `Ingredient` table and returns `ingredient_name` + `grocery_unit` for each match
- `backend/app/services/orchestrator/steps.py` — both email cost block and shopping-list total now use unit conversion
- `backend/tests/test_planner_cost.py` — updated fixture to include new fields
- `backend/tests/test_units.py` — 19 tests covering normalization, within-family, density, and fallback

**Test results:** `test_units.py` 19/19 pass; planner cost/score/select 36 passed.

---

## New session: 2026-05-23

### Context
User observed that the system is constrained to 30 seed recipes and asked whether feedback triggers new recipe discovery. Investigation confirmed:
- **No feedback analysis service exists.** `feedback_text`, `rating`, `denial_reason` are persisted but never read downstream.
- **No recipe discovery pipeline exists.** External recipe APIs (Spoonacular, TheMealDB) are only used for image/description enrichment (`scripts/enrich_recipes_spoonacular.py`), not for discovering new recipes based on preferences.
- **Planner only reads blocklist + recency.** No signal from free-form feedback reaches `score.py` or `generate.py`.

### Proposal written
A comprehensive proposal for **Feedback-Driven Recipe Discovery** has been authored at `docs/proposals/2026-05-23-feedback-driven-recipe-discovery.md` with:
- Feedback Analyzer service (reads feedback → positive/negative signals + discovery queries)
- Recipe Discovery Service (queries Spoonacular/TheMealDB)
- Recipe Ingestion Pipeline (normalizes external recipes → our schema)
- Review Queue table (admin approval gate before recipes enter planner)
- Full architecture diagram, API changes, schema changes, cost analysis, risk matrix

### Files written
- `docs/proposals/2026-05-23-feedback-driven-recipe-discovery.md`

### Files NOT yet modified (blocked on approval)
- No code changes. No schema migrations. No API endpoints added.
- `backend/app/services/feedback_analyzer.py` — planned
- `backend/app/services/recipe_discovery.py` — planned
- `backend/alembic/versions/0010_feedback_analysis_and_review_queue.py` — planned

### Next step
Await user approval on the proposal. If approved, create `.agent/plan.md` and begin Phase A (Feedback Analyzer).