Meal-Planner/.agent/context.md

# Context — Recovery Takeover

## Why this plan exists

Prior agent marked Phases 1, 2, 3, 7 complete and consensus blockers "addressed" in docs, but verification of the repo shows:

1. Auth blocker (review §1.2) closed in docs only — no auth dependency on any router; `/api/admin/scrape` is open.
2. No tests, no CI; verification matrix from `Review/reviewconcensus.md §6` was never run.
3. Review §2.4 explicitly warned: spike scrape + email-approval BEFORE schema/UI commits. Prior agent did the opposite — schema, full API surface, and UI shell first; scrape unverified, email-approval not started.
4. `/api/admin/scrape` runs Playwright synchronously inside the request handler; will time out in production.
5. Phase 7 UI ships above engines (4/5/9) that don't exist — Dashboard renders meal plans the system can't generate.

## Decisions (locked in for this recovery branch)

- **Auth model:** bearer-token admin (single shared `ADMIN_TOKEN` env var) + signed-cookie session for family web UI. Matches what was claimed in ORIENTATION.md "Adversarial Review" section. No public-internet exposure assumed; nginx is sole entrypoint, already correct in `docker-compose.yml`.
- **Path canonicalization (R1-B+D):** dropped `/list` and `/planned` suffixes; routers use `@router.get("")` (no trailing slash) so the canonical paths are `/api/profile`, `/api/recipes`, `/api/recipes/ingredients`, `/api/meals`, `/api/pantry`, `/api/shopping-list`. Frontend `frontend/src/api/index.ts` and smoke tests updated to enforce.
- **Login bootstrap:** `/api/auth/login` signs the family-profile id; if no profile row exists yet, signs literal "bootstrap" so first-run isn't blocked. Cookie validates regardless; downstream code that needs a real id should re-issue after profile creation.
- **Recipe-ingredient:** stay JSONB-only (already chosen). Do not reopen.
- **Household model:** keep `family_member` table (already chosen). Do not reopen.
- **Day-of-week:** ISO (1=Mon). Already chosen.
- **Migrations:** Alembic only. Never `Base.metadata.create_all()` at runtime.
- **Background work:** FastAPI `BackgroundTasks` for the scrape now; APScheduler container with `--workers 1` later (R3-E).

## Open questions to surface to the user, not to assume

- Is `ADMIN_TOKEN` acceptable, or does the user want OIDC/Tailscale-style auth? Default for now: bearer token, easy to swap.
- Email backend for the spike: real SendGrid (needs key) or a console/file backend? Default for spike: console backend, swap to SendGrid in R3-C.

## Verification gate (Phase R1 must pass all)

- `cd backend && pytest` → green
- `docker compose run --rm backend alembic upgrade head` → no error, schema matches models
- `docker compose run --rm backend python -c "from app.main import app; print(app.title)"` → "MealPlanner"
- `docker compose run --rm frontend npm run build` → no error
- `curl -X POST http://localhost/api/admin/scrape` (no token) → 401
- `curl http://localhost/api/profile` (no session) → 200 (read), POST/PUT → 401
- CI workflow runs all of the above on push.

## Phase ordering rule (do not violate)

R1 and R2 are independent and run in parallel. R3 cannot start until BOTH R1 verification and R2 spikes pass. If R2 reveals schema impact, schema changes happen on this branch BEFORE R3-A.

## Swiftly API (R3-0, replaces Playwright path)

- Discovery: `GET https://luckysupermarkets.com/categories` (HTML, no auth). Selector: `<a class="swiftlyCouponCategory" href="/categories/<urlencoded slug>">`. Slug regex: `/categories/(.+)$` then `urllib.parse.unquote`. Fixture (2026-05-05) yielded 17 distinct slugs (e.g. `Product/meat_seafood`, `Product/produce`, ...).
- Products: `GET https://prod.swiftlyapi.net/search/api/v1/products/categories?cat=<slug>&store=757&limit=10000` with `Authorization: Bearer <SWIFTLY_BEARER_TOKEN>`. Response shape: `{"products": {"info": {"count": N}, "items": [...], "facets": [...]}}`. `meat_seafood` returned 256 items.
- Field mapping (item dict → grocery_item):
  - `id` (string) → new `external_id` column (migration 0005)
  - `name` → `name`
  - `description` → `description`
  - `brand` → `brand`
  - `primaryImage.url` → `image_url`
  - `price.ok.regPriceText` (e.g. `"$3.49 /lb"`) → parsed `regular_price` (Decimal) + `unit` (e.g. `"lb"`, may be NULL when no `/unit` suffix)
  - `price.ok.promoArea.promoText` (e.g. `"$2.49 /lb"`) → parsed `sale_price` (Decimal); when present `is_on_sale=True`, else `is_on_sale=False`
  - `price.ok.promoArea.validityText` (e.g. `"Valid 04/29/26 - 05/05/26"`) → ignored for v1 (no migration to add date columns; existing `sale_start_date` / `sale_end_date` left null)
  - aisle: extracted from the queried category slug (`Product/meat_seafood` → `meat_seafood`)
  - `product_url` → NULL (site has no public product page; per R2-A note kept nullable)
- Auth scoping: bearer header is attached ONLY to `prod.swiftlyapi.net` requests, NOT to the public `luckysupermarkets.com` HTML page. Two `requests.Session` objects (one with default UA, one with the bearer header).
- 401 detection: cannot use `BaseScraper._get` because it swallows HTTPError into a `None` return. The new client calls `session.get(...)` directly and checks `resp.status_code == 401` BEFORE `raise_for_status` to raise `SwiftlyAuthError`. Token in `.env.example` expires hourly per spec; on 401 the scraper aborts with a fixed error_message instructing the admin to refresh the token.
- Idempotency key: `(source, external_id)` upserts. Migration 0005 adds `grocery_item.external_id` (nullable text, indexed; not unique because legacy R2-A rows lack one).

---

# Context — Sprint 8 ("Deny" semantics, C + Z, hard-filter escalation)

## Why Sprint 8 exists

User report 2026-06-05 (follow-up to Sprint 7): "one of the meals was the meal that I rejected last week. After you fix the above, lets discuss what rejeccting means." User clarified (exact words): "Hard filter. If it is denied this week twice, it should be considered denied for good."

## Decisions (locked in for Sprint 8)

- **D1. Two-button model:** explicit Approve / Deny this week / Never again on the webui meal card. The "Deny" button is renamed to "Deny this week" so the soft-vs-hard distinction is visible in the UI.
- **D2. Server-side 2-denial auto-escalation:** any "Deny this week" call that finds a prior `denied` row with `denial_expires_at > now()` for the same `(family, recipe)` automatically promotes the recipe to a permanent `NeverSuggest` block. The 2nd-denial toast says "Denied — won't suggest again (denied twice recently)" so the user knows what happened.
- **D3. 90-day decay window** for soft denials (`denial_expires_at = now() + 90d`). Implemented as a partial index for fast lookup; filter is at read time, no cron cleanup needed.
- **D4. Hard filter for both soft + permanent denials.** The planner's `_load_blocklists` returns 3 sets; the soft set is unioned into the `blocked_recipe_ids` filter (per user decision: "Hard filter"). A denied recipe never reappears in the next plan; the user must unblock via the `NeverSuggest` API.
- **D5. `never_again` is the explicit path** to permanent. Always writes a `NeverSuggest` row, regardless of prior denials. Idempotent: re-calling on an already-blocked recipe is a no-op.
- **D6. Email renders 3 direct-action links per recipe** (Approve / Deny this week / Never again). Each link is a one-click GET to the vote page with `?scope=...`, which consumes the token via `submit_vote` and renders a tiny confirmation page. The legacy single-link "Vote on this meal" is preserved as a secondary "Open vote page (all 3 options)" link for completeness.
- **D7. `window.confirm` on "Never again"** to prevent accidental permanent blocks. Soft denials need no confirm.
- **D8. Pre-existing 1 denied row (2026-05-15 day-2 Roasted Sweet Potato and Chickpea Bowl) is left untouched.** Its `denial_expires_at` stays NULL (the filter requires `> now()`), so the recipe is effectively eligible again ~90d from migration time. If the user wants it permanently remembered, the soft-deny cycle auto-escalates it.
- **D9. No "unblock" UI.** The `NeverSuggest` API exists (`DELETE /api/never-suggest/{id}`); no webui button to remove a row. User can use the API directly. Documented as a follow-up.

## Open questions to surface to the user, not to assume

- **Q1. Should the migration reset `denial_expires_at` for the 1 pre-existing denied row?** Default: leave it NULL. Alternative: set it to `now() + 90d` so the row is still soft-active after migration. Asked the user — they said "leave it."
- **Q2. Should "Approve" reset any prior `denial_expires_at`?** The webui approve path (Sprint 3) goes through `approve_meal_item` (POST /api/meals/items/{id}/approve) which sets `approval_status = approved` but **does not clear `denial_expires_at`**. A user who denied a recipe 30 days ago and then approves it 60 days later will see it as `approved`; the soft-deny filter still excludes it for the remaining 30 days. Acceptable as-is; the unblock path is via "Deny this week" twice → "Never again" → manual `NeverSuggest` removal. Documented as a small follow-up.
- **Q3. Pre-existing planner test failure:** `tests/test_planner_filter.py::test_filter_blocks_by_cost` fails on a clean checkout (verified via `git stash` + re-run). Pre-existing, not introduced by Sprint 8. Filed as a pre-existing repo issue.

## Sprint 8 verification gate

- `cd frontend && npm run build` → green
- `cd backend && venv/bin/python -m pytest tests/test_planner_filter.py tests/test_planner_score.py tests/test_planner_select.py --deselect tests/test_planner_filter.py::test_filter_blocks_by_cost` → 21 passed, 1 deselected
- `docker compose exec backend alembic upgrade head` → applies 0016
- `docker compose up -d --build backend frontend` → both up
- API: `POST /api/meals/items/{id}/deny?scope=never_again` returns 200 + `promoted_to_permanent: true`
- API: `GET /api/never-suggest?family_profile_id=...` shows the new row
- Webui: 3 buttons on pending meal cards; "Deny this week" toast reflects `promoted_to_permanent`
- Email: 3 direct-action links per recipe; each is a one-click vote
- `Review/sprint8-verification.md` is the source of truth for the deploy + smoke flow.

## Sprint 8 — does NOT touch

- The `extractErrorMessage` / `showApiError` flow (Sprint 4 F7) — unchanged.
- The keyboard shortcuts (Sprint 5 F2) — unchanged.
- The bulk pantry add (Sprint 6 F3) — unchanged.
- The plan-the-week (Sprint 6 F4) — unchanged.
- The undo-toast (Sprint 3 B12) — unchanged.
- The WeekRangeNav (Sprint 7) — unchanged.
- The `extractErrorMessage` flow now sees the new `denial_expires_at` field if it propagates errors that include item data, but no new error messages.

## Key file:line references

- `backend/alembic/versions/0016_denial_decay_and_scope.py` (NEW)
- `backend/app/models/__init__.py:221-242` (MealPlanItem) + `:250-269` (MealPlanVote)
- `backend/app/schemas/__init__.py:204-219, 248-269`
- `backend/app/api/meals.py:30-138` — helpers (`_apply_denial`, `_ensure_never_suggest_recipe`, `_has_prior_active_soft_denial`)
- `backend/app/api/meals.py:240-330` — `get_vote_page` HTML (3 buttons + `?scope=...` one-click)
- `backend/app/api/meals.py:380-455` — `submit_vote` (handles `never_again` + auto-escalation)
- `backend/app/api/meals.py:486-552` — `deny_meal_item` (`?scope=`)
- `backend/app/services/orchestrator/steps.py:283-300` — email template (3 direct-action links)
- `backend/app/services/planner/generate.py:59-99, 150-194` — `_load_blocklists` returns 3 sets; soft set is hard-filtered
- `frontend/src/api/index.ts:48-58` — `meals.denyItem(itemId, { scope })`
- `frontend/src/pages/Dashboard.tsx:38-50, 385-410` — `MealCard` 3-button voting row
- `Review/sprint8-verification.md` — new file (deploy + smoke)

## Sprint 7 — webui empty-meal-plan fix

## Why Sprint 7 exists

User report 2026-06-05: "Latest meal plans were emails to me this morning, but when I go to the webui, the Meal Planner page is empty." Investigation found a date-semantics mismatch.

## Decisions (locked in for Sprint 7)

- **D1. "This week" = the upcoming Mon-Sun week.** The Friday email advertises the upcoming week; the plan is keyed by the upcoming Monday; the webui opens on the upcoming Monday. Past weeks accessible via the back-arrow. (User asked for a clickable `< Jun 8 — Jun 14 >` style nav, so the range is visible at a glance.)
- **D2. Plan key changes from Friday to Monday.** All future plans are Monday-keyed. Existing 2026-06-05 plan migrated to 2026-06-08 via guarded SQL.
- **D3. Email subject unchanged in form, changes in content.** `step_email` already uses `run.week_start_date` for the subject (verified `steps.py:305`). After D1, subject becomes "Meal plan for week of 2026-06-08" — natural Mon-Sun.
- **D4. Frontend `isoMonday` renamed to `upcomingMonday`.** Same surface (Dashboard + ShoppingList). No backward-compat alias needed; the only callers are within our codebase.
- **D5. New `WeekRangeNav` component is shared between Dashboard and ShoppingList.** Single source of truth for the visual + behavior.
- **D6. The no-op `Generate Meal Plan` CTA at `Dashboard.tsx:415` is still out of scope.** F4 (plan-the-week) and the dead CTA solve different problems. Documented as a follow-up.

## Open questions to surface to the user, not to assume

- **Q1. Migrate the 2026-05-29 plan too?** It's also Friday-keyed. Operator can run a separate guarded UPDATE in the same SQL script. Default for now: include the statement but commented out; user uncomments if they want.
- **Q2. Recency logic in the planner.** `_load_last_cooked` in `planner/generate.py:80-94` compares `MealPlan.week_start_date` across plans. After D2, all values are Mondays, so the comparison is symmetric and "days since last cooked" stays correct. No change needed. (Verified by reading the code.)
- **Q3. Should the email subject line shift by one day (Thu instead of Fri)?** No — the scheduler still fires Fri 02:00..18:00 PT (verified `scheduler/__main__.py`). The deadline (vote by Fri 17:00) still makes sense. The plan key shifts to Mon, the email timing stays Fri. No scheduler change.
- **Q4. Any URL bookmarked with `?week=2026-06-05`?** After the SQL fix, the plan moves to 2026-06-08. Any external link to `?week=2026-06-05` will hit "no plan for that week" (404-ish). Acceptable since the user uses the webui, not external links.

## Sprint 7 verification gate

- `cd frontend && npm run build` → green
- `curl http://100.108.208.56:8082/api/meals?week_start=2026-06-08` (after deploy + SQL) → 3 pending items
- Browser: open `/` (no `?week=` param) on deployment host → header shows `Week of Jun 8, 2026`, 3 meal cards visible
- `curl http://100.108.208.56:8082/api/meals?week_start=2026-06-01` → null (current calendar week has no plan; expected)
- `curl http://100.108.208.56:8082/api/meals?week_start=2026-05-29` → null if user opted in to migrate it, 3 items otherwise
- `Review/sprint7-verification.md` is the source of truth for the deploy + smoke flow.

## Sprint 7 — does NOT touch

- The `extractErrorMessage` / `showApiError` flow (Sprint 4 F7) — unchanged.
- The keyboard shortcuts (Sprint 5 F2) — unchanged. Note: `g d` still navigates to Dashboard at `upcomingMonday()`.
- The bulk pantry add (Sprint 6 F3) — unchanged.
- The plan-the-week (Sprint 6 F4) — unchanged. It still operates on the active plan regardless of week.
- The undo-toast (Sprint 3 B12) — unchanged.
- The aisle-migration (Sprint 2 / Sprint 5 fix) — no migration in S7.

## Key file:line references

- `backend/app/services/orchestrator/runner.py:20-24` — `_current_week_start()` (TO MODIFY)
- `backend/app/scheduler/__main__.py:31-66` — Friday cron schedule (NO CHANGE)
- `backend/app/services/orchestrator/steps.py:305` — `f"Meal plan for week of {run.week_start_date}"` (NO CHANGE; uses upstream value)
- `frontend/src/lib/utils.ts:44-50` — `isoMonday()` (TO RENAME + CHANGE)
- `frontend/src/pages/Dashboard.tsx:316-320` — default-week + navigateWeek (TO UPDATE)
- `frontend/src/pages/ShoppingList.tsx:87-90` — same (TO UPDATE)
- `frontend/src/pages/Dashboard.tsx:479-503` — inline week nav (TO REPLACE with `<WeekRangeNav>`)
- `frontend/src/pages/ShoppingList.tsx:259-283` — same (TO REPLACE)
- `frontend/src/components/` — new `WeekRangeNav.tsx` (TO ADD)
- `backend/scripts/fix_2026_06_05_to_2026_06_08.sql` — new (TO ADD)
- `Review/sprint7-verification.md` — new (TO ADD)

---

# Context — Sprint 9 (F1 Onboarding Tour, H10)

## Why Sprint 9 exists

User direction 2026-06-05: "Proceed with the next phase in the redesign." §Future backlog items: F1 (onboarding tour), F8 (Spoonacular proposal), F9 (Ollama proposal), dead `Generate Meal Plan` CTA. F1 is the only §Future item with a clear UI scope — selected.

## Decisions (locked in for Sprint 9)

- **D1. Hand-rolled tour, no `react-joyride`.** Adding a new npm dep is a 1-line trade-off; the audit's prior principles ("reuse existing components/ui/*", "no new npm deps") win. The tour is 4 steps; the implementation is ~420 lines of focused React.
- **D2. localStorage key `mealplanner:onboarding-complete` (`"1"` once done).** Same shape as the other `mealplanner:` prefixed keys in the codebase (verified by grep).
- **D3. `?reset-tour=1` re-triggers the tour.** Strips the param via `navigate(..., { replace: true })` so a refresh doesn't re-clear. Operator can use this from the browser URL bar; a footer link is a 5-line follow-up if requested.
- **D4. Auto-show on `/` only.** Other routes need a manual trigger (or `?reset-tour=1`). The first-time user lands on `/` (the Dashboard is the only root route), so auto-show on first visit is the natural moment.
- **D5. Tooltip is a real `<div role="dialog" aria-modal="true">`, not a portal.** The 4 anchor elements are all in the same DOM tree as the dialog. The 20-line portal boilerplate was not worth it; a `position: fixed` dialog at the right z-index works fine.
- **D6. rAF polling for the anchor's `getBoundingClientRect`.** Runs only while the tour is open. Cancellable. One DOM read per frame; well under 1% CPU on a 60Hz display.
- **D7. Focus captured on open (primary action), restored on close.** Uses `previouslyFocused.current = document.activeElement` on mount; restores on unmount. Standard focus-trap pattern, minus the trap (the dialog is intentionally non-modal — the user can interact with the page below).
- **D8. The 4 anchor points are stable elements that already exist in the DOM.** The Dashboard's `<Card>` wrapping the Weekly Overview, the Pantry's page header, the Recipes Filters button, the Shopping List page header. Each gets `data-tour="<id>"`. The anchor also has an off-route fallback (centered card + "Open <page>" CTA) so a first-time user who lands on `/pantry` can still see the Dashboard step (with a one-click nav).

## Open questions to surface to the user, not to assume

- **Q1. Should the tour show on every page or only `/`?** Default: `/` only. Other pages need `?reset-tour=1`. If the user lands on a non-root page first, the tour does NOT auto-show. Documented in `Review/sprint9-verification.md` smoke step 2.
- **Q2. Should the tour re-show on logout / new device?** Default: no. The localStorage key is per-browser, not per-family-profile. If the user has multiple devices or shares a device, the tour shows once per browser. A future migration could move the key to the family profile, but that's a Sprint 11+.
- **Q3. Should the tour re-show on a recipe update / catalog change?** Default: no. The tour is a one-shot. New users see it; existing users don't.

## Sprint 9 verification gate

- `cd frontend && npm run build` → green (tsc 0 errors, vite 0 errors)
- Browser smoke (8 steps) on `http://100.108.208.56:8082/` per `Review/sprint9-verification.md`
- No regression in Sprints 1–8

## Sprint 9 — does NOT touch

- The `extractErrorMessage` / `showApiError` flow (Sprint 4 F7) — unchanged.
- The keyboard shortcuts (Sprint 5 F2) — unchanged.
- The bulk pantry add (Sprint 6 F3) — unchanged.
- The plan-the-week (Sprint 6 F4) — unchanged.
- The undo-toast (Sprint 3 B12) — unchanged.
- The WeekRangeNav (Sprint 7) — unchanged.
- The 3-button Sprint 8 voting row — unchanged.
- Pre-existing WIP: `backend/app/api/recipes.py`, `backend/app/schemas/recipe.py`, `nginx/nginx.conf` — untouched.

## Key file:line references

- `frontend/src/components/OnboardingTour.tsx` (NEW) — ~420 lines
- `frontend/src/App.tsx:75-105` — `useOnboarding` + tour mount
- `frontend/src/pages/Dashboard.tsx:602` — `<Card data-tour="dashboard">`
- `frontend/src/pages/Pantry.tsx:185, 208` — header + add-form anchors
- `frontend/src/pages/Recipes.tsx:124` — Filters button anchor
- `frontend/src/pages/ShoppingList.tsx:231` — header anchor
- `Review/sprint9-verification.md` — new file (deploy + 8-step browser smoke + a11y check)