admin/Meal-Planner
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
aeed2a4dc0d3d3f605ace14f5bde1b68f5252cbb
Meal-Planner/docs/HANDOFF.md
T

21 KiB
Raw Blame History

MealPlanner — Agent Handoff

You are taking over a project in mid-flight. Read docs/ORIENTATION.md first for the high-level. This file is the deep dive: what's real, what's stubbed, where the bodies are buried, and what to do next.

Date of handoff: 2026-05-08. Last commit before handoff: Phase 6 SendGrid integration shipped.

TL;DR

The project completed a recovery pass (R1+R2+R3-0) from 2026-05-04 to 2026-05-05 (12 defects from a prior agent fixed), then shipped thin Phase 4 (recipe engine + ingredient↔grocery match layer + 30-recipe seed), Phase 9 (meal-planner generation algorithm), Phase 5 (weekly orchestration cycle — scrape → generate → email → deadline → finalize on a Friday Pacific cadence, driven by a dedicated APScheduler container), and Phase 6 (real SendGrid email delivery + step_reminder pre-deadline nudge).

The project's reason to exist is now real and verified end-to-end. 123/123 pytest tests pass.

Operator toil eliminated. Swiftly bearer JWTs are now auto-minted via Firebase REST anon-signUp (backend/app/services/swiftly_auth.py); the SWIFTLY_BEARER_TOKEN env var is gone. Cache hit ratio in steady state is ~99% (one mint per ~hour). See docs/specs/2026-05-06-swiftly-token-auto-mint.md (status: Implemented).

What is real (verified)

Backend

  • backend/app/main.py imports cleanly with 25 routes wired (22 prior + 3 orchestrate endpoints).
  • Health: GET /health, GET /health/db.
  • Auth: bearer ADMIN_TOKEN for admin; signed-cookie session via app.security.require_session for mutations on family-facing routes; /api/auth/login with shared SESSION_PASSWORD.
  • Routers (backend/app/api/): profile.py, recipes.py, meals.py, pantry.py, shopping_list.py, admin.py, auth.py. CRUD shapes are stubbed/partial — they validate request bodies and persist correctly but business logic is thin.
  • POST /api/admin/scrape enqueues via FastAPI BackgroundTasks, returns 202 + scrape_log_id. Status polled via GET /api/admin/logs/{id}.
  • Lucky California ingestion (backend/app/scraper/lucky_ca_scraper.py) is a requests-based Swiftly JSON API client. Not Playwright — that path was deleted. Discovers 17 categories from https://luckysupermarkets.com/categories, fetches each from prod.swiftlyapi.net/search/api/v1/products/categories?cat=…&store=757&limit=10000. Bearer scoping: token only ever attached to prod.swiftlyapi.net requests, never to the public categories page.
  • Approval flow (backend/app/services/approval.py + meals router): per-voter URLSafeTimedSerializer tokens, TTL, single-use enforced in consume_token, GET renders an HTMLResponse vote page, POST records the vote and applies the rule (any deny → item denied; all approve → item approved; otherwise pending).
  • Email backend (backend/app/services/email.py): Protocol + ConsoleEmailBackend (writes JSONL to backend/var/email_outbox.jsonl) + SendGridEmailBackend (live — uses sendgrid==6.12.0, reads SENDGRID_API_KEY/SENDGRID_FROM_EMAIL/SENDGRID_REPLY_TO from Settings, raises RuntimeError on non-2xx). Selected via EMAIL_BACKEND env (default console).
  • 401 from Swiftly is now rare (we always send a freshly minted JWT). When it does happen, SwiftlyAuthError carries a message pointing at the auto-mint spec; mint failures upstream surface as SwiftlyAuthMintError. Either lands verbatim in ScrapeLog.error_message via the bg runner.
  • Thin Phase 4: ingredient + recipe CRUD endpoints with admin gating; NeverSuggest CRUD (covers both ingredient blocklist and recipe blocklist via the existing schema); ingredient↔grocery_item match layer (rapidfuzz top-3 ranking with confidence threshold 0.75, manual override via /api/admin/ingredients/{id}/matches and /api/admin/ingredient-matches/{id}); 50 canonical ingredients seeded with aliases enriching pre-existing rows from migration 0002; 30 starter recipes spanning chicken/beef/turkey/pork/fish/vegetarian with varied cuisines, all under 45 min for 28/30. Match job runs after each successful scrape; matcher failures don't flip the scrape to FAILED.
  • Phase 9: meal-plan generation. POST /api/admin/meal-plans/generate runs the full filter→score→set-select pipeline against seeded recipes and produces a persisted MealPlan with up to 3 MealPlanItem dinners. Regenerate endpoint accepts relaxed constraint overrides (relax_time_max_minutes, relax_calorie_pct, relax_max_meal_cost) and deletes any prior plan for the same (family, week_start_date) before re-running. Per-meal cost matched against ingredient_grocery_match using the top-confidence grocery row.
  • Phase 5 orchestration (backend/app/services/orchestrator/): six idempotent step functions (step_scrape, step_generate, step_email, step_reminder, step_deadline, step_finalize) chained by runner.run_step() / run_week(). State tracked in weekly_run table (one row per family per week; each step sets its timestamp column on completion — re-firing is a no-op). Scrape failure retries once then proceeds with stale data + admin alert banner in email. Deadline resolves PENDING items per family_profile.pending_approval_policy (default "approve"). Shopping-list email sent at finalize. Admin override endpoints: POST /api/admin/orchestrate/{step}, POST /api/admin/orchestrate/run-week, GET /api/admin/orchestrate/status. New env vars: ADMIN_EMAIL (alert destination, empty = silent), APP_BASE_URL (vote link base, default http://localhost).
  • Phase 6 email (backend/app/services/email.py): SendGridEmailBackend now live. step_reminder (Fri 16:00 PT) queries MealPlanVote to find members who haven't voted on any PENDING item and sends them a "1 hour until cutoff" nudge with fresh vote links. HTML-injection risk in proposal and shopping-list emails closed (html.escape() on all recipe/ingredient/member names). Migration 0009 adds reminded_at TIMESTAMPTZ NULL to weekly_run.
  • Scheduler container (backend/app/scheduler/__main__.py): BlockingScheduler(timezone="America/Los_Angeles") with six CronTrigger jobs — Fri 02:00 scrape, 05:00 generate, 06:00 email, 16:00 reminder, 17:00 deadline, 18:00 finalize. Runs as a separate Docker service (scheduler:) using the same backend image with command: python -m app.scheduler. Started via docker compose up -d scheduler.
  • Swiftly auto-mint (backend/app/services/swiftly_auth.py): get_token() returns a Firebase anon-signUp JWT, cached in process memory until exp − 5min. mint_anonymous_token() fetches firebaseApiKey from luckysupermarkets.com/config.json, posts to identitytoolkit.googleapis.com/v1/accounts:signUp with Origin/Referer set to https://luckysupermarkets.com, validates iss/exp on the returned JWT. LuckyCaliforniaScraper.fetch_category() calls it; mint failures surface as SwiftlyAuthMintError. Live-verified 2026-05-06: 10,928 items scraped in 44s, 29,779 ingredient_grocery_match rows produced.

Database

  • Postgres 15. Nine migrations head-at: 0001_initial_migration, 0002_seed_data, 0003_grocery_item_description, 0004_family_profile_calorie_target, 0005_grocery_item_external_id, 0006_thin_phase4 (ingredient.aliases, recipe.calories_per_serving, ingredient_grocery_match), 0007_seed_canonical_ingredients (50 ingredients + 30 recipes), 0008_phase5_orchestration (weekly_run table + family_profile.pending_approval_policy), 0009_phase6_reminded_at (weekly_run.reminded_at TIMESTAMPTZ NULL).
  • 0001 downgrade now does a DO $$ … DROP TABLE … DROP TYPE … END $$; block that preserves alembic_version. Round-trip works.
  • 0002 seed is idempotent (ON CONFLICT (name_lower) DO NOTHING).
  • Every SQLEnum(...) column carries values_callable=lambda obj: [e.value for e in obj] — without this, name-mode breaks reads against the lowercase Postgres enum values.
  • MealPlan.votes relationship was removed (it had no FK target). Votes are reachable via MealPlan.items[*].votes.
  • grocery_item upsert key is (source, external_id).

Frontend

  • React 18 + TS + Vite + Tailwind. Dashboard / MealDetail / Pantry / ShoppingList pages exist.
  • API client at frontend/src/api/index.ts uses withCredentials: true for cookie-based session auth and exposes auth.login(password) + auth.logout().
  • No login UI yet. No feedback page. No tests.

Tests

  • 123 tests under backend/tests/ — 117 prior + 2 in test_email_backend.py (SendGrid send + error) + 6 new in test_orchestrator.py (step_reminder: idempotent, skips-when-not-emailed, sends-to-non-voter, skips-voter, all-voted, escape). All green when TEST_DATABASE_URL is set; requires_postgres marker auto-skips locally without it.
  • Live spike scripts in scripts/: spike_lucky_scrape.py (R2-A archived path), send_test_approval.py (email round-trip prover, supports --simulate-click {approve|deny}), spike_swiftly_ingest.py (live Swiftly ingestion prover; requires --confirm-live; uses auto-minted JWT — no env var needed).

CI

  • .github/workflows/ci.yml: backend job (postgres:15 service, alembic + pytest) + frontend job (npm ci + build). Triggers on push and pull_request.

What is stubbed or missing

Phase 4 — Recipe Engine (thin slice complete)

  • Thin slice landed: ingredient + recipe CRUD, NeverSuggest CRUD, ingredient↔grocery match layer with rapidfuzz + manual override, 50 canonical ingredients + 30 starter recipes seeded.
  • Phase 4 ingestion source (Spoonacular / TheMealDB / manual-only) — pros/cons table in docs/specs/2026-05-05-meal-planner-algorithm-design.md §6; decision deferred until Phase 9 lands.
  • Still thin: full-text recipe search, advanced tag filtering, bulk import endpoints — punted until the engine demonstrates which surfaces it actually needs.

Phase 5 — Meal-planner orchestration (complete)

  • Weekly cycle ships every Friday (Pacific): 02:00 scrape, 05:00 generate, 06:00 email, 16:00 reminder, 17:00 deadline, 18:00 finalize + shopping list.
  • weekly_run table is the state machine; each step is idempotent on its timestamp column.
  • Pending approval policy is configurable per family (family_profile.pending_approval_policy); default "approve" (silence = ok).
  • Scrape failure retries once then falls back to stale data with a visible banner in the proposal email.

Phase 6 — SendGrid (complete)

  • SendGridEmailBackend live: sendgrid==6.12.0, sends from SENDGRID_FROM_EMAIL with reply_to=SENDGRID_REPLY_TO.
  • step_reminder (Fri 16:00 PT): nudges members who haven't voted yet on any PENDING meal plan item; sends fresh vote-link email; idempotent via weekly_run.reminded_at.
  • All email templates HTML-safe: html.escape() applied to recipe names, ingredient names, and member names.
  • All user-derived strings (recipe names, ingredient names, member names) are HTML-escaped in all email templates.

Phase 8 — Feedback UI (not started)

  • feedback table exists with rating, denial_reason, free-text. No frontend page reads or writes it. No /api/feedback router (folded into meals.py?).
  • The "learn from feedback" loop into Phase 9 is unscoped.

Phase 10 — Images (not started)

  • Recipe images: scrape from source sites first, AI fallback (AI_IMAGE_ENABLED=false flag exists, no implementation).

Phase 11 — Polish (not started)

  • Variety analysis dashboard.
  • Budget tracking.
  • WhatsApp via Twilio (out of MVP scope).
  • Note: APScheduler container is now live (Phase 5). --workers 1 constraint applies to the scheduler service only.

Known caveats and traps

  1. Bootstrap login hatch. app/api/auth.py login: when no family_profile row exists, it signs the literal string "bootstrap" instead of a UUID. Anyone with SESSION_PASSWORD gets a session even with zero data in the DB. Acceptable for self-hosted on a trusted network. Replace with a proper first-run setup gate before exposing the system beyond the LAN/VPN. The decision is documented in .agent/context.md under "Decisions".

  2. Auto-mint failure modes. swiftly_auth.get_token() can fail in three ways: (a) luckysupermarkets.com/config.json becomes non-public; (b) Lucky disables anonymous Firebase auth on the swiftly-lu-prod project (signUp returns 400); (c) Google adds anti-abuse fingerprinting that the REST headers can't satisfy. All three surface as SwiftlyAuthMintError with the upstream status/body in the message and land verbatim in ScrapeLog.error_message. The fallback is to revive the manual-capture flow; the historical scripts/refresh_swiftly_token.py (commit ccfb38a) is in git history if you ever need it.

  3. ScrapeStatus enum reuses STARTED for the queued state. R1-C didn't add a QUEUED value because that would have churned the Postgres enum type. Cosmetic. If you change it, add a migration.

  4. Pytest's transactional db fixture rolls back at teardown. Background tasks open their own SessionLocal() and don't see uncommitted data. test_swiftly_api.py::test_background_runner_writes_failed_with_token_message is the example of how to test bg-task behavior — use a separate non-fixture session, commit, run, verify, clean up explicitly.

  5. alembic downgrade base in 0001 preserves alembic_version table. Don't change this to DROP SCHEMA public CASCADE — that would also drop alembic_version and break the alembic state machine on the next upgrade.

  6. Login bootstrap aside, family_profile is currently empty in any fresh DB. Phase 9 must either seed it during the first-run flow or assume the admin manually created the row. Either way, document it.

  7. Routes use @router.get("") (no trailing slash). FastAPI's redirect_slashes=True (the default) will 307-redirect /api/profile/ to /api/profile. Tests assert canonical paths (no slash). The frontend client matches.

  8. 30 starter recipes seeded. Migration 0007 loads them; enough to exercise Phase 9 against real data. Bulk ingestion source still deferred.

  9. Frontend doesn't have a login UI. Until you build one, the family-facing flows can't actually be exercised by a real user — only by tests. The Dashboard/Pantry/etc. pages assume the cookie is already set.

  10. regenerate.exclude_recipe_ids accepted by the API for forward compat but not yet honored by the orchestrator — only NeverSuggest blocklist applies. ~30-line follow-up.

  11. GET /api/meal-plans/{id} returns persisted items but with score=0, components={}, and zeroed debug — those are only available in the immediate generate response. Acceptable for the email-approval flow which uses the generate response directly. To persist them, add columns to MealPlanItem.

  12. family_profile.calorie_target is treated as per-serving by the planner filter (matches spec §2.1 wording). The family-setup UI/API should clarify per-serving vs per-day to avoid confusion. Test families use ~500 cal/serving for a 4-person household.

  13. Cost estimation treats qty as dimensionless (no unit conversion). Produces a biased-but-monotonic ranking signal; sufficient for current use, revisit if real-dollar accuracy is needed (docs/specs/2026-05-05-meal-planner-algorithm-design.md §7).

Verification commands

Same as docs/ORIENTATION.md:

# Stack up (include scheduler to test the full Phase 5 setup)
docker compose --env-file .env.test up -d db backend scheduler
docker compose --env-file .env.test exec backend alembic upgrade head

# Tests
docker compose --env-file .env.test exec \
  -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
  backend pytest -q tests/   # → 123 passed

# Verify scheduler registered all 6 jobs
docker compose --env-file .env.test logs scheduler | grep Registered

# Frontend
cd frontend && npm ci && npm run build

# Email approval round-trip
docker cp scripts/send_test_approval.py mealplanner-backend-1:/app/send_test_approval.py
docker compose --env-file .env.test exec backend \
  python /app/send_test_approval.py --simulate-click approve

# Live Swiftly ingest (will hit the real API once)
docker cp scripts/spike_swiftly_ingest.py mealplanner-backend-1:/app/spike_swiftly_ingest.py
docker compose --env-file .env.test exec backend \
  python /app/spike_swiftly_ingest.py --confirm-live

Suggested next move

Phase 6 SendGrid shipped. Emails now actually deliver via SendGrid. The weekly Friday cycle is fully automated and observable end-to-end.

Priority order

  1. Frontend login UI. /api/auth/login exists and the cookie-based session works, but no UI consumes it. Until this lands, family-facing flows (Pantry, MealDetail, ShoppingList) can only be exercised by tests.
  2. Phase 8 — feedback UI. Close the learning loop into Phase 9. The feedback table exists and the schema supports it; nothing reads/writes it from a UI.
  3. Phase 11 polish — variety analysis, budget tracking. APScheduler container is live.
  4. Phase 10 — image strategy.

Brainstorm with the user before committing to non-trivial scope. Use the superpowers:brainstorming skill.

Open tasks

ID Subject Priority
#8 ScrapeStatus enum could use a distinct QUEUED value Cosmetic

Other tasks in the recovery session were closed. See .agent/phase-summaries/ for the detailed write-ups of each phase (R1A test harness, R1B+D auth+paths, R1C async scrape, R2A live scrape, R2B email approval, R3-0 Swiftly ingestion).

Useful files map

.agent/
├── plan.md                              recovery plan (R1, R2, R3 phases)
├── context.md                           locked-in decisions
└── phase-summaries/                     per-phase write-ups
    ├── r1-r2-gate-pass.md
    ├── r3-0-gate-pass.md
    ├── r1a-summary.md
    ├── r1bd-summary.md
    ├── r1c-summary.md
    ├── r2a-summary.md
    ├── r2b-summary.md
    ├── r2b-blockers.md
    └── r3-0-summary.md

backend/app/
├── api/
│   ├── admin.py                         scrape trigger + logs (admin-gated)
│   ├── auth.py                          login/logout (R1-B+D)
│   ├── meals.py                         meal plans + vote routes (per-token)
│   ├── pantry.py
│   ├── profile.py
│   ├── recipes.py
│   └── shopping_list.py
├── scraper/
│   ├── base.py                          rate-limited HTTP base
│   └── lucky_ca_scraper.py              Swiftly JSON API client; mints via swiftly_auth.get_token()
├── scheduler/
│   ├── __init__.py
│   └── __main__.py                      APScheduler entry; 6 Friday Pacific jobs (incl. 16:00 reminder)
├── services/
│   ├── approval.py                      per-voter token issue/verify/consume
│   ├── email.py                         Console + SendGrid (live)
│   ├── matcher.py                       ingredient ↔ grocery rapidfuzz scorer
│   ├── orchestrator/                    Phase 5 weekly cycle
│   │   ├── __init__.py                  re-exports run_step / run_week
│   │   ├── alerts.py                    send_admin_alert()
│   │   ├── runner.py                    per-family loop; run_step / run_week
│   │   └── steps.py                     step_scrape/generate/email/reminder/deadline/finalize
│   ├── planner/                         Phase 9 filter / score / select / orchestrator
│   ├── scraper_service.py               enqueue + bg runner; runs matcher post-scrape
│   └── swiftly_auth.py                  Firebase REST anon-signUp; process-local JWT cache
├── security.py                          require_admin / require_session
├── config.py                            pydantic Settings (no SWIFTLY_BEARER_TOKEN — auto-minted)
├── database.py                          engine / SessionLocal / get_db
└── models/__init__.py                   all SQLAlchemy models

backend/alembic/versions/                0001 → 0009
backend/tests/                           123 tests (incl. test_swiftly_auth.py, test_orchestrator.py, test_email_backend.py)
backend/tests/fixtures/lucky_ca/         categories.html, category_meat_seafood.json, weekly_ad.html (R2-A archive)

scripts/
├── send_test_approval.py                email round-trip prover
├── spike_lucky_scrape.py                R2-A archived
└── spike_swiftly_ingest.py              live ingest prover (auto-minted JWT)

docs/specs/
├── 2026-05-05-meal-planner-algorithm-design.md   Phase 9 + thin Phase 4 design
└── 2026-05-06-swiftly-token-auto-mint.md         Implemented; auto-minted JWT replaces SWIFTLY_BEARER_TOKEN

.github/workflows/ci.yml                 backend (postgres + pytest) + frontend (npm build)

Final words

Trust the tests. Trust the live runs. Don't trust prose claims that something is "complete" without running the verification gate yourself. The recovery happened because the prior agent did the latter without the former.

Last updated: 2026-05-08 — Phase 6 SendGrid shipped; 123/123 pytest green; scheduler has 6 Friday Pacific jobs. Next pickup: Frontend login UI.

Reference in New Issue View Git Blame Copy Permalink