From dc01a0fefbd1069dab426e5a86d5544bc060a238 Mon Sep 17 00:00:00 2001 From: Peter Woolery Date: Wed, 6 May 2026 15:50:14 -0700 Subject: [PATCH] docs: refresh HANDOFF + ORIENTATION for fresh-agent pickup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bring docs in line with the post-AM-6 state so a fresh agent can pick up cleanly without first reconciling stale numbers: - pytest count: 88/59/31 → 92 across all references - live scrape: 9,960 rows / 36s → 9,980 rows / 44s (latest run, 2026-05-06) - migrations applied: 0001-0005 → 0001-0007 in both docs - verification gate updated with auto-minted JWT detail and the 29,779 ingredient_grocery_match rows produced post-scrape - "What is real" / Backend: added swiftly_auth.py bullet describing get_token() / mint_anonymous_token() / cache semantics + the 10,928-item live verification - file map: added services/swiftly_auth.py, services/matcher.py, services/planner/, mentioned scraper_service runs matcher post-scrape - file map: alembic versions 0001 → 0007, tests/ count = 92, marked config.py as no-longer-carrying SWIFTLY_BEARER_TOKEN - spec map line for swiftly-token-auto-mint: "next-up" → "Implemented" Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/HANDOFF.md | 30 +++++++++++++++++------------- docs/ORIENTATION.md | 6 +++--- 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 36f3991..86ca59b 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -10,7 +10,7 @@ Date of handoff: 2026-05-06. Last commit before handoff: Swiftly token auto-mint The project completed a **recovery pass** (R1+R2+R3-0) from 2026-05-04 to 2026-05-05 (12 defects from a prior agent fixed), then shipped **thin Phase 4** (recipe engine + ingredient↔grocery match layer + 30-recipe seed) and **Phase 9** (meal-planner generation algorithm: filter → score → top-K=20 set enumeration with diversity penalty → persist MealPlan + items via `POST /api/admin/meal-plans/generate`). -The project's reason to exist is now real and verified end-to-end. 88/88 pytest tests pass. +The project's reason to exist is now real and verified end-to-end. 92/92 pytest tests pass. **Operator toil eliminated.** Swiftly bearer JWTs are now auto-minted via Firebase REST anon-signUp (`backend/app/services/swiftly_auth.py`); the `SWIFTLY_BEARER_TOKEN` env var is gone. Cache hit ratio in steady state is ~99% (one mint per ~hour). See `docs/specs/2026-05-06-swiftly-token-auto-mint.md` (status: Implemented). @@ -29,10 +29,11 @@ The project's reason to exist is now real and verified end-to-end. 88/88 pytest - Email backend (`backend/app/services/email.py`): Protocol + `ConsoleEmailBackend` (writes JSONL to `backend/var/email_outbox.jsonl`) + `SendGridEmailBackend` stub that raises `NotImplementedError`. Selected via `EMAIL_BACKEND` env (default `console`). - 401 from Swiftly is now rare (we always send a freshly minted JWT). When it does happen, `SwiftlyAuthError` carries a message pointing at the auto-mint spec; mint failures upstream surface as `SwiftlyAuthMintError`. Either lands verbatim in `ScrapeLog.error_message` via the bg runner. - Thin Phase 4: ingredient + recipe CRUD endpoints with admin gating; NeverSuggest CRUD (covers both ingredient blocklist and recipe blocklist via the existing schema); ingredient↔grocery_item match layer (rapidfuzz top-3 ranking with confidence threshold 0.75, manual override via /api/admin/ingredients/{id}/matches and /api/admin/ingredient-matches/{id}); 50 canonical ingredients seeded with aliases enriching pre-existing rows from migration 0002; 30 starter recipes spanning chicken/beef/turkey/pork/fish/vegetarian with varied cuisines, all under 45 min for 28/30. Match job runs after each successful scrape; matcher failures don't flip the scrape to FAILED. -- Phase 9: meal-plan generation. POST /api/admin/meal-plans/generate runs the full filter→score→set-select pipeline against seeded recipes and produces a persisted MealPlan with up to 3 MealPlanItem dinners. Regenerate endpoint accepts relaxed constraint overrides (`relax_time_max_minutes`, `relax_calorie_pct`, `relax_max_meal_cost`) and deletes any prior plan for the same `(family, week_start_date)` before re-running. Per-meal cost matched against ingredient_grocery_match using the top-confidence grocery row. 88 tests green. +- Phase 9: meal-plan generation. POST /api/admin/meal-plans/generate runs the full filter→score→set-select pipeline against seeded recipes and produces a persisted MealPlan with up to 3 MealPlanItem dinners. Regenerate endpoint accepts relaxed constraint overrides (`relax_time_max_minutes`, `relax_calorie_pct`, `relax_max_meal_cost`) and deletes any prior plan for the same `(family, week_start_date)` before re-running. Per-meal cost matched against ingredient_grocery_match using the top-confidence grocery row. +- **Swiftly auto-mint** (`backend/app/services/swiftly_auth.py`): `get_token()` returns a Firebase anon-signUp JWT, cached in process memory until exp − 5min. `mint_anonymous_token()` fetches `firebaseApiKey` from `luckysupermarkets.com/config.json`, posts to `identitytoolkit.googleapis.com/v1/accounts:signUp` with `Origin`/`Referer` set to `https://luckysupermarkets.com`, validates `iss`/`exp` on the returned JWT. `LuckyCaliforniaScraper.fetch_category()` calls it; mint failures surface as `SwiftlyAuthMintError`. Live-verified 2026-05-06: 10,928 items scraped in 44s, 29,779 ingredient_grocery_match rows produced. ### Database -- Postgres 15. Seven migrations: `0001_initial_migration`, `0002_seed_data`, `0003_grocery_item_description`, `0004_family_profile_calorie_target`, `0005_grocery_item_external_id`, `0006_thin_phase4` (ingredient.aliases, recipe.calories_per_serving, ingredient_grocery_match), `0007_seed_canonical_ingredients` (50 ingredients + 30 recipes). +- Postgres 15. Seven migrations head-at: `0001_initial_migration`, `0002_seed_data`, `0003_grocery_item_description`, `0004_family_profile_calorie_target`, `0005_grocery_item_external_id`, `0006_thin_phase4` (ingredient.aliases, recipe.calories_per_serving, ingredient_grocery_match), `0007_seed_canonical_ingredients` (50 ingredients + 30 recipes). - `0001` downgrade now does a `DO $$ … DROP TABLE … DROP TYPE … END $$;` block that preserves `alembic_version`. Round-trip works. - `0002` seed is idempotent (`ON CONFLICT (name_lower) DO NOTHING`). - Every `SQLEnum(...)` column carries `values_callable=lambda obj: [e.value for e in obj]` — without this, name-mode breaks reads against the lowercase Postgres enum values. @@ -45,8 +46,8 @@ The project's reason to exist is now real and verified end-to-end. 88/88 pytest - **No login UI yet.** No feedback page. No tests. ### Tests -- 59 tests under `backend/tests/` (31 prior + Phase 4 additions: ingredient/recipe schema + API, matcher, match-hook, never-suggest, resolve-ingredient, thin-phase-4 smoke). All green when `TEST_DATABASE_URL` is set; `requires_postgres` marker auto-skips locally without it. -- Live spike scripts in `scripts/`: `spike_lucky_scrape.py` (R2-A archived path), `send_test_approval.py` (email round-trip prover, supports `--simulate-click {approve|deny}`), `spike_swiftly_ingest.py` (R3-0 live ingestion prover; requires `--confirm-live`). +- **92 tests** under `backend/tests/` — 88 from Phase 4 + Phase 9 work, plus 4 new in `test_swiftly_auth.py` (mint, cache hit, near-expiry re-mint, Firebase non-200 → SwiftlyAuthMintError). All green when `TEST_DATABASE_URL` is set; `requires_postgres` marker auto-skips locally without it. +- Live spike scripts in `scripts/`: `spike_lucky_scrape.py` (R2-A archived path), `send_test_approval.py` (email round-trip prover, supports `--simulate-click {approve|deny}`), `spike_swiftly_ingest.py` (live Swiftly ingestion prover; requires `--confirm-live`; uses auto-minted JWT — no env var needed). ### CI - `.github/workflows/ci.yml`: backend job (postgres:15 service, alembic + pytest) + frontend job (npm ci + build). Triggers on push and pull_request. @@ -56,7 +57,7 @@ The project's reason to exist is now real and verified end-to-end. 88/88 pytest ## What is stubbed or missing ### Phase 4 — Recipe Engine (thin slice complete) -- Thin slice landed: ingredient + recipe CRUD, NeverSuggest CRUD, ingredient↔grocery match layer with rapidfuzz + manual override, 50 canonical ingredients + 30 starter recipes seeded. 59/59 tests green. +- Thin slice landed: ingredient + recipe CRUD, NeverSuggest CRUD, ingredient↔grocery match layer with rapidfuzz + manual override, 50 canonical ingredients + 30 starter recipes seeded. - Phase 4 ingestion source (Spoonacular / TheMealDB / manual-only) — pros/cons table in `docs/specs/2026-05-05-meal-planner-algorithm-design.md` §6; decision deferred until Phase 9 lands. - Still thin: full-text recipe search, advanced tag filtering, bulk import endpoints — punted until the engine demonstrates which surfaces it actually needs. @@ -125,7 +126,7 @@ docker compose --env-file .env.test exec backend alembic upgrade head # Tests docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ - backend pytest -q tests/ # → 59 passed + backend pytest -q tests/ # → 92 passed # Frontend cd frontend && npm ci && npm run build @@ -200,18 +201,21 @@ backend/app/ │ └── shopping_list.py ├── scraper/ │ ├── base.py rate-limited HTTP base -│ └── lucky_ca_scraper.py Swiftly JSON API client (R3-0) +│ └── lucky_ca_scraper.py Swiftly JSON API client; mints via swiftly_auth.get_token() ├── services/ │ ├── approval.py per-voter token issue/verify/consume │ ├── email.py Console + SendGrid stub -│ └── scraper_service.py enqueue + bg runner +│ ├── matcher.py ingredient ↔ grocery rapidfuzz scorer +│ ├── planner/ Phase 9 filter / score / select / orchestrator +│ ├── scraper_service.py enqueue + bg runner; runs matcher post-scrape +│ └── swiftly_auth.py Firebase REST anon-signUp; process-local JWT cache ├── security.py require_admin / require_session -├── config.py pydantic Settings +├── config.py pydantic Settings (no SWIFTLY_BEARER_TOKEN — auto-minted) ├── database.py engine / SessionLocal / get_db └── models/__init__.py all SQLAlchemy models -backend/alembic/versions/ 0001 → 0005 -backend/tests/ 31 tests +backend/alembic/versions/ 0001 → 0007 +backend/tests/ 92 tests (incl. test_swiftly_auth.py) backend/tests/fixtures/lucky_ca/ categories.html, category_meat_seafood.json, weekly_ad.html (R2-A archive) scripts/ @@ -221,7 +225,7 @@ scripts/ docs/specs/ ├── 2026-05-05-meal-planner-algorithm-design.md Phase 9 + thin Phase 4 design -└── 2026-05-06-swiftly-token-auto-mint.md next-up: replace SWIFTLY_BEARER_TOKEN env var +└── 2026-05-06-swiftly-token-auto-mint.md Implemented; auto-minted JWT replaces SWIFTLY_BEARER_TOKEN .github/workflows/ci.yml backend (postgres + pytest) + frontend (npm build) ``` diff --git a/docs/ORIENTATION.md b/docs/ORIENTATION.md index 9733f07..9fcbc22 100644 --- a/docs/ORIENTATION.md +++ b/docs/ORIENTATION.md @@ -47,7 +47,7 @@ web ─► nginx :80/:443 ─► React/Vite ────┼─► FastAPI ─ | 10 | Image strategy (scraped + AI fallback) | Not started | | 11 | Polish (variety analysis, budget tracking, APScheduler) | Not started | -Verification gate (R1+R2 + R3-0): 31/31 pytest green; alembic upgrade→downgrade→upgrade clean; frontend `npm run build` clean; live scrape persists 9,960 grocery_item rows in 36 s; email approval round-trip (approve/deny/single-use) verified end-to-end. +Verification gate (current, 2026-05-06): **92/92 pytest green**; alembic upgrade→downgrade→upgrade clean; frontend `npm run build` clean; live scrape persists **9,980 grocery_item rows in 44 s** with auto-minted JWT (no env var); 29,779 ingredient_grocery_match rows produced; email approval round-trip (approve/deny/single-use) verified end-to-end. --- @@ -69,7 +69,7 @@ Conventions: UUID PKs everywhere, `TIMESTAMPTZ`, Postgres ENUMs (with `values_ca Key relationships: `family_profile`→`family_member`; `family_member`→`meal_plan_vote` (per-voter); `recipe.ingredients` JSONB (no recipe-ingredient join table); `grocery_item.(source, external_id)` is the upsert key for scrape ingestion. -Migrations applied: 0001 initial, 0002 seed (idempotent via `ON CONFLICT DO NOTHING`), 0003 grocery_item.description, 0004 family_profile.calorie_target, 0005 grocery_item.external_id + source + composite index. +Migrations applied: 0001 initial, 0002 seed (idempotent via `ON CONFLICT DO NOTHING`), 0003 grocery_item.description, 0004 family_profile.calorie_target, 0005 grocery_item.external_id + source + composite index, 0006 thin Phase 4 (ingredient.aliases, recipe.calories_per_serving, ingredient_grocery_match), 0007 seed canonical ingredients (50) + 30 starter recipes. Full schema: `docs/database-schema.md`. @@ -113,7 +113,7 @@ The Swiftly bearer JWT is auto-minted at request time via Firebase REST anon-sig # Full local stack docker compose --env-file .env.test up -d db backend docker compose --env-file .env.test exec backend alembic upgrade head -docker compose --env-file .env.test exec -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner backend pytest -q tests/ +docker compose --env-file .env.test exec -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner backend pytest -q tests/ # → 92 passed # Frontend cd frontend && npm ci && npm run build