Scraper: remove price guard in map_product so produce items without a
catalog price (e.g. Fresh Garlic, Lime sold by weight) are saved to
grocery_item with current_price=NULL rather than skipped.
Matcher:
- Add exact-name fast path: build a lowercase-trimmed name→index map
and skip fuzzy search entirely when the ingredient name matches a
grocery item exactly. Lime → Lime (confidence 1.0), Garlic → Fresh
Garlic from fuzzy (confidence 1.0).
- Add exclusion words: juice, gelatin to prevent beverage/dessert
products from matching cooking ingredients.
- Increase fuzzy candidate limit 20→100 so exact-name items buried in
large tie groups are not missed.
- Add 'juice' to exclusion: prevents '100% Lime Juice' from winning
over plain 'Lime'.
Result: all recipe ingredients now match correct Lucky CA products or
show '—' (no match); zero category cross-contamination remaining.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add exclusion words: soda, rotisserie, tuna/tonno/salmon/sardine/anchovy
to prevent beverages, prepared poultry, and seafood-in-oil from matching
raw cooking ingredients
- Add min precision floor (0.45): grocery sig-word count must be ≤ 2× the
ingredient's sig-word count, catching long branded products that pass
the word-overlap recall check but are clearly wrong category matches
(e.g. "Garlic Herb Rotisserie Chicken" precision=0.25 now rejected)
Result: all previously wrong matches now show '—' (no match) rather than
a wrong product; correct matches unchanged
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Flip matching direction: iterate ingredients, search grocery items
(previously: iterate grocery items → false positives from partial word
overlap, e.g. Pampers Wipes matched Ginger Fresh via the word "Fresh")
- Score = partial_token_sort_ratio × (ingredient_sig / grocery_sig_words)
— precision term penalises long branded products where the ingredient
word appears incidentally ("Vermicelli, Garlic & Olive Oil" now scores
lower than a pure olive oil SKU)
- 100% recall guard: every significant ingredient word must appear in the
grocery name (eliminates cross-category noise completely)
- Stop-word list strips generic qualifiers so "boneless skinless" in an
ingredient name doesn't block "Chicken Thighs Boneless" in the grocery
- ON CONFLICT DO NOTHING preserves manual matches on re-run
Benchmark on today's Lucky CA weekly ad (10,965 items):
Before: ~25% correct (Pampers→Ginger, Red Wine→Bell Pepper, etc.)
After: ~80% correct; remaining misses are data gaps (Lucky has no
standalone garlic or olive oil in this week's ad)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Vote email: recipe cards now include a collapsible <details> block with
numbered cooking steps (recipe.instructions ARRAY)
- Shopping list email: ingredients now grouped under each meal heading
instead of a flat deduplicated list
- step_finalize: fix grocery price lookup (.price -> .current_price)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Preloads Ingredient names from the DB (ingredients JSONB has no name field),
deduplicates by ingredient name, looks up top-confidence IngredientGroceryMatch
per ingredient, and renders a rich 4-column HTML table (Ingredient | Qty | Unit |
At Lucky | Price) with an estimated total.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
recipe.ingredients JSONB has ingredient_id but no name field; preload
names in a single bulk query before the per-member loop so ing_rows,
shopping preview, and cost lookup all render real ingredient names.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Wraps the SessionLocal body in try/finally so db.close() is always
called, preventing connection leaks on exception. Updates the
test monkeypatch to use a no-op close() proxy so the transactional
fixture stays live after run_step returns.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Bring docs in line with the post-AM-6 state so a fresh agent can pick
up cleanly without first reconciling stale numbers:
- pytest count: 88/59/31 → 92 across all references
- live scrape: 9,960 rows / 36s → 9,980 rows / 44s (latest run, 2026-05-06)
- migrations applied: 0001-0005 → 0001-0007 in both docs
- verification gate updated with auto-minted JWT detail and the 29,779
ingredient_grocery_match rows produced post-scrape
- "What is real" / Backend: added swiftly_auth.py bullet describing
get_token() / mint_anonymous_token() / cache semantics + the
10,928-item live verification
- file map: added services/swiftly_auth.py, services/matcher.py,
services/planner/, mentioned scraper_service runs matcher post-scrape
- file map: alembic versions 0001 → 0007, tests/ count = 92, marked
config.py as no-longer-carrying SWIFTLY_BEARER_TOKEN
- spec map line for swiftly-token-auto-mint: "next-up" → "Implemented"
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fetch_category() now calls swiftly_auth.get_token() to mint a fresh
Firebase JWT on demand when no explicit bearer_token override is
pinned by tests. The cache short-circuit means the per-call mint
overhead is ~zero in the steady state.
- Removed the empty-token short-circuit; auto-mint makes it moot
- Updated _AUTH_ERROR_MESSAGE: 401-after-mint now points at the spec
(Lucky tightening anon-auth) rather than asking for manual capture
- Replaced test_swiftly_auth_error_when_token_missing with a positive
test that verifies fetch_category mints when bearer_token is None
- bearer_token constructor arg preserved for the 401-path test
Full suite: 92/92 green. Live verification via
scripts/spike_swiftly_ingest.py --confirm-live deferred to next step
per HANDOFF AM-2 halt boundary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the static SWIFTLY_BEARER_TOKEN env-var lookup with a JIT
mint via the Firebase Identity Toolkit signUp endpoint, gated by the
firebaseApiKey published in luckysupermarkets.com/config.json.
- get_token(): returns cached JWT if exp > now+300s, else mints
- mint_anonymous_token(): fetches API key, posts signUp with
Origin/Referer headers, validates iss + exp on the returned JWT
- SwiftlyAuthMintError surfaces verbatim to ScrapeLog.error_message
- Process-local cache only; threading.Lock around mutate
Tests: 4 unit tests covering fresh mint, cache hit, near-expiry
re-mint, and Firebase non-200. Full suite: 92/92 green.
Spec: docs/specs/2026-05-06-swiftly-token-auto-mint.md
Wiring into lucky_ca_scraper deferred to AM-2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Breaks the auto-mint spec into 6 ordered, sized tasks with an explicit
halt-for-approval boundary at AM-2 (live scrape verification before
removing the env var). Restates the verified prerequisites (config.json
publicly readable; Firebase signUp returns valid JWT with proper headers;
Swiftly accepts the minted token) so a fresh agent doesn't have to
re-discover them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Discovery: luckysupermarkets.com/config.json is publicly readable and
exposes firebaseApiKey. With proper Origin/Referer headers, Firebase
Identity Toolkit's anonymous-signup REST endpoint mints the same JWT
shape (iss=swiftly-lu-prod, aud=swiftly-lu-prod, anon provider, 3600s
TTL) that Swiftly accepts. Verified end-to-end on 2026-05-06.
This eliminates the manual hourly token-capture toil and supersedes
the seleniumbase-based scripts/refresh_swiftly_token.py (commit
ccfb38a) which had partial UI selector issues.
- New spec: docs/specs/2026-05-06-swiftly-token-auto-mint.md
- HANDOFF.md TL;DR refreshed (Phase 9 shipped); caveat #2 + #3
rewritten to point to the auto-mint redesign; suggested-next-move
reordered to put the redesign first
- ORIENTATION.md env-var section flags SWIFTLY_BEARER_TOKEN as
scheduled-for-removal; "Where to look" lists both specs;
last-updated footer refreshed
Implementation deferred — this commit captures the design and routing
only. Estimated 2-3 hours of focused work to ship per the spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drives luckysupermarkets.com in stealth CDP mode: opens the store
locator, types the zip code, clicks the target store, then triggers a
category page navigation. A fetch + XHR interceptor (installed via JS)
captures the first Authorization header sent to a Swiftly host. The
captured JWT is validated (iss + exp), then written into the env file.
Runs on the host (not docker) since seleniumbase needs a real Chrome.
Defaults to .env.test, headless, zip 94806, store 757. Flags:
--debug visible Chrome window
--restart-backend rerun docker compose to pick up the new token
--env-file PATH override target env file
--zip / --store override location
Selectors are intentionally JS-based and tolerant of UI changes
(querySelectorAll fallthrough by attribute heuristics + textContent
substring match) so first-attempt failures degrade to clear errors
rather than silent breakage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST/PATCH validate every ingredient_id against the ingredient table
and return 422 with the missing list when refs don't resolve. Replaces
the prior recipes.py stub. Public read routes + admin write routes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8-task TDD plan covering planner package (config, types, cost,
filter, score, select, generate orchestrator), generate/regenerate/
get endpoints, and end-to-end smoke test against seeded data.
Builds on the thin phase 4 plan; uses NeverSuggest for blocklists,
ingredient_grocery_match for sale-aware cost estimation, top-K=20
set enumeration with diversity penalty for protein/cuisine repeats.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
15-task TDD plan covering migration 0006 (ingredient.aliases,
recipe.calories_per_serving, ingredient_grocery_match), CRUD endpoints
for ingredient/recipe/never-suggest, rapidfuzz matcher service hooked
into scrape success, manual match override, resolve-ingredient assist,
and migration 0007 seeding 30 starter recipes.
Reuses existing NeverSuggest table for ingredient + recipe blocking
rather than adding a new family_ingredient_block table.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures brainstorm decisions for the meal-planner algorithm and the
minimum recipe-engine surface needed to feed it:
- 6 hard constraints (blocklist, never_suggest, recency N=4, calories
+/-20%, time <=45min, cost <=$30)
- Top-K=20 set enumeration with diversity penalty for protein/cuisine
- Ingredient<->grocery_item matching as a cacheable layer (rapidfuzz +
manual override) rather than per-run fuzzy work
- Thin phase 4: recipe CRUD, ingredient CRUD, resolve-ingredient
assist, manual match override, match job, 30-recipe seed
- Recipe ingestion source pros/cons (TheMealDB + Spoonacular + manual)
documented; decision deferred until phase 4 + phase 9 work end-to-end
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates ORIENTATION.md and HANDOFF.md to reflect actual state as of
commit 8e89f79: phases 1/2/3/7 complete (verified, not just claimed),
phases 4/5/6/8/9/10/11 not started. Documents the auth model, the
bootstrap login hatch, the SWIFTLY_BEARER_TOKEN expiration handling,
the Swiftly JSON API ingestion path that replaced Playwright, the
canonicalized API paths, and the verification commands to reproduce
the 31/31 pytest gate locally and in CI.
HANDOFF.md is intended for fresh agents and points at .agent/
phase-summaries for the per-phase write-ups. Surfaces 10 caveats and
traps the next agent will hit if they skim, and recommends Phase 9
(meal-planner generation algorithm) as the next move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add types for all API models (MealPlan, Recipe, Ingredient, etc.)
- Add API client with mealPlannerApi wrapper for all endpoints
- Implement Dashboard with weekly meal plan grid view
- Implement Pantry page with add/remove functionality
- Implement MealDetail page with recipe display
- Implement ShoppingList page with aisle grouping
- Add ShoppingList route to App.tsx
- Add vite-env.d.ts for Vite env type support
Backend (FastAPI):
- docker-compose with all 4 services
- FastAPI app with health endpoints
- SQLAlchemy models for all tables
- Placeholder API endpoints for all routes
- Config and database modules
- requirements.txt with all dependencies
Frontend (React):
- package.json with React, Tailwind, React Query, React Router
- Vite config with API proxy
- Tailwind and TypeScript configs
- Basic App with routing skeleton
- Placeholder pages (Dashboard, MealDetail, Pantry)
Infrastructure:
- nginx config for reverse proxy
- Dockerfile for backend and frontend
adminandClaude Sonnet 4.6