Commit Graph
52 Commits
Author SHA1 Message Date
admin 6323eadc86 fix: test-email actually sends; vote page pre-checks existing votes; lowercase MealPlanItemStatus everywhere
- /api/admin/test-email now calls get_email_backend().send() instead of only logging.
- /api/meals/vote/{id} GET now queries MealPlanVote and renders 'already voted' confirmation if found.
- api/meals.py: fix remaining uppercase MealPlanItemStatus enum ref (DENIED, APPROVED, PENDING).
- Fixes the 'all meals show as pending' status regression and the 'Error: Already voted' bug.
2026-05-14 12:17:44 -07:00
admin 26d9985433 feat: remove login requirements for internal home-network use
- backend/app/security.py: require_session() now auto-authenticates by
  returning the first family_profile_id from the DB. No cookie or password
  needed. Falls back to "bootstrap" sentinel if no FamilyProfile exists.
  Admin routes (require_admin) still protected by bearer token.
- frontend/src/api/index.ts: removed 401→/login redirect interceptor
- frontend/src/App.tsx: removed Sign out button, removed /login route and
  Login page import
- Login page kept on disk (unused) for potential future re-enablement
2026-05-14 11:27:09 -07:00
admin f7ed10651b feat: Phase 8 Feedback UI + API endpoints
- New backend/app/api/feedback.py: GET/POST for meal_plan_item feedback
- MealDetail.tsx: star rating, never-suggest checkbox, reason dropdown,
  free-text comments, displays saved feedback
- frontend/src/api/index.ts + types: feedback API + TypeScript interface
- backend/app/schemas/__init__.py: model_validator maps qty→quantity for
  RecipeIngredient (fixes Pydantic validation on recipe JSONB)
- docs/HANDOFF.md: mark Phase 8 complete, update file map and date
2026-05-14 09:54:25 -07:00
adminandClaude Sonnet 4.6 b52276056c fix: cast qty/unit to str before html.escape in vote email shopping preview
Recipe ingredient qty fields in JSONB are stored as floats (e.g. 1.5),
not strings. html.escape() requires str input — AttributeError otherwise.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 11:40:55 -07:00
adminandClaude Sonnet 4.6 dbc26bcc30 feat: LLM-powered second-pass ingredient matcher + matcher improvements
Matcher improvements (matcher.py):
- Plural normalization: 'tortillas'→'tortilla', 'thighs'→'thigh' so
  subset recall check works without stemmer
- Precision floor lowered 0.45→0.30: allows 'Bacon'→'Wright Brand Bacon'
  (1/3=0.33) while exclusion words still block category contaminants
- _EXCLUSION_WORDS now normalized through same singularizer for consistency

LLM second-pass (llm_matcher.py):
- run_llm_match_job(): for each still-unmatched ingredient, collects top-12
  candidates from grocery catalog ranked by fuzzy×precision (same metric as
  AUTO matcher), then asks Ollama to pick the best match
- Candidate scoring: combined = (partial_token_sort_ratio/100) × precision
  ensures "McCormick Black Pepper" outranks "Dr Pepper" for 'Black Pepper'
- Stores picks as source='auto_llm' (confidence=0.750)
- Ollama Cloud endpoint: https://ollama.com/v1, model: kimi-k2.6:cloud

Migration 0010: adds 'auto_llm' to ingredient_match_source_enum

Config: OLLAMA_BASE_URL / OLLAMA_API_KEY / OLLAMA_MODEL settings
Docker-compose: wires all three Ollama + Spoonacular env vars to backend/scheduler
Scraper service: calls run_llm_match_job after run_match_job on every scrape

Results: AUTO matcher went from 36→25 unmatched (plural normalization fix),
LLM added 3 more (Black Pepper, Zucchini, Chicken Thighs).
Remaining 22 are genuine Lucky CA catalog gaps (standalone olive oil,
dried spices, etc. not in Swiftly weekly ad).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 09:37:58 -07:00
adminandClaude Sonnet 4.6 2373883fe7 fix: exact-name fast path in matcher + save priceless produce in scraper
Scraper: remove price guard in map_product so produce items without a
catalog price (e.g. Fresh Garlic, Lime sold by weight) are saved to
grocery_item with current_price=NULL rather than skipped.

Matcher:
- Add exact-name fast path: build a lowercase-trimmed name→index map
  and skip fuzzy search entirely when the ingredient name matches a
  grocery item exactly. Lime → Lime (confidence 1.0), Garlic → Fresh
  Garlic from fuzzy (confidence 1.0).
- Add exclusion words: juice, gelatin to prevent beverage/dessert
  products from matching cooking ingredients.
- Increase fuzzy candidate limit 20→100 so exact-name items buried in
  large tie groups are not missed.
- Add 'juice' to exclusion: prevents '100% Lime Juice' from winning
  over plain 'Lime'.

Result: all recipe ingredients now match correct Lucky CA products or
show '—' (no match); zero category cross-contamination remaining.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 20:17:03 -07:00
adminandClaude Sonnet 4.6 d7a3f5c081 fix: matcher exclusion words + precision floor for clean grocery matching
- Add exclusion words: soda, rotisserie, tuna/tonno/salmon/sardine/anchovy
  to prevent beverages, prepared poultry, and seafood-in-oil from matching
  raw cooking ingredients
- Add min precision floor (0.45): grocery sig-word count must be ≤ 2× the
  ingredient's sig-word count, catching long branded products that pass
  the word-overlap recall check but are clearly wrong category matches
  (e.g. "Garlic Herb Rotisserie Chicken" precision=0.25 now rejected)

Result: all previously wrong matches now show '—' (no match) rather than
a wrong product; correct matches unchanged

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 12:00:19 -07:00
adminandClaude Sonnet 4.6 ac2b575f6b fix: rewrite matcher as ingredient-centric with precision×recall scoring
- Flip matching direction: iterate ingredients, search grocery items
  (previously: iterate grocery items → false positives from partial word
  overlap, e.g. Pampers Wipes matched Ginger Fresh via the word "Fresh")
- Score = partial_token_sort_ratio × (ingredient_sig / grocery_sig_words)
  — precision term penalises long branded products where the ingredient
  word appears incidentally ("Vermicelli, Garlic & Olive Oil" now scores
  lower than a pure olive oil SKU)
- 100% recall guard: every significant ingredient word must appear in the
  grocery name (eliminates cross-category noise completely)
- Stop-word list strips generic qualifiers so "boneless skinless" in an
  ingredient name doesn't block "Chicken Thighs Boneless" in the grocery
- ON CONFLICT DO NOTHING preserves manual matches on re-run

Benchmark on today's Lucky CA weekly ad (10,965 items):
  Before: ~25% correct (Pampers→Ginger, Red Wine→Bell Pepper, etc.)
  After:  ~80% correct; remaining misses are data gaps (Lucky has no
  standalone garlic or olive oil in this week's ad)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 11:38:59 -07:00
adminandClaude Sonnet 4.6 aeed2a4dc0 feat: add cooking instructions to vote email; shopping list grouped by meal; fix current_price field
- Vote email: recipe cards now include a collapsible <details> block with
  numbered cooking steps (recipe.instructions ARRAY)
- Shopping list email: ingredients now grouped under each meal heading
  instead of a flat deduplicated list
- step_finalize: fix grocery price lookup (.price -> .current_price)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 09:55:52 -07:00
adminandClaude Sonnet 4.6 bc85b9b617 fix: step_finalize — resolve ingredient names and prices in shopping list email
Preloads Ingredient names from the DB (ingredients JSONB has no name field),
deduplicates by ingredient name, looks up top-confidence IngredientGroceryMatch
per ingredient, and renders a rich 4-column HTML table (Ingredient | Qty | Unit |
At Lucky | Price) with an estimated total.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-09 20:48:46 -07:00
adminandClaude Sonnet 4.6 a03152e51a fix: resolve ingredient names from Ingredient table in email template
recipe.ingredients JSONB has ingredient_id but no name field; preload
names in a single bulk query before the per-member loop so ing_rows,
shopping preview, and cost lookup all render real ingredient names.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-09 14:47:48 -07:00
adminandClaude Sonnet 4.6 0739a688dd feat: enrich proposal email with ingredients, cost, shopping preview
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-09 13:40:05 -07:00
admin aea47d8365 fix: escape member.name in step_email; add all-voted reminder test 2026-05-08 21:40:00 -07:00
adminandClaude Sonnet 4.6 3b28ad0e1c fix: html-escape recipe/ingredient names in email templates (#P5-a, #P5-b)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-08 21:37:19 -07:00
admin 9a484d39c3 feat: wire step_reminder into runner, admin, and scheduler (Fri 16:00 PT) 2026-05-08 21:29:53 -07:00
adminandClaude Sonnet 4.6 d002485c10 feat: step_reminder — 1-hour pre-deadline nudge for non-voters
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-08 18:00:16 -07:00
adminandClaude Sonnet 4.6 5a41402644 feat: wire SendGridEmailBackend with from_email/reply_to settings
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-08 12:12:05 -07:00
admin 63ad306a61 feat: migration 0009 — add reminded_at to weekly_run 2026-05-08 06:48:36 -07:00
adminandClaude Sonnet 4.6 7972ecb948 feat: APScheduler entry point — 5-job weekly cadence (Pacific)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:46:39 -07:00
adminandClaude Sonnet 4.6 14a0ed1ced feat: admin orchestrate endpoints — run-week, per-step, status
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:44:41 -07:00
adminandClaude Sonnet 4.6 914fdbdb51 fix: close DB session in run_step finally block
Wraps the SessionLocal body in try/finally so db.close() is always
called, preventing connection leaks on exception. Updates the
test monkeypatch to use a no-op close() proxy so the transactional
fixture stays live after run_step returns.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:42:37 -07:00
adminandClaude Sonnet 4.6 e3ca8b7a96 feat: orchestrator runner — run_step / run_week per-family loop
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:40:36 -07:00
adminandClaude Sonnet 4.6 0a097da18e feat: orchestrator step_generate
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:30:50 -07:00
adminandClaude Sonnet 4.6 b2cbdd1533 feat: orchestrator step_scrape with retry + stale-data fallback
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:27:27 -07:00
admin 0135b74c03 feat: orchestrator package + alerts.send_admin_alert 2026-05-07 06:24:55 -07:00
admin f8ab448cbd feat: add ADMIN_EMAIL and APP_BASE_URL settings 2026-05-07 06:23:10 -07:00
adminandClaude Sonnet 4.6 03ced28ead feat: WeeklyRun model + FamilyProfile.pending_approval_policy
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 06:21:40 -07:00
adminandClaude Opus 4.7 95b8e0c1b5 feat: AM-3..AM-6 strip SWIFTLY_BEARER_TOKEN env var, delete superseded script, refresh docs
AM-3: SWIFTLY_BEARER_TOKEN removed from .env.example, .env.test (local),
docker-compose.yml service env, and Settings (backend/app/config.py).
The scraper docstring is updated to reflect the auto-mint path.

AM-4: scripts/refresh_swiftly_token.py (commit ccfb38a, seleniumbase
click-through capture) deleted; superseded by swiftly_auth.py.

AM-5: docs refreshed.
- spec status header → "Implemented 2026-05-06" with live-verification
  evidence
- HANDOFF.md TL;DR + caveats #2/#3 collapsed; replaced with the
  auto-mint failure-modes caveat; "Suggested next move" rewritten
  pointing to Phase 5 orchestration; file-map and last-updated touched
- ORIENTATION.md env-var section updated (no bearer var) + footer

AM-6 verification gate (run 2026-05-06):
- pytest -q tests/ → 92/92 green (88 prior + 4 new swiftly_auth)
- POST /api/admin/scrape → status=success, items_scraped=10928 in 44s
- grocery_item rows: 9980 (after dedup-by external_id)
- ingredient_grocery_match rows: 29779 (matcher post-hook populated)
- Container env confirmed clean of SWIFTLY_BEARER_TOKEN

The system now scrapes, matches, and generates plans without any
operator-managed credential. Live JWT lifecycle: Firebase REST anon
signUp → cache for ~55min → re-mint as needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 15:47:15 -07:00
adminandClaude Opus 4.7 dfd79a9d08 feat: AM-2 wire swiftly_auth.get_token() into LuckyCaliforniaScraper
fetch_category() now calls swiftly_auth.get_token() to mint a fresh
Firebase JWT on demand when no explicit bearer_token override is
pinned by tests. The cache short-circuit means the per-call mint
overhead is ~zero in the steady state.

- Removed the empty-token short-circuit; auto-mint makes it moot
- Updated _AUTH_ERROR_MESSAGE: 401-after-mint now points at the spec
  (Lucky tightening anon-auth) rather than asking for manual capture
- Replaced test_swiftly_auth_error_when_token_missing with a positive
  test that verifies fetch_category mints when bearer_token is None
- bearer_token constructor arg preserved for the 401-path test

Full suite: 92/92 green. Live verification via
scripts/spike_swiftly_ingest.py --confirm-live deferred to next step
per HANDOFF AM-2 halt boundary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 15:27:37 -07:00
adminandClaude Opus 4.7 5e0a49e4ae feat: AM-1 swiftly_auth module — Firebase REST anon-signUp + process cache
Replaces the static SWIFTLY_BEARER_TOKEN env-var lookup with a JIT
mint via the Firebase Identity Toolkit signUp endpoint, gated by the
firebaseApiKey published in luckysupermarkets.com/config.json.

- get_token(): returns cached JWT if exp > now+300s, else mints
- mint_anonymous_token(): fetches API key, posts signUp with
  Origin/Referer headers, validates iss + exp on the returned JWT
- SwiftlyAuthMintError surfaces verbatim to ScrapeLog.error_message
- Process-local cache only; threading.Lock around mutate

Tests: 4 unit tests covering fresh mint, cache hit, near-expiry
re-mint, and Firebase non-200. Full suite: 92/92 green.

Spec: docs/specs/2026-05-06-swiftly-token-auto-mint.md
Wiring into lucky_ca_scraper deferred to AM-2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 15:19:30 -07:00
admin 75e4bdb7a6 feat: POST /api/admin/meal-plans/generate + regenerate + get endpoints 2026-05-06 09:15:16 -07:00
admin 3f92e1f641 feat: planner orchestrator - load, filter, score, select, persist 2026-05-06 09:11:53 -07:00
admin 63e292a995 feat: planner top-K set enumeration with protein/cuisine diversity penalty 2026-05-06 06:48:26 -07:00
admin 77813cc7d3 feat: planner per-recipe scoring with 5 weighted signals 2026-05-06 06:44:58 -07:00
admin 95396137c6 feat: planner hard-constraint filter for the 6 spec constraints 2026-05-06 06:42:49 -07:00
admin bf0a327561 feat: planner cost+savings estimator against ingredient_grocery_match 2026-05-06 06:40:33 -07:00
admin c86321908c feat: planner config (weights, thresholds, K) and shared types 2026-05-06 06:38:26 -07:00
admin 1f7b9bac23 feat: never-suggest CRUD endpoints (ingredient and recipe blocklist) 2026-05-06 06:26:16 -07:00
admin 3d5f0c2668 feat: manual match pin/unpin endpoints 2026-05-06 06:24:17 -07:00
admin db4b01337e feat: run matcher after successful scrape; failures don't flip scrape status 2026-05-06 06:21:54 -07:00
admin 6dfb84310f feat: rapidfuzz-based ingredient<->grocery matcher with manual-pin preservation 2026-05-06 06:18:18 -07:00
admin 489ee03574 feat: POST /api/admin/recipes/resolve-ingredient with rapidfuzz top-3 2026-05-06 06:16:29 -07:00
adminandClaude Opus 4.7 f16a2f8710 feat: recipe CRUD endpoints with canonical ingredient validation
POST/PATCH validate every ingredient_id against the ingredient table
and return 422 with the missing list when refs don't resolve. Replaces
the prior recipes.py stub. Public read routes + admin write routes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 06:14:37 -07:00
admin b1ea011d49 feat: ingredient CRUD endpoints with admin gating 2026-05-05 20:54:31 -07:00
admin be7f698779 feat: add RecipeCreate/Update/Read schemas with canonical ingredient refs 2026-05-05 20:50:07 -07:00
admin c8382b37e7 feat: add IngredientCreate/Update/Read and IngredientGroceryMatchRead schemas 2026-05-05 20:48:49 -07:00
admin 8ac27d843d feat: add IngredientGroceryMatch model and Ingredient.aliases / Recipe.calories_per_serving 2026-05-05 20:47:31 -07:00
adminandClaude Opus 4.7 8e89f793d5 feat: phase r1+r2 recovery + r3-0 swiftly api ingestion
R1 stabilization: pytest harness with transactional db fixture, smoke
+ alembic + auth + scrape + approval + swiftly tests, github actions
ci yaml. Bearer-token admin auth + signed-cookie session for family
ui mutations. Async POST /api/admin/scrape (BackgroundTasks, returns
202). Path canonicalization (no /list, /planned suffixes). DATABASE_URL
fail-fast on empty.

R2 deferred-risk spikes: live lucky california fetch (R2-A), full
email+per-voter approval click round trip with single-use enforcement
(R2-B, console email backend, sendgrid stub).

R3-0 phase 3 redesign: replaced playwright html scraper with requests
based swiftly json api client. 17 categories, ~10k products per scrape,
upsert by (source, external_id). 401 surfaces actionable token-refresh
message via ScrapeLog.error_message.

Pre-existing defects fixed: shopping_list.py syntax error blocking app
import, MealPlan.votes orphan relationship, JSONB(astext=True) invalid
kwarg, missing requests dep, calorie_target schema drift, every SQLEnum
needed values_callable, 0001 had empty downgrade(), seed had duplicate
ingredient rows.

Migrations added: 0003 grocery_item.description, 0004 family_profile.
calorie_target, 0005 grocery_item.external_id + source + composite index.

Verified: 31/31 pytest green, alembic upgrade->downgrade->upgrade clean,
frontend npm run build clean, live scrape 9,960 grocery_item rows in 36s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:08:19 -07:00
admin 933a0cc9db feat: implement Lucky California scraper with Playwright + BeautifulSoup
- Add BaseScraper with rate limiting, retries, session management
- Add LuckyCaliforniaScraper with Playwright for dynamic content
- Add ScraperService to save scraped items to grocery_item table
- Connect /api/admin/scrape to ScraperService
- Update ORIENTATION.md phase table
2026-05-04 20:50:27 -07:00
admin c735d21661 feat: implement Phase 2 - Alembic migrations, Pydantic schemas, and real API endpoints
- Add initial Alembic migration with full PostgreSQL schema (enums, tables, indexes, constraints)
- Add seed data migration with basic ingredients (70+) and family profile
- Add Pydantic schemas for all models (FamilyProfile, Recipe, MealPlan, etc.)
- Implement /api/profile endpoints (CRUD, family member management)
- Implement /api/recipes endpoints (CRUD, ingredients, filtering)
- Implement /api/meals endpoints (meal plans, voting, approval tokens)
- Implement /api/pantry endpoints (CRUD for home pantry)
- Implement /api/shopping-list endpoints (aggregation, print-ready HTML)
- Implement /api/admin endpoints (scrape trigger, logs, stats)
- Update ORIENTATION.md with Phase 2 progress
2026-05-04 20:21:20 -07:00