feat: LLM-powered second-pass ingredient matcher + matcher improvements

Matcher improvements (matcher.py):
- Plural normalization: 'tortillas'→'tortilla', 'thighs'→'thigh' so
  subset recall check works without stemmer
- Precision floor lowered 0.45→0.30: allows 'Bacon'→'Wright Brand Bacon'
  (1/3=0.33) while exclusion words still block category contaminants
- _EXCLUSION_WORDS now normalized through same singularizer for consistency

LLM second-pass (llm_matcher.py):
- run_llm_match_job(): for each still-unmatched ingredient, collects top-12
  candidates from grocery catalog ranked by fuzzy×precision (same metric as
  AUTO matcher), then asks Ollama to pick the best match
- Candidate scoring: combined = (partial_token_sort_ratio/100) × precision
  ensures "McCormick Black Pepper" outranks "Dr Pepper" for 'Black Pepper'
- Stores picks as source='auto_llm' (confidence=0.750)
- Ollama Cloud endpoint: https://ollama.com/v1, model: kimi-k2.6:cloud

Migration 0010: adds 'auto_llm' to ingredient_match_source_enum

Config: OLLAMA_BASE_URL / OLLAMA_API_KEY / OLLAMA_MODEL settings
Docker-compose: wires all three Ollama + Spoonacular env vars to backend/scheduler
Scraper service: calls run_llm_match_job after run_match_job on every scrape

Results: AUTO matcher went from 36→25 unmatched (plural normalization fix),
LLM added 3 more (Black Pepper, Zucchini, Chicken Thighs).
Remaining 22 are genuine Lucky CA catalog gaps (standalone olive oil,
dried spices, etc. not in Swiftly weekly ad).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-12 09:37:58 -07:00
co-authored by Claude Sonnet 4.6
parent b03e7f8070
commit dbc26bcc30
7 changed files with 282 additions and 10 deletions
+2
View File
@@ -89,6 +89,8 @@ def _run_scrape_in_background(
try:
from app.services.matcher import run_match_job
run_match_job(db, source_filter="lucky_california")
from app.services.llm_matcher import run_llm_match_job
run_llm_match_job(db, source_filter="lucky_california")
except Exception as e: # matcher failure must not flip scrape to FAILED
import logging
logging.exception("matcher failed after successful scrape: %s", e)