Matcher improvements (matcher.py):
- Plural normalization: 'tortillas'→'tortilla', 'thighs'→'thigh' so
subset recall check works without stemmer
- Precision floor lowered 0.45→0.30: allows 'Bacon'→'Wright Brand Bacon'
(1/3=0.33) while exclusion words still block category contaminants
- _EXCLUSION_WORDS now normalized through same singularizer for consistency
LLM second-pass (llm_matcher.py):
- run_llm_match_job(): for each still-unmatched ingredient, collects top-12
candidates from grocery catalog ranked by fuzzy×precision (same metric as
AUTO matcher), then asks Ollama to pick the best match
- Candidate scoring: combined = (partial_token_sort_ratio/100) × precision
ensures "McCormick Black Pepper" outranks "Dr Pepper" for 'Black Pepper'
- Stores picks as source='auto_llm' (confidence=0.750)
- Ollama Cloud endpoint: https://ollama.com/v1, model: kimi-k2.6:cloud
Migration 0010: adds 'auto_llm' to ingredient_match_source_enum
Config: OLLAMA_BASE_URL / OLLAMA_API_KEY / OLLAMA_MODEL settings
Docker-compose: wires all three Ollama + Spoonacular env vars to backend/scheduler
Scraper service: calls run_llm_match_job after run_match_job on every scrape
Results: AUTO matcher went from 36→25 unmatched (plural normalization fix),
LLM added 3 more (Black Pepper, Zucchini, Chicken Thighs).
Remaining 22 are genuine Lucky CA catalog gaps (standalone olive oil,
dried spices, etc. not in Swiftly weekly ad).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Scraper: remove price guard in map_product so produce items without a
catalog price (e.g. Fresh Garlic, Lime sold by weight) are saved to
grocery_item with current_price=NULL rather than skipped.
Matcher:
- Add exact-name fast path: build a lowercase-trimmed name→index map
and skip fuzzy search entirely when the ingredient name matches a
grocery item exactly. Lime → Lime (confidence 1.0), Garlic → Fresh
Garlic from fuzzy (confidence 1.0).
- Add exclusion words: juice, gelatin to prevent beverage/dessert
products from matching cooking ingredients.
- Increase fuzzy candidate limit 20→100 so exact-name items buried in
large tie groups are not missed.
- Add 'juice' to exclusion: prevents '100% Lime Juice' from winning
over plain 'Lime'.
Result: all recipe ingredients now match correct Lucky CA products or
show '—' (no match); zero category cross-contamination remaining.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add exclusion words: soda, rotisserie, tuna/tonno/salmon/sardine/anchovy
to prevent beverages, prepared poultry, and seafood-in-oil from matching
raw cooking ingredients
- Add min precision floor (0.45): grocery sig-word count must be ≤ 2× the
ingredient's sig-word count, catching long branded products that pass
the word-overlap recall check but are clearly wrong category matches
(e.g. "Garlic Herb Rotisserie Chicken" precision=0.25 now rejected)
Result: all previously wrong matches now show '—' (no match) rather than
a wrong product; correct matches unchanged
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Flip matching direction: iterate ingredients, search grocery items
(previously: iterate grocery items → false positives from partial word
overlap, e.g. Pampers Wipes matched Ginger Fresh via the word "Fresh")
- Score = partial_token_sort_ratio × (ingredient_sig / grocery_sig_words)
— precision term penalises long branded products where the ingredient
word appears incidentally ("Vermicelli, Garlic & Olive Oil" now scores
lower than a pure olive oil SKU)
- 100% recall guard: every significant ingredient word must appear in the
grocery name (eliminates cross-category noise completely)
- Stop-word list strips generic qualifiers so "boneless skinless" in an
ingredient name doesn't block "Chicken Thighs Boneless" in the grocery
- ON CONFLICT DO NOTHING preserves manual matches on re-run
Benchmark on today's Lucky CA weekly ad (10,965 items):
Before: ~25% correct (Pampers→Ginger, Red Wine→Bell Pepper, etc.)
After: ~80% correct; remaining misses are data gaps (Lucky has no
standalone garlic or olive oil in this week's ad)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
adminandClaude Sonnet 4.6