From 71e2317462bd14ced444ed8eb39e49194b859071 Mon Sep 17 00:00:00 2001 From: Peter Woolery Date: Tue, 5 May 2026 20:03:18 -0700 Subject: [PATCH] docs: phase 9 + thin phase 4 design spec Captures brainstorm decisions for the meal-planner algorithm and the minimum recipe-engine surface needed to feed it: - 6 hard constraints (blocklist, never_suggest, recency N=4, calories +/-20%, time <=45min, cost <=$30) - Top-K=20 set enumeration with diversity penalty for protein/cuisine - Ingredient<->grocery_item matching as a cacheable layer (rapidfuzz + manual override) rather than per-run fuzzy work - Thin phase 4: recipe CRUD, ingredient CRUD, resolve-ingredient assist, manual match override, match job, 30-recipe seed - Recipe ingestion source pros/cons (TheMealDB + Spoonacular + manual) documented; decision deferred until phase 4 + phase 9 work end-to-end Co-Authored-By: Claude Opus 4.7 (1M context) --- ...026-05-05-meal-planner-algorithm-design.md | 256 ++++++++++++++++++ 1 file changed, 256 insertions(+) create mode 100644 docs/specs/2026-05-05-meal-planner-algorithm-design.md diff --git a/docs/specs/2026-05-05-meal-planner-algorithm-design.md b/docs/specs/2026-05-05-meal-planner-algorithm-design.md new file mode 100644 index 0000000..f352473 --- /dev/null +++ b/docs/specs/2026-05-05-meal-planner-algorithm-design.md @@ -0,0 +1,256 @@ +# Meal Planner Algorithm — Design Spec + +Date: 2026-05-05 +Phase: 9 (planner algorithm) + thin Phase 4 (recipe engine) +Status: Draft for user review + +--- + +## 1. Goal + +Generate a weekly proposal of **3 dinner recipes** for the family, biased toward this week's Lucky California sale prices, while respecting hard household constraints. Output is a `MealPlan` with 3 `MealPlanItem` rows ready for the existing email-approval flow. + +Scope is intentionally narrow: dinners only, 3 nights/week. Lunch, breakfast, leftovers, and additional dinner slots are explicitly out of scope and can be added later without re-architecting. + +--- + +## 2. Algorithm shape + +**Two-stage: filter then rank.** + +1. **Hard-constraint filter** removes any recipe that violates a binary rule. Survivors form the *feasible set*. +2. **Top-K set enumeration** ranks the feasible set by individual score, takes the top K = 20, enumerates all C(20, 3) = 1,140 possible 3-recipe sets, and picks the set with the highest combined score after diversity adjustment. + +Why this shape: hard constraints are explainable ("rejected because cooking time > 45 min"), and top-K enumeration over a tiny K avoids the greedy failure mode where the first pick locks in a bad triple. 1,140 set evaluations is trivial — the whole pass should run in well under a second. + +### 2.1 Hard constraints + +A recipe enters the feasible set only if **all** of the following hold: + +| # | Constraint | Default | +|---|---|---| +| 1 | No ingredient on the family-level ingredient blocklist | — | +| 2 | `recipe.never_suggest` is false | — | +| 3 | Not cooked in the last N weeks (per `MealPlanItem` history) | N = 4 | +| 4 | Calories per serving within ±X% of `family_profile.calorie_target` (recipes with NULL `calories_per_serving` fail this constraint) | X = 20% | +| 5 | `prep_time_min + cook_time_min` ≤ T minutes | T = 45 | +| 6 | Estimated meal cost ≤ B dollars (current sale prices, scaled to servings) | B = $30 | + +If the feasible set has fewer than 3 recipes, the planner returns a partial plan and an explanatory `error_message` rather than relaxing constraints silently. The user can rerun with relaxed bounds via API params. + +### 2.2 Scoring signals + +Each recipe in the feasible set gets an individual score: + +``` +score(recipe) = + w_savings * normalize(savings_dollars) + + w_coverage * sale_coverage_pct + + w_pantry * pantry_hit_pct + + w_time * time_bonus + + w_recency * recency_bonus +``` + +| Signal | Definition | Default weight | +|---|---|---| +| `savings_dollars` | sum over recipe ingredients of `(regular_price − current_price)` * (recipe_qty / package_qty) | **0.30** | +| `sale_coverage_pct` | fraction of recipe ingredients whose linked `grocery_item.is_on_sale = true` | **0.25** | +| `pantry_hit_pct` | fraction of recipe ingredients present in `home_pantry` | **0.10** | +| `time_bonus` | `1.0` if total_time ≤ 25 min; linearly decays to `0.0` at 45 min | **0.15** | +| `recency_bonus` | `1.0` if never cooked or > R weeks ago; linearly decays toward `0.0` at the N-week cutoff. R = 12 default, configurable | **0.20** | + +Weights sum to 1.0. They live in `app/services/planner/config.py` so future tuning doesn't require schema changes. `normalize(savings_dollars)` is a min-max scaling across the feasible set (so a slow-sale week still produces meaningful ranking). + +### 2.3 Set-selection diversity + +Set score = `sum(individual_scores) − diversity_penalty`. + +Diversity penalty applies pairwise across the 3-set: + +``` +diversity_penalty = sum over each pair (a, b): + p_protein if a.protein == b.protein + + p_cuisine if a.cuisine == b.cuisine +``` + +Defaults: `p_protein = 0.15`, `p_cuisine = 0.10`. A set of three chicken dinners pays a 0.45 protein penalty (3 pairs × 0.15) — usually enough to lose to a more varied set even if individual scores are slightly lower. + +`recipe.protein` and `recipe.cuisine` are required (single-value, low-cardinality enums on the recipe model — see §4.2). + +--- + +## 3. Ingredient ↔ grocery_item matching layer + +The savings/coverage signals are only as good as the join from a recipe's ingredients to the scraped `grocery_item` rows. The matching is a separate, cacheable layer rather than fuzzy work done at every Phase 9 run. + +### 3.1 Schema + +**`ingredient`** (already exists, fleshed out): + +| Column | Type | Notes | +|---|---|---| +| `id` | UUID PK | | +| `canonical_name` | CITEXT NOT NULL UNIQUE | e.g., "chicken thighs, boneless skinless" | +| `aliases` | TEXT[] NOT NULL DEFAULT '{}' | e.g., `{"chicken thigh", "BSL chicken thighs"}` | +| `category` | TEXT | freeform: "protein", "produce", "pantry", "dairy", … | +| `default_unit` | TEXT | "lb", "oz", "ea", "cup" | + +**`ingredient_grocery_match`** (new): + +| Column | Type | Notes | +|---|---|---| +| `id` | UUID PK | | +| `ingredient_id` | UUID FK → ingredient | | +| `grocery_item_id` | UUID FK → grocery_item | | +| `confidence` | FLOAT NOT NULL | rapidfuzz `WRatio(name, canonical_name + aliases) / 100` | +| `source` | ENUM('auto','manual') NOT NULL | | +| `updated_at` | TIMESTAMPTZ NOT NULL | | +| | | UNIQUE(ingredient_id, grocery_item_id) | + +Composite index on `(ingredient_id, confidence DESC)` for fast top-3 lookup. + +### 3.2 Match job + +Runs after each successful scrape (hooked into `scraper_service._run_scrape_in_background`'s success path): + +1. For each `Ingredient`, build a query string: `canonical_name` + each alias. +2. Fuzzy-rank against `grocery_item.name` (and `description` when present), filtered to `source = 'lucky_california'`. +3. Keep top 3 by confidence ≥ 0.75 threshold. +4. Upsert into `ingredient_grocery_match` with `source = 'auto'`. Skip rows whose existing `source = 'manual'` (admin overrides win). + +Implementation: `rapidfuzz.process.extract` is fast enough for ~500 ingredients × ~10k grocery items in-memory; no need for Postgres trigram extensions yet. + +### 3.3 Manual override + +`POST /api/admin/ingredient-matches` and `DELETE …/{id}` let the admin pin or unpin matches when the auto-job picks the wrong row. Pinned rows have `source = 'manual'` and are never overwritten by the job. + +### 3.4 Recipe-create assist + +`POST /api/admin/recipes/resolve-ingredient` — body: `{"text": "1 lb chicken thighs"}` → returns top-3 `Ingredient` suggestions by fuzzy score plus a "create new" affordance with parsed `qty`/`unit`. Admin UI uses this to bind each freeform line to an `ingredient_id` at recipe-create time. No recipe is saved until every ingredient is bound. + +--- + +## 4. Thin Phase 4 surface + +Just enough recipe engine to feed Phase 9. Ingestion source (Spoonacular et al.) is explicitly deferred — see §6. + +### 4.1 In scope + +1. Flesh out `ingredient` columns per §3.1 +2. New `ingredient_grocery_match` table per §3.1 +3. New `family_ingredient_block(family_profile_id, ingredient_id)` table for hard constraint #1 +4. Recipe model field additions (see §4.2) +5. Recipe CRUD endpoints (admin-gated) +6. Ingredient CRUD endpoints (admin-gated) +7. Recipe-create assist endpoint (§3.4) +8. Manual match override endpoints (§3.3) +9. Match job hook into scraper_service +10. Seed migration with 30 hand-curated recipes + their canonical ingredients (starter set proposed by implementer; user edits later via admin UI) + +### 4.2 Recipe model fields (additions to existing `recipe` table) + +| Column | Type | Notes | +|---|---|---| +| `never_suggest` | BOOLEAN NOT NULL DEFAULT false | hard constraint #2 | +| `prep_time_min` | INTEGER NOT NULL | for hard constraint #5 + scoring | +| `cook_time_min` | INTEGER NOT NULL | | +| `servings` | INTEGER NOT NULL | for cost scaling | +| `cuisine` | TEXT NOT NULL | e.g., "italian", "mexican", "thai" | +| `protein` | TEXT NOT NULL | e.g., "chicken", "beef", "fish", "vegetarian" | +| `tags` | TEXT[] NOT NULL DEFAULT '{}' | freeform | +| `calories_per_serving` | INTEGER | for hard constraint #4 | + +`recipe.ingredients` JSONB shape (locked-in: option B from brainstorm): + +```json +[ + {"ingredient_id": "", "qty": 1.0, "unit": "lb"}, + {"ingredient_id": "", "qty": 2.0, "unit": "tbsp", "notes": "minced"} +] +``` + +No freeform `text` field. Recipes that fail to resolve every ingredient cannot be saved. + +### 4.3 Out of scope (deferred) + +- Recipe ingestion source (Spoonacular / scrape NYT Cooking / manual-only). Tracked in §6. +- Per-member ingredient blocklists (household-level only for now) +- Recipe search/filter UI +- Recipe import from URL +- Photo upload (Phase 10) +- Tag taxonomy management (admins enter freeform; consolidate later if needed) + +--- + +## 5. Phase 9 API surface + +``` +POST /api/admin/meal-plans/generate + body: { week_start_date: "YYYY-MM-DD" } + returns: 201 { meal_plan_id, items: [...], debug: { feasible_count, picks, rejected_summary } } + +GET /api/meal-plans/{id} + returns: full meal plan with items, scores, and savings breakdown + +POST /api/admin/meal-plans/{id}/regenerate + body: { exclude_recipe_ids?: [uuid], relax: { time_max?: 60, calorie_pct?: 30 } } + returns: same as generate +``` + +Generation is synchronous (target < 1 s wall time). It does NOT email — that's Phase 5's orchestration concern. Phase 9 produces the plan; Phase 5 chains scrape → generate → email → vote → finalize. + +--- + +## 6. Recipe source comparison (deferred decision) + +Deferred from Phase 4 scope; lands here so the decision is in one place when we revisit. + +| Source | Cost | License / TOS | Quality | API ergonomics | Notes | +|---|---|---|---|---|---| +| **Spoonacular** | Free tier: 150 req/day; paid $29–$249/mo for ingestion volumes | Commercial-friendly with attribution | High; structured ingredients, nutrition, images | REST, well-documented | The pragmatic default for an MVP. Free tier is enough for personal use after initial ingestion. | +| **Edamam Recipe Search API** | Free tier: 10 req/min, 10k/mo; paid from $9/mo | Personal-use free; commercial requires paid | High; nutrition focus | REST | Tighter rate limits but generous monthly cap. | +| **TheMealDB** | Free | Commercial-friendly | Modest (~300 recipes); ingredient strings unstructured | REST | Good for early dev/seeding; not enough variety long-term. | +| **MealieDB / Tandoor (self-hosted)** | Free (self-hosted) | Open source | Bring-your-own-recipes; you import | n/a | Useful as a frontend for manual recipe entry, not a source. | +| **Scrape NYT Cooking / Serious Eats / etc.** | Free in dollars | TOS-violating for most major sites; brittle | Excellent when it works | Custom per site | Recommend against. Same trap as the original Lucky scraper — fragile + legal gray zone. | +| **Open Recipes Project / Recipe1M+** | Free | Research datasets, mixed licensing | Variable; ingredient parsing required | Static dumps | Useful for bulk seeding once; not an ongoing feed. | +| **Manual entry only** | Free | n/a | Whatever you put in | n/a | Highest friction; best fit if family preferences narrow. | + +**Recommendation when we revisit:** start with **TheMealDB** (free, commercial-OK) for dev seeding, plan to add **Spoonacular free tier** when we need real volume + nutrition data, and keep manual entry as the primary intake path indefinitely (a 4-person household doesn't need 50k recipes — it needs 100 good ones). + +This decision can wait until the thin Phase 4 + Phase 9 are working end-to-end. The seed migration covers initial development. + +--- + +## 7. Open questions / deferred + +- **Tunable weights UI.** §2.2 weights live in code config. A future admin UI to tune them is reasonable but not urgent. +- **Cost estimation when an ingredient has no `ingredient_grocery_match` row.** Current plan: treat as zero savings, but include the ingredient in the `B = $30` cap using the previous-scrape average of similarly-categorized items. If even that isn't available, the recipe is rejected from the feasible set with `reason = "missing_cost_data"`. Documented; can revisit if too aggressive. +- **Per-member preferences.** Schema has `family_member`; current design only consults household-level data. A future signal could be "rejected by member X last time we tried this" → reduces score. Not in this phase. +- **Leftover modeling.** When we expand from 3 → more nights/week, recipes need a "scales well as leftover" flag. Out of scope. + +--- + +## 8. Implementation order + +1. **Phase 4-thin schema migrations** (ingredient flesh-out, ingredient_grocery_match, family_ingredient_block, recipe field additions) +2. **Phase 4-thin endpoints** (recipe CRUD, ingredient CRUD, resolve-ingredient assist, manual match override) +3. **Match job** wired into scraper_service success path +4. **Seed migration** with ~30 recipes + their ingredients +5. **Phase 9 planner module** (`app/services/planner/`) — filter, score, set-select, persist as MealPlan +6. **Phase 9 endpoints** (generate, regenerate) +7. **Tests**: unit for filter/score/set-select; integration for end-to-end generate against seeded data; the existing `requires_postgres` marker pattern + +--- + +## 9. Out of scope for this spec + +- Phase 5 orchestration (generate → email → vote → finalize chaining) +- Phase 6 SendGrid implementation +- Phase 8 feedback UI +- Phase 10 image strategy +- Phase 11 polish (APScheduler, variety analytics, budget tracking dashboards) +- Frontend admin UI for recipe management (separate spec; thin Phase 4 ships API only) + +--- + +Last updated: 2026-05-05