docs: phase 9 + thin phase 4 design spec

Captures brainstorm decisions for the meal-planner algorithm and the minimum recipe-engine surface needed to feed it: - 6 hard constraints (blocklist, never_suggest, recency N=4, calories +/-20%, time <=45min, cost <=$30) - Top-K=20 set enumeration with diversity penalty for protein/cuisine - Ingredient<->grocery_item matching as a cacheable layer (rapidfuzz + manual override) rather than per-run fuzzy work - Thin phase 4: recipe CRUD, ingredient CRUD, resolve-ingredient assist, manual match override, match job, 30-recipe seed - Recipe ingestion source pros/cons (TheMealDB + Spoonacular + manual) documented; decision deferred until phase 4 + phase 9 work end-to-end Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 20:03:18 -07:00
parent c953111395
commit 71e2317462
1 changed files with 256 additions and 0 deletions
@@ -0,0 +1,256 @@
+# Meal Planner Algorithm — Design Spec
+
+Date: 2026-05-05
+Phase: 9 (planner algorithm) + thin Phase 4 (recipe engine)
+Status: Draft for user review
+
+---
+
+## 1. Goal
+
+Generate a weekly proposal of **3 dinner recipes** for the family, biased toward this week's Lucky California sale prices, while respecting hard household constraints. Output is a `MealPlan` with 3 `MealPlanItem` rows ready for the existing email-approval flow.
+
+Scope is intentionally narrow: dinners only, 3 nights/week. Lunch, breakfast, leftovers, and additional dinner slots are explicitly out of scope and can be added later without re-architecting.
+
+---
+
+## 2. Algorithm shape
+
+**Two-stage: filter then rank.**
+
+1. **Hard-constraint filter** removes any recipe that violates a binary rule. Survivors form the *feasible set*.
+2. **Top-K set enumeration** ranks the feasible set by individual score, takes the top K = 20, enumerates all C(20, 3) = 1,140 possible 3-recipe sets, and picks the set with the highest combined score after diversity adjustment.
+
+Why this shape: hard constraints are explainable ("rejected because cooking time > 45 min"), and top-K enumeration over a tiny K avoids the greedy failure mode where the first pick locks in a bad triple. 1,140 set evaluations is trivial — the whole pass should run in well under a second.
+
+### 2.1 Hard constraints
+
+A recipe enters the feasible set only if **all** of the following hold:
+
+| # | Constraint | Default |
+|---|---|---|
+| 1 | No ingredient on the family-level ingredient blocklist | — |
+| 2 | `recipe.never_suggest` is false | — |
+| 3 | Not cooked in the last N weeks (per `MealPlanItem` history) | N = 4 |
+| 4 | Calories per serving within ±X% of `family_profile.calorie_target` (recipes with NULL `calories_per_serving` fail this constraint) | X = 20% |
+| 5 | `prep_time_min + cook_time_min` ≤ T minutes | T = 45 |
+| 6 | Estimated meal cost ≤ B dollars (current sale prices, scaled to servings) | B = $30 |
+
+If the feasible set has fewer than 3 recipes, the planner returns a partial plan and an explanatory `error_message` rather than relaxing constraints silently. The user can rerun with relaxed bounds via API params.
+
+### 2.2 Scoring signals
+
+Each recipe in the feasible set gets an individual score:
+
+```
+score(recipe) =
+    w_savings   * normalize(savings_dollars)
+  + w_coverage  * sale_coverage_pct
+  + w_pantry    * pantry_hit_pct
+  + w_time      * time_bonus
+  + w_recency   * recency_bonus
+```
+
+| Signal | Definition | Default weight |
+|---|---|---|
+| `savings_dollars` | sum over recipe ingredients of `(regular_price − current_price)` * (recipe_qty / package_qty) | **0.30** |
+| `sale_coverage_pct` | fraction of recipe ingredients whose linked `grocery_item.is_on_sale = true` | **0.25** |
+| `pantry_hit_pct` | fraction of recipe ingredients present in `home_pantry` | **0.10** |
+| `time_bonus` | `1.0` if total_time ≤ 25 min; linearly decays to `0.0` at 45 min | **0.15** |
+| `recency_bonus` | `1.0` if never cooked or > R weeks ago; linearly decays toward `0.0` at the N-week cutoff. R = 12 default, configurable | **0.20** |
+
+Weights sum to 1.0. They live in `app/services/planner/config.py` so future tuning doesn't require schema changes. `normalize(savings_dollars)` is a min-max scaling across the feasible set (so a slow-sale week still produces meaningful ranking).
+
+### 2.3 Set-selection diversity
+
+Set score = `sum(individual_scores) − diversity_penalty`.
+
+Diversity penalty applies pairwise across the 3-set:
+
+```
+diversity_penalty = sum over each pair (a, b):
+    p_protein  if a.protein  == b.protein
+  + p_cuisine  if a.cuisine  == b.cuisine
+```
+
+Defaults: `p_protein = 0.15`, `p_cuisine = 0.10`. A set of three chicken dinners pays a 0.45 protein penalty (3 pairs × 0.15) — usually enough to lose to a more varied set even if individual scores are slightly lower.
+
+`recipe.protein` and `recipe.cuisine` are required (single-value, low-cardinality enums on the recipe model — see §4.2).
+
+---
+
+## 3. Ingredient ↔ grocery_item matching layer
+
+The savings/coverage signals are only as good as the join from a recipe's ingredients to the scraped `grocery_item` rows. The matching is a separate, cacheable layer rather than fuzzy work done at every Phase 9 run.
+
+### 3.1 Schema
+
+**`ingredient`** (already exists, fleshed out):
+
+| Column | Type | Notes |
+|---|---|---|
+| `id` | UUID PK | |
+| `canonical_name` | CITEXT NOT NULL UNIQUE | e.g., "chicken thighs, boneless skinless" |
+| `aliases` | TEXT[] NOT NULL DEFAULT '{}' | e.g., `{"chicken thigh", "BSL chicken thighs"}` |
+| `category` | TEXT | freeform: "protein", "produce", "pantry", "dairy", … |
+| `default_unit` | TEXT | "lb", "oz", "ea", "cup" |
+
+**`ingredient_grocery_match`** (new):
+
+| Column | Type | Notes |
+|---|---|---|
+| `id` | UUID PK | |
+| `ingredient_id` | UUID FK → ingredient | |
+| `grocery_item_id` | UUID FK → grocery_item | |
+| `confidence` | FLOAT NOT NULL | rapidfuzz `WRatio(name, canonical_name + aliases) / 100` |
+| `source` | ENUM('auto','manual') NOT NULL | |
+| `updated_at` | TIMESTAMPTZ NOT NULL | |
+| | | UNIQUE(ingredient_id, grocery_item_id) |
+
+Composite index on `(ingredient_id, confidence DESC)` for fast top-3 lookup.
+
+### 3.2 Match job
+
+Runs after each successful scrape (hooked into `scraper_service._run_scrape_in_background`'s success path):
+
+1. For each `Ingredient`, build a query string: `canonical_name` + each alias.
+2. Fuzzy-rank against `grocery_item.name` (and `description` when present), filtered to `source = 'lucky_california'`.
+3. Keep top 3 by confidence ≥ 0.75 threshold.
+4. Upsert into `ingredient_grocery_match` with `source = 'auto'`. Skip rows whose existing `source = 'manual'` (admin overrides win).
+
+Implementation: `rapidfuzz.process.extract` is fast enough for ~500 ingredients × ~10k grocery items in-memory; no need for Postgres trigram extensions yet.
+
+### 3.3 Manual override
+
+`POST /api/admin/ingredient-matches` and `DELETE …/{id}` let the admin pin or unpin matches when the auto-job picks the wrong row. Pinned rows have `source = 'manual'` and are never overwritten by the job.
+
+### 3.4 Recipe-create assist
+
+`POST /api/admin/recipes/resolve-ingredient` — body: `{"text": "1 lb chicken thighs"}` → returns top-3 `Ingredient` suggestions by fuzzy score plus a "create new" affordance with parsed `qty`/`unit`. Admin UI uses this to bind each freeform line to an `ingredient_id` at recipe-create time. No recipe is saved until every ingredient is bound.
+
+---
+
+## 4. Thin Phase 4 surface
+
+Just enough recipe engine to feed Phase 9. Ingestion source (Spoonacular et al.) is explicitly deferred — see §6.
+
+### 4.1 In scope
+
+1. Flesh out `ingredient` columns per §3.1
+2. New `ingredient_grocery_match` table per §3.1
+3. New `family_ingredient_block(family_profile_id, ingredient_id)` table for hard constraint #1
+4. Recipe model field additions (see §4.2)
+5. Recipe CRUD endpoints (admin-gated)
+6. Ingredient CRUD endpoints (admin-gated)
+7. Recipe-create assist endpoint (§3.4)
+8. Manual match override endpoints (§3.3)
+9. Match job hook into scraper_service
+10. Seed migration with 30 hand-curated recipes + their canonical ingredients (starter set proposed by implementer; user edits later via admin UI)
+
+### 4.2 Recipe model fields (additions to existing `recipe` table)
+
+| Column | Type | Notes |
+|---|---|---|
+| `never_suggest` | BOOLEAN NOT NULL DEFAULT false | hard constraint #2 |
+| `prep_time_min` | INTEGER NOT NULL | for hard constraint #5 + scoring |
+| `cook_time_min` | INTEGER NOT NULL | |
+| `servings` | INTEGER NOT NULL | for cost scaling |
+| `cuisine` | TEXT NOT NULL | e.g., "italian", "mexican", "thai" |
+| `protein` | TEXT NOT NULL | e.g., "chicken", "beef", "fish", "vegetarian" |
+| `tags` | TEXT[] NOT NULL DEFAULT '{}' | freeform |
+| `calories_per_serving` | INTEGER | for hard constraint #4 |
+
+`recipe.ingredients` JSONB shape (locked-in: option B from brainstorm):
+
+```json
+[
+  {"ingredient_id": "<uuid>", "qty": 1.0, "unit": "lb"},
+  {"ingredient_id": "<uuid>", "qty": 2.0, "unit": "tbsp", "notes": "minced"}
+]
+```
+
+No freeform `text` field. Recipes that fail to resolve every ingredient cannot be saved.
+
+### 4.3 Out of scope (deferred)
+
+- Recipe ingestion source (Spoonacular / scrape NYT Cooking / manual-only). Tracked in §6.
+- Per-member ingredient blocklists (household-level only for now)
+- Recipe search/filter UI
+- Recipe import from URL
+- Photo upload (Phase 10)
+- Tag taxonomy management (admins enter freeform; consolidate later if needed)
+
+---
+
+## 5. Phase 9 API surface
+
+```
+POST /api/admin/meal-plans/generate
+  body: { week_start_date: "YYYY-MM-DD" }
+  returns: 201 { meal_plan_id, items: [...], debug: { feasible_count, picks, rejected_summary } }
+
+GET /api/meal-plans/{id}
+  returns: full meal plan with items, scores, and savings breakdown
+
+POST /api/admin/meal-plans/{id}/regenerate
+  body: { exclude_recipe_ids?: [uuid], relax: { time_max?: 60, calorie_pct?: 30 } }
+  returns: same as generate
+```
+
+Generation is synchronous (target < 1 s wall time). It does NOT email — that's Phase 5's orchestration concern. Phase 9 produces the plan; Phase 5 chains scrape → generate → email → vote → finalize.
+
+---
+
+## 6. Recipe source comparison (deferred decision)
+
+Deferred from Phase 4 scope; lands here so the decision is in one place when we revisit.
+
+| Source | Cost | License / TOS | Quality | API ergonomics | Notes |
+|---|---|---|---|---|---|
+| **Spoonacular** | Free tier: 150 req/day; paid $29–$249/mo for ingestion volumes | Commercial-friendly with attribution | High; structured ingredients, nutrition, images | REST, well-documented | The pragmatic default for an MVP. Free tier is enough for personal use after initial ingestion. |
+| **Edamam Recipe Search API** | Free tier: 10 req/min, 10k/mo; paid from $9/mo | Personal-use free; commercial requires paid | High; nutrition focus | REST | Tighter rate limits but generous monthly cap. |
+| **TheMealDB** | Free | Commercial-friendly | Modest (~300 recipes); ingredient strings unstructured | REST | Good for early dev/seeding; not enough variety long-term. |
+| **MealieDB / Tandoor (self-hosted)** | Free (self-hosted) | Open source | Bring-your-own-recipes; you import | n/a | Useful as a frontend for manual recipe entry, not a source. |
+| **Scrape NYT Cooking / Serious Eats / etc.** | Free in dollars | TOS-violating for most major sites; brittle | Excellent when it works | Custom per site | Recommend against. Same trap as the original Lucky scraper — fragile + legal gray zone. |
+| **Open Recipes Project / Recipe1M+** | Free | Research datasets, mixed licensing | Variable; ingredient parsing required | Static dumps | Useful for bulk seeding once; not an ongoing feed. |
+| **Manual entry only** | Free | n/a | Whatever you put in | n/a | Highest friction; best fit if family preferences narrow. |
+
+**Recommendation when we revisit:** start with **TheMealDB** (free, commercial-OK) for dev seeding, plan to add **Spoonacular free tier** when we need real volume + nutrition data, and keep manual entry as the primary intake path indefinitely (a 4-person household doesn't need 50k recipes — it needs 100 good ones).
+
+This decision can wait until the thin Phase 4 + Phase 9 are working end-to-end. The seed migration covers initial development.
+
+---
+
+## 7. Open questions / deferred
+
+- **Tunable weights UI.** §2.2 weights live in code config. A future admin UI to tune them is reasonable but not urgent.
+- **Cost estimation when an ingredient has no `ingredient_grocery_match` row.** Current plan: treat as zero savings, but include the ingredient in the `B = $30` cap using the previous-scrape average of similarly-categorized items. If even that isn't available, the recipe is rejected from the feasible set with `reason = "missing_cost_data"`. Documented; can revisit if too aggressive.
+- **Per-member preferences.** Schema has `family_member`; current design only consults household-level data. A future signal could be "rejected by member X last time we tried this" → reduces score. Not in this phase.
+- **Leftover modeling.** When we expand from 3 → more nights/week, recipes need a "scales well as leftover" flag. Out of scope.
+
+---
+
+## 8. Implementation order
+
+1. **Phase 4-thin schema migrations** (ingredient flesh-out, ingredient_grocery_match, family_ingredient_block, recipe field additions)
+2. **Phase 4-thin endpoints** (recipe CRUD, ingredient CRUD, resolve-ingredient assist, manual match override)
+3. **Match job** wired into scraper_service success path
+4. **Seed migration** with ~30 recipes + their ingredients
+5. **Phase 9 planner module** (`app/services/planner/`) — filter, score, set-select, persist as MealPlan
+6. **Phase 9 endpoints** (generate, regenerate)
+7. **Tests**: unit for filter/score/set-select; integration for end-to-end generate against seeded data; the existing `requires_postgres` marker pattern
+
+---
+
+## 9. Out of scope for this spec
+
+- Phase 5 orchestration (generate → email → vote → finalize chaining)
+- Phase 6 SendGrid implementation
+- Phase 8 feedback UI
+- Phase 10 image strategy
+- Phase 11 polish (APScheduler, variety analytics, budget tracking dashboards)
+- Frontend admin UI for recipe management (separate spec; thin Phase 4 ships API only)
+
+---
+
+Last updated: 2026-05-05