Meal-Planner/docs/specs/2026-05-05-meal-planner-algorithm-design.md

# Meal Planner Algorithm — Design Spec

Date: 2026-05-05
Phase: 9 (planner algorithm) + thin Phase 4 (recipe engine)
Status: Draft for user review

---

## 1. Goal

Generate a weekly proposal of **3 dinner recipes** for the family, biased toward this week's Lucky California sale prices, while respecting hard household constraints. Output is a `MealPlan` with 3 `MealPlanItem` rows ready for the existing email-approval flow.

Scope is intentionally narrow: dinners only, 3 nights/week. Lunch, breakfast, leftovers, and additional dinner slots are explicitly out of scope and can be added later without re-architecting.

---

## 2. Algorithm shape

**Two-stage: filter then rank.**

1. **Hard-constraint filter** removes any recipe that violates a binary rule. Survivors form the *feasible set*.
2. **Top-K set enumeration** ranks the feasible set by individual score, takes the top K = 20, enumerates all C(20, 3) = 1,140 possible 3-recipe sets, and picks the set with the highest combined score after diversity adjustment.

Why this shape: hard constraints are explainable ("rejected because cooking time > 45 min"), and top-K enumeration over a tiny K avoids the greedy failure mode where the first pick locks in a bad triple. 1,140 set evaluations is trivial — the whole pass should run in well under a second.

### 2.1 Hard constraints

A recipe enters the feasible set only if **all** of the following hold:

| # | Constraint | Default |
|---|---|---|
| 1 | No ingredient on the family-level ingredient blocklist | — |
| 2 | `recipe.never_suggest` is false | — |
| 3 | Not cooked in the last N weeks (per `MealPlanItem` history) | N = 4 |
| 4 | Calories per serving within ±X% of `family_profile.calorie_target` (recipes with NULL `calories_per_serving` fail this constraint) | X = 20% |
| 5 | `prep_time_min + cook_time_min` ≤ T minutes | T = 45 |
| 6 | Estimated meal cost ≤ B dollars (current sale prices, scaled to servings) | B = $30 |

If the feasible set has fewer than 3 recipes, the planner returns a partial plan and an explanatory `error_message` rather than relaxing constraints silently. The user can rerun with relaxed bounds via API params.

### 2.2 Scoring signals

Each recipe in the feasible set gets an individual score:

```
score(recipe) =
    w_savings   * normalize(savings_dollars)
  + w_coverage  * sale_coverage_pct
  + w_pantry    * pantry_hit_pct
  + w_time      * time_bonus
  + w_recency   * recency_bonus
```

| Signal | Definition | Default weight |
|---|---|---|
| `savings_dollars` | sum over recipe ingredients of `(regular_price − current_price)` * (recipe_qty / package_qty) | **0.30** |
| `sale_coverage_pct` | fraction of recipe ingredients whose linked `grocery_item.is_on_sale = true` | **0.25** |
| `pantry_hit_pct` | fraction of recipe ingredients present in `home_pantry` | **0.10** |
| `time_bonus` | `1.0` if total_time ≤ 25 min; linearly decays to `0.0` at 45 min | **0.15** |
| `recency_bonus` | `1.0` if never cooked or > R weeks ago; linearly decays toward `0.0` at the N-week cutoff. R = 12 default, configurable | **0.20** |

Weights sum to 1.0. They live in `app/services/planner/config.py` so future tuning doesn't require schema changes. `normalize(savings_dollars)` is a min-max scaling across the feasible set (so a slow-sale week still produces meaningful ranking).

### 2.3 Set-selection diversity

Set score = `sum(individual_scores) − diversity_penalty`.

Diversity penalty applies pairwise across the 3-set:

```
diversity_penalty = sum over each pair (a, b):
    p_protein  if a.protein  == b.protein
  + p_cuisine  if a.cuisine  == b.cuisine
```

Defaults: `p_protein = 0.15`, `p_cuisine = 0.10`. A set of three chicken dinners pays a 0.45 protein penalty (3 pairs × 0.15) — usually enough to lose to a more varied set even if individual scores are slightly lower.

`recipe.protein` and `recipe.cuisine` are required (single-value, low-cardinality enums on the recipe model — see §4.2).

---

## 3. Ingredient ↔ grocery_item matching layer

The savings/coverage signals are only as good as the join from a recipe's ingredients to the scraped `grocery_item` rows. The matching is a separate, cacheable layer rather than fuzzy work done at every Phase 9 run.

### 3.1 Schema

**`ingredient`** (already exists, fleshed out):

| Column | Type | Notes |
|---|---|---|
| `id` | UUID PK | |
| `canonical_name` | CITEXT NOT NULL UNIQUE | e.g., "chicken thighs, boneless skinless" |
| `aliases` | TEXT[] NOT NULL DEFAULT '{}' | e.g., `{"chicken thigh", "BSL chicken thighs"}` |
| `category` | TEXT | freeform: "protein", "produce", "pantry", "dairy", … |
| `default_unit` | TEXT | "lb", "oz", "ea", "cup" |

**`ingredient_grocery_match`** (new):

| Column | Type | Notes |
|---|---|---|
| `id` | UUID PK | |
| `ingredient_id` | UUID FK → ingredient | |
| `grocery_item_id` | UUID FK → grocery_item | |
| `confidence` | FLOAT NOT NULL | rapidfuzz `WRatio(name, canonical_name + aliases) / 100` |
| `source` | ENUM('auto','manual') NOT NULL | |
| `updated_at` | TIMESTAMPTZ NOT NULL | |
| | | UNIQUE(ingredient_id, grocery_item_id) |

Composite index on `(ingredient_id, confidence DESC)` for fast top-3 lookup.

### 3.2 Match job

Runs after each successful scrape (hooked into `scraper_service._run_scrape_in_background`'s success path):

1. For each `Ingredient`, build a query string: `canonical_name` + each alias.
2. Fuzzy-rank against `grocery_item.name` (and `description` when present), filtered to `source = 'lucky_california'`.
3. Keep top 3 by confidence ≥ 0.75 threshold.
4. Upsert into `ingredient_grocery_match` with `source = 'auto'`. Skip rows whose existing `source = 'manual'` (admin overrides win).

Implementation: `rapidfuzz.process.extract` is fast enough for ~500 ingredients × ~10k grocery items in-memory; no need for Postgres trigram extensions yet.

### 3.3 Manual override

`POST /api/admin/ingredient-matches` and `DELETE …/{id}` let the admin pin or unpin matches when the auto-job picks the wrong row. Pinned rows have `source = 'manual'` and are never overwritten by the job.

### 3.4 Recipe-create assist

`POST /api/admin/recipes/resolve-ingredient` — body: `{"text": "1 lb chicken thighs"}` → returns top-3 `Ingredient` suggestions by fuzzy score plus a "create new" affordance with parsed `qty`/`unit`. Admin UI uses this to bind each freeform line to an `ingredient_id` at recipe-create time. No recipe is saved until every ingredient is bound.

---

## 4. Thin Phase 4 surface

Just enough recipe engine to feed Phase 9. Ingestion source (Spoonacular et al.) is explicitly deferred — see §6.

### 4.1 In scope

1. Flesh out `ingredient` columns per §3.1
2. New `ingredient_grocery_match` table per §3.1
3. New `family_ingredient_block(family_profile_id, ingredient_id)` table for hard constraint #1
4. Recipe model field additions (see §4.2)
5. Recipe CRUD endpoints (admin-gated)
6. Ingredient CRUD endpoints (admin-gated)
7. Recipe-create assist endpoint (§3.4)
8. Manual match override endpoints (§3.3)
9. Match job hook into scraper_service
10. Seed migration with 30 hand-curated recipes + their canonical ingredients (starter set proposed by implementer; user edits later via admin UI)

### 4.2 Recipe model fields (additions to existing `recipe` table)

| Column | Type | Notes |
|---|---|---|
| `never_suggest` | BOOLEAN NOT NULL DEFAULT false | hard constraint #2 |
| `prep_time_min` | INTEGER NOT NULL | for hard constraint #5 + scoring |
| `cook_time_min` | INTEGER NOT NULL | |
| `servings` | INTEGER NOT NULL | for cost scaling |
| `cuisine` | TEXT NOT NULL | e.g., "italian", "mexican", "thai" |
| `protein` | TEXT NOT NULL | e.g., "chicken", "beef", "fish", "vegetarian" |
| `tags` | TEXT[] NOT NULL DEFAULT '{}' | freeform |
| `calories_per_serving` | INTEGER | for hard constraint #4 |

`recipe.ingredients` JSONB shape (locked-in: option B from brainstorm):

```json
[
  {"ingredient_id": "<uuid>", "qty": 1.0, "unit": "lb"},
  {"ingredient_id": "<uuid>", "qty": 2.0, "unit": "tbsp", "notes": "minced"}
]
```

No freeform `text` field. Recipes that fail to resolve every ingredient cannot be saved.

### 4.3 Out of scope (deferred)

- Recipe ingestion source (Spoonacular / scrape NYT Cooking / manual-only). Tracked in §6.
- Per-member ingredient blocklists (household-level only for now)
- Recipe search/filter UI
- Recipe import from URL
- Photo upload (Phase 10)
- Tag taxonomy management (admins enter freeform; consolidate later if needed)

---

## 5. Phase 9 API surface

```
POST /api/admin/meal-plans/generate
  body: { week_start_date: "YYYY-MM-DD" }
  returns: 201 { meal_plan_id, items: [...], debug: { feasible_count, picks, rejected_summary } }

GET /api/meal-plans/{id}
  returns: full meal plan with items, scores, and savings breakdown

POST /api/admin/meal-plans/{id}/regenerate
  body: { exclude_recipe_ids?: [uuid], relax: { time_max?: 60, calorie_pct?: 30 } }
  returns: same as generate
```

Generation is synchronous (target < 1 s wall time). It does NOT email — that's Phase 5's orchestration concern. Phase 9 produces the plan; Phase 5 chains scrape → generate → email → vote → finalize.

---

## 6. Recipe source comparison (deferred decision)

Deferred from Phase 4 scope; lands here so the decision is in one place when we revisit.

| Source | Cost | License / TOS | Quality | API ergonomics | Notes |
|---|---|---|---|---|---|
| **Spoonacular** | Free tier: 150 req/day; paid $29–$249/mo for ingestion volumes | Commercial-friendly with attribution | High; structured ingredients, nutrition, images | REST, well-documented | The pragmatic default for an MVP. Free tier is enough for personal use after initial ingestion. |
| **Edamam Recipe Search API** | Free tier: 10 req/min, 10k/mo; paid from $9/mo | Personal-use free; commercial requires paid | High; nutrition focus | REST | Tighter rate limits but generous monthly cap. |
| **TheMealDB** | Free | Commercial-friendly | Modest (~300 recipes); ingredient strings unstructured | REST | Good for early dev/seeding; not enough variety long-term. |
| **MealieDB / Tandoor (self-hosted)** | Free (self-hosted) | Open source | Bring-your-own-recipes; you import | n/a | Useful as a frontend for manual recipe entry, not a source. |
| **Scrape NYT Cooking / Serious Eats / etc.** | Free in dollars | TOS-violating for most major sites; brittle | Excellent when it works | Custom per site | Recommend against. Same trap as the original Lucky scraper — fragile + legal gray zone. |
| **Open Recipes Project / Recipe1M+** | Free | Research datasets, mixed licensing | Variable; ingredient parsing required | Static dumps | Useful for bulk seeding once; not an ongoing feed. |
| **Manual entry only** | Free | n/a | Whatever you put in | n/a | Highest friction; best fit if family preferences narrow. |

**Recommendation when we revisit:** start with **TheMealDB** (free, commercial-OK) for dev seeding, plan to add **Spoonacular free tier** when we need real volume + nutrition data, and keep manual entry as the primary intake path indefinitely (a 4-person household doesn't need 50k recipes — it needs 100 good ones).

This decision can wait until the thin Phase 4 + Phase 9 are working end-to-end. The seed migration covers initial development.

---

## 7. Open questions / deferred

- **Tunable weights UI.** §2.2 weights live in code config. A future admin UI to tune them is reasonable but not urgent.
- **Cost estimation when an ingredient has no `ingredient_grocery_match` row.** Current plan: treat as zero savings, but include the ingredient in the `B = $30` cap using the previous-scrape average of similarly-categorized items. If even that isn't available, the recipe is rejected from the feasible set with `reason = "missing_cost_data"`. Documented; can revisit if too aggressive.
- **Per-member preferences.** Schema has `family_member`; current design only consults household-level data. A future signal could be "rejected by member X last time we tried this" → reduces score. Not in this phase.
- **Leftover modeling.** When we expand from 3 → more nights/week, recipes need a "scales well as leftover" flag. Out of scope.

---

## 8. Implementation order

1. **Phase 4-thin schema migrations** (ingredient flesh-out, ingredient_grocery_match, family_ingredient_block, recipe field additions)
2. **Phase 4-thin endpoints** (recipe CRUD, ingredient CRUD, resolve-ingredient assist, manual match override)
3. **Match job** wired into scraper_service success path
4. **Seed migration** with ~30 recipes + their ingredients
5. **Phase 9 planner module** (`app/services/planner/`) — filter, score, set-select, persist as MealPlan
6. **Phase 9 endpoints** (generate, regenerate)
7. **Tests**: unit for filter/score/set-select; integration for end-to-end generate against seeded data; the existing `requires_postgres` marker pattern

---

## 9. Out of scope for this spec

- Phase 5 orchestration (generate → email → vote → finalize chaining)
- Phase 6 SendGrid implementation
- Phase 8 feedback UI
- Phase 10 image strategy
- Phase 11 polish (APScheduler, variety analytics, budget tracking dashboards)
- Frontend admin UI for recipe management (separate spec; thin Phase 4 ships API only)

---

Last updated: 2026-05-05