Meal-Planner/docs/proposals/2026-05-23-feedback-driven-recipe-discovery.md

Proposal: Feedback-Driven Recipe Discovery

Date: 2026-05-23 Status: Draft — awaiting approval Author: Agent handoff

---

1. Problem Statement

The system currently has 30 hardcoded seed recipes and no automated mechanism to grow the recipe pool based on family feedback. Feedback (rating, feedback_text, denial_reason) is collected via the Phase 8 UI but never read by any downstream service. As the family uses the system, preferences evolve ("too spicy", "loved the Thai one", "boring chicken again"), but the planner cannot discover new recipes that better match these signals.

Current state:

• 30 recipes → planner selects 3/week → cycle repeats every 10 weeks
• Feedback is write-only: stored but not analyzed
• Manual recipe creation is the only growth path

Desired state:

• Feedback text/ratings automatically analyzed
• Low-rated recipes or categories trigger external discovery
• New recipes fetched, normalized, and added to the database
• Family sees increasing variety aligned with their tastes

---

2. Design Principles

1. Privacy-first: External API calls only when necessary; no bulk uploads of family data
2. Idempotent: Re-running discovery with the same feedback produces no duplicate recipes
3. Admin-gated: New recipes enter as needs_review status; admin approves before the planner sees them
4. Budget-aware: Free tiers prioritized; paid quotas tracked and logged
5. Graceful degradation: If external APIs fail, the system continues with existing recipes

---

3. Proposed Architecture

┌─────────────────────────────────────────────────────────────┐
│  FEEDBACK-DRIVEN RECIPE DISCOVERY                           │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │ Feedback     │    │ Feedback     │    │ Recipe       │  │
│  │ Collector    │───▶│ Analyzer     │───▶│ Discovery    │  │
│  │ (exists)     │    │ (new)        │    │ Service      │  │
│  └──────────────┘    └──────────────┘    └──────────────┘  │
│                              │                              │
│                              ▼                              │
│                       ┌──────────────┐                     │
│                       │ Recipe       │                     │
│                       │ Ingestion    │                     │
│                       │ Pipeline     │                     │
│                       └──────────────┘                     │
│                              │                              │
│                              ▼                              │
│                       ┌──────────────┐                     │
│                       │ Review Queue │                     │
│                       │ (new table)  │                     │
│                       └──────────────┘                     │
└─────────────────────────────────────────────────────────────┘

3.1 Feedback Analyzer

Periodic job (weekly, post-finalize) that reads all feedback from the past N weeks and produces a structured analysis.

Inputs:

• feedback table rows for the lookback window
• recipe metadata (tags, ingredients, cuisine)
• meal_plan_item history

Outputs (feedback_analysis JSONB on weekly_run):

{
  "negative_signals": [
    {"type": "avoid_tag': "dietary:vegetarian", "count": 3, "source": "member_A"},
    {"type": "avoid_ingredient": "mushroom", "count": 2, "source": "member_B"},
    {"type": "too_spicy", "count": 1, "source": "feedback_text"}
  ],
  "positive_signals": [
    {"type": "prefer_cuisine": "mexican", "avg_rating": 5.0, "count": 4},
    {"type": "prefer_protein": "shrimp", "avg_rating": 4.5, "count": 3}
  ],
  "discovery_queries": [
    "mexican shrimp",
    "mild thai chicken",
    "vegetarian pasta"
  ],
  "confidence": 0.82
}

3.2 Recipe Discovery Service

Queries external recipe APIs using the discovery queries from the Analyzer.

Primary source: Spoonacular (/recipes/complexSearch)

• Quota: 150 req/day free tier (1 query per recipe page of 10 results = 15 calls/day)
• Rate limit: built-in via API key

Fallback source: TheMealDB

• Free, no auth required for basic usage
• ~300 recipes total (limited variety)
• Good for backup or initial seeding

Query construction from feedback signals:

def build_queries(analysis: dict) -> list[str]:
    """Convert positive signals into API search queries."""
    queries = []
    
    # Combine preferred cuisines + proteins
    for cuisine in analysis.positive_signals["cuisines"]:
        for protein in analysis.positive_signals["proteins"]:
            queries.append(f"{cuisine} {protein}")
    
    # Weight by rating: high-rated specific recipes → "similar to X"
    for recipe in analysis.top_rated_recipes:
        queries.append(recipe.name)  # Spoonacular search by name
    
    # Apply negative filters: exclude avoided ingredients/tags
    for avoid in analysis.negative_signals:
        if avoid.type == "avoid_ingredient":
            queries.append(f"-{avoid.value}")  # Spoonacular supports excludeIngredients
    
    return queries[:5]  # Cap at 5 queries per run to stay within free quota

3.3 Recipe Ingestion Pipeline

Transforms external API responses into our recipe schema.

Spoonacular → Normalized Recipe:

@dataclass
class ExternalRecipe:
    name: str
    source_url: str
    image_url: str
    description: str
    prep_time_minutes: int
    cook_time_minutes: int
    servings: int
    cuisine_tags: list[str]
    dietary_tags: list[str]
    protein_type: str  # inferred from ingredients
    calories_per_serving: int
    ingredients: list[dict]  # {ingredient_id?, name, qty, unit}
    instructions: list[str]
    external_source: str  # "spoonacular"
    external_id: str  # spoonacular recipe ID

Challenges & Mitigations:

Challenge	Mitigation
External ingredients use different names than our canonical ingredient table	LLM-assisted mapping (reuse llm_matcher.py pattern) or fuzzy match + admin review
Units differ (cups vs grams)	Phase 9 spec §7 already flags unit conversion as follow-up; defer ingestion-side conversion
Conflicting cuisine/protein tags	Normalize via small mapping table ("mexican"↔"mexico", "chicken breast"→"chicken")
Duplicate external recipes	ON CONFLICT on external_source + external_id
Image licensing	Store as URLs only (per ARCHITECTURE.md §6.1), hotlink with attribution

3.4 Review Queue

New table to hold externally discovered recipes pending admin approval:

CREATE TABLE recipe_review_queue (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    recipe JSONB NOT NULL,  -- full normalized recipe payload
    external_source TEXT NOT NULL,
    external_id TEXT,
    discovery_reason TEXT,  -- e.g., "positive_signal: mexican shrimp"
    status TEXT NOT NULL CHECK (status IN ('pending', 'approved', 'rejected')),
    similarity_score FLOAT,  -- cosine/embedding similarity to existing recipes
    created_at TIMESTAMPTZ DEFAULT now(),
    reviewed_at TIMESTAMPTZ,
    reviewed_by UUID REFERENCES family_member(id)  -- who approved/rejected
);

CREATE INDEX idx_recipe_review_status ON recipe_review_queue(status);

Admin API:

• GET /api/admin/recipes/review-queue — list pending
• POST /api/admin/recipes/review-queue/{id}/approve — move to recipe table
• POST /api/admin/recipes/review-queue/{id}/reject — mark rejected (never re-discover)

Similarity gate: Before adding to queue, compute embedding similarity against existing recipes. Reject if >0.90 similar (exact or near-duplicate). This prevents re-adding recipes the family already has.

---

4. Data Flow

1. Weekly finalize step completes
   → triggers `analyze_feedback()`

2. Analyzer reads feedback from past 8 weeks
   → produces positive/negative signals + discovery_queries

3. If confidence > 0.5 and queries exist:
   → call Spoonacular complexSearch for each query
   → normalize responses
   → dedupe by (external_source, external_id)
   → similarity check vs existing recipes
   → insert into recipe_review_queue with status='pending'

4. Admin (Peter) reviews queue via UI
   → approves → recipe moved to `recipe` table, `is_manually_added=false`
   → rejects → stays in queue, marked rejected

5. Next week's planner generation
   → `db.query(Recipe).all()` now includes new approved recipes

---

5. API Changes

New Endpoints

Method	Path	Auth	Description
POST	/api/admin/recipes/discover	admin	Trigger manual discovery run
GET	/api/admin/recipes/review-queue	admin	List pending recipes
POST	/api/admin/recipes/review-queue/{id}/approve	admin	Approve a queued recipe
POST	/api/admin/recipes/review-queue/{id}/reject	admin	Reject a queued recipe
GET	/api/analytics/feedback	admin	Feedback analysis summary

Schema Changes

1. recipe table additions:

   • external_source TEXT — e.g., "spoonacular", "themealdb"
   • external_id TEXT — ID in the external system
   • discovery_reason TEXT — why this recipe was found

2. weekly_run table additions:

   • feedback_analysis JSONB — output of the analyzer

3. recipe_review_queue table: (new)

   • As defined in §3.4

---

6. Implementation Phases

Phase A — Feedback Analyzer (1–2 days)

• Implement FeedbackAnalyzer service
• Add feedback_analysis JSONB column to weekly_run
• Unit tests for analysis logic (mocked feedback data)
• Hook into step_finalize (orchestrator)

Phase B — Recipe Discovery + Ingestion (2–3 days)

• Implement RecipeDiscoveryService (Spoonacular client)
• Implement ingestion normalization pipeline
• Add recipe_review_queue table + CRUD API
• Add external_source/external_id/discovery_reason to recipe
• Rate-limiting and quota tracking
• Unit + integration tests (mocked Spoonacular responses)

Phase C — Review Queue UI (1–2 days)

• Admin page: list pending recipes with cards (image, title, ingredients)
• Approve/Reject actions
• Filter by status, discovery reason

Phase D — Orchestrator Integration (0.5 day)

• Weekly finalize step calls analyzer → triggers discovery if conditions met
• Configuration: enable/disable auto-discovery, quota limits

Phase E — Verification & Hardening (1 day)

• End-to-end test: submit feedback → run discovery → approve recipe → verify in planner
• Quota exhaustion handling
• Duplicate detection accuracy

---

7. Cost & Quota Analysis

Source	Free Tier	Proposed Usage	Cost
Spoonacular	150 req/day	5 queries × 1 call = 5/day	Free
TheMealDB	100% free	Backup only	Free
Ollama (ingredient mapping)	Self-hosted	~20 LLM calls per batch of recipes	Free

Spoonacular points math:

• complexSearch = 1 point + 0.01 per result
• 5 queries × 10 results = 5 + 0.5 = 5.5 points/day
• Weekly total: ~38.5 points / 150 free = well within limits
• If we need more, paid tiers start at $29/mo (5,000 points/day)

---

8. Risks & Mitigations

Risk	Severity	Mitigation
Spoonacular API changes/breaks schema	Medium	TheMealDB fallback; version-pinned client; mocked fixtures in CI
Ingredient mapping is inaccurate	Medium	LLM-assisted + admin review queue gate; never auto-add without review
Recipes not matched to our grocery catalog	Medium	Same as above — review queue lets admin see if ingredients are shoppable
Family data leaked to external APIs	Low	Only recipe search queries (keywords) leave the system; no family data
Duplicate recipes slip through	Low	Embedding similarity check + human review
Free quota exhausted	Low	5.5/day usage; explicit quota tracker; admin alert on 80%

---

9. Files to Create / Modify

New Files

backend/app/services/feedback_analyzer.py
backend/app/services/recipe_discovery.py
backend/app/api/recipe_discovery.py  # or merge into recipes.py
backend/alembic/versions/0010_feedback_analysis_and_review_queue.py
docs/specs/2026-05-23-recipe-discovery-pipeline.md

Modified Files

backend/app/services/orchestrator/steps.py       # hook analyzer into finalize
backend/app/models/__init__.py                   # new columns + RecipeReviewQueue
backend/app/schemas/__init__.py                  # response schemas
backend/app/api/admin.py                         # new admin endpoints
backend/app/api/recipes.py                       # external_source fields
backend/app/models/...                           # Recipe additions

---

10. Open Questions

1. Admin review cadence: Should new recipes email Peter, or is a dashboard badge enough?
2. Ollama endpoint: What's the exact model + base URL Peter uses? (Confirmed in .agent/context.md: Ollama Cloud)
3. Embedding model: Use Ollama embeddings or a lightweight local model (sentence-transformers)?
4. Feedback lookback: 8 weeks feels right for a 4-person family; confirm.
5. Auto-approval threshold: Should very high-confidence discoveries (spoonacular score > 90, all ingredients mapped) skip the review queue? (Recommended: no — always review.)

---

11. Next Step

Await approval on this proposal. Once approved, agent will create the implementation plan (.agent/plan.md) and execute Phase A.