admin/Meal-Planner
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refresh ORIENTATION + HANDOFF after r1+r2 recovery and r3-0

Browse Source 
Updates ORIENTATION.md and HANDOFF.md to reflect actual state as of
commit 8e89f79: phases 1/2/3/7 complete (verified, not just claimed),
phases 4/5/6/8/9/10/11 not started. Documents the auth model, the
bootstrap login hatch, the SWIFTLY_BEARER_TOKEN expiration handling,
the Swiftly JSON API ingestion path that replaced Playwright, the
canonicalized API paths, and the verification commands to reproduce
the 31/31 pytest gate locally and in CI.

HANDOFF.md is intended for fresh agents and points at .agent/
phase-summaries for the per-phase write-ups. Surfaces 10 caveats and
traps the next agent will hit if they skim, and recommends Phase 9
(meal-planner generation algorithm) as the next move.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
admin
2026-05-05 14:41:19 -07:00
co-authored by Claude Opus 4.7
parent 8e89f793d5
commit c953111395
2 changed files with 305 additions and 515 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ORIENTATION.md
+99 -220
View File
@@ -1,273 +1,152 @@
# Meal Planner - Orientation Guide
# Meal Planner — Orientation
This document is the **first stop** for any agent resuming work on this project after context compaction. Read it before doing anything else.
First stop for any agent resuming work. Read this, then `docs/HANDOFF.md` for the deep dive.
---
## Project Overview
## What this project is
**MealPlanner** is a self-hosted meal planning system for a family of 4 (2 adults, 2 children) that integrates with Lucky California grocery store to source ingredients from weekly sales, sends meal proposals via email, generates shopping lists, and learns from feedback.
Self-hosted meal planning for one family of 4. Pulls weekly grocery prices from Lucky California (San Pablo, store 757) via the Swiftly JSON API, generates a 7-day meal plan, emails per-member approval links, builds a shopping list grouped by aisle. Replaces meal-kit subscriptions (Blue Apron / Sunbasket / etc.) which marked up ingredients ~3× and produced repetitive meals.
### Key Problem Being Solved
The family has been using meal kit services (Blue Apron → EveryPlate → HungryRoot → Sunbasket) which suffer from:
- 3x ingredient markup in cost
- Repetitive meals and sauces
- Forcing app login to manage meal selection
- No home pantry integration
### Current Status
**Phase**: Phase 2 (Database & Models) IN PROGRESS. Migrations created, schemas implemented, API endpoints implemented.
**Adversarial Review Completed**: 2026-05-04
- All consensus blockers (§1.1 - §1.8) addressed
- All high-risk gaps (§2.1 - §2.8) addressed
- Key fixes applied:
- Recipe-ingredient: JSONB only (removed join table)
- Family member: Added `family_member` table for per-voter tracking
- Approval flow: Redesigned with confirmation page + POST + per-voter tokens + TTL
- Auth: VPN-only for admin endpoints, session-based for family web UI
- Schema: ENUMs, CHECKs, CITEXT for ingredients, ISO day_of_week (1=Mon)
- Lucky URL: Fixed to `https://luckysupermarkets.com`
- Docker: Hardened (no direct port exposure to backend/frontend)
- Alembic: Configured with migration policy
**Phase 2 Progress** (2026-05-04):
- [x] Initial Alembic migration (0001_initial_migration.py)
- [x] Seed data migration (0002_seed_data.py)
- [x] Pydantic schemas for all models
- [x] /api/profile endpoints (CRUD, family members)
- [x] /api/recipes endpoints (CRUD, ingredients)
- [x] /api/meals endpoints (meal plans, voting, approval tokens)
- [x] /api/pantry endpoints (CRUD)
- [x] /api/shopping-list endpoints (aggregation, print)
- [x] /api/admin endpoints (scrape trigger, logs, stats)
**Phase 3 Progress** (2026-05-04):
- [x] Base scraper with rate limiting, retries, session management
- [x] LuckyCaliforniaScraper with BeautifulSoup + Playwright
- [x] ScraperService to save scraped items to grocery_item table
- [x] /api/admin/scrape endpoint connected to ScraperService
**Phase 7 Progress** (2026-05-04):
- [x] Types: FamilyProfile, Recipe, MealPlan, Ingredient, ShoppingList, etc.
- [x] API client: mealPlannerApi wrapper for all endpoints
- [x] Dashboard: weekly meal plan grid view with day/meal columns
- [x] Pantry page: add/remove items with ingredient selection
- [x] MealDetail page: recipe display with ingredients and instructions
- [x] ShoppingList page: grouped by aisle with sale highlighting
Constraint that drives the design: 3 of 4 members do not like mushrooms; family is calorie/budget conscious; no allergies. Must work for non-technical wife + 2 kids; technical owner self-hosts.
---
## Architecture Summary
## Architecture (current)
```
User (email) ──► SendGrid ───────────────────────────────┐
User (web) ───► React UI ──► nginx ──► FastAPI ──────────┼──► PostgreSQL
│ │
└──► Lucky CA scraper ──┘
(luckysupermarkets.com)
email ─► SendGrid (R3-C, not yet wired) ──┐
web ─► nginx :80/:443 ─► React/Vite ────┼─► FastAPI ──► PostgreSQL 15
│ │
│ └─► Swiftly JSON API
│ (prod.swiftlyapi.net)
│
└─► /api/admin/scrape (BackgroundTasks)
```
### Services (Docker Compose)
- **backend**: FastAPI Python app (port 8000, internal only)
- **frontend**: React + Tailwind (port 3000, internal only via nginx)
- **db**: PostgreSQL 15 (internal only)
- **nginx**: Reverse proxy with SSL (ports 80/443)
### Tech Stack
- Backend: Python 3.11, FastAPI, SQLAlchemy 2.0, Alembic
- Frontend: React 18, TypeScript, Tailwind CSS, React Query, Vite
- Database: PostgreSQL 15 with ENUMs and CITEXT
- Scraping: Playwright, BeautifulSoup
- Email: SendGrid
- Hosting: Docker Compose, nginx
- **backend** (FastAPI 0.109, SQLAlchemy 2.0, Alembic) — internal only, `expose: 8000`
- **frontend** (React 18 + TS + Vite + Tailwind) — internal only via nginx
- **db** (Postgres 15-alpine) — internal only
- **nginx** — sole external entry, ports 80/443
---
## Family Profile
## Phase status (2026-05-05)
### Household
- 2 adults, 2 children
- 3 of 4 members do NOT like mushrooms
- No allergies
- Goals: Calorie, budget, and health conscious; tasty but not expensive
| # | Phase | Status |
|---|---|---|
| 1 | Infra (Docker, FastAPI, React, nginx, Postgres) | **Complete** |
| 2 | DB & models (Alembic, Pydantic schemas, API endpoints) | **Complete** (real, verified) |
| 3 | Lucky California ingestion (Swiftly JSON API) | **Complete** — 17 categories, ~10k products live |
| 4 | Recipe engine (CRUD, search, tagging, never-suggest filter) | Not started |
| 5 | Meal planner orchestration (generate → email → vote → finalize) | Not started |
| 6 | SendGrid email integration (proposal/reminder/confirmation) | Stub only — `app/services/email.py::SendGridEmailBackend` raises NotImplementedError |
| 7 | Web UI core (Dashboard / Meal Detail / Pantry / Shopping List) | **Complete** (no auth UI yet) |
| 8 | Web UI feedback portal | Not started |
| 9 | **Meal-planner generation algorithm** | Not started — *the core of the project* |
| 10 | Image strategy (scraped + AI fallback) | Not started |
| 11 | Polish (variety analysis, budget tracking, APScheduler) | Not started |
### Family Members
| Member | Role | Mushroom Preference |
|--------|------|-------------------|
| Adult 1 | Adult | Does NOT like mushrooms |
| Adult 2 | Adult | Likes mushrooms |
| Child 1 | Child | Does NOT like mushrooms |
| Child 2 | Child | OK with mushrooms |
### Approval Workflow (REDESIGNED)
1. System generates 7-day meal plan (Sunday)
2. Email sent to all adults with meals, images, approval page links
3. Email link → confirmation page (GET, not auto-approve)
4. Adult clicks Approve/Deny → POST with reason
5. Per-member token (single-use, 72h TTL)
6. Majority approve → meal confirmed; any deny → swap
7. After approval → shopping list generated
Verification gate (R1+R2 + R3-0): 31/31 pytest green; alembic upgrade→downgrade→upgrade clean; frontend `npm run build` clean; live scrape persists 9,960 grocery_item rows in 36 s; email approval round-trip (approve/deny/single-use) verified end-to-end.
---
## Database Schema Highlights
## Auth model (R1-B+D, locked in)
### Core Tables
- `family_profile` - Household configuration with household_size CHECK
- `family_member` - Individual family members with email, role, mushroom preference
- `recipe` - Recipes with JSONB ingredients (not join table)
- `ingredient` - Master list with name_lower (CITEXT for case-insensitive matching)
- `meal_plan` - Weekly plan with ISO day_of_week (1=Mon, 7=Sun)
- `meal_plan_item` - Individual meal with approval status
- `meal_plan_vote` - Per-member votes (one vote per member per meal)
- `approval_token` - Single-use tokens with TTL and status tracking
- `home_pantry` - Family's on-hand ingredients
- `feedback` - Ratings, denial reasons, never-suggest flags
- `grocery_item` - Scraped Lucky California items with FK to ingredient
- `scrape_log` / `email_log` - Operation history
### Key Relationships
- `family_profile` 1:N `family_member`
- `family_profile` 1:N `meal_plan`
- `family_member` 1:N `meal_plan_vote` (per-voter tracking)
- `recipe` 1:N `meal_plan_item`
- `meal_plan` 1:N `meal_plan_item`
- `meal_plan_item` 1:N `meal_plan_vote`
- `meal_plan_item` 1:N `approval_token`
- `grocery_item` → `ingredient` (FK)
- Bearer token `ADMIN_TOKEN` for every `/api/admin/*` route.
- Signed-cookie session (itsdangerous, key=`SECRET_KEY`) for all NON-GET routes on profile/pantry/recipes/meals/shopping-list. GET reads stay open inside the trusted network.
- `/api/auth/login` accepts `{password}` matching `SESSION_PASSWORD`. Sets the cookie.
- `/api/meals/vote/{item_id}?token=…` keeps its per-voter token flow; not session-gated.
- **Bootstrap hatch**: when no `family_profile` row exists, login signs the literal string `"bootstrap"` instead of a UUID. First-run convenience only — replace with a real setup gate before any non-trusted exposure.
---
## Implementation Phases
## Database schema highlights
| Phase | Description | Status |
|-------|-------------|--------|
| 1 | Infrastructure (Docker, PostgreSQL, FastAPI, React, nginx) | **Complete** |
| 2 | Database & Models (Alembic migrations, Pydantic schemas, API endpoints) | **Complete** |
| 3 | Lucky California Scraper (BeautifulSoup + Playwright, ScraperService) | **Complete** |
| 4 | Recipe Engine (CRUD, tagging, search) | Not Started |
| 5 | Meal Planner Engine (generation algorithm, substitutions) | Not Started |
| 6 | SendGrid Email Integration | Not Started |
| 7 | Web UI - Core (Dashboard, Meal Detail, Pantry, Shopping List) | **Complete** |
| 8 | Web UI - Feedback (Feedback Portal, Learning) | Not Started |
| 9 | Meal Planner Generation Algorithm | Not Started |
| 10 | Image Strategy (scraped + AI fallback) | Not Started |
| 11 | Polish & Future (variety analysis, budget tracking) | Not Started |
Core tables: `family_profile`, `family_member`, `recipe`, `ingredient`, `meal_plan`, `meal_plan_item`, `meal_plan_vote`, `approval_token`, `home_pantry`, `feedback`, `grocery_item`, `scrape_log`, `email_log`.
Conventions: UUID PKs everywhere, `TIMESTAMPTZ`, Postgres ENUMs (with `values_callable=lambda obj: [e.value for e in obj]` on every SQLEnum — name-mode silently breaks otherwise), ISO day-of-week (1=Mon).
Key relationships: `family_profile`→`family_member`; `family_member`→`meal_plan_vote` (per-voter); `recipe.ingredients` JSONB (no recipe-ingredient join table); `grocery_item.(source, external_id)` is the upsert key for scrape ingestion.
Migrations applied: 0001 initial, 0002 seed (idempotent via `ON CONFLICT DO NOTHING`), 0003 grocery_item.description, 0004 family_profile.calorie_target, 0005 grocery_item.external_id + source + composite index.
Full schema: `docs/database-schema.md`.
---
## Environment Variables
## Environment variables (verified)
```bash
# Database
DATABASE_URL=postgresql://mealplanner:password@db:5432/mealplanner
POSTGRES_PASSWORD=secure_password
# SendGrid
SENDGRID_API_KEY=SG.xxx
# Family
FAMILY_EMAIL_1=user@example.com
FAMILY_EMAIL_2=spouse@example.com
# Scraping
LUCKY_CA_URL=https://luckysupermarkets.com
# AI Images (optional)
AI_IMAGE_ENABLED=false
POSTGRES_PASSWORD=...
DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner
# Auth
SECRET_KEY=change-me-in-production
SECRET_KEY=... # signs session cookies + approval tokens
ADMIN_TOKEN=... # bearer for /api/admin/*
SESSION_PASSWORD=... # family-shared password for /api/auth/login
# Email
EMAIL_BACKEND=console # 'console' (default) or 'sendgrid'
SENDGRID_API_KEY=... # only when EMAIL_BACKEND=sendgrid (R3-C)
# Lucky / Swiftly
LUCKY_STORE_ID=757 # Lucky California — San Pablo
SWIFTLY_API_BASE=https://prod.swiftlyapi.net
SWIFTLY_CATEGORIES_URL=https://luckysupermarkets.com/categories
SWIFTLY_BEARER_TOKEN=... # Firebase anon JWT, expires hourly
# Other
LUCKY_CA_URL=https://luckysupermarkets.com
AI_IMAGE_ENABLED=false
LOG_LEVEL=INFO
```
`.env.example` carries a literal expiring bearer token — rotate before any real run. On expiry, the next scrape's `ScrapeLog.error_message` reads `SWIFTLY_BEARER_TOKEN expired — request a fresh token from the user (capture from luckysupermarkets.com network tab on a /search/api/v1 request)`. Fix: capture a fresh `Authorization: Bearer …` from devtools, update env, restart backend, re-trigger.
---
## Next Steps
### IMMEDIATE: Verify skeleton with verification matrix
## Verification commands
```bash
docker compose config
docker compose build backend
docker compose build frontend
docker compose up -d db
docker compose run --rm backend python -c "from app.main import app; print(app.title)"
docker compose run --rm backend alembic upgrade head
docker compose run --rm frontend npm run build
# Full local stack
docker compose --env-file .env.test up -d db backend
docker compose --env-file .env.test exec backend alembic upgrade head
docker compose --env-file .env.test exec -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner backend pytest -q tests/
# Frontend
cd frontend && npm ci && npm run build
# CI: .github/workflows/ci.yml runs both jobs on push/PR.
```
If any of these fail, fix before proceeding.
### After Verification
1. Phase 4: Recipe Engine (CRUD, tagging, search)
2. Phase 5: Meal Planner Engine (generation algorithm, substitutions)
3. Phase 6: SendGrid Email Integration (meal proposal emails with approval links)
A `.env.test` template lives in the repo root (gitignored) for local stack runs. Pytest uses `TEST_DATABASE_URL`; alembic uses `DATABASE_URL`.
---
## Current Git State
## Conventions
```bash
git log --oneline -10
27bca0e docs: update ORIENTATION.md phase table
08e196b feat: implement frontend Web UI pages
933a0cc feat: implement Lucky California scraper with Playwright + BeautifulSoup
c735d21 feat: implement Phase 2 - Alembic migrations, Pydantic schemas, and real API endpoints
e8706d3 docs: final ORIENTATION update
624b516 docs: update ORIENTATION.md for Phase 1 complete
1328ec3 feat: add Phase 1 infrastructure skeleton
0c5b0aa docs: add complete project documentation
```
- Python: Black + isort. TypeScript: Prettier + ESLint.
- Conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`).
- API paths: no trailing slash, no `/list`/`/planned` suffixes.
- Tests: pytest; `requires_postgres` marker auto-skips locally without `TEST_DATABASE_URL`.
- Migrations: Alembic only. Never `Base.metadata.create_all()` at runtime.
- Background work: FastAPI `BackgroundTasks` (current). APScheduler with `--workers 1` planned for Phase 11.
---
## Important Conventions
## Where to look
### Code Style
- Python: Black formatter, isort for imports
- TypeScript: Prettier, ESLint
- No comments unless explaining non-obvious logic
### Git Commits
- Conventional commits: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`
- One logical change per commit where possible
### API Design
- RESTful endpoints with proper HTTP methods and status codes
- Pydantic schemas for request/response validation
- Email approval links use GET → confirmation page → POST
- Per-voter tokens, not shared household tokens
### Database
- Always use UUIDs for primary keys
- Timestamps with timezone (TIMESTAMPTZ)
- Use ENUMs for status fields (not loose VARCHAR)
- Use CITEXT or lowercase-on-write for name matching
### Testing
- Write unit tests for services (pytest)
- Integration tests for API endpoints
- Frontend: component tests with React Testing Library
- `docs/HANDOFF.md` — comprehensive handoff for fresh agents (start here for non-trivial work).
- `docs/SPEC.md` — product spec.
- `docs/ARCHITECTURE.md` — system design.
- `docs/database-schema.md` — full DDL reference.
- `docs/implementation-plan.md` — original phased plan.
- `docs/RUNNING.md` — local dev workflow.
- `.agent/plan.md`, `.agent/context.md`, `.agent/phase-summaries/` — recovery decisions and per-phase summaries from the R1+R2+R3-0 work.
- `Review/reviewconcensus.md` — the adversarial review that drove the recovery.
---
## Known Issues / Open Questions
1. **Lucky California scraping**: Feasibility not yet spiked. URL is `luckysupermarkets.com`.
2. **AI image generation**: Config flag to enable/disable. Disabled by default.
3. **WhatsApp integration**: Planned for future via Twilio. Out of scope for MVP.
4. **Planned scheduler**: APScheduler with `--workers 1` to avoid duplicate fires.
---
## Contact / Context
- **Primary user**: Peter (tech-savvy, hosts the system)
- **Wife**: Non-technical, will use web UI and email
- **Children**: 2, eating habits vary (one OK with mushrooms)
Last updated: 2026-05-04 (all adversarial review fixes committed)
Last updated: 2026-05-05 — after R1+R2 stabilization + R3-0 Swiftly ingestion (commit `8e89f79`).