# Phase 5 — Weekly Orchestration Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Chain scrape → generate → email → deadline → finalize into a Friday weekly cycle, driven by a dedicated APScheduler container, with per-step idempotency via a new `weekly_run` DB table. **Architecture:** A `scheduler` container (same Docker image as `backend`, different command) fires five APScheduler jobs every Friday in Pacific time. Each job calls `app.services.orchestrator.runner.run_step(step_name)`, which iterates all `family_profile` rows, upserts a `weekly_run` record, and delegates to a step function in `steps.py`. Steps are idempotent: a non-null timestamp column short-circuits re-execution. Manual override via `POST /api/admin/orchestrate/*` endpoints dispatches to the same functions via FastAPI `BackgroundTasks`. **Tech Stack:** APScheduler 3.10.4 (already in requirements.txt), existing FastAPI/SQLAlchemy/Alembic stack, existing `scraper_service.ScraperService`, `planner.generate.generate_meal_plan`, `approval.issue_token`, `email.get_email_backend`. --- ## File Map | Path | Action | Purpose | |---|---|---| | `backend/alembic/versions/0008_phase5_orchestration.py` | Create | `weekly_run` table + `family_profile.pending_approval_policy` column | | `backend/app/models/__init__.py` | Modify | Add `WeeklyRun` model + `FamilyProfile.pending_approval_policy` + relationship | | `backend/app/config.py` | Modify | Add `ADMIN_EMAIL` + `APP_BASE_URL` settings | | `backend/app/services/orchestrator/__init__.py` | Create | Public `run_step` / `run_week` re-exports | | `backend/app/services/orchestrator/alerts.py` | Create | `send_admin_alert(subject, body)` | | `backend/app/services/orchestrator/steps.py` | Create | Five step functions (scrape/generate/email/deadline/finalize) | | `backend/app/services/orchestrator/runner.py` | Create | Per-family loop + `run_step` / `run_week` | | `backend/app/api/admin.py` | Modify | Add `/orchestrate/*` endpoints | | `backend/app/scheduler/__init__.py` | Create | Empty package marker | | `backend/app/scheduler/__main__.py` | Create | APScheduler entry point | | `docker-compose.yml` | Modify | Add `scheduler` service | | `backend/tests/test_orchestrator.py` | Create | Unit tests for all orchestrator components | --- ## Task 1 — Alembic migration 0008 **Files:** - Create: `backend/alembic/versions/0008_phase5_orchestration.py` - [ ] **Step 1: Create the migration file** ```python # backend/alembic/versions/0008_phase5_orchestration.py """Phase 5 orchestration: weekly_run table + pending_approval_policy Revision ID: 0008 Revises: 0007 Create Date: 2026-05-07 """ from alembic import op import sqlalchemy as sa from sqlalchemy.dialects import postgresql revision = "0008" down_revision = "0007" branch_labels = None depends_on = None def upgrade() -> None: op.add_column( "family_profile", sa.Column( "pending_approval_policy", sa.String(10), nullable=False, server_default="approve", ), ) op.create_table( "weekly_run", sa.Column( "id", postgresql.UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()"), ), sa.Column( "family_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("family_profile.id", ondelete="CASCADE"), nullable=False, ), sa.Column("week_start_date", sa.Date, nullable=False), sa.Column("status", sa.String(20), nullable=False, server_default="pending"), sa.Column("scraped_at", sa.DateTime(timezone=True), nullable=True), sa.Column("generated_at", sa.DateTime(timezone=True), nullable=True), sa.Column("emailed_at", sa.DateTime(timezone=True), nullable=True), sa.Column("deadline_passed_at", sa.DateTime(timezone=True), nullable=True), sa.Column("finalized_at", sa.DateTime(timezone=True), nullable=True), sa.Column( "used_stale_data", sa.Boolean, nullable=False, server_default="false", ), sa.Column("error_step", sa.String(50), nullable=True), sa.Column("error_message", sa.Text, nullable=True), sa.Column( "created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()"), ), sa.UniqueConstraint( "family_id", "week_start_date", name="uq_weekly_run_family_week" ), ) def downgrade() -> None: op.drop_table("weekly_run") op.drop_column("family_profile", "pending_approval_policy") ``` - [ ] **Step 2: Apply migration and round-trip verify** ```bash docker compose --env-file .env.test exec backend alembic upgrade head ``` Expected: `Running upgrade 0007 -> 0008, Phase 5 orchestration` ```bash docker compose --env-file .env.test exec backend alembic downgrade -1 docker compose --env-file .env.test exec backend alembic upgrade head ``` Expected: both run without error. - [ ] **Step 3: Commit** ```bash git add backend/alembic/versions/0008_phase5_orchestration.py git commit -m "feat: migration 0008 — weekly_run table + pending_approval_policy" ``` --- ## Task 2 — WeeklyRun model + FamilyProfile update **Files:** - Modify: `backend/app/models/__init__.py` - [ ] **Step 1: Add `WeeklyRun` model and update `FamilyProfile`** In `backend/app/models/__init__.py`, add after the `IngredientGroceryMatch` class: ```python class WeeklyRun(Base): __tablename__ = "weekly_run" id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) family_id = Column( UUID(as_uuid=True), ForeignKey("family_profile.id", ondelete="CASCADE"), nullable=False, ) week_start_date = Column(Date, nullable=False) status = Column(String(20), nullable=False, default="pending") scraped_at = Column(DateTime(timezone=True)) generated_at = Column(DateTime(timezone=True)) emailed_at = Column(DateTime(timezone=True)) deadline_passed_at = Column(DateTime(timezone=True)) finalized_at = Column(DateTime(timezone=True)) used_stale_data = Column(Boolean, nullable=False, default=False) error_step = Column(String(50)) error_message = Column(Text) created_at = Column(DateTime(timezone=True), server_default=func.now()) __table_args__ = ( UniqueConstraint( "family_id", "week_start_date", name="uq_weekly_run_family_week" ), ) family_profile = relationship("FamilyProfile", back_populates="weekly_runs") ``` In `FamilyProfile`, add the column after `calorie_target` and add the relationship: ```python # column (after calorie_target line): pending_approval_policy = Column(String(10), nullable=False, server_default="approve") # relationship (after the existing relationships): weekly_runs = relationship( "WeeklyRun", back_populates="family_profile", cascade="all, delete-orphan" ) ``` - [ ] **Step 2: Verify import is clean** ```bash docker compose --env-file .env.test exec backend python -c "from app.models import WeeklyRun, FamilyProfile; print('ok')" ``` Expected: `ok` - [ ] **Step 3: Commit** ```bash git add backend/app/models/__init__.py git commit -m "feat: WeeklyRun model + FamilyProfile.pending_approval_policy" ``` --- ## Task 3 — Settings: ADMIN_EMAIL + APP_BASE_URL **Files:** - Modify: `backend/app/config.py` - Modify: `docker-compose.yml` - [ ] **Step 1: Add settings** In `backend/app/config.py`, add after `SESSION_PASSWORD`: ```python ADMIN_EMAIL: str = "" APP_BASE_URL: str = "http://localhost" ``` - [ ] **Step 2: Add env vars to docker-compose.yml backend service** In `docker-compose.yml`, under `backend.environment`, append: ```yaml - ADMIN_EMAIL=${ADMIN_EMAIL:-} - APP_BASE_URL=${APP_BASE_URL:-http://localhost} ``` - [ ] **Step 3: Verify** ```bash docker compose --env-file .env.test exec backend python -c "from app.config import settings; print(settings.ADMIN_EMAIL, settings.APP_BASE_URL)" ``` Expected: ` http://localhost` (empty ADMIN_EMAIL, default APP_BASE_URL) - [ ] **Step 4: Commit** ```bash git add backend/app/config.py docker-compose.yml git commit -m "feat: add ADMIN_EMAIL and APP_BASE_URL settings" ``` --- ## Task 4 — alerts.py **Files:** - Create: `backend/app/services/orchestrator/alerts.py` - Test: `backend/tests/test_orchestrator.py` (initial section) - [ ] **Step 1: Write failing tests** Create `backend/tests/test_orchestrator.py`: ```python """Tests for the Phase 5 weekly orchestration.""" import pytest from datetime import date, datetime, timezone from uuid import uuid4 import pytest pytestmark = pytest.mark.requires_postgres # --------------------------------------------------------------------------- # Fixtures # --------------------------------------------------------------------------- WEEK = date(2026, 5, 8) # a Friday @pytest.fixture() def family(db): from app.models import FamilyProfile fp = FamilyProfile( id=uuid4(), name="Test Family", household_size=4, adult_count=2, child_count=2, pending_approval_policy="approve", ) db.add(fp) db.flush() return fp @pytest.fixture() def member(db, family): from app.models import FamilyMember, FamilyMemberRole m = FamilyMember( id=uuid4(), family_profile_id=family.id, name="Alice", email="alice@example.com", role=FamilyMemberRole.ADULT, ) db.add(m) db.flush() return m @pytest.fixture() def weekly_run(db, family): from app.models import WeeklyRun r = WeeklyRun(family_id=family.id, week_start_date=WEEK, status="pending") db.add(r) db.flush() return r @pytest.fixture() def weekly_run_scraped(db, family): from app.models import WeeklyRun r = WeeklyRun( family_id=family.id, week_start_date=WEEK, status="running", scraped_at=datetime.now(timezone.utc), ) db.add(r) db.flush() return r @pytest.fixture() def weekly_run_generated(db, family): from app.models import WeeklyRun r = WeeklyRun( family_id=family.id, week_start_date=WEEK, status="running", scraped_at=datetime.now(timezone.utc), generated_at=datetime.now(timezone.utc), ) db.add(r) db.flush() return r @pytest.fixture() def meal_plan(db, family): from app.models import MealPlan, MealPlanStatus mp = MealPlan( id=uuid4(), family_profile_id=family.id, week_start_date=WEEK, status=MealPlanStatus.PENDING_APPROVAL, ) db.add(mp) db.flush() return mp @pytest.fixture() def pending_item(db, meal_plan): from app.models import MealPlanItem, MealPlanItemStatus, MealType, Recipe recipe = Recipe( id=uuid4(), name="Test Pasta", servings=4, ingredients=[{"name": "pasta", "qty": "400", "unit": "g"}], instructions=["boil water", "cook pasta"], ) db.add(recipe) db.flush() item = MealPlanItem( id=uuid4(), meal_plan_id=meal_plan.id, recipe_id=recipe.id, day_of_week=5, meal_type=MealType.DINNER, approval_status=MealPlanItemStatus.PENDING, ) db.add(item) db.flush() return item @pytest.fixture() def weekly_run_emailed(db, family): from app.models import WeeklyRun r = WeeklyRun( family_id=family.id, week_start_date=WEEK, status="running", scraped_at=datetime.now(timezone.utc), generated_at=datetime.now(timezone.utc), emailed_at=datetime.now(timezone.utc), ) db.add(r) db.flush() return r # --------------------------------------------------------------------------- # alerts.py tests # --------------------------------------------------------------------------- def test_send_admin_alert_noop_when_email_unset(monkeypatch): monkeypatch.setattr( "app.services.orchestrator.alerts.settings", type("S", (), {"ADMIN_EMAIL": ""})(), ) from app.services.orchestrator.alerts import send_admin_alert send_admin_alert("test subject", "test body") # must not raise def test_send_admin_alert_calls_backend(monkeypatch): sent = [] class FakeBackend: def send(self, **kwargs): sent.append(kwargs) monkeypatch.setattr( "app.services.orchestrator.alerts.settings", type("S", (), {"ADMIN_EMAIL": "admin@example.com"})(), ) monkeypatch.setattr( "app.services.orchestrator.alerts.get_email_backend", lambda: FakeBackend(), ) from app.services.orchestrator.alerts import send_admin_alert send_admin_alert("boom", "details here") assert len(sent) == 1 assert "[MealPlanner ALERT] boom" in sent[0]["subject"] assert sent[0]["to"] == "admin@example.com" ``` - [ ] **Step 2: Run to verify they fail** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py::test_send_admin_alert_noop_when_email_unset \ tests/test_orchestrator.py::test_send_admin_alert_calls_backend -v ``` Expected: `ImportError` or `ModuleNotFoundError` — `app.services.orchestrator.alerts` does not exist yet. - [ ] **Step 3: Create the package and alerts.py** ```bash mkdir -p backend/app/services/orchestrator touch backend/app/services/orchestrator/__init__.py ``` Create `backend/app/services/orchestrator/alerts.py`: ```python import logging from app.config import settings from app.services.email import get_email_backend logger = logging.getLogger(__name__) def send_admin_alert(subject: str, body: str) -> None: if not settings.ADMIN_EMAIL: logger.warning("ADMIN_EMAIL not set; dropping alert: %s", subject) return try: get_email_backend().send( to=settings.ADMIN_EMAIL, subject=f"[MealPlanner ALERT] {subject}", html=f"
{body}",
text=body,
)
except Exception:
logger.exception("Failed to send admin alert: %s", subject)
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest tests/test_orchestrator.py::test_send_admin_alert_noop_when_email_unset \
tests/test_orchestrator.py::test_send_admin_alert_calls_backend -v
```
Expected: `2 passed`
- [ ] **Step 5: Commit**
```bash
git add backend/app/services/orchestrator/ backend/tests/test_orchestrator.py
git commit -m "feat: orchestrator package + alerts.send_admin_alert"
```
---
## Task 5 — step_scrape
**Files:**
- Modify: `backend/app/services/orchestrator/steps.py` (create)
- Modify: `backend/tests/test_orchestrator.py` (append)
- [ ] **Step 1: Write failing tests** — append to `test_orchestrator.py`:
```python
# ---------------------------------------------------------------------------
# step_scrape tests
# ---------------------------------------------------------------------------
def test_step_scrape_idempotent(db, weekly_run_scraped):
original_ts = weekly_run_scraped.scraped_at
from app.services.orchestrator.steps import step_scrape
step_scrape(weekly_run_scraped, db)
assert weekly_run_scraped.scraped_at == original_ts
def test_step_scrape_success(db, weekly_run, monkeypatch):
monkeypatch.setattr(
"app.services.orchestrator.steps.ScraperService.run_scrape",
lambda self, **kw: {"status": "success", "items_scraped": 42},
)
from app.services.orchestrator.steps import step_scrape
step_scrape(weekly_run, db)
assert weekly_run.scraped_at is not None
assert weekly_run.used_stale_data is False
def test_step_scrape_both_fail_marks_stale(db, weekly_run, monkeypatch):
monkeypatch.setattr(
"app.services.orchestrator.steps.ScraperService.run_scrape",
lambda self, **kw: {"status": "failed", "error": "timeout"},
)
monkeypatch.setattr(
"app.services.orchestrator.steps.send_admin_alert",
lambda subject, body: None,
)
from app.services.orchestrator.steps import step_scrape
step_scrape(weekly_run, db)
assert weekly_run.used_stale_data is True
assert weekly_run.scraped_at is not None
```
- [ ] **Step 2: Run to verify they fail**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest tests/test_orchestrator.py -k "step_scrape" -v
```
Expected: `ImportError` — `app.services.orchestrator.steps` does not exist yet.
- [ ] **Step 3: Create steps.py with step_scrape**
Create `backend/app/services/orchestrator/steps.py`:
```python
"""
Orchestrator step functions.
Each step receives a WeeklyRun row and an open Session. Steps are idempotent:
a non-null timestamp column causes an immediate return. Steps commit their own
changes before returning.
All service imports are at module level so tests can monkeypatch via
`app.services.orchestrator.steps.Note: Grocery prices in this plan may be a few " "days old — the Friday scrape failed and stale data was used.
" ) backend = get_email_backend() for member in members: item_html_parts = [] for item in plan.items: if item.approval_status != MealPlanItemStatus.PENDING: continue token = issue_token(item.id, member.id) vote_url = ( f"{settings.APP_BASE_URL}/api/meals/vote/{item.id}?token={token}" ) recipe_name = item.recipe.name if item.recipe else str(item.recipe_id) item_html_parts.append( f'Hi {member.name}, please vote on this week's meals by Fri 17:00 PT:
" f"Silence = approved. Any denial removes that meal.
" ) backend.send( to=member.email, subject=f"Meal plan for week of {run.week_start_date}", html=html, ) logger.info("step_email: sent to %s", member.email) run.emailed_at = datetime.now(timezone.utc) db.commit() ``` - [ ] **Step 4: Run to verify pass** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "step_email" -v ``` Expected: `3 passed` - [ ] **Step 5: Commit** ```bash git add backend/app/services/orchestrator/steps.py backend/tests/test_orchestrator.py git commit -m "feat: orchestrator step_email with per-member vote links + stale banner" ``` --- ## Task 8 — step_deadline **Files:** - Modify: `backend/app/services/orchestrator/steps.py` - Modify: `backend/tests/test_orchestrator.py` - [ ] **Step 1: Write failing tests** — append to `test_orchestrator.py`: ```python # --------------------------------------------------------------------------- # step_deadline tests # --------------------------------------------------------------------------- def test_step_deadline_idempotent(db, weekly_run_emailed): weekly_run_emailed.deadline_passed_at = datetime.now(timezone.utc) db.flush() original_ts = weekly_run_emailed.deadline_passed_at from app.services.orchestrator.steps import step_deadline step_deadline(weekly_run_emailed, db) assert weekly_run_emailed.deadline_passed_at == original_ts def test_step_deadline_resolves_pending_to_approved(db, weekly_run_emailed, meal_plan, pending_item, family): from app.models import MealPlanItemStatus from app.services.orchestrator.steps import step_deadline family.pending_approval_policy = "approve" db.flush() step_deadline(weekly_run_emailed, db) db.refresh(pending_item) assert pending_item.approval_status == MealPlanItemStatus.APPROVED assert weekly_run_emailed.deadline_passed_at is not None def test_step_deadline_resolves_pending_to_denied(db, weekly_run_emailed, meal_plan, pending_item, family): from app.models import MealPlanItemStatus from app.services.orchestrator.steps import step_deadline family.pending_approval_policy = "deny" db.flush() step_deadline(weekly_run_emailed, db) db.refresh(pending_item) assert pending_item.approval_status == MealPlanItemStatus.DENIED ``` - [ ] **Step 2: Run to verify they fail** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "step_deadline" -v ``` Expected: `AttributeError` — `step_deadline` not defined. - [ ] **Step 3: Add step_deadline to steps.py** Append to `backend/app/services/orchestrator/steps.py`: ```python def step_deadline(run: "WeeklyRun", db: "Session") -> None: if run.deadline_passed_at is not None: logger.info("step_deadline: already done for %s", run.week_start_date) return plan = ( db.query(MealPlan) .filter( MealPlan.family_profile_id == run.family_id, MealPlan.week_start_date == run.week_start_date, ) .first() ) if plan is None: run.deadline_passed_at = datetime.now(timezone.utc) db.commit() return family = db.query(FamilyProfile).filter(FamilyProfile.id == run.family_id).first() policy = getattr(family, "pending_approval_policy", "approve") if family else "approve" resolved = 0 for item in plan.items: if item.approval_status == MealPlanItemStatus.PENDING: item.approval_status = ( MealPlanItemStatus.APPROVED if policy == "approve" else MealPlanItemStatus.DENIED ) resolved += 1 run.deadline_passed_at = datetime.now(timezone.utc) db.commit() logger.info( "step_deadline: resolved %d pending items with policy=%s", resolved, policy ) ``` - [ ] **Step 4: Run to verify pass** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "step_deadline" -v ``` Expected: `3 passed` - [ ] **Step 5: Commit** ```bash git add backend/app/services/orchestrator/steps.py backend/tests/test_orchestrator.py git commit -m "feat: orchestrator step_deadline with configurable pending policy" ``` --- ## Task 9 — step_finalize **Files:** - Modify: `backend/app/services/orchestrator/steps.py` - Modify: `backend/tests/test_orchestrator.py` - [ ] **Step 1: Write failing tests** — append to `test_orchestrator.py`: ```python # --------------------------------------------------------------------------- # step_finalize tests # --------------------------------------------------------------------------- @pytest.fixture() def weekly_run_deadline_passed(db, family): from app.models import WeeklyRun r = WeeklyRun( family_id=family.id, week_start_date=WEEK, status="running", scraped_at=datetime.now(timezone.utc), generated_at=datetime.now(timezone.utc), emailed_at=datetime.now(timezone.utc), deadline_passed_at=datetime.now(timezone.utc), ) db.add(r) db.flush() return r @pytest.fixture() def approved_item(db, meal_plan): from app.models import MealPlanItem, MealPlanItemStatus, MealType, Recipe recipe = Recipe( id=uuid4(), name="Approved Chicken", servings=4, ingredients=[{"name": "chicken", "qty": "500", "unit": "g"}], instructions=["cook it"], ) db.add(recipe) db.flush() item = MealPlanItem( id=uuid4(), meal_plan_id=meal_plan.id, recipe_id=recipe.id, day_of_week=6, meal_type=MealType.DINNER, approval_status=MealPlanItemStatus.APPROVED, ) db.add(item) db.flush() return item def test_step_finalize_idempotent(db, weekly_run_deadline_passed): weekly_run_deadline_passed.finalized_at = datetime.now(timezone.utc) db.flush() original_ts = weekly_run_deadline_passed.finalized_at from app.services.orchestrator.steps import step_finalize step_finalize(weekly_run_deadline_passed, db) assert weekly_run_deadline_passed.finalized_at == original_ts def test_step_finalize_sends_shopping_list(db, weekly_run_deadline_passed, meal_plan, approved_item, member, monkeypatch): sent = [] class FakeBackend: def send(self, **kwargs): sent.append(kwargs) monkeypatch.setattr( "app.services.orchestrator.steps.get_email_backend", lambda: FakeBackend(), ) from app.services.orchestrator.steps import step_finalize step_finalize(weekly_run_deadline_passed, db) assert weekly_run_deadline_passed.finalized_at is not None assert weekly_run_deadline_passed.status == "completed" assert len(sent) == 1 assert "Shopping list" in sent[0]["subject"] assert "chicken" in sent[0]["html"].lower() def test_step_finalize_no_approved_sends_empty_message(db, weekly_run_deadline_passed, meal_plan, pending_item, member, monkeypatch): sent = [] class FakeBackend: def send(self, **kwargs): sent.append(kwargs) monkeypatch.setattr( "app.services.orchestrator.steps.get_email_backend", lambda: FakeBackend(), ) from app.services.orchestrator.steps import step_finalize step_finalize(weekly_run_deadline_passed, db) assert weekly_run_deadline_passed.finalized_at is not None assert "No meals were approved" in sent[0]["html"] ``` - [ ] **Step 2: Run to verify they fail** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "step_finalize" -v ``` Expected: `AttributeError` — `step_finalize` not defined. - [ ] **Step 3: Add step_finalize to steps.py** Append to `backend/app/services/orchestrator/steps.py`: ```python def step_finalize(run: "WeeklyRun", db: "Session") -> None: if run.finalized_at is not None: logger.info("step_finalize: already done for %s", run.week_start_date) return plan = ( db.query(MealPlan) .filter( MealPlan.family_profile_id == run.family_id, MealPlan.week_start_date == run.week_start_date, ) .first() ) approved_items = [ item for item in (plan.items if plan else []) if item.approval_status == MealPlanItemStatus.APPROVED ] members = ( db.query(FamilyMember) .filter( FamilyMember.family_profile_id == run.family_id, FamilyMember.email.isnot(None), ) .all() ) if approved_items: rows_html = "".join( f"{len(approved_items)} meal(s) approved.
" f"| Recipe | Ingredient | Qty |
|---|
No meals were approved for week of {run.week_start_date}.
" backend = get_email_backend() for member in members: backend.send( to=member.email, subject=f"Shopping list — week of {run.week_start_date}", html=html, ) logger.info("step_finalize: shopping list sent to %s", member.email) run.finalized_at = datetime.now(timezone.utc) run.status = "completed" db.commit() logger.info("step_finalize: done, %d approved meals", len(approved_items)) ``` - [ ] **Step 4: Run to verify pass** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "step_finalize" -v ``` Expected: `4 passed` - [ ] **Step 5: Commit** ```bash git add backend/app/services/orchestrator/steps.py backend/tests/test_orchestrator.py git commit -m "feat: orchestrator step_finalize builds and emails shopping list" ``` --- ## Task 10 — runner.py + orchestrator/__init__.py **Files:** - Create: `backend/app/services/orchestrator/runner.py` - Modify: `backend/app/services/orchestrator/__init__.py` - Modify: `backend/tests/test_orchestrator.py` - [ ] **Step 1: Write failing tests** — append to `test_orchestrator.py`: ```python # --------------------------------------------------------------------------- # runner tests # --------------------------------------------------------------------------- def test_run_step_upserts_weekly_run(db, family, monkeypatch): monkeypatch.setattr( "app.services.orchestrator.steps.ScraperService.run_scrape", lambda self, **kw: {"status": "success", "items_scraped": 0}, ) # Override SessionLocal to return our test db session monkeypatch.setattr( "app.services.orchestrator.runner.SessionLocal", lambda: db, ) from app.services.orchestrator.runner import run_step run_step("scrape", WEEK) from app.models import WeeklyRun run = db.query(WeeklyRun).filter(WeeklyRun.family_id == family.id).first() assert run is not None assert run.scraped_at is not None def test_run_step_invalid_raises(monkeypatch): monkeypatch.setattr( "app.services.orchestrator.runner.SessionLocal", lambda: None, ) from app.services.orchestrator.runner import run_step with pytest.raises(ValueError, match="Unknown step"): run_step("bogus", WEEK) def test_run_week_calls_all_steps(monkeypatch): called = [] def fake_run_step(step, week): called.append(step) monkeypatch.setattr( "app.services.orchestrator.runner.run_step", fake_run_step, ) from app.services.orchestrator.runner import run_week run_week(WEEK) assert called == ["scrape", "generate", "email", "deadline", "finalize"] ``` - [ ] **Step 2: Run to verify they fail** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "run_step or run_week" -v ``` Expected: `ImportError` — `app.services.orchestrator.runner` not found. - [ ] **Step 3: Create runner.py** Create `backend/app/services/orchestrator/runner.py`: ```python """ Orchestrator runner. run_step(step_name, week_start_date) — execute one step for all families run_week(week_start_date) — execute all steps in sequence """ from __future__ import annotations import logging from datetime import date, timedelta from typing import Optional from app.database import SessionLocal logger = logging.getLogger(__name__) STEPS = ("scrape", "generate", "email", "deadline", "finalize") def _current_week_start() -> date: """Return the most recent Friday (today if today is Friday).""" today = date.today() days_since_friday = (today.weekday() - 4) % 7 return today - timedelta(days=days_since_friday) def _get_or_create_run(db, family_id, week_start_date: date): from app.models import WeeklyRun run = ( db.query(WeeklyRun) .filter( WeeklyRun.family_id == family_id, WeeklyRun.week_start_date == week_start_date, ) .first() ) if run is None: run = WeeklyRun( family_id=family_id, week_start_date=week_start_date, status="pending", ) db.add(run) db.commit() db.refresh(run) return run def run_step(step_name: str, week_start_date: Optional[date] = None) -> None: from app.models import FamilyProfile from app.services.orchestrator import steps as s step_fns = { "scrape": s.step_scrape, "generate": s.step_generate, "email": s.step_email, "deadline": s.step_deadline, "finalize": s.step_finalize, } if step_name not in step_fns: raise ValueError(f"Unknown step: {step_name!r}. Valid: {list(step_fns)}") week = week_start_date or _current_week_start() db = SessionLocal() try: families = db.query(FamilyProfile).all() if not families: logger.warning("run_step(%s): no family profiles, skipping", step_name) return for family in families: run = _get_or_create_run(db, family.id, week) try: step_fns[step_name](run, db) except Exception: logger.exception( "run_step(%s) failed for family %s", step_name, family.id ) finally: db.close() def run_week(week_start_date: Optional[date] = None) -> None: week = week_start_date or _current_week_start() for step in STEPS: run_step(step, week) ``` - [ ] **Step 4: Update orchestrator/__init__.py** ```python # backend/app/services/orchestrator/__init__.py from app.services.orchestrator.runner import run_step, run_week __all__ = ["run_step", "run_week"] ``` - [ ] **Step 5: Run to verify pass** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "run_step or run_week" -v ``` Expected: `3 passed` - [ ] **Step 6: Run full orchestrator test suite** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -v ``` Expected: all tests pass (target: ~20 tests). - [ ] **Step 7: Commit** ```bash git add backend/app/services/orchestrator/ backend/tests/test_orchestrator.py git commit -m "feat: orchestrator runner — run_step / run_week per-family loop" ``` --- ## Task 11 — Admin orchestrate endpoints **Files:** - Modify: `backend/app/api/admin.py` - Modify: `backend/tests/test_orchestrator.py` - [ ] **Step 1: Write failing tests** — append to `test_orchestrator.py`: ```python # --------------------------------------------------------------------------- # Admin endpoint tests # --------------------------------------------------------------------------- @pytest.fixture() def admin_client(db): from fastapi.testclient import TestClient from app.main import app from app.database import get_db app.dependency_overrides[get_db] = lambda: db client = TestClient(app) yield client app.dependency_overrides.clear() ADMIN_HEADERS = {"Authorization": "Bearer test-admin-token"} def test_orchestrate_status_returns_list(admin_client, family, monkeypatch): from app.config import settings monkeypatch.setattr(settings, "ADMIN_TOKEN", "test-admin-token") resp = admin_client.get("/api/admin/orchestrate/status", headers=ADMIN_HEADERS) assert resp.status_code == 200 assert "runs" in resp.json() def test_orchestrate_step_invalid_returns_400(admin_client, monkeypatch): from app.config import settings monkeypatch.setattr(settings, "ADMIN_TOKEN", "test-admin-token") resp = admin_client.post( "/api/admin/orchestrate/bogus", headers=ADMIN_HEADERS ) assert resp.status_code == 400 def test_orchestrate_run_week_returns_202(admin_client, monkeypatch): from app.config import settings monkeypatch.setattr(settings, "ADMIN_TOKEN", "test-admin-token") monkeypatch.setattr( "app.api.admin.run_week", lambda week=None: None, ) resp = admin_client.post( "/api/admin/orchestrate/run-week", headers=ADMIN_HEADERS ) assert resp.status_code == 202 ``` - [ ] **Step 2: Run to verify they fail** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "orchestrate" -v ``` Expected: `404` or import errors — endpoints not wired yet. - [ ] **Step 3: Add orchestrate endpoints to admin.py** Append to `backend/app/api/admin.py` (add the new imports at the top with existing imports, add routes at the bottom): New imports to add at the top: ```python from datetime import date as DateType from app.services.orchestrator import run_step, run_week from app.models import WeeklyRun ``` New routes at the bottom: ```python _VALID_STEPS = {"scrape", "generate", "email", "deadline", "finalize"} @router.get("/orchestrate/status") def orchestrate_status(limit: int = 10, db: Session = Depends(get_db)): runs = ( db.query(WeeklyRun) .order_by(WeeklyRun.week_start_date.desc()) .limit(limit) .all() ) return { "runs": [ { "id": str(r.id), "family_id": str(r.family_id), "week_start_date": r.week_start_date.isoformat(), "status": r.status, "scraped_at": r.scraped_at.isoformat() if r.scraped_at else None, "generated_at": r.generated_at.isoformat() if r.generated_at else None, "emailed_at": r.emailed_at.isoformat() if r.emailed_at else None, "deadline_passed_at": r.deadline_passed_at.isoformat() if r.deadline_passed_at else None, "finalized_at": r.finalized_at.isoformat() if r.finalized_at else None, "used_stale_data": r.used_stale_data, "error_step": r.error_step, "error_message": r.error_message, } for r in runs ] } @router.post("/orchestrate/run-week", status_code=202) def orchestrate_run_week( background_tasks: BackgroundTasks, week_start: Optional[str] = None, ): week = DateType.fromisoformat(week_start) if week_start else None background_tasks.add_task(run_week, week) return {"status": "queued", "week_start": str(week or "current")} @router.post("/orchestrate/{step}", status_code=202) def orchestrate_step( step: str, background_tasks: BackgroundTasks, week_start: Optional[str] = None, ): if step not in _VALID_STEPS: raise HTTPException( status_code=400, detail=f"Unknown step {step!r}. Valid: {sorted(_VALID_STEPS)}", ) week = DateType.fromisoformat(week_start) if week_start else None background_tasks.add_task(run_step, step, week) return {"status": "queued", "step": step, "week_start": str(week or "current")} ``` **Important:** The `GET /orchestrate/status` route must be registered **before** `POST /orchestrate/{step}` in the file, and both before any general catch-all. FastAPI matches routes in order — `run-week` in the POST path must also come before `{step}`. The order above (status GET, run-week POST, {step} POST) is correct. - [ ] **Step 4: Run to verify pass** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest tests/test_orchestrator.py -k "orchestrate" -v ``` Expected: `3 passed` - [ ] **Step 5: Run full test suite to check for regressions** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest -q tests/ ``` Expected: all prior tests pass + new orchestrator tests pass. - [ ] **Step 6: Commit** ```bash git add backend/app/api/admin.py backend/tests/test_orchestrator.py git commit -m "feat: admin orchestrate endpoints — run-week, per-step, status" ``` --- ## Task 12 — Scheduler entry point **Files:** - Create: `backend/app/scheduler/__init__.py` - Create: `backend/app/scheduler/__main__.py` - [ ] **Step 1: Create package and entry point** ```bash mkdir -p backend/app/scheduler touch backend/app/scheduler/__init__.py ``` Create `backend/app/scheduler/__main__.py`: ```python """ APScheduler entry point for the scheduler container. Run: python -m app.scheduler Weekly cadence (America/Los_Angeles): Fri 02:00 — scrape (Lucky weekend prices live by ~midnight) Fri 05:00 — generate (fresh grocery data) Fri 06:00 — email (per-voter approval links) Fri 17:00 — deadline (pending items resolved per family policy) Fri 18:00 — finalize (shopping-list email) """ import logging from apscheduler.schedulers.blocking import BlockingScheduler from apscheduler.triggers.cron import CronTrigger from app.services.orchestrator.runner import run_step logging.basicConfig( level="INFO", format="%(asctime)s %(levelname)s %(name)s %(message)s", ) logger = logging.getLogger(__name__) TZ = "America/Los_Angeles" scheduler = BlockingScheduler(timezone=TZ) scheduler.add_job( lambda: run_step("scrape"), CronTrigger(day_of_week="fri", hour=2, minute=0, timezone=TZ), id="weekly_scrape", name="Weekly scrape (Fri 02:00 PT)", ) scheduler.add_job( lambda: run_step("generate"), CronTrigger(day_of_week="fri", hour=5, minute=0, timezone=TZ), id="weekly_generate", name="Weekly generate (Fri 05:00 PT)", ) scheduler.add_job( lambda: run_step("email"), CronTrigger(day_of_week="fri", hour=6, minute=0, timezone=TZ), id="weekly_email", name="Weekly email (Fri 06:00 PT)", ) scheduler.add_job( lambda: run_step("deadline"), CronTrigger(day_of_week="fri", hour=17, minute=0, timezone=TZ), id="weekly_deadline", name="Weekly deadline (Fri 17:00 PT)", ) scheduler.add_job( lambda: run_step("finalize"), CronTrigger(day_of_week="fri", hour=18, minute=0, timezone=TZ), id="weekly_finalize", name="Weekly finalize (Fri 18:00 PT)", ) if __name__ == "__main__": logger.info("Starting MealPlanner scheduler (tz=%s)", TZ) for job in scheduler.get_jobs(): logger.info(" Registered: %s", job.name) try: scheduler.start() except (KeyboardInterrupt, SystemExit): logger.info("Scheduler stopped") ``` - [ ] **Step 2: Smoke-test the import** ```bash docker compose --env-file .env.test exec backend python -c " import app.scheduler.__main__ as m jobs = m.scheduler.get_jobs() print([j.id for j in jobs]) " ``` Expected: `['weekly_scrape', 'weekly_generate', 'weekly_email', 'weekly_deadline', 'weekly_finalize']` - [ ] **Step 3: Commit** ```bash git add backend/app/scheduler/ git commit -m "feat: APScheduler entry point — 5-job weekly cadence (Pacific)" ``` --- ## Task 13 — Docker Compose: scheduler service **Files:** - Modify: `docker-compose.yml` - [ ] **Step 1: Add scheduler service** In `docker-compose.yml`, add after the `backend` service block (before `frontend`): ```yaml scheduler: build: context: ./backend dockerfile: Dockerfile command: python -m app.scheduler environment: - DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner - SENDGRID_API_KEY=${SENDGRID_API_KEY} - LUCKY_CA_URL=${LUCKY_CA_URL:-https://luckysupermarkets.com} - LUCKY_STORE_ID=${LUCKY_STORE_ID:-757} - SWIFTLY_API_BASE=${SWIFTLY_API_BASE:-https://prod.swiftlyapi.net} - SWIFTLY_CATEGORIES_URL=${SWIFTLY_CATEGORIES_URL:-https://luckysupermarkets.com/categories} - AI_IMAGE_ENABLED=${AI_IMAGE_ENABLED:-false} - LOG_LEVEL=${LOG_LEVEL:-INFO} - SECRET_KEY=${SECRET_KEY} - ADMIN_TOKEN=${ADMIN_TOKEN} - SESSION_PASSWORD=${SESSION_PASSWORD} - EMAIL_BACKEND=${EMAIL_BACKEND:-console} - ADMIN_EMAIL=${ADMIN_EMAIL:-} - APP_BASE_URL=${APP_BASE_URL:-http://localhost} depends_on: db: condition: service_healthy restart: unless-stopped ``` Also add `ADMIN_EMAIL` and `APP_BASE_URL` to the existing `backend` service environment block (they're needed there too for the admin endpoints). - [ ] **Step 2: Build and start scheduler container** ```bash docker compose --env-file .env.test build scheduler docker compose --env-file .env.test up -d scheduler ``` Expected: container starts without error. - [ ] **Step 3: Verify scheduler logs show registered jobs** ```bash docker compose --env-file .env.test logs scheduler ``` Expected output contains: ``` Registered: Weekly scrape (Fri 02:00 PT) Registered: Weekly generate (Fri 05:00 PT) Registered: Weekly email (Fri 06:00 PT) Registered: Weekly deadline (Fri 17:00 PT) Registered: Weekly finalize (Fri 18:00 PT) ``` - [ ] **Step 4: Commit** ```bash git add docker-compose.yml git commit -m "feat: add scheduler container to docker-compose" ``` --- ## Final verification - [ ] **Run full test suite** ```bash docker compose --env-file .env.test exec \ -e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \ backend pytest -q tests/ ``` Expected: all tests pass (92 prior + ~20 new orchestrator tests). - [ ] **Smoke-test manual step trigger via API** ```bash # Seed a family_profile row first if DB is fresh curl -s -X POST http://localhost:8000/api/admin/orchestrate/run-week \ -H "Authorization: Bearer ${ADMIN_TOKEN}" | jq . ``` Expected: `{"status": "queued", "week_start": "current"}` ```bash curl -s http://localhost:8000/api/admin/orchestrate/status \ -H "Authorization: Bearer ${ADMIN_TOKEN}" | jq .runs[0] ``` Expected: JSON with `week_start_date`, step timestamps, and `status`. - [ ] **Final commit** ```bash git commit --allow-empty -m "feat: Phase 5 weekly orchestration complete" ``` --- ## Environment variables added | Var | Default | Purpose | |---|---|---| | `ADMIN_EMAIL` | `""` | Alert destination; empty = alerts dropped silently | | `APP_BASE_URL` | `http://localhost` | Base URL for vote links in emails |