# Phase 6 — SendGrid Email Integration Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the `ConsoleEmailBackend` stub with a live SendGrid backend, add a pre-deadline reminder step, and fix two hygiene issues from Phase 5.
**Architecture:** `SendGridEmailBackend.send()` is wired using the `sendgrid` Python library already present in `requirements.txt`. Sender identity (`from_email`, `reply_to`) comes from two new Settings fields. A new `step_reminder` orchestrator step (idempotent via `reminded_at` column, added by migration 0009) fires at Fri 16:00 PT and emails only members who have un-voted PENDING items. The `STEPS` tuple in `runner.py` grows from 5 to 6; admin and scheduler get matching updates.
**Tech Stack:** Python 3.11, FastAPI 0.109, SQLAlchemy 2.0, Alembic 1.13, sendgrid 6.12, APScheduler 3.10, pytest 7.4. All tests in `backend/tests/`.
---
## File map
| File | Action |
|---|---|
| `backend/alembic/versions/0009_phase6_reminded_at.py` | **Create** — add `reminded_at TIMESTAMPTZ NULL` to `weekly_run` |
| `backend/app/models/__init__.py` | **Modify** — add `reminded_at` column to `WeeklyRun` |
| `backend/app/services/email.py` | **Modify** — wire `SendGridEmailBackend`; keep `ConsoleEmailBackend` unchanged |
| `backend/app/config.py` | **Modify** — add `SENDGRID_FROM_EMAIL` and `SENDGRID_REPLY_TO` |
| `backend/tests/test_email_backend.py` | **Create** — 2 unit tests for `SendGridEmailBackend` |
| `backend/app/services/orchestrator/steps.py` | **Modify** — add `step_reminder`; add `html` + `MealPlanVote` imports; html-escape (#P5-a); fix getattr (#P5-b) |
| `backend/app/services/orchestrator/runner.py` | **Modify** — add `"reminder"` to `STEPS` tuple and `step_fns` dict |
| `backend/app/api/admin.py` | **Modify** — add `"reminder"` to `_VALID_STEPS` |
| `backend/app/scheduler/__main__.py` | **Modify** — add 6th job Fri 16:00 PT; update docstring |
| `backend/tests/test_orchestrator.py` | **Modify** — add `weekly_run_reminded` fixture + 4 `step_reminder` tests |
---
## Task 1: Migration 0009 — add `reminded_at` to `weekly_run`
**Files:**
- Create: `backend/alembic/versions/0009_phase6_reminded_at.py`
- Modify: `backend/app/models/__init__.py`
### Context
`WeeklyRun` is defined at the bottom of `backend/app/models/__init__.py`. It currently has `scraped_at`, `generated_at`, `emailed_at`, `deadline_passed_at`, `finalized_at`. The last migration is `0008`. Alembic is run inside the Docker container via `docker compose exec backend alembic upgrade head`.
- [ ] **Step 1: Create the migration file**
Create `backend/alembic/versions/0009_phase6_reminded_at.py`:
```python
"""Phase 6: add reminded_at to weekly_run
Revision ID: 0009
Revises: 0008
Create Date: 2026-05-07
"""
from alembic import op
import sqlalchemy as sa
revision = "0009"
down_revision = "0008"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"weekly_run",
sa.Column("reminded_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
op.drop_column("weekly_run", "reminded_at")
```
- [ ] **Step 2: Add `reminded_at` to the `WeeklyRun` model**
Open `backend/app/models/__init__.py`. Find the `WeeklyRun` class (near the bottom of the file). After the `emailed_at` column line, add the new column. The existing block looks like:
```python
scraped_at = Column(DateTime(timezone=True))
generated_at = Column(DateTime(timezone=True))
emailed_at = Column(DateTime(timezone=True))
deadline_passed_at = Column(DateTime(timezone=True))
finalized_at = Column(DateTime(timezone=True))
```
Change it to:
```python
scraped_at = Column(DateTime(timezone=True))
generated_at = Column(DateTime(timezone=True))
emailed_at = Column(DateTime(timezone=True))
reminded_at = Column(DateTime(timezone=True))
deadline_passed_at = Column(DateTime(timezone=True))
finalized_at = Column(DateTime(timezone=True))
```
- [ ] **Step 3: Apply migration inside Docker and verify round-trip**
```bash
docker compose --env-file .env.test up -d db backend
docker compose --env-file .env.test exec backend alembic upgrade head
# Expected: INFO [alembic.runtime.migration] Running upgrade 0008 -> 0009 ...
docker compose --env-file .env.test exec backend alembic downgrade -1
# Expected: INFO [alembic.runtime.migration] Running downgrade 0009 -> 0008 ...
docker compose --env-file .env.test exec backend alembic upgrade head
# Expected: Running upgrade 0008 -> 0009 again, clean
```
- [ ] **Step 4: Verify existing tests still pass**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/
```
Expected: `115 passed`
- [ ] **Step 5: Commit**
```bash
git add backend/alembic/versions/0009_phase6_reminded_at.py \
backend/app/models/__init__.py
git commit -m "feat: migration 0009 — add reminded_at to weekly_run"
```
---
## Task 2: Wire `SendGridEmailBackend`
**Files:**
- Modify: `backend/app/services/email.py`
- Modify: `backend/app/config.py`
- Create: `backend/tests/test_email_backend.py`
### Context
`backend/app/services/email.py` already has an `EmailBackend` Protocol, `ConsoleEmailBackend`, and a `SendGridEmailBackend` stub that raises `NotImplementedError`. The `sendgrid==6.12.0` package is already in `requirements.txt`. The `settings` object in `email.py` comes from `from app.config import settings` at module level — tests monkeypatch it at `app.services.email.SendGridAPIClient`.
`backend/app/config.py` has `SENDGRID_API_KEY: Optional[str] = None` already. Two new fields are needed.
- [ ] **Step 1: Write the failing tests**
Create `backend/tests/test_email_backend.py`:
```python
"""Unit tests for SendGridEmailBackend — no Postgres needed."""
import pytest
def test_sendgrid_send_calls_api(monkeypatch):
calls = []
class FakeResponse:
status_code = 202
body = b""
class FakeClient:
def send(self, msg):
calls.append(msg)
return FakeResponse()
monkeypatch.setattr("app.services.email.SendGridAPIClient", lambda _key: FakeClient())
from app.services.email import SendGridEmailBackend
backend = SendGridEmailBackend()
backend.send(to="to@example.com", subject="Hello", html="
Hi
")
assert len(calls) == 1
def test_sendgrid_raises_on_4xx(monkeypatch):
class FakeResponse:
status_code = 400
body = b"bad request"
class FakeClient:
def send(self, msg):
return FakeResponse()
monkeypatch.setattr("app.services.email.SendGridAPIClient", lambda _key: FakeClient())
from app.services.email import SendGridEmailBackend
backend = SendGridEmailBackend()
with pytest.raises(RuntimeError, match="400"):
backend.send(to="to@example.com", subject="Hello", html="Hi
")
```
- [ ] **Step 2: Run tests to confirm they fail**
```bash
docker compose --env-file .env.test exec backend pytest -q tests/test_email_backend.py -v
```
Expected: `FAILED` — `NotImplementedError: Wire SendGrid in R3-C`
- [ ] **Step 3: Add `SENDGRID_FROM_EMAIL` and `SENDGRID_REPLY_TO` to Settings**
Open `backend/app/config.py`. After `SENDGRID_API_KEY`, add:
```python
SENDGRID_FROM_EMAIL: str = "peter@research.bike"
SENDGRID_REPLY_TO: str = "peter@research.bike"
```
The full relevant block becomes:
```python
SENDGRID_API_KEY: Optional[str] = None
SENDGRID_FROM_EMAIL: str = "peter@research.bike"
SENDGRID_REPLY_TO: str = "peter@research.bike"
EMAIL_BACKEND: str = "console"
```
- [ ] **Step 4: Wire `SendGridEmailBackend.send()`**
Open `backend/app/services/email.py`. Make two changes:
**4a — Add module-level sendgrid imports** after the existing stdlib imports and before `from app.config import settings`. The full import block at the top of `email.py` should look like:
```python
from __future__ import annotations
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional, Protocol
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail, ReplyTo
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail, ReplyTo
from app.config import settings
```
Remove the `import os` line if present (unused). Keep everything else the same.
**4b — Replace the `SendGridEmailBackend` class** (find the stub class that raises `NotImplementedError` and replace the entire class body):
```python
class SendGridEmailBackend:
def __init__(self) -> None:
self._client = SendGridAPIClient(settings.SENDGRID_API_KEY)
self._from_email = settings.SENDGRID_FROM_EMAIL
self._reply_to = settings.SENDGRID_REPLY_TO
def send(
self,
to: str,
subject: str,
html: str,
text: Optional[str] = None,
) -> None:
message = Mail(
from_email=self._from_email,
to_emails=to,
subject=subject,
html_content=html,
plain_text_content=text,
)
message.reply_to = ReplyTo(self._reply_to)
response = self._client.send(message)
if response.status_code >= 400:
raise RuntimeError(
f"SendGrid error {response.status_code}: {response.body}"
)
```
`SendGridAPIClient` and `Mail`/`ReplyTo` are module-level names, so the test's `monkeypatch.setattr("app.services.email.SendGridAPIClient", ...)` patches them correctly.
- [ ] **Step 5: Run tests to confirm they pass**
```bash
docker compose --env-file .env.test exec backend pytest -q tests/test_email_backend.py -v
```
Expected: `2 passed`
- [ ] **Step 6: Run full suite to confirm no regressions**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/
```
Expected: `117 passed` (115 existing + 2 new)
- [ ] **Step 7: Commit**
```bash
git add backend/app/services/email.py \
backend/app/config.py \
backend/tests/test_email_backend.py
git commit -m "feat: wire SendGridEmailBackend with from_email/reply_to settings"
```
---
## Task 3: `step_reminder` + tests
**Files:**
- Modify: `backend/app/services/orchestrator/steps.py`
- Modify: `backend/tests/test_orchestrator.py`
### Context
`steps.py` has module-level imports from `app.models` and other services. Tests monkeypatch via `app.services.orchestrator.steps.`. The `MealPlanVote` model is in `app.models` and has columns `meal_plan_item_id` (FK), `family_member_id` (FK), `vote` (Boolean). `WeeklyRun` now has `reminded_at` (added in Task 1).
The `weekly_run_emailed` fixture (in `test_orchestrator.py`) has `scraped_at`, `generated_at`, and `emailed_at` all set — it's the right base for reminder tests.
- [ ] **Step 1: Write the failing tests**
In `backend/tests/test_orchestrator.py`, add the following. First, add a new fixture after the existing `weekly_run_emailed` fixture:
```python
@pytest.fixture()
def weekly_run_reminded(db, family):
from app.models import WeeklyRun
r = WeeklyRun(
family_id=family.id,
week_start_date=WEEK,
status="running",
scraped_at=datetime.now(timezone.utc),
generated_at=datetime.now(timezone.utc),
emailed_at=datetime.now(timezone.utc),
reminded_at=datetime.now(timezone.utc),
)
db.add(r)
db.flush()
return r
```
Then add the four tests (these can go after the existing `step_email` tests):
```python
# ── step_reminder ──────────────────────────────────────────────────────────
def test_step_reminder_idempotent(db, weekly_run_reminded):
original_ts = weekly_run_reminded.reminded_at
from app.services.orchestrator.steps import step_reminder
step_reminder(weekly_run_reminded, db)
assert weekly_run_reminded.reminded_at == original_ts
def test_step_reminder_skips_when_not_emailed(db, weekly_run_generated, monkeypatch):
sent = []
class FakeBackend:
def send(self, **kwargs):
sent.append(kwargs)
monkeypatch.setattr(
"app.services.orchestrator.steps.get_email_backend",
lambda: FakeBackend(),
)
from app.services.orchestrator.steps import step_reminder
step_reminder(weekly_run_generated, db)
# emailed_at is None on weekly_run_generated — should be a no-op
assert len(sent) == 0
assert weekly_run_generated.reminded_at is None
def test_step_reminder_sends_to_non_voter(
db, weekly_run_emailed, meal_plan, pending_item, member, monkeypatch
):
sent = []
class FakeBackend:
def send(self, **kwargs):
sent.append(kwargs)
monkeypatch.setattr(
"app.services.orchestrator.steps.get_email_backend",
lambda: FakeBackend(),
)
from app.services.orchestrator.steps import step_reminder
step_reminder(weekly_run_emailed, db)
assert weekly_run_emailed.reminded_at is not None
assert len(sent) == 1
assert sent[0]["to"] == "alice@example.com"
assert "1 hour" in sent[0]["subject"].lower() or "closes" in sent[0]["subject"].lower()
def test_step_reminder_skips_voter(
db, weekly_run_emailed, meal_plan, pending_item, member, monkeypatch
):
from app.models import MealPlanVote
vote = MealPlanVote(
meal_plan_item_id=pending_item.id,
family_member_id=member.id,
vote=True,
)
db.add(vote)
db.flush()
sent = []
class FakeBackend:
def send(self, **kwargs):
sent.append(kwargs)
monkeypatch.setattr(
"app.services.orchestrator.steps.get_email_backend",
lambda: FakeBackend(),
)
from app.services.orchestrator.steps import step_reminder
step_reminder(weekly_run_emailed, db)
# member already voted — no reminder
assert len(sent) == 0
assert weekly_run_emailed.reminded_at is not None
```
- [ ] **Step 2: Run tests to confirm they fail**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/test_orchestrator.py::test_step_reminder_idempotent \
tests/test_orchestrator.py::test_step_reminder_skips_when_not_emailed \
tests/test_orchestrator.py::test_step_reminder_sends_to_non_voter \
tests/test_orchestrator.py::test_step_reminder_skips_voter -v
```
Expected: `4 errors` — `ImportError: cannot import name 'step_reminder'`
- [ ] **Step 3: Add module-level imports to `steps.py`**
Open `backend/app/services/orchestrator/steps.py`. At the top of the file, after the other standard library imports, add `import html`:
```python
from __future__ import annotations
import html
import logging
from datetime import datetime, timezone
from typing import TYPE_CHECKING
```
In the `from app.models import ...` line, add `MealPlanVote`:
```python
from app.models import FamilyMember, FamilyProfile, MealPlan, MealPlanItemStatus, MealPlanVote
```
- [ ] **Step 4: Add `step_reminder` to `steps.py`**
Add the following function at the end of `steps.py`, after `step_finalize`:
```python
def step_reminder(run: "WeeklyRun", db: "Session") -> None:
if run.reminded_at is not None:
logger.info("step_reminder: already done for %s", run.week_start_date)
return
if run.emailed_at is None:
logger.info("step_reminder: proposal not sent yet for %s, skipping", run.week_start_date)
return
plan = (
db.query(MealPlan)
.filter(
MealPlan.family_profile_id == run.family_id,
MealPlan.week_start_date == run.week_start_date,
)
.first()
)
if plan is None:
run.reminded_at = datetime.now(timezone.utc)
db.commit()
return
pending_item_ids = [
item.id
for item in plan.items
if item.approval_status == MealPlanItemStatus.PENDING
]
if not pending_item_ids:
run.reminded_at = datetime.now(timezone.utc)
db.commit()
return
members = (
db.query(FamilyMember)
.filter(
FamilyMember.family_profile_id == run.family_id,
FamilyMember.email.isnot(None),
)
.all()
)
backend = get_email_backend()
for member in members:
voted_ids = {
v.meal_plan_item_id
for v in db.query(MealPlanVote)
.filter(
MealPlanVote.meal_plan_item_id.in_(pending_item_ids),
MealPlanVote.family_member_id == member.id,
)
.all()
}
unvoted = [
item
for item in plan.items
if item.approval_status == MealPlanItemStatus.PENDING
and item.id not in voted_ids
]
if not unvoted:
continue
item_html_parts = []
for item in unvoted:
token = issue_token(item.id, member.id)
vote_url = (
f"{settings.APP_BASE_URL}/api/meals/vote/{item.id}?token={token}"
)
recipe_name = html.escape(
item.recipe.name if item.recipe else str(item.recipe_id)
)
item_html_parts.append(
f'{recipe_name} — Vote'
)
email_html = (
f"Vote closes in 1 hour!
"
f"Hi {html.escape(member.name)}, the meal plan vote closes at Fri 17:00 PT. "
f"You haven't voted on:
"
f"{''.join(item_html_parts)}
"
)
backend.send(
to=member.email,
subject=f"Meal plan vote closes in 1 hour — week of {run.week_start_date}",
html=email_html,
)
logger.info("step_reminder: sent to %s", member.email)
run.reminded_at = datetime.now(timezone.utc)
db.commit()
logger.info("step_reminder: done for %s", run.week_start_date)
```
- [ ] **Step 5: Run tests to confirm they pass**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/test_orchestrator.py::test_step_reminder_idempotent \
tests/test_orchestrator.py::test_step_reminder_skips_when_not_emailed \
tests/test_orchestrator.py::test_step_reminder_sends_to_non_voter \
tests/test_orchestrator.py::test_step_reminder_skips_voter -v
```
Expected: `4 passed`
- [ ] **Step 6: Run full suite**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/
```
Expected: `121 passed` (117 from Task 2 + 4 new)
- [ ] **Step 7: Commit**
```bash
git add backend/app/services/orchestrator/steps.py \
backend/tests/test_orchestrator.py
git commit -m "feat: step_reminder — 1-hour pre-deadline nudge for non-voters"
```
---
## Task 4: Wire reminder into runner, admin, and scheduler
**Files:**
- Modify: `backend/app/services/orchestrator/runner.py`
- Modify: `backend/app/api/admin.py`
- Modify: `backend/app/scheduler/__main__.py`
### Context
`runner.py` has `STEPS = ("scrape", "generate", "email", "deadline", "finalize")` and a `step_fns` dict inside `run_step`. `admin.py` has `_VALID_STEPS = {"scrape", "generate", "email", "deadline", "finalize"}`. The scheduler has 5 jobs; the docstring lists 5 lines. All three files need `"reminder"` added.
- [ ] **Step 1: Update `runner.py`**
Open `backend/app/services/orchestrator/runner.py`. Change the `STEPS` tuple:
```python
STEPS = ("scrape", "generate", "email", "reminder", "deadline", "finalize")
```
Inside `run_step`, find the `step_fns` dict and add the `"reminder"` entry:
```python
step_fns = {
"scrape": s.step_scrape,
"generate": s.step_generate,
"email": s.step_email,
"reminder": s.step_reminder,
"deadline": s.step_deadline,
"finalize": s.step_finalize,
}
```
- [ ] **Step 2: Update `admin.py`**
Open `backend/app/api/admin.py`. Change `_VALID_STEPS`:
```python
_VALID_STEPS = {"scrape", "generate", "email", "reminder", "deadline", "finalize"}
```
- [ ] **Step 3: Update scheduler `__main__.py`**
Open `backend/app/scheduler/__main__.py`. After the `weekly_email` job block and before the `weekly_deadline` job block, add:
```python
scheduler.add_job(
lambda: run_step("reminder"),
CronTrigger(day_of_week="fri", hour=16, minute=0, timezone=TZ),
id="weekly_reminder",
name="Weekly reminder (Fri 16:00 PT)",
)
```
Also update the module docstring to include the new step:
```
Weekly cadence (America/Los_Angeles):
Fri 02:00 — scrape (Lucky weekend prices live by ~midnight)
Fri 05:00 — generate (fresh grocery data)
Fri 06:00 — email (per-voter approval links)
Fri 16:00 — reminder (nudge non-voters, 1h before deadline)
Fri 17:00 — deadline (pending items resolved per family policy)
Fri 18:00 — finalize (shopping-list email)
```
- [ ] **Step 4: Run full test suite**
```bash
docker compose --env-file .env.test exec \
-e TEST_DATABASE_URL=postgresql://mealplanner:${POSTGRES_PASSWORD}@db:5432/mealplanner \
backend pytest -q tests/
```
Expected: `121 passed` (unchanged count — no new tests in this task)
- [ ] **Step 5: Verify scheduler registers 6 jobs**
```bash
docker compose --env-file .env.test up -d scheduler
docker compose --env-file .env.test logs scheduler | grep "Registered"
```
Expected: 6 lines — scrape / generate / email / reminder / deadline / finalize
- [ ] **Step 6: Commit**
```bash
git add backend/app/services/orchestrator/runner.py \
backend/app/api/admin.py \
backend/app/scheduler/__main__.py
git commit -m "feat: wire step_reminder into runner, admin, and scheduler (Fri 16:00 PT)"
```
---
## Task 5: HTML-escape (#P5-a) and `getattr` cleanup (#P5-b)
**Files:**
- Modify: `backend/app/services/orchestrator/steps.py`
### Context
`import html` was already added in Task 3. This task applies `html.escape()` to the two existing functions that inline user-derived strings into HTML, and removes a dead `getattr` default in `step_deadline`.
**`step_email`** currently builds recipe name like:
```python
recipe_name = item.recipe.name if item.recipe else str(item.recipe_id)
item_html_parts.append(f'{recipe_name} — Vote')
```
**`step_finalize`** currently builds rows like:
```python
f"| {item.recipe.name} | "
f"{ing.get('name', '')} | "
f"{ing.get('qty', '')} {ing.get('unit', '')} |
"
```
**`step_deadline`** currently has:
```python
policy = getattr(family, "pending_approval_policy", "approve") if family else "approve"
```
- [ ] **Step 1: Write a test to verify HTML-escaping in `step_email`**
Add to `backend/tests/test_orchestrator.py` (after the existing `step_email` tests):
```python
def test_step_email_escapes_recipe_name(
db, weekly_run_generated, meal_plan, pending_item, member, monkeypatch
):
# Inject a recipe name with HTML-special chars
pending_item.recipe.name = ""
db.flush()
sent = []
class FakeBackend:
def send(self, **kwargs):
sent.append(kwargs)
monkeypatch.setattr(
"app.services.orchestrator.steps.get_email_backend",
lambda: FakeBackend(),
)
from app.services.orchestrator.steps import step_email
step_email(weekly_run_generated, db)
assert len(sent) == 1
assert "