Meal-Planner/docs/superpowers/specs/2026-05-07-phase6-sendgrid-design.md

# Phase 6 — SendGrid Email Integration Design

> Status: Approved 2026-05-07. Implementation plan forthcoming.

---

## Goal

Replace the `ConsoleEmailBackend` JSONL stub with a live SendGrid backend, add a pre-deadline reminder email, and close two minor hygiene issues from Phase 5.

---

## Scope

1. Wire `SendGridEmailBackend.send()` using the `sendgrid` Python library.
2. Add `SENDGRID_FROM_EMAIL` / `SENDGRID_REPLY_TO` to Settings.
3. New `step_reminder` orchestrator step — 1-hour pre-deadline nudge for members who have not yet voted.
4. Alembic migration 0009 — add `reminded_at TIMESTAMPTZ NULL` to `weekly_run`.
5. 6th scheduler job — Fri 16:00 PT → `run_step("reminder")`.
6. #P5-a — HTML-escape recipe/ingredient names in `step_email` and `step_finalize` templates.
7. #P5-b — Remove dead `getattr` default in `step_deadline`.

Out of scope: SendGrid sandbox mode, retry logic, denial notification email, per-family sender config, HTML template module.

---

## Architecture

No structural changes. `email.py` keeps its three-class layout (Protocol + Console + SendGrid) and `get_email_backend()` factory. The `EmailBackend` Protocol signature is unchanged — `send(to, subject, html, text=None)`. Sender identity (`from_email`, `reply_to`) is read from Settings inside `SendGridEmailBackend.__init__`, invisible to callers.

`step_reminder` is a standard orchestrator step: lives in `steps.py`, has its own `reminded_at` idempotency column, is registered in `STEPS` (between `email` and `deadline`), and is reachable via the existing admin `POST /api/admin/orchestrate/reminder` endpoint.

---

## Components

### `backend/app/services/email.py` (modify)

Replace `SendGridEmailBackend.send()` stub with:

```python
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail, ReplyTo

class SendGridEmailBackend:
    def __init__(self):
        self._client = SendGridAPIClient(settings.SENDGRID_API_KEY)
        self._from_email = settings.SENDGRID_FROM_EMAIL
        self._reply_to = settings.SENDGRID_REPLY_TO

    def send(self, to: str, subject: str, html: str, text: Optional[str] = None) -> None:
        message = Mail(
            from_email=self._from_email,
            to_emails=to,
            subject=subject,
            html_content=html,
            plain_text_content=text,
        )
        message.reply_to = ReplyTo(self._reply_to)
        response = self._client.send(message)
        if response.status_code >= 400:
            raise RuntimeError(
                f"SendGrid error {response.status_code}: {response.body}"
            )
```

### `backend/app/config.py` (modify)

```python
SENDGRID_FROM_EMAIL: str = "peter@research.bike"
SENDGRID_REPLY_TO: str = "peter@research.bike"
```

### `backend/app/services/orchestrator/steps.py` (modify)

**`step_reminder(run, db)`** — new function:

1. Return immediately if `run.reminded_at is not None`.
2. Return immediately if `run.emailed_at is None` (proposal never sent).
3. Load the `MealPlan` for `(run.family_id, run.week_start_date)`. If none, return.
4. Collect PENDING `MealPlanItem` IDs.
5. For each `FamilyMember` with a non-null email: query `MealPlanVote` for votes by this member on the pending items. If no un-voted items remain, skip.
6. For un-voted items, issue new tokens via `issue_token(item.id, member.id)` and build vote links.
7. Send reminder email:
   - Subject: `f"Meal plan vote closes in 1 hour — week of {run.week_start_date}"`
   - Body: member name, list of un-voted meals with vote links, deadline note.
8. Set `run.reminded_at = datetime.now(timezone.utc)`, commit.

**HTML-escape (#P5-a):** Wrap every user-derived string (recipe name, ingredient name) in `html.escape()` before interpolating into f-string HTML in `step_email` and `step_finalize`.

**`getattr` cleanup (#P5-b):** In `step_deadline`, replace:
```python
policy = getattr(family, "pending_approval_policy", "approve")
```
with:
```python
policy = family.pending_approval_policy
```

### `backend/alembic/versions/0009_phase6_reminded_at.py` (create)

```python
def upgrade():
    op.add_column("weekly_run", sa.Column("reminded_at", sa.DateTime(timezone=True), nullable=True))

def downgrade():
    op.drop_column("weekly_run", "reminded_at")
```

### `backend/app/models/__init__.py` (modify)

Add `reminded_at = Column(DateTime(timezone=True), nullable=True)` to `WeeklyRun`.

### `backend/app/services/orchestrator/runner.py` (modify)

Add `"reminder"` to `STEPS` list between `"email"` and `"deadline"`:

```python
STEPS = ["scrape", "generate", "email", "reminder", "deadline", "finalize"]
```

### `backend/app/api/admin.py` (modify)

Add `"reminder"` to `_VALID_STEPS`:

```python
_VALID_STEPS = {"scrape", "generate", "email", "reminder", "deadline", "finalize"}
```

### `backend/app/scheduler/__main__.py` (modify)

Add 6th job:

```python
scheduler.add_job(
    lambda: run_step("reminder"),
    CronTrigger(day_of_week="fri", hour=16, minute=0, timezone="America/Los_Angeles"),
    id="reminder",
    name="step_reminder",
)
```

### `requirements.txt` (verify / add)

Ensure `sendgrid` (or `sendgrid-python`) is present. Add if missing.

---

## Data flow

```
Fri 02:00  step_scrape       → weekly_run.scraped_at
Fri 05:00  step_generate     → weekly_run.generated_at
Fri 06:00  step_email        → weekly_run.emailed_at       (proposal + vote links)
Fri 16:00  step_reminder     → weekly_run.reminded_at      (nudge for non-voters)
Fri 17:00  step_deadline     → weekly_run.deadline_passed_at
Fri 18:00  step_finalize     → weekly_run.finalized_at     (shopping list)
```

---

## Error handling

`SendGridEmailBackend.send()` raises `RuntimeError` on non-2xx. The orchestrator's existing per-step error handler in `runner.py` catches it, sets `weekly_run.error_step` / `weekly_run.error_message`, and logs it. No additional retry logic.

---

## Testing

- `test_sendgrid_backend_send`: monkeypatch `SendGridAPIClient.send` to return a mock 202 response; assert `Mail` object has correct `to`, `from_email`, `reply_to`, `subject`, `html_content`.
- `test_sendgrid_backend_raises_on_error`: mock returns 400; assert `RuntimeError` raised.
- `test_step_reminder_idempotent`: `reminded_at` already set → no sends.
- `test_step_reminder_skips_if_no_proposal`: `emailed_at` is None → no sends.
- `test_step_reminder_sends_only_to_non_voters`: one member voted, one didn't → only one send.
- `test_step_reminder_no_pending_items`: all items approved/denied → no sends.
- Existing `test_orchestrator.py` tests for `step_email` / `step_finalize` pass unchanged (they monkeypatch `get_email_backend`, not the SendGrid client directly).

Target: ~125 tests total (115 existing + ~10 new).

---

## Environment variables

```bash
# Required for EMAIL_BACKEND=sendgrid
SENDGRID_API_KEY=SG.xxx
SENDGRID_FROM_EMAIL=peter@research.bike
SENDGRID_REPLY_TO=peter@research.bike
```

`EMAIL_BACKEND=console` remains the default; no SendGrid calls are made unless explicitly set.

---

## Files touched

| File | Action |
|---|---|
| `backend/app/services/email.py` | Modify — wire SendGridEmailBackend |
| `backend/app/config.py` | Modify — add SENDGRID_FROM_EMAIL / SENDGRID_REPLY_TO |
| `backend/app/services/orchestrator/steps.py` | Modify — add step_reminder; #P5-a html.escape; #P5-b getattr cleanup |
| `backend/app/services/orchestrator/runner.py` | Modify — add "reminder" to STEPS |
| `backend/app/api/admin.py` | Modify — add "reminder" to _VALID_STEPS |
| `backend/app/scheduler/__main__.py` | Modify — add 6th Fri 16:00 job |
| `backend/app/models/__init__.py` | Modify — add reminded_at to WeeklyRun |
| `backend/alembic/versions/0009_phase6_reminded_at.py` | Create — add reminded_at column |
| `backend/requirements.txt` | Verify/add sendgrid |
| `backend/tests/test_orchestrator.py` | Modify — add ~10 new tests |

Last updated: 2026-05-07