docs: design Swiftly token auto-mint and queue it as next implementation pass

Discovery: luckysupermarkets.com/config.json is publicly readable and
exposes firebaseApiKey. With proper Origin/Referer headers, Firebase
Identity Toolkit's anonymous-signup REST endpoint mints the same JWT
shape (iss=swiftly-lu-prod, aud=swiftly-lu-prod, anon provider, 3600s
TTL) that Swiftly accepts. Verified end-to-end on 2026-05-06.

This eliminates the manual hourly token-capture toil and supersedes
the seleniumbase-based scripts/refresh_swiftly_token.py (commit
ccfb38a) which had partial UI selector issues.

- New spec: docs/specs/2026-05-06-swiftly-token-auto-mint.md
- HANDOFF.md TL;DR refreshed (Phase 9 shipped); caveat #2 + #3
  rewritten to point to the auto-mint redesign; suggested-next-move
  reordered to put the redesign first
- ORIENTATION.md env-var section flags SWIFTLY_BEARER_TOKEN as
  scheduled-for-removal; "Where to look" lists both specs;
  last-updated footer refreshed

Implementation deferred — this commit captures the design and routing
only. Estimated 2-3 hours of focused work to ship per the spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/HANDOFF.md

@@ -2,17 +2,17 @@
 
 You are taking over a project in mid-flight. Read `docs/ORIENTATION.md` first for the high-level. This file is the deep dive: what's real, what's stubbed, where the bodies are buried, and what to do next.
 
-Date of handoff: 2026-05-06. Last commit before handoff: thin Phase 4 (P4-15).
+Date of handoff: 2026-05-06. Last commit before handoff: Swiftly auto-mint design spec (post-Phase 9).
 
 ---
 
 ## TL;DR
 
-The project completed a **recovery pass** (R1+R2+R3-0) from 2026-05-04 to 2026-05-05 because the prior agent had marked Phases 2/3/7 "complete" while leaving 12 distinct defects that prevented the app from importing, prevented migrations from upgrading on Postgres, prevented authentication from existing, and prevented any actual grocery items from being persisted.
+The project completed a **recovery pass** (R1+R2+R3-0) from 2026-05-04 to 2026-05-05 (12 defects from a prior agent fixed), then shipped **thin Phase 4** (recipe engine + ingredient↔grocery match layer + 30-recipe seed) and **Phase 9** (meal-planner generation algorithm: filter → score → top-K=20 set enumeration with diversity penalty → persist MealPlan + items via `POST /api/admin/meal-plans/generate`).
 
-That is now fixed. The app imports, the schema upgrades cleanly, auth works, scraping persists ~10k rows live in 36 s, and the email approval round-trip works end-to-end. 31/31 pytest tests pass.
+The project's reason to exist is now real and verified end-to-end. 88/88 pytest tests pass.
 
-The project's reason to exist — the meal-planner generation algorithm — is **still not started**. That is your top priority. See "Suggested next move" at the bottom.
+**Top remaining toil:** `SWIFTLY_BEARER_TOKEN` expires hourly, requiring manual devtools capture. A complete redesign is specced in `docs/specs/2026-05-06-swiftly-token-auto-mint.md` (Path B: pure-HTTP token minting via Firebase REST, validated on 2026-05-06). That should ship first — see "Suggested next move".
 
 ---
 
@@ -87,9 +87,9 @@ The project's reason to exist — the meal-planner generation algorithm — is *
 
 1. **Bootstrap login hatch.** `app/api/auth.py` login: when no `family_profile` row exists, it signs the literal string `"bootstrap"` instead of a UUID. Anyone with `SESSION_PASSWORD` gets a session even with zero data in the DB. Acceptable for self-hosted on a trusted network. Replace with a proper first-run setup gate before exposing the system beyond the LAN/VPN. The decision is documented in `.agent/context.md` under "Decisions".
 
-2. **`SWIFTLY_BEARER_TOKEN` expires hourly.** It's a Firebase anonymous-auth JWT scoped to `swiftly-lu-prod`. The user explicitly chose to surface a request when it expires rather than mint new tokens automatically. On 401, `ScrapeLog.error_message` carries the actionable message. The user captures a fresh token from luckysupermarkets.com devtools and updates the env var. Don't try to automate Firebase auth unless the user asks.
+2. **`SWIFTLY_BEARER_TOKEN` expires hourly — and there's a designed-but-not-yet-built fix.** Today the env var holds a Firebase anonymous-auth JWT for `swiftly-lu-prod` that expires hourly; on 401 the user manually captures a fresh one from devtools. **The replacement is fully designed in `docs/specs/2026-05-06-swiftly-token-auto-mint.md`** (~2-3 hours of work). The discovery: `https://luckysupermarkets.com/config.json` is publicly readable and exposes `firebaseApiKey`; with that, anonymous Firebase signUp via REST mints fresh tokens in <1 second. Verified end-to-end on 2026-05-06. **Do this redesign first** — it eliminates the only operator-toil step in the system. The `scripts/refresh_swiftly_token.py` (commit `ccfb38a`) seleniumbase approach is superseded; leave it for historical context but don't extend it.
 
-3. **`.env.example` ships a real (expiring) token.** Per user authorization. If it's already expired by the time you read this, that's expected — surface the refresh request to the user. Do not log it.
+3. **`.env.example` ships a real (expiring) token.** Per user authorization. If it's already expired by the time you read this, that's expected — surface the refresh request to the user, OR (preferably) ship the auto-mint redesign per caveat #2 and remove the env var entirely.
 
 4. **`ScrapeStatus` enum reuses `STARTED` for the queued state.** R1-C didn't add a `QUEUED` value because that would have churned the Postgres enum type. Cosmetic. If you change it, add a migration.
 
@@ -147,18 +147,18 @@ docker compose --env-file .env.test exec backend \
 
 ## Suggested next move
 
-**Phase 9 first — meal-planner generation algorithm.**
+**Swiftly token auto-mint first.** Phases 4 (thin slice) and 9 are both shipped — the project's reason to exist is real. The remaining operator toil is the hourly bearer-token refresh. The fix is fully designed in `docs/specs/2026-05-06-swiftly-token-auto-mint.md`: ~2-3 hours, no new deps, replaces `SWIFTLY_BEARER_TOKEN` env var with a process-local cache that mints fresh JWTs from Firebase via REST. Eliminates manual capture entirely. Do this before extending to Phase 5 (orchestration) so the weekly cycle can run unattended.
 
-Why: the schema is provably real now. The grocery feed is live. The approval flow works. Email and recipe ingestion can be developed against fixtures while you build the engine. Without Phase 9 the project has no reason to exist.
+After that, in priority order:
 
-Sketch:
+1. **Phase 5 — meal-planner orchestration.** Chain scrape → generate → email → vote → finalize on a weekly cadence. APScheduler container with `--workers 1` was the original plan. All the parts exist (scrape, generate, email-stub, approval round-trip); nothing chains them.
+2. **Phase 6 — SendGrid.** Replace the `ConsoleEmailBackend` JSONL stub with real SendGrid. Templates: meal proposal, T-24h reminder, confirmation, denial. `from_email`/`reply_to` config still needs adding to Settings.
+3. **Frontend login UI.** `/api/auth/login` exists and the cookie-based session works, but no UI consumes it. Until this lands, family-facing flows (Pantry, MealDetail, ShoppingList) can only be exercised by tests.
+4. **Phase 8 — feedback UI.** Close the learning loop into Phase 9. The `feedback` table exists and the schema supports it; nothing reads/writes it from a UI.
+5. **Phase 11 polish — APScheduler, variety analysis, budget tracking.**
+6. **Phase 10 — image strategy.**
 
-1. **Phase 4 (recipe ingestion) just enough to feed Phase 9.** Cheapest path: a CSV / JSON seed of ~30 recipes the family already knows + `POST /api/recipes/import` that takes a JSON body. Punt scraping recipe sites until later.
+Brainstorm with the user before committing to non-trivial scope. Use the `superpowers:brainstorming` skill.
-2. **Phase 9 algorithm.** Inputs listed above. Output: 7 `MealPlanItem` rows. Start dumb — random selection respecting `never_suggest` + mushroom rule. Iterate to add variety, sales bias, pantry bias, budget.
-3. **Phase 5 orchestration + Phase 6 SendGrid** — chain everything.
-4. **Phase 8 feedback UI** — close the learning loop.
-
-Brainstorm with the user before committing to the algorithm shape. Use the `superpowers:brainstorming` skill.
 
 ---
 
@@ -217,7 +217,13 @@ backend/tests/fixtures/lucky_ca/         categories.html, category_meat_seafood.
 scripts/
 ├── send_test_approval.py                email round-trip prover
 ├── spike_lucky_scrape.py                R2-A archived
-└── spike_swiftly_ingest.py              R3-0 live ingest prover
+├── spike_swiftly_ingest.py              R3-0 live ingest prover
+└── refresh_swiftly_token.py             SUPERSEDED — seleniumbase token capture
+                                         (replaced by docs/specs/2026-05-06-swiftly-token-auto-mint.md)
+
+docs/specs/
+├── 2026-05-05-meal-planner-algorithm-design.md   Phase 9 + thin Phase 4 design
+└── 2026-05-06-swiftly-token-auto-mint.md         next-up: replace SWIFTLY_BEARER_TOKEN env var
 
 .github/workflows/ci.yml                 backend (postgres + pytest) + frontend (npm build)
 ```

docs/ORIENTATION.md

@@ -96,6 +96,9 @@ LUCKY_STORE_ID=757                 # Lucky California — San Pablo
 SWIFTLY_API_BASE=https://prod.swiftlyapi.net
 SWIFTLY_CATEGORIES_URL=https://luckysupermarkets.com/categories
 SWIFTLY_BEARER_TOKEN=...           # Firebase anon JWT, expires hourly
+                                   # → scheduled for removal: see
+                                   # docs/specs/2026-05-06-swiftly-token-auto-mint.md
+                                   # (auto-mint via Firebase REST, no manual capture)
 
 # Other
 LUCKY_CA_URL=https://luckysupermarkets.com
@@ -144,9 +147,11 @@ A `.env.test` template lives in the repo root (gitignored) for local stack runs.
 - `docs/database-schema.md` — full DDL reference.
 - `docs/implementation-plan.md` — original phased plan.
 - `docs/RUNNING.md` — local dev workflow.
+- `docs/specs/2026-05-05-meal-planner-algorithm-design.md` — Phase 9 + thin Phase 4 design.
+- `docs/specs/2026-05-06-swiftly-token-auto-mint.md` — next-up redesign that eliminates the only operator-toil step.
 - `.agent/plan.md`, `.agent/context.md`, `.agent/phase-summaries/` — recovery decisions and per-phase summaries from the R1+R2+R3-0 work.
 - `Review/reviewconcensus.md` — the adversarial review that drove the recovery.
 
 ---
 
-Last updated: 2026-05-06 — Phase 9 complete (meal-plan generation algorithm: filter→score→set-select pipeline; persisted MealPlan + items via POST /api/admin/meal-plans/generate). 88/88 pytest green.
+Last updated: 2026-05-06 — Phase 9 shipped (88/88 pytest green); Swiftly token auto-mint designed (`docs/specs/2026-05-06-swiftly-token-auto-mint.md`) and queued as the next implementation pass.

docs/specs/2026-05-06-swiftly-token-auto-mint.md

@@ -0,0 +1,145 @@
+# Swiftly Token Auto-Mint — Design Spec
+
+Date: 2026-05-06
+Status: **Designed, not yet implemented.** The current scraper still reads a static `SWIFTLY_BEARER_TOKEN` from the env. This spec replaces that with an automatic, HTTP-only refresh path.
+
+---
+
+## Why this matters
+
+The Lucky California (Swiftly) JSON API requires `Authorization: Bearer <jwt>`. The JWT is a Firebase anonymous-auth token from the `swiftly-lu-prod` project, expires hourly. Today the user manually captures one from luckysupermarkets.com devtools whenever a scrape fails. This is the only piece of operator toil in the system; eliminating it converts the project to "scrape just works forever."
+
+A previous attempt (`scripts/refresh_swiftly_token.py`, commit `ccfb38a`) used seleniumbase to automate the manual capture. That path is **superseded by this spec** — leave the script in place as historical context but don't build on it.
+
+---
+
+## Discovery
+
+`https://luckysupermarkets.com/config.json` is publicly readable and returns the full Firebase config:
+
+```json
+{
+  "firebaseApiKey": "AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U",
+  "firebaseAuthDomain": "luckysupermarkets.com",
+  "firebaseProjectId": "swiftly-lu-prod",
+  "firebaseAppId": "1:166967617165:web:e177179df9be229d2fe4c4",
+  "apiBaseURL": "https://prod.swiftlyapi.net",
+  "chainId": "a3b11717-f4ca-4196-b670-e5142c205dee"
+}
+```
+
+The `firebaseApiKey` is a Firebase Web API key — **public by design** (Google's docs explicitly say so) but protected by HTTP-referrer restrictions. With proper `Origin` and `Referer` headers, anyone can call Firebase Identity Toolkit's anonymous-signup endpoint to mint a fresh JWT.
+
+**Verified working** (2026-05-06):
+
+```bash
+curl -X POST \
+  -H 'Content-Type: application/json' \
+  -H 'Origin: https://luckysupermarkets.com' \
+  -H 'Referer: https://luckysupermarkets.com/' \
+  -d '{"returnSecureToken": true}' \
+  "https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U"
+```
+
+Returns a JWT with `iss=https://securetoken.google.com/swiftly-lu-prod`, `aud=swiftly-lu-prod`, `provider=anonymous`, `expires_in=3600s`. The Swiftly API accepts it (verified: 400 "Category is required" on a malformed test request, NOT 401).
+
+---
+
+## Design
+
+### New module: `backend/app/services/swiftly_auth.py`
+
+```python
+def get_token() -> str:
+    """Return a valid Swiftly bearer JWT, minting one if no cached token has
+    >5 minutes of life remaining. Caches in process memory only — no DB."""
+```
+
+Internals:
+1. On first call, GET `https://luckysupermarkets.com/config.json` to fetch `firebaseApiKey`. Cache for the process lifetime.
+2. POST `https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=<API_KEY>` with body `{"returnSecureToken": true}` and headers `Origin`/`Referer` set to `https://luckysupermarkets.com`.
+3. Parse the returned `idToken`. Decode the JWT payload, validate `iss == https://securetoken.google.com/swiftly-lu-prod` and `exp > now + 60s`. Cache (token, exp) in a module-level variable.
+4. On subsequent calls, return the cached token if `exp > now + 300s`, otherwise mint a new one.
+5. Process-local cache only — no Redis, no DB. A backend restart re-mints. Restarts are rare and minting is cheap (<500ms typical).
+
+### New errors
+
+`SwiftlyAuthMintError` — raised when:
+- `config.json` fetch fails
+- Firebase signUp returns non-200
+- Returned JWT fails validation
+
+The existing `SwiftlyAuthError` is repurposed for the (now rare) case where the Swiftly API itself returns 401 even with a freshly minted token (shouldn't happen, but defensive).
+
+### Scraper integration
+
+`backend/app/scraper/lucky_ca_scraper.py`:
+
+Today:
+```python
+self.bearer_token = settings.SWIFTLY_BEARER_TOKEN
+```
+
+After:
+```python
+from app.services.swiftly_auth import get_token
+
+# In the request method or per-call:
+self.api_session.headers["Authorization"] = f"Bearer {get_token()}"
+```
+
+The `get_token()` call short-circuits 99% of the time (cached), so latency overhead per scrape is ~zero.
+
+### Config changes
+
+- **Remove:** `SWIFTLY_BEARER_TOKEN` from `.env.example`, `.env.test`, `docker-compose.yml`, `app/config.py` Settings, `.github/workflows/ci.yml`.
+- **Optional new setting:** `SWIFTLY_FIREBASE_CONFIG_URL` (defaults to `https://luckysupermarkets.com/config.json`) — only useful if Lucky moves the file.
+- The hardcoded `firebaseApiKey` is fetched at runtime from config.json, NOT pinned in code. If Lucky rotates it, fetch picks up the new value automatically.
+
+### Tests
+
+- `backend/tests/test_swiftly_auth.py` (new):
+  1. Unit test: mock the Firebase signUp HTTP call → assert get_token returns the idToken
+  2. Unit test: cached token is returned without re-minting when exp > now+5min
+  3. Unit test: expired/near-expired cached token triggers re-mint
+  4. Unit test: Firebase 403/non-200 raises `SwiftlyAuthMintError`
+  5. Integration test (gated by `--live` marker): hit the real Firebase + Swiftly endpoints, mint a token, hit the Swiftly products API, confirm a 200 with non-empty product list. Skipped in CI; run manually to verify Lucky hasn't broken the path.
+
+### Caveats this resolves
+
+- HANDOFF.md caveat **#2 (`SWIFTLY_BEARER_TOKEN` expires hourly)** — eliminated.
+- HANDOFF.md caveat **#3 (`.env.example` ships a real expiring token)** — variable removed.
+
+---
+
+## Implementation order (small, single sitting)
+
+Each step is a separate commit.
+
+1. **`swiftly_auth.py` + unit tests** (~80 lines + 4 tests). Mock Firebase, validate JWT decode, validate cache logic. ~1 hour.
+2. **Wire into `lucky_ca_scraper.py`**: replace `SWIFTLY_BEARER_TOKEN` env lookup with `get_token()`. Run the existing live spike (`scripts/spike_swiftly_ingest.py --confirm-live`) to verify end-to-end. ~30 min.
+3. **Remove env var + config**: drop `SWIFTLY_BEARER_TOKEN` from all env files, Settings, docker-compose, CI. Update `.env.example` to not ship a token. ~15 min.
+4. **Delete the seleniumbase script + its dependency footprint.** `scripts/refresh_swiftly_token.py` was an interim. ~5 min.
+5. **Update HANDOFF.md and ORIENTATION.md**: remove the manual-token caveats, note the auto-mint path is live, update env-var docs.
+6. **Run the full pytest suite + live scrape end-to-end.** Confirm 88+/88+ green, fresh scrape persists rows, generate produces a meal plan.
+
+Total: ~2-3 hours of focused work. No new dependencies.
+
+---
+
+## Risks
+
+- **Firebase API key rotation.** If Lucky rotates the key, our fetch of `config.json` picks up the new value automatically — zero impact.
+- **Anonymous auth disabled.** If Lucky disables anon auth on the Firebase project, signUp returns 400. Detect via `SwiftlyAuthMintError` with a clear message: "Lucky disabled anonymous Firebase auth — manual capture required as fallback." This is the only failure mode that requires the original manual-capture flow.
+- **HTTP-referrer restriction tightened.** If Google adds anti-abuse checks (e.g., requiring a real browser fingerprint), the REST call breaks. Same fallback: revert to the manual capture, surface a clear error.
+- **`config.json` made non-public.** Same fallback.
+
+All three risks are hypothetical and result in scrape failures with actionable error messages, not silent breakage. The error is surfaced via `ScrapeLog.error_message` exactly the same way the current expired-token error is surfaced today.
+
+---
+
+## Out of scope
+
+- Persisting tokens across backend restarts (process-local cache is fine).
+- Refreshing tokens via Firebase's refresh-token flow (not needed — anon signUp is cheap and we get a fresh `idToken` directly).
+- Auto-recovery on the rare 401 from Swiftly with a fresh token (defensive code for "shouldn't happen" edge case; revisit if observed).