Meal-Planner/docs/specs/2026-05-06-swiftly-token-auto-mint.md

# Swiftly Token Auto-Mint — Design Spec

Date: 2026-05-06
Status: **Implemented 2026-05-06.** Live-verified end-to-end: 260 products fetched from `Product/meat_seafood` on a fresh-mint scrape with `SWIFTLY_BEARER_TOKEN` removed from the env. Module: `backend/app/services/swiftly_auth.py`; wiring: `backend/app/scraper/lucky_ca_scraper.py`. Original design follows for context.

---

## Why this matters

The Lucky California (Swiftly) JSON API requires `Authorization: Bearer <jwt>`. The JWT is a Firebase anonymous-auth token from the `swiftly-lu-prod` project, expires hourly. Today the user manually captures one from luckysupermarkets.com devtools whenever a scrape fails. This is the only piece of operator toil in the system; eliminating it converts the project to "scrape just works forever."

A previous attempt (`scripts/refresh_swiftly_token.py`, commit `ccfb38a`) used seleniumbase to automate the manual capture. That path is **superseded by this spec** — leave the script in place as historical context but don't build on it.

---

## Discovery

`https://luckysupermarkets.com/config.json` is publicly readable and returns the full Firebase config:

```json
{
  "firebaseApiKey": "AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U",
  "firebaseAuthDomain": "luckysupermarkets.com",
  "firebaseProjectId": "swiftly-lu-prod",
  "firebaseAppId": "1:166967617165:web:e177179df9be229d2fe4c4",
  "apiBaseURL": "https://prod.swiftlyapi.net",
  "chainId": "a3b11717-f4ca-4196-b670-e5142c205dee"
}
```

The `firebaseApiKey` is a Firebase Web API key — **public by design** (Google's docs explicitly say so) but protected by HTTP-referrer restrictions. With proper `Origin` and `Referer` headers, anyone can call Firebase Identity Toolkit's anonymous-signup endpoint to mint a fresh JWT.

**Verified working** (2026-05-06):

```bash
curl -X POST \
  -H 'Content-Type: application/json' \
  -H 'Origin: https://luckysupermarkets.com' \
  -H 'Referer: https://luckysupermarkets.com/' \
  -d '{"returnSecureToken": true}' \
  "https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U"
```

Returns a JWT with `iss=https://securetoken.google.com/swiftly-lu-prod`, `aud=swiftly-lu-prod`, `provider=anonymous`, `expires_in=3600s`. The Swiftly API accepts it (verified: 400 "Category is required" on a malformed test request, NOT 401).

---

## Design

### New module: `backend/app/services/swiftly_auth.py`

```python
def get_token() -> str:
    """Return a valid Swiftly bearer JWT, minting one if no cached token has
    >5 minutes of life remaining. Caches in process memory only — no DB."""
```

Internals:
1. On first call, GET `https://luckysupermarkets.com/config.json` to fetch `firebaseApiKey`. Cache for the process lifetime.
2. POST `https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=<API_KEY>` with body `{"returnSecureToken": true}` and headers `Origin`/`Referer` set to `https://luckysupermarkets.com`.
3. Parse the returned `idToken`. Decode the JWT payload, validate `iss == https://securetoken.google.com/swiftly-lu-prod` and `exp > now + 60s`. Cache (token, exp) in a module-level variable.
4. On subsequent calls, return the cached token if `exp > now + 300s`, otherwise mint a new one.
5. Process-local cache only — no Redis, no DB. A backend restart re-mints. Restarts are rare and minting is cheap (<500ms typical).

### New errors

`SwiftlyAuthMintError` — raised when:
- `config.json` fetch fails
- Firebase signUp returns non-200
- Returned JWT fails validation

The existing `SwiftlyAuthError` is repurposed for the (now rare) case where the Swiftly API itself returns 401 even with a freshly minted token (shouldn't happen, but defensive).

### Scraper integration

`backend/app/scraper/lucky_ca_scraper.py`:

Today:
```python
self.bearer_token = settings.SWIFTLY_BEARER_TOKEN
```

After:
```python
from app.services.swiftly_auth import get_token

# In the request method or per-call:
self.api_session.headers["Authorization"] = f"Bearer {get_token()}"
```

The `get_token()` call short-circuits 99% of the time (cached), so latency overhead per scrape is ~zero.

### Config changes

- **Remove:** `SWIFTLY_BEARER_TOKEN` from `.env.example`, `.env.test`, `docker-compose.yml`, `app/config.py` Settings, `.github/workflows/ci.yml`.
- **Optional new setting:** `SWIFTLY_FIREBASE_CONFIG_URL` (defaults to `https://luckysupermarkets.com/config.json`) — only useful if Lucky moves the file.
- The hardcoded `firebaseApiKey` is fetched at runtime from config.json, NOT pinned in code. If Lucky rotates it, fetch picks up the new value automatically.

### Tests

- `backend/tests/test_swiftly_auth.py` (new):
  1. Unit test: mock the Firebase signUp HTTP call → assert get_token returns the idToken
  2. Unit test: cached token is returned without re-minting when exp > now+5min
  3. Unit test: expired/near-expired cached token triggers re-mint
  4. Unit test: Firebase 403/non-200 raises `SwiftlyAuthMintError`
  5. Integration test (gated by `--live` marker): hit the real Firebase + Swiftly endpoints, mint a token, hit the Swiftly products API, confirm a 200 with non-empty product list. Skipped in CI; run manually to verify Lucky hasn't broken the path.

### Caveats this resolves

- HANDOFF.md caveat **#2 (`SWIFTLY_BEARER_TOKEN` expires hourly)** — eliminated.
- HANDOFF.md caveat **#3 (`.env.example` ships a real expiring token)** — variable removed.

---

## Implementation order (small, single sitting)

Each step is a separate commit.

1. **`swiftly_auth.py` + unit tests** (~80 lines + 4 tests). Mock Firebase, validate JWT decode, validate cache logic. ~1 hour.
2. **Wire into `lucky_ca_scraper.py`**: replace `SWIFTLY_BEARER_TOKEN` env lookup with `get_token()`. Run the existing live spike (`scripts/spike_swiftly_ingest.py --confirm-live`) to verify end-to-end. ~30 min.
3. **Remove env var + config**: drop `SWIFTLY_BEARER_TOKEN` from all env files, Settings, docker-compose, CI. Update `.env.example` to not ship a token. ~15 min.
4. **Delete the seleniumbase script + its dependency footprint.** `scripts/refresh_swiftly_token.py` was an interim. ~5 min.
5. **Update HANDOFF.md and ORIENTATION.md**: remove the manual-token caveats, note the auto-mint path is live, update env-var docs.
6. **Run the full pytest suite + live scrape end-to-end.** Confirm 88+/88+ green, fresh scrape persists rows, generate produces a meal plan.

Total: ~2-3 hours of focused work. No new dependencies.

---

## Risks

- **Firebase API key rotation.** If Lucky rotates the key, our fetch of `config.json` picks up the new value automatically — zero impact.
- **Anonymous auth disabled.** If Lucky disables anon auth on the Firebase project, signUp returns 400. Detect via `SwiftlyAuthMintError` with a clear message: "Lucky disabled anonymous Firebase auth — manual capture required as fallback." This is the only failure mode that requires the original manual-capture flow.
- **HTTP-referrer restriction tightened.** If Google adds anti-abuse checks (e.g., requiring a real browser fingerprint), the REST call breaks. Same fallback: revert to the manual capture, surface a clear error.
- **`config.json` made non-public.** Same fallback.

All three risks are hypothetical and result in scrape failures with actionable error messages, not silent breakage. The error is surfaced via `ScrapeLog.error_message` exactly the same way the current expired-token error is surfaced today.

---

## Out of scope

- Persisting tokens across backend restarts (process-local cache is fine).
- Refreshing tokens via Firebase's refresh-token flow (not needed — anon signUp is cheap and we get a fresh `idToken` directly).
- Auto-recovery on the rare 401 from Swiftly with a fresh token (defensive code for "shouldn't happen" edge case; revisit if observed).