Meal-Planner/docs/specs/2026-05-06-swiftly-token-auto-mint.md

Swiftly Token Auto-Mint — Design Spec

Date: 2026-05-06 Status: Implemented 2026-05-06. Live-verified end-to-end: 260 products fetched from Product/meat_seafood on a fresh-mint scrape with SWIFTLY_BEARER_TOKEN removed from the env. Module: backend/app/services/swiftly_auth.py; wiring: backend/app/scraper/lucky_ca_scraper.py. Original design follows for context.

---

Why this matters

The Lucky California (Swiftly) JSON API requires Authorization: Bearer <jwt>. The JWT is a Firebase anonymous-auth token from the swiftly-lu-prod project, expires hourly. Today the user manually captures one from luckysupermarkets.com devtools whenever a scrape fails. This is the only piece of operator toil in the system; eliminating it converts the project to "scrape just works forever."

A previous attempt (scripts/refresh_swiftly_token.py, commit ccfb38a) used seleniumbase to automate the manual capture. That path is superseded by this spec — leave the script in place as historical context but don't build on it.

---

Discovery

https://luckysupermarkets.com/config.json is publicly readable and returns the full Firebase config:

{
  "firebaseApiKey": "AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U",
  "firebaseAuthDomain": "luckysupermarkets.com",
  "firebaseProjectId": "swiftly-lu-prod",
  "firebaseAppId": "1:166967617165:web:e177179df9be229d2fe4c4",
  "apiBaseURL": "https://prod.swiftlyapi.net",
  "chainId": "a3b11717-f4ca-4196-b670-e5142c205dee"
}

The firebaseApiKey is a Firebase Web API key — public by design (Google's docs explicitly say so) but protected by HTTP-referrer restrictions. With proper Origin and Referer headers, anyone can call Firebase Identity Toolkit's anonymous-signup endpoint to mint a fresh JWT.

Verified working (2026-05-06):

curl -X POST \
  -H 'Content-Type: application/json' \
  -H 'Origin: https://luckysupermarkets.com' \
  -H 'Referer: https://luckysupermarkets.com/' \
  -d '{"returnSecureToken": true}' \
  "https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=AIzaSyCnG97lkCEUvVTcRdSEJ6looOPQgX0WE2U"

Returns a JWT with iss=https://securetoken.google.com/swiftly-lu-prod, aud=swiftly-lu-prod, provider=anonymous, expires_in=3600s. The Swiftly API accepts it (verified: 400 "Category is required" on a malformed test request, NOT 401).

---

Design

New module: backend/app/services/swiftly_auth.py

def get_token() -> str:
    """Return a valid Swiftly bearer JWT, minting one if no cached token has
    >5 minutes of life remaining. Caches in process memory only — no DB."""

Internals:

1. On first call, GET https://luckysupermarkets.com/config.json to fetch firebaseApiKey. Cache for the process lifetime.
2. POST https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=<API_KEY> with body {"returnSecureToken": true} and headers Origin/Referer set to https://luckysupermarkets.com.
3. Parse the returned idToken. Decode the JWT payload, validate iss == https://securetoken.google.com/swiftly-lu-prod and exp > now + 60s. Cache (token, exp) in a module-level variable.
4. On subsequent calls, return the cached token if exp > now + 300s, otherwise mint a new one.
5. Process-local cache only — no Redis, no DB. A backend restart re-mints. Restarts are rare and minting is cheap (<500ms typical).

New errors

SwiftlyAuthMintError — raised when:

• config.json fetch fails
• Firebase signUp returns non-200
• Returned JWT fails validation

The existing SwiftlyAuthError is repurposed for the (now rare) case where the Swiftly API itself returns 401 even with a freshly minted token (shouldn't happen, but defensive).

Scraper integration

backend/app/scraper/lucky_ca_scraper.py:

Today:

self.bearer_token = settings.SWIFTLY_BEARER_TOKEN

After:

from app.services.swiftly_auth import get_token

# In the request method or per-call:
self.api_session.headers["Authorization"] = f"Bearer {get_token()}"

The get_token() call short-circuits 99% of the time (cached), so latency overhead per scrape is ~zero.

Config changes

• Remove: SWIFTLY_BEARER_TOKEN from .env.example, .env.test, docker-compose.yml, app/config.py Settings, .github/workflows/ci.yml.
• Optional new setting: SWIFTLY_FIREBASE_CONFIG_URL (defaults to https://luckysupermarkets.com/config.json) — only useful if Lucky moves the file.
• The hardcoded firebaseApiKey is fetched at runtime from config.json, NOT pinned in code. If Lucky rotates it, fetch picks up the new value automatically.

Tests

• backend/tests/test_swiftly_auth.py (new):
  1. Unit test: mock the Firebase signUp HTTP call → assert get_token returns the idToken
  2. Unit test: cached token is returned without re-minting when exp > now+5min
  3. Unit test: expired/near-expired cached token triggers re-mint
  4. Unit test: Firebase 403/non-200 raises SwiftlyAuthMintError
  5. Integration test (gated by --live marker): hit the real Firebase + Swiftly endpoints, mint a token, hit the Swiftly products API, confirm a 200 with non-empty product list. Skipped in CI; run manually to verify Lucky hasn't broken the path.

Caveats this resolves

• HANDOFF.md caveat #2 (SWIFTLY_BEARER_TOKEN expires hourly) — eliminated.
• HANDOFF.md caveat #3 (.env.example ships a real expiring token) — variable removed.

---

Implementation order (small, single sitting)

Each step is a separate commit.

1. swiftly_auth.py + unit tests (~80 lines + 4 tests). Mock Firebase, validate JWT decode, validate cache logic. ~1 hour.
2. Wire into lucky_ca_scraper.py: replace SWIFTLY_BEARER_TOKEN env lookup with get_token(). Run the existing live spike (scripts/spike_swiftly_ingest.py --confirm-live) to verify end-to-end. ~30 min.
3. Remove env var + config: drop SWIFTLY_BEARER_TOKEN from all env files, Settings, docker-compose, CI. Update .env.example to not ship a token. ~15 min.
4. Delete the seleniumbase script + its dependency footprint. scripts/refresh_swiftly_token.py was an interim. ~5 min.
5. Update HANDOFF.md and ORIENTATION.md: remove the manual-token caveats, note the auto-mint path is live, update env-var docs.
6. Run the full pytest suite + live scrape end-to-end. Confirm 88+/88+ green, fresh scrape persists rows, generate produces a meal plan.

Total: ~2-3 hours of focused work. No new dependencies.

---

Risks

• Firebase API key rotation. If Lucky rotates the key, our fetch of config.json picks up the new value automatically — zero impact.
• Anonymous auth disabled. If Lucky disables anon auth on the Firebase project, signUp returns 400. Detect via SwiftlyAuthMintError with a clear message: "Lucky disabled anonymous Firebase auth — manual capture required as fallback." This is the only failure mode that requires the original manual-capture flow.
• HTTP-referrer restriction tightened. If Google adds anti-abuse checks (e.g., requiring a real browser fingerprint), the REST call breaks. Same fallback: revert to the manual capture, surface a clear error.
• config.json made non-public. Same fallback.

All three risks are hypothetical and result in scrape failures with actionable error messages, not silent breakage. The error is surfaced via ScrapeLog.error_message exactly the same way the current expired-token error is surfaced today.

---

Out of scope

• Persisting tokens across backend restarts (process-local cache is fine).
• Refreshing tokens via Firebase's refresh-token flow (not needed — anon signUp is cheap and we get a fresh idToken directly).
• Auto-recovery on the rare 401 from Swiftly with a fresh token (defensive code for "shouldn't happen" edge case; revisit if observed).