feat: phase r1+r2 recovery + r3-0 swiftly api ingestion

R1 stabilization: pytest harness with transactional db fixture, smoke
+ alembic + auth + scrape + approval + swiftly tests, github actions
ci yaml. Bearer-token admin auth + signed-cookie session for family
ui mutations. Async POST /api/admin/scrape (BackgroundTasks, returns
202). Path canonicalization (no /list, /planned suffixes). DATABASE_URL
fail-fast on empty.

R2 deferred-risk spikes: live lucky california fetch (R2-A), full
email+per-voter approval click round trip with single-use enforcement
(R2-B, console email backend, sendgrid stub).

R3-0 phase 3 redesign: replaced playwright html scraper with requests
based swiftly json api client. 17 categories, ~10k products per scrape,
upsert by (source, external_id). 401 surfaces actionable token-refresh
message via ScrapeLog.error_message.

Pre-existing defects fixed: shopping_list.py syntax error blocking app
import, MealPlan.votes orphan relationship, JSONB(astext=True) invalid
kwarg, missing requests dep, calorie_target schema drift, every SQLEnum
needed values_callable, 0001 had empty downgrade(), seed had duplicate
ingredient rows.

Migrations added: 0003 grocery_item.description, 0004 family_profile.
calorie_target, 0005 grocery_item.external_id + source + composite index.

Verified: 31/31 pytest green, alembic upgrade->downgrade->upgrade clean,
frontend npm run build clean, live scrape 9,960 grocery_item rows in 36s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.agent/phase-summaries/r3-0-summary.md

@@ -0,0 +1,64 @@
+# R3-0 — Swiftly product API client (replaces Playwright path)
+
+## Schema migration
+
+Added `0005_grocery_item_external_id.py` (`down_revision='0004'`):
+
+- `grocery_item.external_id` (`String(100)`, nullable, indexed)
+- `grocery_item.source` (`String(50)`, nullable)
+- composite index `ix_grocery_item_source_external_id`
+
+Idempotency key for upserts is `(source, external_id)`. Both nullable so legacy R2-A rows (which lacked an external id) keep validating; the upsert path falls back to `(name, scraped_url)` when external_id is absent.
+
+## Discovery + sample counts
+
+- Categories page: 17 distinct slugs (e.g. `Product/meat_seafood`, `Product/produce`, `Product/dairy_eggs_cheese`, ...). Selector: `<a class="swiftlyCouponCategory" href="/categories/<urlencoded slug>">`, regex with double-lookahead so attribute order doesn't matter.
+- `Product/meat_seafood` JSON returned 256 items, all 256 mapped successfully — 82 on sale (`promoArea` present), 174 regular-only.
+
+## Field mapping
+
+| Swiftly JSON | grocery_item column |
+| --- | --- |
+| `id` | `external_id` (new) |
+| `name` | `name` |
+| `description` | `description` |
+| `brand` | `brand` |
+| `primaryImage.url` | `image_url` |
+| `price.ok.regPriceText` (e.g. `"$3.49 /lb"`) | `regular_price`, `unit` |
+| `price.ok.promoArea.promoText` | `sale_price`, `is_on_sale=True` |
+| (queried slug → tail) | `aisle` (e.g. `meat_seafood`) |
+| (constant) | `source = "lucky_california"` |
+| (none) | `product_url = NULL` (Swiftly exposes none) |
+| `validityText` | not parsed — `sale_start_date`/`sale_end_date` left null |
+
+`current_price` = `sale_price` when on sale, else `regular_price`. Prices stored as `Decimal`.
+
+## 401 handling
+
+`SwiftlyAuthError` is a custom exception raised:
+
+1. Up-front when `SWIFTLY_BEARER_TOKEN` is empty (no HTTP call).
+2. On `response.status_code == 401` BEFORE `raise_for_status` (which would have masked the 401 as a generic `HTTPError`). The new client uses `requests.Session.get` directly — `BaseScraper._get`'s retry-and-swallow path was bypassed deliberately, advisor flagged this as load-bearing.
+
+`_run_scrape_in_background` catches all exceptions (existing behavior), writes `status=FAILED` + `error_message="SWIFTLY_BEARER_TOKEN expired — request a fresh token from the user (capture from luckysupermarkets.com network tab on a /search/api/v1 request)"`. Admin sees it via `GET /api/admin/logs/<id>`.
+
+## Auth scoping
+
+Two `requests.Session` objects: `public_session` (no auth, hits `luckysupermarkets.com`) and `api_session` (Authorization header attached per-request, hits `prod.swiftlyapi.net`). Bearer is never sent to the public host.
+
+## Verification
+
+- `pytest -q tests/test_swiftly_api.py` → 5 passed, 1 skipped (Postgres-only).
+- `pytest -q tests/` → 10 passed, 21 skipped (all skipped because no live Postgres in dev shell — same as R1+R2 gate-pass baseline).
+- `python scripts/spike_swiftly_ingest.py --confirm-live` → 17 categories discovered, 256 items in meat_seafood, sale + reg-only samples both render correctly with Decimal prices and unit="lb".
+- Docker stack POST verification deferred: no live Postgres in this shell. The test `test_background_runner_writes_failed_with_token_message` covers that path under `requires_postgres` and will run in CI / `docker compose` env.
+
+## Caveats
+
+- **Token in `.env.example` is a real (expiring) credential**, per task spec. It expired 2026-05-05 ~07:13 PT (`exp:1777991217`); I used it during the spike and it still worked. The user explicitly authorized this. Recommend gitignoring `.env.example` or rotating to a placeholder in a future cleanup task.
+- Old test `tests/test_lucky_ca_scraper.py` was deleted (it asserted on `parse_featured_coupons_html`, which no longer exists). R2-A's fixtures `weekly_ad.html`, `weekly_ad.png`, `META.md` retained per task spec.
+- `BaseScraper` and `SeleniumScraper` classes in `app/scraper/base.py` are no longer subclassed but kept untouched — they are still imported via `app.scraper.__init__` and may be useful for a future second store. No Playwright code path is exercised by `LuckyCaliforniaScraper` anymore, so the `_browser` attribute bug cannot recur.
+- Rate limit set to 0.75s/req in the new client (between the spec's 1–2 req/sec). Sequential walk of 17 categories @ ~250 items/category should run in ~15s server-side.
+- `_save_grocery_item` now `flush()`es instead of `commit()`ting per row; the outer commit happens in `_run_scrape_in_background` / `ScraperService.run_scrape`. Trade-off: a single bad row aborts the whole scrape's transaction. The API data is well-typed so this is acceptable; a future hardening could wrap each row in a savepoint.
+- Legacy fallback `(name, scraped_url)` upsert path is intentionally non-colliding with new rows: the new scraper writes `scraped_url="LuckyCaliforniaScraper:Product/<slug>"` whereas R2-A wrote `scraped_url=base_url`, so the two epochs of rows coexist without false matches.
+- Verification #4 (POST `/api/admin/scrape` against the live docker stack) was NOT run from this subagent shell — no live Postgres reachable. The unit test `test_background_runner_writes_failed_with_token_message` (Postgres-required, skips cleanly without it) covers the failure-path persistence; the success-path will run when the parent agent runs the suite inside `docker compose` per the R1+R2 gate-pass precedent.