# R3-0 (Swiftly API ingestion) — GATE PASSED

Date: 2026-05-05

## Live end-to-end scrape

```
POST /api/admin/scrape (Bearer test-admin-token) → 202 + scrape_log_id
[bg] Swiftly: discovered 17 categories
[bg] Background scrape complete: 10908/10908 items saved
final status: success
items_scraped: 10908
duration_seconds: 36
grocery_item count (after upsert): 9960
```

## Tests

```
pytest -q tests/  →  31 passed in 2.41s
```

Includes:
- 26 tests from R1+R2 gate
- 5 new Swiftly tests: parse categories fixture, parse category JSON, mapping invariants, 401 raises `SwiftlyAuthError`, background runner writes FAILED with actionable message.

## Schema

Migration `0005_grocery_item_external_id.py` adds:
- `grocery_item.external_id TEXT NULLABLE` (Swiftly product id)
- `grocery_item.source TEXT NULLABLE`
- composite index `ix_grocery_item_source_external_id`

Idempotency key for upserts: `(source, external_id)` with fallback `(name, scraped_url)` for legacy R2-A rows.

## Resolved

- Task #11 (LuckyCaliforniaScraper `_browser` attr bug): obsoleted — the new client uses `requests`, no Playwright lifecycle.
- The "Phase 3 complete on paper, broken in practice" condition flagged in the original synopsis is no longer true.

## Behavior to know

- `SWIFTLY_BEARER_TOKEN` is read from env at scraper init. The token in `.env.example` and `.env.test` is the user-supplied one and will expire (~hourly). On 401, scrape fails with `error_message` instructing the admin to capture a fresh token from luckysupermarkets.com's devtools network tab and update the env var.
- Bearer header is scoped to `prod.swiftlyapi.net` only; never sent to the public categories page.
- `LUCKY_STORE_ID=757` (Lucky California in San Pablo). Switch via env to target a different store.
- Playwright is still pinned in `requirements.txt` (other future scrapers may want it). Removable later if no other use case lands.

## Open items

- Task #8: ScrapeStatus lacks distinct queued/started states. Cosmetic.
- `.env.example` carries the live bearer token; user authorized. Refresh whenever the token expires.

## Phase 3 status

Was: Playwright scraper, 11 coupons, broken `_browser` attribute, 0 rows persisted via the endpoint.
Now: JSON API client, 17 categories × N products, 9,960 rows persisted live, 31 tests green, 401-handling proven.