# R1-C — Async scrape + grocery_item.description ## What changed - `backend/app/api/admin.py` — `POST /api/admin/scrape` now returns **202** with body `{"status": "queued", "scrape_log_id": ""}`. Endpoint delegates to `enqueue_scrape(db, source, scrape_type, background_tasks)`. - `backend/app/services/scraper_service.py` — split: - `enqueue_scrape()` inserts a `ScrapeLog` row (status=`STARTED`), `db.commit()`, then `background_tasks.add_task(_run_scrape_in_background, log.id, ...)`. Returns the refreshed log row. - `_run_scrape_in_background(log_id, source, scrape_type)` opens its OWN `SessionLocal()` (request session is gone by then), runs the scraper, sets terminal status (`SUCCESS`/`FAILED` + `error_message` + `duration_seconds`), `db.close()` in `finally`. Rolls back before writing FAILED to avoid detached instances. - `ScraperService.run_scrape` retained for direct/test callers; both paths share `_do_scrape()` and `_save_grocery_item()`. - `_save_grocery_item` now maps `item_data["description"]` → `GroceryItem.description` on both insert and update. - `backend/app/models/__init__.py` — added `description = Column(Text, nullable=True)` to `GroceryItem`. - `backend/alembic/versions/0003_grocery_item_description.py` — `revision="0003"`, `down_revision="0002"`. `op.add_column("grocery_item", sa.Column("description", sa.Text(), nullable=True))` / `op.drop_column` on downgrade. - `backend/tests/test_scrape_endpoint.py` — monkeypatches `_run_scrape_in_background` to a no-op, POSTs with valid bearer, asserts 202 + `scrape_log_id`, reads back the row through the test session, asserts `status == ScrapeStatus.STARTED` and `completed_at is None`. ## Endpoint contract `POST /api/admin/scrape` (admin bearer required): - Request: optional query `source` (default `lucky_california`), `scrape_type` (default `weekly_ad`). - Response: **202** + `{"status": "queued", "scrape_log_id": ""}`. - Poll `GET /api/admin/logs/{id}` for terminal state. ## Status enum note Existing `ScrapeStatus` enum is `started|success|failed` only — no `pending`/`running`/`completed` members. Added migration would have to alter the PG enum type (and seed data assumes 3-value). Reused `STARTED` as the queued/in-flight state; spec semantics map cleanly. If you want explicit `PENDING`/`RUNNING`/`COMPLETED`, that's a separate enum migration. ## DB session in the bg task Imports `SessionLocal` from `app.database` at function call time and opens a fresh session per task; the request-scoped session is closed when the 202 is sent. ## Verification - `pytest tests/test_lucky_ca_scraper.py -q` → 1 passed (R2-A still green). - `tests/test_scrape_endpoint.py` parses and is wired correctly; runs only with `TEST_DATABASE_URL` set (other tests follow the same convention). - `python -c "from app.services.scraper_service import enqueue_scrape, _run_scrape_in_background, ScraperService; from app.models import GroceryItem, ScrapeStatus"` → ok. ## Blocker (pre-existing, OUT OF R1-C SCOPE) `backend/app/api/shopping_list.py:59` has a syntax error from commit c735d21: `Ingredient.id.in_ all_ingredient_ids` — missing parens, should be `Ingredient.id.in_(all_ingredient_ids)`. This blocks `from app.main import app` and therefore `tests/test_smoke.py::test_app_imports`. Untouched by R1-A, R1-B+D, R2-A, or R1-C — they all couldn't actually exercise the import path. Recommend a one-character fix in the next round (it's outside my owned files).