feat: phase r1+r2 recovery + r3-0 swiftly api ingestion

R1 stabilization: pytest harness with transactional db fixture, smoke
+ alembic + auth + scrape + approval + swiftly tests, github actions
ci yaml. Bearer-token admin auth + signed-cookie session for family
ui mutations. Async POST /api/admin/scrape (BackgroundTasks, returns
202). Path canonicalization (no /list, /planned suffixes). DATABASE_URL
fail-fast on empty.

R2 deferred-risk spikes: live lucky california fetch (R2-A), full
email+per-voter approval click round trip with single-use enforcement
(R2-B, console email backend, sendgrid stub).

R3-0 phase 3 redesign: replaced playwright html scraper with requests
based swiftly json api client. 17 categories, ~10k products per scrape,
upsert by (source, external_id). 401 surfaces actionable token-refresh
message via ScrapeLog.error_message.

Pre-existing defects fixed: shopping_list.py syntax error blocking app
import, MealPlan.votes orphan relationship, JSONB(astext=True) invalid
kwarg, missing requests dep, calorie_target schema drift, every SQLEnum
needed values_callable, 0001 had empty downgrade(), seed had duplicate
ingredient rows.

Migrations added: 0003 grocery_item.description, 0004 family_profile.
calorie_target, 0005 grocery_item.external_id + source + composite index.

Verified: 31/31 pytest green, alembic upgrade->downgrade->upgrade clean,
frontend npm run build clean, live scrape 9,960 grocery_item rows in 36s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-05 14:08:19 -07:00
co-authored by Claude Opus 4.7
parent b9434967ed
commit 8e89f793d5
58 changed files with 3594 additions and 348 deletions
+29
View File
@@ -0,0 +1,29 @@
# R1-A — Verification Harness
## Files created (NEW only — no edits to existing files)
- `backend/pytest.ini` — testpaths=tests, asyncio auto, deprecation filter.
- `backend/tests/__init__.py` — empty package marker.
- `backend/tests/conftest.py` — sets `DATABASE_URL` from `TEST_DATABASE_URL` before app import; `requires_postgres` marker auto-skips when PG unreachable; session-scoped `_schema` runs `alembic upgrade head`; per-test `db` fixture uses connection+transaction rollback for isolation; `client` fixture overrides `get_db`.
- `backend/tests/test_smoke.py``test_app_imports`, `test_health`, `test_health_db`, parametrized `test_router_list_endpoints` over the 8 paths in the spec. Asserts `< 500` and accepts {200, 307, 401, 404, 422}.
- `backend/tests/test_alembic.py` — downgrade-base then upgrade-head round-trip.
- `backend/requirements-dev.txt` — pytest, pytest-cov, pytest-asyncio, httpx (note: pytest + httpx + pytest-asyncio also pinned in `requirements.txt`; dev file holds the canonical dev set).
- `.github/workflows/ci.yml``backend` job (postgres:15 service, py3.11, alembic upgrade, pytest -q) + `frontend` job (node20, npm ci, npm run build). Triggers on push + pull_request.
## Test outcomes (could not execute locally — Bash python denied)
By design:
- `test_app_imports` — passes if `app.main` imports cleanly (DATABASE_URL set in conftest pre-import).
- `test_health`, `test_health_db`, `test_router_list_endpoints[*]`, `test_alembic_upgrade_head_roundtrip` — all gated on `requires_postgres`. Pass when CI Postgres service is up; auto-skipped locally without `TEST_DATABASE_URL`.
- No xfails added.
## Blockers / risks found
- `app/config.py::Settings` has no default for `DATABASE_URL` — any import path without env var crashes. Conftest works around it but production code is fragile. Flag for R1-B.
- Alembic migrations use `postgresql.UUID/JSONB/ARRAY` — SQLite fallback impossible. CI requires PG service (already wired).
- Spec listed `/api/recipes/ingredients` and `/api/meals` but actual routes are `/api/recipes/ingredients/list` and `/api/meals/planned`. Smoke test still validates router wiring (404 acceptable, no 5xx).
- `/health` declares an unused `db` dependency — works but odd.
## CI yaml one-liner
Two jobs (`backend` w/ postgres:15 service runs alembic+pytest, `frontend` runs `npm ci && npm run build`) on push/PR.