feat: phase r1+r2 recovery + r3-0 swiftly api ingestion

R1 stabilization: pytest harness with transactional db fixture, smoke
+ alembic + auth + scrape + approval + swiftly tests, github actions
ci yaml. Bearer-token admin auth + signed-cookie session for family
ui mutations. Async POST /api/admin/scrape (BackgroundTasks, returns
202). Path canonicalization (no /list, /planned suffixes). DATABASE_URL
fail-fast on empty.

R2 deferred-risk spikes: live lucky california fetch (R2-A), full
email+per-voter approval click round trip with single-use enforcement
(R2-B, console email backend, sendgrid stub).

R3-0 phase 3 redesign: replaced playwright html scraper with requests
based swiftly json api client. 17 categories, ~10k products per scrape,
upsert by (source, external_id). 401 surfaces actionable token-refresh
message via ScrapeLog.error_message.

Pre-existing defects fixed: shopping_list.py syntax error blocking app
import, MealPlan.votes orphan relationship, JSONB(astext=True) invalid
kwarg, missing requests dep, calorie_target schema drift, every SQLEnum
needed values_callable, 0001 had empty downgrade(), seed had duplicate
ingredient rows.

Migrations added: 0003 grocery_item.description, 0004 family_profile.
calorie_target, 0005 grocery_item.external_id + source + composite index.

Verified: 31/31 pytest green, alembic upgrade->downgrade->upgrade clean,
frontend npm run build clean, live scrape 9,960 grocery_item rows in 36s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-05 14:08:19 -07:00
co-authored by Claude Opus 4.7
parent b9434967ed
commit 8e89f793d5
58 changed files with 3594 additions and 348 deletions
+13
View File
@@ -13,6 +13,15 @@ RECIPES_EMAIL=you@example.com
# Lucky California
LUCKY_CA_URL=https://luckysupermarkets.com
# Swiftly product API (Lucky CA backend). Token expires hourly; capture a
# fresh one from the luckysupermarkets.com network panel on any
# /search/api/v1 request and paste it here. The scraper aborts with a
# FAILED ScrapeLog row whose error_message asks for a refresh on 401.
LUCKY_STORE_ID=757
SWIFTLY_API_BASE=https://prod.swiftlyapi.net
SWIFTLY_CATEGORIES_URL=https://luckysupermarkets.com/categories
SWIFTLY_BEARER_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IjJiMzZhYjQxYTczOTJlMTRlNjM1ZmRlM2M2YWYwOWZlYmFhM2YyZDYiLCJ0eXAiOiJKV1QifQ.eyJzd2lmdGx5U2hvcHBlcklkIjoiYWQ2NjMyYjItOGY1Ny00NzgyLTgwZDAtOTJjMDZjNDI4YjY0IiwiYmFubmVySWQiOiJhM2IxMTcxNy1mNGNhLTQxOTYtYjY3MC1lNTE0MmMyMDVkZWUiLCJzZWxlY3RlZFN0b3JlIjoiNzU3IiwicHJvdmlkZXJfaWQiOiJhbm9ueW1vdXMiLCJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vc3dpZnRseS1sdS1wcm9kIiwiYXVkIjoic3dpZnRseS1sdS1wcm9kIiwiYXV0aF90aW1lIjoxNzc3OTg3NTcwLCJ1c2VyX2lkIjoicHJWUkh0Ujd5OFBKNEpyQTJtaGw4RW04VkpzMSIsInN1YiI6InByVlJIdFI3eThQSjRKckEybWhsOEVtOFZKczEiLCJpYXQiOjE3Nzc5ODc2MTcsImV4cCI6MTc3Nzk5MTIxNywiZmlyZWJhc2UiOnsiaWRlbnRpdGllcyI6e30sInNpZ25faW5fcHJvdmlkZXIiOiJhbm9ueW1vdXMifX0.VjlmFQVKrHy9RAqqqXpA8anaOhtYrgR9Odi77qlDcArnxb_4k_6imwNm48yuoH4umCMRKJSb3qcbWyBWSvcSjwtAKFhZLfH_QGX5VYMuyrJXtov8Hyy26jA12zLWxGxmOwmhow30ZTiElwqS0FSrA2GNPNF3l7ZLCIPnED8ijQo4XGb7NnZJ7egFdcvHwbpiZ6a4pFgY-74VBM411c9f4NQBamc3S5Xwez5T7bSA3KJ2ST6MpJ3G5_fIG-53ah1XqfsDBllqM4KxgQ0c1Nbux37kW4U3kVzNx9rCx8JZHBp8gHe_mDsbNO0bSXAzicQJuLIKGRPQtNLcZdtckMAetA
# AI Image Generation (optional)
AI_IMAGE_ENABLED=false
AI_IMAGE_PROVIDER=openai
@@ -21,3 +30,7 @@ AI_IMAGE_API_KEY=sk-your-api-key
# Application
LOG_LEVEL=INFO
SECRET_KEY=change-me-to-a-random-secret-key
# Auth
ADMIN_TOKEN=change-me-to-a-random-admin-token
SESSION_PASSWORD=change-me-to-the-family-shared-password