admin/Meal-Planner
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement Lucky California scraper with Playwright + BeautifulSoup

Browse Source 
- Add BaseScraper with rate limiting, retries, session management
- Add LuckyCaliforniaScraper with Playwright for dynamic content
- Add ScraperService to save scraped items to grocery_item table
- Connect /api/admin/scrape to ScraperService
- Update ORIENTATION.md phase table
This commit is contained in:
admin
2026-05-04 20:50:27 -07:00
parent c735d21661
commit 933a0cc9db
6 changed files with 500 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ORIENTATION.md
+16 -11
View File
@@ -44,6 +44,12 @@ The family has been using meal kit services (Blue Apron → EveryPlate → Hungr
- [x] /api/shopping-list endpoints (aggregation, print)
- [x] /api/admin endpoints (scrape trigger, logs, stats)
**Phase 3 Progress** (2026-05-04):
- [x] Base scraper with rate limiting, retries, session management
- [x] LuckyCaliforniaScraper with BeautifulSoup + Playwright
- [x] ScraperService to save scraped items to grocery_item table
- [x] /api/admin/scrape endpoint connected to ScraperService
---
## Architecture Summary
@@ -132,17 +138,16 @@ User (web) ───► React UI ──► nginx ──► FastAPI ────
| Phase | Description | Status |
|-------|-------------|--------|
| 1 | Infrastructure (Docker, PostgreSQL, FastAPI, React, nginx) | **Complete** |
| 2 | Database & Models (SQLAlchemy models, Alembic migrations) | **Post-review fixes applied** |
| 3 | API Endpoints (CRUD, meal plans, shopping list, feedback) | Not Started |
| 4 | Lucky California Scraper (weekly ad, Playwright) | Not Started |
| 5 | Recipe Engine (CRUD, tagging, search) | Not Started |
| 6 | Meal Planner Engine (generation algorithm, substitutions) | Not Started |
| 7 | SendGrid Email Integration | Not Started |
| 8 | Web UI - Core (Dashboard, Meal Detail, Approval, Pantry) | Not Started |
| 9 | Web UI - Feedback (Feedback Portal, Learning) | Not Started |
| 10 | Shopping List & Print | Not Started |
| 11 | Image Strategy (scraped + AI fallback) | Not Started |
| 12 | Polish & Future (variety analysis, budget tracking) | Not Started |
| 2 | Database & Models (Alembic migrations, Pydantic schemas, API endpoints) | **Complete** |
| 3 | Lucky California Scraper (BeautifulSoup + Playwright, ScraperService) | **Complete** |
| 4 | Recipe Engine (CRUD, tagging, search) | Not Started |
| 5 | Meal Planner Engine (generation algorithm, substitutions) | Not Started |
| 6 | SendGrid Email Integration | Not Started |
| 7 | Web UI - Core (Dashboard, Meal Detail, Approval, Pantry) | Not Started |
| 8 | Web UI - Feedback (Feedback Portal, Learning) | Not Started |
| 9 | Shopping List & Print | Not Started |
| 10 | Image Strategy (scraped + AI fallback) | Not Started |
| 11 | Polish & Future (variety analysis, budget tracking) | Not Started |
---