Meal-Planner/.agent/phase-summaries/recipe-catalog-import-phase-2.md

Recipe Catalog Import Phase 2

Scope

Added a dedicated synchronous Wikibooks staging service and offline tests without changing recipe ingestion, recipe APIs, or existing dirty recipe tests.

Implementation

• Fixed https://en.wikibooks.org/w/api.php client with bounded category continuation, required caller User-Agent, sequential rate limiting, timeouts, streamed payload cap, HTTP Retry-After retries, and MediaWiki maxlag/ratelimited exponential backoff.
• Validated MediaWiki response shapes and restricted page fetches to bounded Cookbook: titles.
• Normalized rendered HTML into text-only recipe candidates: intro, infobox yields/times, ingredient and ordered instruction subsections, quantity/unit/source fields, and no images.
• Added deterministic validation and quarantine for invalid servings, insufficient ingredients/instructions, invalid quantities, placeholders, and distinctive title-token ingredient mismatches.
• Staged exact API parse payloads with canonical checksums, immutable provenance inputs, CC BY-SA 4.0 attribution/history, and source/page/revision deduplication. The service flushes only.

Verification

• Offline focused run: 22 passed, 2 deselected in 1.60s.
• Full run against disposable PostgreSQL: 24 passed in 3.08s, including the two-session concurrent deduplication test.
• The disposable database was removed after verification.
• backend/venv/bin/python -m py_compile backend/app/services/wikibooks_import.py backend/tests/test_wikibooks_import.py: passed.
• Scoped git diff --check: clean for the service, test, and two fixtures.

Status

Phase 2 is verified. Phase 3 has not started.