Recipe Catalog Operations

MealPlanner acquires reusable recipe text from English Wikibooks through a staging queue. Acquisition never publishes directly to the live catalog. An authenticated operator reviews each candidate before publication.

Source And License

Source: English Wikibooks Cookbook through the MediaWiki API.
Text license recorded per source revision: CC BY-SA 4.0.
Each staged row retains the source page, revision, acquisition time, attribution, license URL, exact API payload, and canonical SHA-256 checksum.
Imported images are intentionally excluded. An image may be reused only after its individual file-page license is verified and its attribution requirements are implemented.
Normalization is an adaptation of source text. Preserve the staging record and source link, identify Wikibooks contributors, link the CC BY-SA 4.0 license, and indicate that formatting/normalization changed the source when displaying or redistributing it.
Do not add another source until its durable-storage, transformation, attribution, image, deletion, and exit rights are documented.

Configuration

Set these values for both the backend and scheduler containers:

ADMIN_TOKEN=<random shared bearer token>
ADMIN_EMAIL=operator@example.com
WIKIBOOKS_USER_AGENT=MealPlanner/1.0 (https://your-host.example; operator@example.com)

The User-Agent must identify the deployment and provide a working contact URL or email. ADMIN_EMAIL becomes the reviewer identity. If it is empty, reviews are recorded as admin; a shared token cannot distinguish multiple operators.

Apply database migrations before acquisition:

cd backend
alembic upgrade head

Workflow

Feedback-driven weekly finalization searches Wikibooks and stages candidates. It does not create recipes.
An operator can also call POST /api/admin/trigger-discovery with family_profile_id, optional limit=1..25, and optional force=true. Forced discovery without feedback queries samples the broad Cookbook category.
Review GET /api/admin/recipe-imports?status=pending and GET /api/admin/recipe-imports/{id} using the configured admin bearer token.
Approve with POST /api/admin/recipe-imports/{id}/approve, or reject with POST /api/admin/recipe-imports/{id}/reject and JSON { "reason": "..." }.
Approval re-reads the immutable payload, verifies its checksum, normalizes and validates it again, maps exact canonical ingredient names, and commits the recipe and review record atomically.

Quarantined candidates cannot be approved until source data produces a valid new revision. Rejected and approved actions are idempotent where their existing state is still valid. If an approved Recipe is deleted, approving its retained staging row republishes and relinks it.

The retired /api/recipes/search and /api/recipes/import endpoints return HTTP 410. Historical Spoonacular scripts refuse write mode.

Bundled Starter Catalog

The app-owned app/data/starter_recipes.json catalog is separate from Wikibooks staging. Its operator loader validates all 50 recipes before opening its single write transaction, creates only globally scoped active manual recipes, and never modifies legacy or imported rows. It upserts ingredients only by their exact canonical name and emits a JSON summary.

cd backend
python scripts/load_starter_recipes.py --dry-run
python scripts/load_starter_recipes.py

Re-running the loader creates no additional marked recipes. Do not use this command as part of a production replacement without the separately approved backup and replacement procedure.

Validation

Candidates remain quarantined when they have invalid servings or durations, fewer than two ingredients or substantive instructions, missing/nonpositive/ambiguous quantities, placeholder instructions, malformed fractions, or distinctive title ingredients missing from the ingredient list. PostgreSQL integer bounds are checked before publication.

Published recipes start active. Catalog consumers select only active recipes. Quarantined recipes remain available in historical meal plans but cannot be recommended, randomly selected, added to a new plan, sent for image generation, or included in LLM planning.

Existing Catalog Audit

Run a report without changing data:

cd backend
python scripts/audit_recipe_catalog.py > recipe-catalog-audit.json

Review the JSON findings, then apply the same deterministic decisions atomically:

python scripts/audit_recipe_catalog.py --apply > recipe-catalog-audit-applied.json

The audit quarantines recipes with structural defects or unverified provenance. Spoonacular rows are marked unlicensed_external_source; non-manual rows with no source are marked missing_provenance. Manual recipes may remain active without external provenance. A Wikibooks recipe is trusted only when an approved staging row matches its source, external ID, and source URL.

Back up PostgreSQL before --apply. The command rolls back on errors, but an accepted quarantine decision changes catalog availability immediately.

Safe Verification

Never run API tests with only TEST_DATABASE_URL set when Compose already supplies DATABASE_URL; request-scoped sessions read DATABASE_URL. Point both variables to a disposable database:

export TEST_DATABASE_URL=postgresql://mealplanner:<password>@db:5432/mealplanner_catalog_test
export DATABASE_URL="$TEST_DATABASE_URL"
python -m pytest tests/test_starter_recipe_catalog.py tests/test_recipe_import_staging.py tests/test_wikibooks_import.py tests/test_recipe_import_admin.py tests/test_recipe_catalog_audit.py

Require a database name ending in _test for migration round-trip tests. Drop the disposable database after verification.

5.7 KiB Raw Blame History