From f5fb7558c417b8fd1f564b1e883e4c288aeaaca8 Mon Sep 17 00:00:00 2001 From: Peter Woolery Date: Wed, 3 Jun 2026 17:41:59 -0700 Subject: [PATCH] fix(migration): simplify aisle migration + add persistent backup script - Drop the empty batch_alter_table block and the meaningless set_config call from migration 0015. Temp tables still persist for the migration's session (Alembic's transactional_ddl). - New backend/scripts/persist_aisle_backup.sql creates public.ingredient_aisle_backup_0015 and public.grocery_item_aisle_backup_0015 permanent tables for operators who want a recoverable record beyond the migration. - Update Review/sprint2-verification.md, Review/ui-nielsen-audit.md and fix-ui-audit.md with the correct container-based deploy steps: docker compose exec db psql -U mealplanner -d mealplanner -f /dev/stdin < ...sql. Host psql is not available on the deployment host; the db runs inside the container. --- Review/sprint2-verification.md | 33 +++++++++--- Review/ui-nielsen-audit.md | 18 +++++-- .../versions/0015_normalize_pantry_aisles.py | 50 +++++++++---------- backend/scripts/persist_aisle_backup.sql | 24 +++++++++ fix-ui-audit.md | 11 +++- 5 files changed, 96 insertions(+), 40 deletions(-) create mode 100644 backend/scripts/persist_aisle_backup.sql diff --git a/Review/sprint2-verification.md b/Review/sprint2-verification.md index b0a20fe..6f96c8a 100644 --- a/Review/sprint2-verification.md +++ b/Review/sprint2-verification.md @@ -18,24 +18,43 @@ | `frontend/src/pages/ShoppingList.tsx` | B9: `AISLE_LABEL` map + `aisleDisplay()`; S3.3: 3-col grid on all viewports with compact mobile sizing. | | `frontend/src/pages/Recipes.tsx` | B11: `applied`/`pending` filter state, Apply/Reset buttons, active-count chip on Filters button, `role="region"`. | -## How to verify on the deployment host +## How to deploy on the remote host (the database is in a container) + +The dev DB runs inside `mealplanner-db-1`. No host `psql` is required. All commands run via `docker compose exec` from the project root. ```bash -cd /path/to/MealPlanner +cd ~/MealPlanner git pull -# 1. Backend migration (one-off, dry-run first) -psql "$DATABASE_URL" -f backend/scripts/dry_run_aisle_migration.sql -docker compose exec backend alembic upgrade head +# 1. (Optional but strongly recommended) Take a persistent backup of +# aisle values BEFORE the migration runs. This stays in the DB +# even after the migration session ends. +docker compose exec -T db psql -U mealplanner -d mealplanner \ + -f /dev/stdin < backend/scripts/persist_aisle_backup.sql -# 2. Frontend rebuild + restart +# 2. Dry-run preview — counts rows that would change, no writes. +docker compose exec -T db psql -U mealplanner -d mealplanner \ + -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql +# Expected output (example): +# tbl | rows_to_change | distinct_old_values +# --------------+----------------+--------------------- +# grocery_item | 10539 | 16 +# ingredient | 10657 | 27 + +# 3. If the dry-run row counts look sane, apply the migration: +docker compose exec backend alembic upgrade head +# Expected: "Running upgrade 0014 -> 0015, normalize_pantry_aisles" + +# 4. Frontend rebuild + restart (Sprint 2 UI changes are already merged). docker compose -f docker-compose.yml up -d --build frontend -# 3. Smoke-check the live site +# 5. Smoke-check the live site curl -s -o /dev/null -w "%{http_code}\n" http://100.108.208.56:8082/ # Expect: 200 ``` +If the dry-run row count is high (>1k on ingredient) and you want to inspect what would change before applying, uncomment the second query at the bottom of `dry_run_aisle_migration.sql` and re-run. + ## Manual smoke checks - `/` (mobile 390 px): Generate button visible on every empty slot; new meal title clamps to 2 lines without `B..`. diff --git a/Review/ui-nielsen-audit.md b/Review/ui-nielsen-audit.md index 0a89fe0..bc55429 100644 --- a/Review/ui-nielsen-audit.md +++ b/Review/ui-nielsen-audit.md @@ -30,13 +30,23 @@ The app looks polished on the surface (Tailwind palette, clean cards, working to > - **B11** Recipes filters: refactored to `pending`/`applied` state with explicit Apply / Reset buttons. `Filters` button shows active-count chip when filters are set. Wrapped in `role="region" aria-label="Filters"`. > - **S3.3** Shopping list stat cards: now `grid-cols-3` on all viewports with compact mobile sizing. > -> Deployment command: +> Deployment commands (run on the deployment host — DB is in a container, no host psql needed): > ```bash -> # On the deployment host: +> cd ~/MealPlanner > git pull -> # Apply the backend migration (one-off): +> +> # Optional: persistent backup of aisle values BEFORE the migration +> docker compose exec -T db psql -U mealplanner -d mealplanner \ +> -f /dev/stdin < backend/scripts/persist_aisle_backup.sql +> +> # Dry-run preview (no writes) +> docker compose exec -T db psql -U mealplanner -d mealplanner \ +> -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql +> +> # Apply the migration > docker compose exec backend alembic upgrade head -> # Rebuild & restart frontend: +> +> # Rebuild & restart frontend > docker compose -f docker-compose.yml up -d --build frontend > ``` diff --git a/backend/alembic/versions/0015_normalize_pantry_aisles.py b/backend/alembic/versions/0015_normalize_pantry_aisles.py index 47b9109..a30cc36 100644 --- a/backend/alembic/versions/0015_normalize_pantry_aisles.py +++ b/backend/alembic/versions/0015_normalize_pantry_aisles.py @@ -3,10 +3,28 @@ Revision ID: 0015 Revises: 0014 Create Date: 2026-06-02 + +Normalizes free-text aisle values on `ingredient.aisle` and `grocery_item.aisle` +to a fixed canonical set. Runs in a single transaction (Alembic default); both +op.execute calls share the same session, so the temp backup tables persist +for the duration of the upgrade. + +Backup tables: the temp tables `ingredient_aisle_backup` and +`grocery_item_aisle_backup` are created for the migration's session. They +auto-drop when the session ends. If you need a persistent backup, run +`backend/scripts/persist_aisle_backup.sql` BEFORE this migration. + +Deploy via Docker (the db runs inside a container; no host psql required): + + # 1. dry-run preview: + docker compose exec db psql -U mealplanner -d mealplanner \\ + -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql + + # 2. apply: + docker compose exec backend alembic upgrade head """ from typing import Sequence, Union -import sqlalchemy as sa from alembic import op # revision identifiers, used by Alembic. @@ -16,20 +34,6 @@ branch_labels: Union[Sequence[str], None] = None depends_on: Union[Sequence[str], None] = None -CANONICAL_AISLES = ( - "Produce", - "Meat & Seafood", - "Dairy & Eggs", - "Pantry", - "Frozen", - "Bakery", - "Beverages", - "Spices", - "Other", -) - -# Map from lowercased source value to canonical label. Keep the rule -# order narrow -> broad; longest matches win via SQL CASE. NORMALIZATION_RULES = [ ("canned goods", "Pantry"), ("canned", "Pantry"), @@ -78,23 +82,15 @@ def _normalize(table: str) -> None: def upgrade() -> None: - bind = op.get_bind() - with op.batch_alter_table("ingredient") as batch: - pass _normalize("ingredient") _normalize("grocery_item") - bind.execute( - sa.text( - "SELECT set_config('app.aisle_backup_retention', 'aisle_migration_0015', false)" - ) - ) def downgrade() -> None: - # Best-effort downgrade: the backup temp tables only exist within the - # upgrade transaction. Restoring the pre-normalization state is not - # possible from this migration alone. Operators must restore from a - # database snapshot taken before upgrade. + # The temp backup tables only exist for the migration's session. + # Restoring the pre-normalization state is not possible from this + # migration alone. Operators must restore from a database snapshot + # taken before upgrade. raise NotImplementedError( "Cannot reverse aisle normalization without an external backup. " "Restore the database from a snapshot taken before 0015 was applied." diff --git a/backend/scripts/persist_aisle_backup.sql b/backend/scripts/persist_aisle_backup.sql new file mode 100644 index 0000000..4508a4e --- /dev/null +++ b/backend/scripts/persist_aisle_backup.sql @@ -0,0 +1,24 @@ +-- Persist a permanent backup of aisle values BEFORE running migration 0015. +-- Run via Docker (no host psql required): +-- +-- docker compose exec db psql -U mealplanner -d mealplanner \ +-- -f /dev/stdin < backend/scripts/persist_aisle_backup.sql +-- +-- Creates two permanent backup tables in the public schema. They are NOT +-- auto-dropped; you can DROP them manually after you confirm the migration +-- is correct: +-- +-- DROP TABLE public.ingredient_aisle_backup_0015; +-- DROP TABLE public.grocery_item_aisle_backup_0015; + +CREATE TABLE IF NOT EXISTS public.ingredient_aisle_backup_0015 AS + SELECT id, aisle, created_at FROM public.ingredient WHERE aisle IS NOT NULL; + +CREATE TABLE IF NOT EXISTS public.grocery_item_aisle_backup_0015 AS + SELECT id, aisle FROM public.grocery_item WHERE aisle IS NOT NULL; + +SELECT 'ingredient backup' AS table_name, COUNT(*) AS rows_backed_up + FROM public.ingredient_aisle_backup_0015 +UNION ALL +SELECT 'grocery_item backup' AS table_name, COUNT(*) AS rows_backed_up + FROM public.grocery_item_aisle_backup_0015; diff --git a/fix-ui-audit.md b/fix-ui-audit.md index f9735f3..f9d5b90 100644 --- a/fix-ui-audit.md +++ b/fix-ui-audit.md @@ -115,10 +115,17 @@ Resolve the 14 issues (5 P0, 6 P1, 3 P2) from `Review/ui-nielsen-audit.md` in th - Creates a `TEMP` backup table for each of `ingredient.aisle` and `grocery_item.aisle` (so a DBA can recover via `SELECT * FROM pg_temp.ingredient_aisle_backup` if needed). - `UPDATE`s both columns via a generated `CASE LOWER(COALESCE(aisle,'')) WHEN ... END` mapping. Mapped variants: `canned goods`/`canned` → `Pantry`, `freezer`/`frozen` → `Frozen`, `dairy`/`eggs`/`cheese`/`milk`/`yogurt` → `Dairy & Eggs`, `meat`/`seafood`/`fish`/`chicken`/`beef`/`pork`/`meat_seafood` → `Meat & Seafood`, `bakery`/`bread` → `Bakery`, `beverage`/`beverages`/`drinks` → `Beverages`, `spice`/`spices`/`seasoning` → `Spices`, `pantry`/`dry`/`snack`/`snacks` → `Pantry`, anything else → `Other`. NULL stays NULL. - **Downgrade:** raises `NotImplementedError` — operator must restore from a pre-migration snapshot. Documented in migration docstring. -- **Dry-run SQL helper (`backend/scripts/dry_run_aisle_migration.sql`):** standalone `psql` query that counts rows that *would* change per table, no writes. +- **Dry-run SQL helper (`backend/scripts/dry_run_aisle_migration.sql`):** standalone SQL that counts rows that *would* change per table, no writes. Run via `docker compose exec -T db psql -U mealplanner -d mealplanner -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql` (no host psql needed; the db runs in a container). +- **Persistent backup helper (`backend/scripts/persist_aisle_backup.sql`):** creates `public.ingredient_aisle_backup_0015` and `public.grocery_item_aisle_backup_0015` permanent tables. Run BEFORE the migration if you want a recoverable record beyond the migration's session. - **Verify (on dev DB):** ```bash - psql "$DATABASE_URL" -f backend/scripts/dry_run_aisle_migration.sql + # Persistent backup (optional, recommended) + docker compose exec -T db psql -U mealplanner -d mealplanner \ + -f /dev/stdin < backend/scripts/persist_aisle_backup.sql + # Dry-run + docker compose exec -T db psql -U mealplanner -d mealplanner \ + -f /dev/stdin < backend/scripts/dry_run_aisle_migration.sql + # Apply docker compose exec backend alembic upgrade head ``` Add a new item with aisle "pantry" → stored as `Pantry`. Open Pantry list → all rows show sentence-case canonical labels. Frontend `npm run build` clean.