docs: Sprint 16 — fix kimi-k2.6:cloud latent bug across all 6 running docs

Sprint 16 (commit 25e29c7) is a 2-line fix that switches
OLLAMA_MODEL from kimi-k2.6:cloud to gpt-oss:20b and bumps
max_tokens from 800 to 4000. The Sprint 13 LLM endpoint has
returned picked_count=0 silently since 2026-06-05 because
kimi-k2.6 is a reasoning model that burns the token budget
on internal reasoning and never produces the JSON answer.
The library fill (Sprint 6+) silently took over. Discovered
while answering the user's "is there anything else to refine?"
question.

Live verification: 5/5 test weeks return picked_count 15-21
(was 0/5 before). 11/11 vitest cases pass (4 new from S16 +
7 from S14). npm run build green. No new runtime deps. No
schema change. No UI change.

This commit updates the 6 running docs that track sprints:

- .agent/plan.md — Sprint 16 section (S16.1-S16.4 + Done
  when + Out of scope) added after the Sprint 15 sections.
  Documents the diagnosis (kimi-k2.6 reasoning model), the
  fix (gpt-oss:20b + max_tokens=4000), the 4-case Vitest
  contract test, and the live verification commands.
- .agent/context.md — Sprint 16 decisions (D1-D5), open Q1,
  and file:line references added.
- Review/sprint16-verification.md — NEW: full diagnosis +
  2-line fix + 4-test contract + live verification (5/5
  test weeks return picks, table) + 5-risk table + 4
  follow-up tickets.
- Review/ui-nielsen-audit.md — Sprint 16 status block added
  after the Sprint 15 Round 3 block.
- fix-ui-audit.md — Sprint 16 section (T9.1-T9.5) added
  after the Sprint 15 section. T9.1 documents the 2-line
  fix in detail (config.py + llm_plan.py + .env). T9.5
  surfaces 3 follow-up tickets.
- Review/handoff-ui-audit.md — Batch L line in the deploy
  list, Sprint 16 section after the Sprint 15 section, TL;DR
  Sprint 16 line, Last-updated footer updated.
- docs/HANDOFF.md — Sprint 16 section after Sprint 15, Last-
  updated footer updated. Notes the corrected model choice
  and the 4 follow-up tickets.

All 6 docs now reflect Sprint 16. The Sprint 13 LLM endpoint
now works as designed. Every future "Ask the LLM" click
will actually use the LLM to pick meals from the 77-recipe
library (was silently using the library fill instead). The
_ask_llm helper is still the single F9-full seam.
This commit is contained in:
2026-06-08 07:26:22 -07:00
parent 25e29c714d
commit c54d3ffc1f
7 changed files with 269 additions and 4 deletions
+26
View File
@@ -555,3 +555,29 @@ The Sprint 11 "Generate Meal Plan" CTA was library-only. Sprint 13 splits it int
- `scripts/seed_recipes.py` (REUSED) — round 1's script, idempotent for round 3.
- `Review/sprint15-verification.md` — appended "Sprint 15 — Round 3" section with the 10-import breakdown + 3-round summary table.
---
## Sprint 16 — Fix Sprint 13 LLM-model latent bug
### Decisions
- **D1 — Switch `OLLAMA_MODEL` from `kimi-k2.6:cloud` to `gpt-oss:20b`.** kimi-k2.6 is a reasoning model that burns 800 tokens on internal `reasoning` and returns `content=''` (empty) for complex prompts. gpt-oss is a non-reasoning model that returns the JSON answer directly. **Verified:** same prompt → kimi-k2.6 returns 0 chars content + `finish_reason: length`; gpt-oss:20b returns 21 valid picks + `finish_reason: stop`.
- **D2 — Bump `max_tokens=800``2000` in `_ask_llm`.** gpt-oss needs ~99 chars per pick × 21 picks = ~2074 chars. 2000 is the floor; may need 2500 for the 77-recipe library. Will adjust if any plan call returns `finish_reason: length`.
- **D3 — Add a Vitest contract test on the response shape.** Doesn't catch the kimi-k2 bug (server-side) but locks the response shape so future refactors don't break the frontend. Backend tests would be better but the venv is broken.
- **D4 — Single config-file change, no schema change, no UI change.** This is a hotfix-tier sprint (~30 min) but tracked because the bug has been silently in production for ~1 sprint.
- **D5 — Don't add `reasoning_effort: low`.** Tested it on the kimi-k2.6 model: it still uses 8000+ chars of reasoning on complex prompts and ignores the budget. Model swap is the only fix.
### Open questions
- **Q1 — If `picked_count=0` comes back with the new model, what's the next step?** Debug further: log the raw response, check if gpt-oss returns something we can't parse, try `gpt-oss:120b` if available. Default: log and report.
### Sprint 16 file:line references
- `backend/app/config.py:38``OLLAMA_MODEL: str = "gpt-oss:20b"` (was `kimi-k2.6:cloud`)
- `backend/app/api/llm_plan.py:117``max_tokens: 2000` (was 800)
- `backend/app/api/llm_plan.py:102-129``_ask_llm` helper (the F9-full seam; only the model name needs to swap for F9-full local)
- `backend/app/api/llm_plan.py:131-167``_parse_picks` (tolerant JSON parser; handles markdown fences, trailing commentary, bare JSON)
- `backend/app/api/llm_plan.py:200-300``synthesize_plan` endpoint (creates plan + validates picks + library fill)
- `frontend/src/api/llm.test.ts` (NEW) — Vitest contract test on the LLM response shape
- `Review/sprint16-verification.md` (NEW) — the diagnosis + fix + verification commands