Public Access
fix(llm): Sprint 16 — switch OLLAMA_MODEL from kimi-k2.6:cloud to gpt-oss:20b
Sprint 13 (commit bae9403) set OLLAMA_MODEL=kimi-k2.6:cloud.
kimi-k2.6 is a reasoning model that burns the entire max_tokens=800
budget on internal reasoning and never produces the JSON answer
for the Sprint 13 prompt. Every /api/llm/plan call has returned
picked_count=0 since 2026-06-05. The library fill (Sprint 6+)
silently took over, masking the bug. Every "Ask the LLM" click
paid Ollama costs for nothing.
Discovered while answering the user's "is there anything else to
refine?" question. Added a temp debug log to _ask_llm, saw
raw_response='' with finish_reason: length. Verified on Ollama
Cloud: gpt-oss:20b (OpenAI's open-source 20B non-reasoning
model) returns 21 valid picks in 2074 chars on the same prompt.
finish_reason: stop. Reasoning field is 239 chars vs kimi-k2.6's
8206+ chars.
Two-line fix:
- backend/app/config.py:38 — OLLAMA_MODEL: str = "gpt-oss:20b"
(was "kimi-k2.6:cloud")
- backend/app/api/llm_plan.py:117 — max_tokens: 4000 (was 800).
21 picks × ~100 chars + reasoning + boilerplate ≈ 2100+ chars;
4000 gives 2x headroom.
Plus the host's .env (or docker-compose env) was also set to
OLLAMA_MODEL=gpt-oss:20b — pydantic settings read env first, so
the .env change is what actually fixed the running container. The
config.py default is a backup for new deploys.
Plus frontend/src/api/llm.test.ts (NEW, 4 cases) — Vitest
contract test on the LLM response shape. Locks plan_id (UUID),
picked_count / filled_count / failed_count (non-negative integers
summing to ≤ 21), and reasoning (string|null). Catches
response-shape regressions so a future model swap that breaks
the JSON contract is caught at npm test time. The 4 cases: 8a
(POST to /llm/plan with payload), 8b (response.plan_id is a
valid UUID), 8c (counts are non-negative integers summing to
≤ 21), 8d (reasoning is string or null).
Verified: 11/11 vitest cases pass (4 new from S16 + 7 from S14).
npm run build green. Live API: 5/5 test weeks return picked_count
15-21 (was 0/5 before). Backend env verified:
docker exec mealplanner-backend-1 env | grep OLLAMA_MODEL →
gpt-oss:20b. No new runtime dependencies. No migration. No
schema change. No UI change.
Deploy: git pull + docker compose up -d --build backend frontend.
The .env change should already be in place; verify with
docker exec mealplanner-backend-1 env | grep OLLAMA_MODEL.
This commit is contained in:
@@ -114,7 +114,7 @@ def _ask_llm(prompt: str) -> Optional[str]:
|
||||
json={
|
||||
"model": settings.OLLAMA_MODEL,
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"max_tokens": 800, # kimi-k2 reasons before answering; 21 picks need headroom
|
||||
"max_tokens": 4000, # 47-recipe library: 21 picks × ~100 chars + reasoning + boilerplate ≈ 2100+ chars; 4000 gives 2x headroom
|
||||
"temperature": 0,
|
||||
},
|
||||
timeout=_LLM_TIMEOUT_SECS,
|
||||
|
||||
Reference in New Issue
Block a user