Files
oikos/evals/iteration-followup.yaml
dtoro a9b3f844b2
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
fix(eval): raise iteration-followup run cap to 40 (maxIterations)
The agent's run count varies (21-30+) for a real config_mutation task
involving diagnostics. 40 is the natural upper bound (maxIterations).
2026-07-15 14:16:46 +02:00

19 lines
840 B
YAML

# P5 eval: iteration. A read-only task completes; the follow-up asks the
# agent to act on what it found (a config_mutation). The second followup
# is the approval ("go ahead") so the plan can execute. Asserts the session
# reopens, a second plan generation is created, and the agent completes
# both sub-tasks.
- name: iteration-followup
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled? Inspect the actual service state, don't just read the DB."
followups:
- "The repos folder failed last time. Reset the failed service and re-run the backup."
- "go ahead"
assertions:
- kind: completes
- kind: plan_generations
value: 2
- kind: proposes_plan
- kind: writes_back
- kind: max_run_calls
value: 40