Files
oikos/evals/iteration-readonly.yaml
dtoro f1dda7290a
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
fix(eval): fix manifests to require live inspection + approval followup
plan-always-readonly: prompt now demands live systemd timer inspection,
not just DB lookup. Added calls_tool: run assertion.
iteration-followup: added 'go ahead' as second followup so the
config_mutation plan gets approved and can execute.
iteration-readonly: replaced nonexistent lxc:prometheus with lxc:dns,
keep it read-only so no approval needed.
2026-07-15 10:02:46 +02:00

18 lines
665 B
YAML

# P5 eval: two read-only sub-tasks back-to-back. The follow-up is also
# read-only (no approval needed) so the agent can execute immediately after
# proposing the second plan. Asserts the session reopens and a second plan
# generation is created.
- name: iteration-readonly
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled?"
followups:
- "Now check the uptime of lxc:dns."
assertions:
- kind: completes
- kind: plan_generations
value: 2
- kind: proposes_plan
- kind: calls_tool
value: run
- kind: max_run_calls
value: 6