dtoro
3d7fa99560
proposePlan: mark pending steps as 'replaced' instead of DELETE, so the generation counter (MAX+1) sees prior generations. Without this, a first plan that was proposed but never executed would be wiped, resetting the counter — a follow-up's plan would look like generation 1 instead of 2. plan-always-readonly: raise max_run_calls from 3 to 6 (agent inspects thoroughly).
15 lines
630 B
YAML
15 lines
630 B
YAML
# P5 eval: a read-only question that requires live inspection (not just DB
|
|
# lookup). Asserts the plan-first gate works: the agent must propose_plan
|
|
# before run, even for a trivial read-only task.
|
|
- name: plan-always-readonly
|
|
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled? Inspect the actual service state, don't just read the DB."
|
|
assertions:
|
|
- kind: completes
|
|
- kind: proposes_plan
|
|
- kind: plan_before_run
|
|
- kind: calls_tool
|
|
value: run
|
|
- kind: writes_back
|
|
- kind: max_run_calls
|
|
value: 6
|