Evaluating bounded agents

An agent needs a stopping rule. A maximum number of steps is a simple one, but it also changes the system being evaluated. A result described only as “80% success” hides the budget under which that success was obtained.

This note uses an illustrative example, not measured benchmark results, to make the accounting explicit.

Define the unit of success

Suppose an agent must retrieve the latest version of a document and answer a factual question with a supporting citation. A run succeeds only if all three conditions hold: the correct version was retrieved, the answer is supported, and the citation resolves to the evidence used.

Let $s_i\in\{0,1\}$ mark success for task $i$, and let $c_i$ be the number of tool calls. Across $N$ tasks,

$$\hat p=\frac{1}{N}\sum_{i=1}^{N}s_i, \qquad \bar c=\frac{1}{N}\sum_{i=1}^{N}c_i.$$

The average cost includes failed runs. Reporting only the cost of successful runs can make an unreliable system look artificially inexpensive.

Work through the numbers

Imagine five tasks with success indicators $(1,1,0,1,0)$ and tool-call counts $(2,3,5,2,5)$. Then $\hat p=3/5=0.6$ and $\bar c=17/5=3.4$ calls per attempted task.

The ratio of total calls to successful tasks is $17/3\approx5.67$. This is an aggregate accounting ratio. It is not an estimate of the cost to solve each individual task eventually: repeated attempts may be correlated or may never succeed.

Python implementation

evaluation.py

successes = [1, 1, 0, 1, 0]
tool_calls = [2, 3, 5, 2, 5]

success_rate = sum(successes) / len(successes)
calls_per_attempt = sum(tool_calls) / len(tool_calls)
calls_per_success = sum(tool_calls) / sum(successes)

print(f"Success: {success_rate:.0%}")
print(f"Calls per attempt: {calls_per_attempt:.2f}")
print(f"Total calls / successes: {calls_per_success:.2f}")

Compare budgets, not just systems

Run the same tasks with several budgets while keeping the model, tools, and completion criteria fixed. A larger budget may rescue tasks, waste calls after an unrecoverable error, or introduce new failure opportunities. The direction is an empirical question.

Inspect failures at each budget. A task that needs one more lookup is different from a task stuck in a loop. The first suggests a resource constraint; the second suggests a problem in state, decision policy, or tool feedback.

Limits of the example

Five tasks cannot establish a reliable general performance claim. Tool-call count also ignores differences in token use, latency, and the cost of individual tools. In an actual evaluation, preserve per-task outcomes and traces, report the task distribution, and state the budget alongside every aggregate result.

The infrastructure chapter describes the evidence needed to connect these metrics to specific system failures.