1 The architecture of an agent

An agent is easiest to understand as a program with a model inside its control loop. The model proposes what to do. Application code decides whether that proposal is valid, performs the operation, and records what happened.

This separation gives us a useful design principle: keep decisions flexible, and make the consequences of those decisions explicit.

A minimal system

Four components are enough to describe a useful starting architecture:

1. A policy chooses an action from the available context. In a language-model system, this usually involves a model call and structured output.
2. An orchestrator validates the proposal and controls execution.
3. Tools expose narrow operations such as searching documents or reading a file.
4. State records the task, observations, decisions, and remaining budget.

The diagram describes an implementation pattern, rather than a requirement to adopt a particular framework. A small loop can be easier to inspect than a large abstraction whose state transitions are hidden.

Decisions and observations

Let $s_t$ be the application state at step $t$, $c_t$ the context assembled for the model, and $a_t$ its proposed action. One abstract description is:

$$\begin{aligned} c_t &= C(s_t), \\ a_t &\sim \pi_\theta(\,\cdot\mid c_t), \\ o_{t+1} &= E(a_t), \\ s_{t+1} &= U(s_t, a_t, o_{t+1}). \end{aligned} \tag{1.1}$$(1.1)

In (1.1), $C$ selects context, $E$ executes an allowed action, and $U$ updates state. Validation occurs before $E$: an invalid proposal should produce a controlled error, rather than an unintended operation.

The ReAct paper studied interleaving model-generated reasoning and actions, using observations from an environment to inform later steps. It offers a concrete research example of this feedback pattern; it does not imply that every application needs to store or display a model’s private reasoning. Read the original paper.

State is larger than context

The application can retain a complete event log while sending only a small selection to the model. This distinction matters when tool responses are large or a task spans many steps.

For a document assistant, persistent state might include document identifiers, validated citations, completed searches, and errors. Context might contain the current question, a short summary, and the most relevant passages. A summary is a lossy representation; keep the underlying evidence available for inspection.

Transformer attention, summarized in (2.3), operates over the context supplied in a particular call. It does not by itself provide an application’s durable memory.

A bounded execution loop

This Python sketch shows the control boundary without depending on an agent library. policy and each tool are application-provided functions. A real model adapter would parse and validate its structured response before returning an action.

Python implementation

agent_loop.py

def run_agent(task, policy, tools, max_steps=8):
    events = [{"kind": "task", "text": task}]

    for step in range(max_steps):
        action = policy(tuple(events))
        kind = action.get("kind")

        if kind == "finish":
            return {"status": "complete", "answer": action["answer"]}
        if kind != "tool" or action.get("name") not in tools:
            raise ValueError("Policy proposed an unsupported action")

        arguments = action.get("arguments", {})
        if not isinstance(arguments, dict):
            raise ValueError("Tool arguments must be an object")

        # Each tool validates its own argument schema and authorization.
        observation = tools[action["name"]](**arguments)
        events.append({"kind": "result", "step": step,
                       "action": action, "value": observation})

    return {"status": "budget_exhausted", "events": events}

The step limit prevents an unbounded number of iterations. It does not impose a wall-clock deadline: model calls and tools also need timeouts. A production runner should define how it records exceptions, cancels work, and resumes after interruption.

Budgets are part of the state

A useful budget has more than one dimension. For example, remaining tokens and tool calls can be represented as:

$$\mathbf{b}_t = \begin{bmatrix} B_{\mathrm{tokens}} - \sum_{i=0}^{t-1} k_i \\ B_{\mathrm{calls}} - t \end{bmatrix}, \qquad r_t = \frac{k_t}{B_{\mathrm{tokens}}}. \tag{1.2}$$(1.2)

Here $k_i$ counts the tokens charged at step $i$, including the chosen accounting convention for input and output. Stop before starting an operation that cannot fit the remaining allowance. The fraction $r_t$ is simply a diagnostic share of the initial token budget.

Tools define the authority boundary

Treat a tool call as untrusted input to an application interface. Validate its schema and check authorization in application code. A tool named read_document should expose exactly the document access the task requires.

For operations that change external state, make retries deliberate. An idempotency key can identify one intended operation so that retrying after a network failure does not create a second copy. Retrieved text belongs in the evidence channel; it should not acquire the authority of application instructions merely because the model can read it.

Evaluate the complete run

A plausible final paragraph is only one observable. A document assistant should also produce citations that resolve to supporting evidence, respect its access scope, and terminate within its budget.

Start with a small set of representative tasks and explicit expected outcomes. Record action names, timings, result status, and evidence identifiers so failures can be reproduced. Avoid collecting sensitive tool payloads by default. The infrastructure section connects this event trail to operational monitoring.

References

• Yao et al. (2023), ReAct: Synergizing Reasoning and Acting in Language Models. A research example of reasoning, action, and observation interleaved in a language-model system.