BlogAI agents

Designing an agent harness: loop, tools, context, permissions

A harness has six parts worth designing on purpose: the loop and its stop conditions, the tools and the shape of their results, the code that assembles context for each call, the permission layer, the sandbox, and the hooks that log and enforce policy. Keep the loop small, put the rules in code where the model cannot talk its way past them, and test the whole thing against a fixed set of tasks every time you change it.

Start with the loop and its stop conditions

We explain what a harness is, and why it matters as much as the model, in what is an agent harness. This article is about building one.

The loop is the smallest part. Anthropic’s guide to building effective agents describes agents as models using tools in a loop, steered by feedback from their environment, and the code is about that short: call the model, run the tools it asks for, append the results, repeat. The design work is in deciding when the loop ends.

type Stop = "done" | "max_turns" | "budget" | "timeout" | "stuck";

async function run(task: Task, limits: Limits): Promise<Result> {
  const state = await loadState(task.id); // history, turn count, spend

  while (true) {
    if (state.turns >= limits.maxTurns) return finish(state, "max_turns");
    if (state.spendUsd >= limits.maxUsd) return finish(state, "budget");
    if (Date.now() >= limits.deadline) return finish(state, "timeout");

    const reply = await callModel(assembleContext(state)); // retries 429, 5xx
    state.add(reply);

    if (reply.toolCalls.length === 0) return finish(state, "done");
    if (repeatsEarlierCalls(state, reply)) return finish(state, "stuck");

    for (const call of reply.toolCalls) {
      const decision = await checkPermission(call, state); // allow | deny | ask
      if (decision === "ask") return pause(state, call); // resume on approval
      const result =
        decision === "deny" ? refusal(call) : await runTool(call).catch(toToolError);
      state.add(formatResult(call, result)); // capped and readable
    }
    await saveState(state);
  }
}

The normal exit is a reply with no tool calls. Every other exit is a limit the model cannot override: a turn cap, a spend cap, a deadline, and a check for the same call made with the same arguments several times running. None of these is optional. The Claude Agent SDK’s loop documentation lists its turn and budget limits as unlimited unless you set them, and recommends a budget as the default for production agents. Check the defaults of whatever you build on.

Two details save trouble later. Return a distinct stop reason for each exit, because “finished” and “ran out of turns” need different handling by whatever called the agent. And treat a request for approval as a pause, not an ending. OpenAI’s guide to running agents gives the same advice: an approval is a paused run, not a new turn. You save the state, wait for the person, and resume the same run with its history and counters intact.

What should tools return?

Most of an agent’s context is tool output, so tool design is context design. Three rules cover most of what we have learned.

Build tools for jobs, not endpoints. A tool for every API endpoint makes the model do the joining. One find_customer_orders tool that accepts a name or an email and returns the last ten orders beats three tools the model has to chain. Anthropic’s write-up on writing tools for agents makes the same point and adds a useful test: describe each tool as you would to a new colleague, with the unwritten context spelled out.

Shape the result for a reader on a budget. Return names and dates in place of internal IDs where you can. Offer a short form by default and a detailed form on request. Cap the size of every result and, when you truncate, say so and say how to narrow the request. The same article notes that Claude Code limits tool responses to 25,000 tokens by default. Pick a number for yours. An uncapped tool will one day return a 3 MB log file.

Write errors the model can act on. “Invalid date: use YYYY-MM-DD, for example 2026-05-18” lets the next call succeed. A stack trace wastes a turn and several thousand tokens.

One loop-level decision belongs here too. When the model asks for several tools at once, run the read-only ones in parallel and the ones that change state one after another. Mark each tool as read-only or not in its definition, so the harness does not have to guess.

What goes into the context, and what gets dropped?

Before each model call the harness assembles the request: system prompt, tool definitions, history so far. We cover the principles in context engineering. The harness is where they become code.

Order for caching. Providers bill a repeated prompt prefix at a reduced rate only when it is identical, so the fixed parts go first and nothing volatile, such as a timestamp, goes near the top.

Drop what has been used. A file the agent read forty turns ago and has since edited is dead weight. Replace old tool results with a one-line placeholder and keep the most recent few in full.

Compact before the limit, not at it. Compaction replaces older history with a summary. Anthropic’s API offers it on the server (in beta when we checked), agent SDKs do it automatically, and you can write your own. Whichever you use, decide what the summary must keep: the goal and acceptance criteria, decisions made, files touched, errors still open.

Compaction has a failure mode that catches people. Instructions given early in the conversation can vanish into the summary. Anything that must hold for the whole run, such as the rules and the definition of done, belongs in the system prompt or in a file the harness re-reads on every call. It should never live only in the first user message.

Permission modes and approval prompts

The model proposes an action and the harness decides whether it runs. That separation is the point. Claude Code’s permissions documentation is explicit that its rules are enforced by the software and not by the model: instructions in a prompt shape what the model attempts and change nothing about what is allowed. Build yours the same way. A rule that lives only in the system prompt is a request.

A workable design has three rule lists checked in a fixed order: deny first, then ask, then allow. A broad deny cannot be undone by a narrower allow. On top of the rules sit modes, which set what happens to every action the rules do not mention. The set below is close to the one Claude Code ships, and it transfers well.

Mode Actions with no matching rule Use it for
Plan Reads run, changes do not Reviewing an approach before work starts
Ask A person approves each one Interactive use, new agents
Accept edits File edits in the workspace run, the rest ask Trusted work in an isolated directory
Classifier A second model reviews each one Long runs where prompts would stall the work
Deny by default Refused without asking Scheduled runs with nobody watching
Bypass Everything runs Disposable containers only

Take an agent that investigates failed nightly data jobs. Reading logs and running read-only SQL with a row limit are allowed. Re-running a job asks the first time and can be remembered for the session. Editing pipeline code asks every time. Dropping a table is denied in every mode.

Approval prompts deserve design too. Show the exact command or request, the system it touches, and whether it can be undone. Then ration them. A person who is asked forty times an hour stops reading, and from then on the prompt protects nothing. If a prompt is nearly always approved, turn it into an allow rule or move the action into a sandbox. Our article on guardrails for AI agents covers which actions should always need a person.

Sandboxing code and file access

Rules match the text of a command. They cannot know what a script does once it runs. If your agent executes code, it needs an operating-system boundary as well.

A sandbox has two layers and needs both. Filesystem isolation limits what a command can write, and ideally what it can read. Network isolation limits where it can send data. With only the first, a compromised command can still post your source code to a server. With only the second, it can rewrite a file that a later, unsandboxed step will execute.

Claude Code’s sandbox is a readable reference design. It uses Seatbelt on macOS and bubblewrap on Linux, limits writes by default to the working directory and a temp directory, and sends all traffic through a proxy whose list of allowed domains starts empty. Reads are much broader by default and include credential files until you deny them, which is the kind of default to look for in anything you adopt.

Two lessons carry over to any harness. First, the sandbox buys back autonomy: commands that run inside it need fewer approval prompts, because the damage they can do is bounded. Second, check what sits outside it. In that design the sandbox covers shell commands, while the built-in file tools and MCP servers run outside and rely on the permission rules. Draw the same map for your own system.

For server-side agents, give each run its own container or micro-VM with a short-lived, narrowly scoped credential, and throw it away afterwards. This is ordinary platform engineering, and it is cheaper to do before launch than after an incident.

Sub-agents and hooks

A sub-agent is a second loop with a fresh context, started by the first, that returns only its final answer. Its value is what stays out of the main context: the forty search results and six files it read to produce a five-line summary.

Use one when the side task is self-contained, produces a lot of intermediate output, or should have a narrower tool set than its parent, such as a reviewer that can read and cannot write. Avoid one when the work depends on the conversation so far, because a sub-agent sees only the brief you write for it, and when latency matters, because it starts cold. Each sub-agent fills a context of its own, so count its tokens against the parent’s budget and cap how deep the nesting can go. The wider trade-offs are in multi-agent systems.

Hooks are functions the harness calls at fixed points: before a tool runs, after it returns, before compaction, when the run stops. They let logging and policy live outside the loop. A before-tool hook can block a call or rewrite its arguments. An after-tool hook can redact a result or write an audit record. A stop hook can refuse to let the agent finish until the tests pass.

Decide one thing explicitly: what happens when a hook itself crashes or times out. For a logging hook, carry on. For a policy hook, block. Do not assume your framework’s default matches.

Errors, retries and a fixed task set

Split failures into two kinds. Transient ones, such as rate limits, timeouts and a 503 from the model API, are the harness’s job: retry with backoff a few times, without the model seeing it. Everything else goes back to the model as a tool result marked as an error, so that it can change approach. Count consecutive tool errors and stop the run after a handful. An agent retrying a broken call fifty times is a billing incident.

A retried write must also be safe to repeat. That, and what changes when a run lasts hours, is the subject of long-running AI agents.

Then test the harness as you would any other software, against a fixed set of tasks.

  1. Collect thirty or so real tasks, including some that should be refused or escalated.
  2. Give each a starting state and a check on the end state: the ticket exists, the file compiles, no table was dropped.
  3. Run every task several times, since the same input does not always take the same path.
  4. Record the pass rate, turns, tokens, tool errors, approvals requested and stop reasons.
  5. Re-run the set on every change to a prompt, a tool description, a permission rule or the model version.

Grade what the agent produced, not the route it took. And read the transcripts of the failures. Most turn out to be a confusing tool result or a missing piece of context, which are faults in the harness and not in the model. We go further in how to evaluate AI agents.

Where to start

In the first week, write the loop above with all its stop conditions and three tools, and run it against ten tasks. Log every request in full.

The first decision is build or adopt. If a provider’s agent SDK fits your tools and your approval flow, use it and spend your time on tools, rules and tests. Write your own loop when you need control over context assembly or want to change models freely.

In both cases, write the permission table before the system prompt. Listing every action the agent can take, and whether it is allowed, asked or denied, exposes most design problems while they are still on paper. It is the first thing we draw in an agent build at WeStack AI.

Questions

Asked often.

What are the main parts of an agent harness?

A loop that calls the model and runs the tools it asks for, a set of tool definitions and handlers, code that assembles and trims the context for each call, a permission layer that decides which actions run, a sandbox for code and files, and hooks for logging. Each part can be designed and tested separately from the model.

How should an agent loop decide when to stop?

The normal exit is a model reply with no tool calls. Add limits the model cannot override: a maximum number of turns, a spend budget, a wall-clock deadline and a check for the same call repeated. Return a distinct stop reason for each, so the caller knows whether the task finished or ran out of room.

Should I build my own harness or use an agent SDK?

Start with a provider's agent SDK if your task fits its tools and permission model, because the loop, compaction and sandboxing are already written. Build your own when you need control over context assembly, a custom approval flow or several model providers. In both cases the tools, permission rules and test tasks are yours to write.

Do approval prompts make an agent safe?

Not on their own. People stop reading prompts they see too often, so a prompt on every action trains them to click yes. Use rules to allow safe actions and deny dangerous ones outright, a sandbox to contain code execution, and keep prompts for the few actions that are irreversible or leave your systems.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.