Login: dewitt                            Name: DeWitt Clinton
Directory: /home/dewitt                  Shell: /bin/zsh
On since Sat Nov 22 14:00 (PST) on ttys000, idle 
No unread mail.
Plan:

Agent Harness 101

Vivek Trivedy wrote that an agent is a model + a harness.

If you would like a refresher on what a model is and how they are built then no one demonstrates it better than Andrej Karpathy.

This post attempts to cover what a harness is in the simplest terms possible.

We’ll walk through creating a harness here step by step using the Anthropic Messages API as a teaching example. The full code is available in a single file harness.py.

The code here is simple, but the concepts are effectively the same ones behind Claude Code, Cursor, Codex, LangChain, ADK, etc, differing only in sophistication and scale.

Inner agentic loop

Fundamentally, an agent is a loop of interactions with the outside world (models, tools, subagents, humans, etc).

What an agentic harness does is execute that loop and facilitate those interactions.

The core inner loop is straightforward:

def run(messages: list, tools: dict = {}, limit: int = 20) -> list:
  messages = list(messages)
  for _ in range(limit):
    reply = generate(messages, tools)
    messages.append(reply)
    tool_uses = [block for block in reply["content"]
                 if block["type"] == "tool_use"]
    if not tool_uses:
      break
    messages.append({"role": "user",
                     "content": [execute(tools, use) for use in tool_uses]})
  return messages

System interaction

The first type of interaction is with the system itself, where the agent’s code gives the seed instructions for the model.

Note: The Anthropic API expects the system prompt as a top-level field, so we move it out of the chain when we generate the API call.

def system(text: str) -> dict:
    return {"role": "system", "content": [{"type": "text", "text": text}]}

User interaction

The second type of interaction is with the user of an agent.

For example, if we are writing a chat bot, this is a message sent by the user (“Whadda they do when it’s murky in Albuquerque?”).

Or if it’s a coding assistant, instruction given by the developer at the keyboard (“Write me a Fortnite clone in Rust.”). And so on.

def user(text: str) -> dict:
    return {"role": "user", "content": [{"type": "text", "text": text}]}

Model interaction

The third type of interaction is with the large language model (LLM).

What any given LLM does during this interaction is potentially diverse, ranging from returning trivial text continuations to the execution of extremely sophisticated multi-modal deep reasoning graphs, chains of thought, and hidden tool calls on the model engine’s side.

But those details are encapsulated behind the basic model interaction API of the form “context in, response out”.

Embedded within those responses is not only the text or media for replies to the user, but also recommendations for additional tool calls on the agent’s side, requests for more information from the system, or metadata about the interaction itself.

Tool definitions

The LLM needs a list of tools that the agent harness is willing to execute locally if asked to. The schema for the tool list varies from LLM API to LLM API, but is represented here as:

(Credit to Opus 5 for figuring this out – I was lost.)

JSON_TYPE = {str: "string", int: "integer", float: "number", bool: "boolean"}

def tool_definition(name: str, fn: callable) -> dict:
    parameters = inspect.signature(fn).parameters
    return {"name": name, "description": inspect.getdoc(fn),
            "input_schema": {
                "type": "object",
                "required": list(parameters),
                "properties": {field: {"type": JSON_TYPE.get(p.annotation, "string")}
                               for field, p in parameters.items()}}}

The API network interface

The network call itself is specific to the particular model API provider, but in all cases at least similar to:

def post(body: dict) -> dict:
  request = urllib.request.Request(
      API_URL, json.dumps(body).encode(),
      {"content-type": "application/json",
      "anthropic-version": "2023-06-01",
      "x-api-key": os.environ["ANTHROPIC_API_KEY"]})
  with urllib.request.urlopen(request) as response:
    return json.load(response)

The LLM call

When put together, this turns into a call to the LLM:

def generate(messages: list, tools: dict = {}) -> dict:
  reply = post({"model": MODEL, "max_tokens": 4096,
               "system": "\n\n".join(block["text"] for message in messages
                                        if message["role"] == "system"
                                        for block in message["content"]),
                "messages": [message for message in messages if message["role"] != "system"],
                "tools": [tool_definition(name, fn) for name, fn in tools.items()]})
  return {"role": "assistant", "content": reply["content"]}

Tool interaction

The fourth type of interaction is tools or function calls made by the agent harness at the request of the model or the user.

These tool calls, which can range from simple inline function calls (like we do here), to sandboxed program execution, to subagent interactions, all serve to inject additional information into the context for the next call and response to the LLM.

def execute(tools: dict, use: dict) -> dict:
    return {"type": "tool_result", "tool_use_id": use["id"],
            "content": str(tools[use["name"]](**use["input"]))}

Stopping

The loop needs to know when the work is done, resources are exhausted, or that it is time to check in with the user, and that signal can come from many sources, from the model itself to token usage to wall clock time to user or subagent interruption. For these purposes, let’s continue the inner loop until the LLM stops asking us to call further tools on our end.

Outer application loop

Surrounding the inner loop are basically ordinary applications, such as a CLI or TUI-based REPL, a cron job, a shell script, that simply call run().

The calling application would be responsble for saving and restoring session information, evaluating end to end trajectories, handling user interaction, dealing with failure states, or anything beyond what is handled by the core inner loop.

Extracting responses

For sake of convenience, a helper function to extract the last reply from the LLM to the user.

def answer(messages: list) -> str:
    return "\n".join(block["text"] for block in messages[-1]["content"]
                     if block["type"] == "text")

Runnable example

Assembled into a runnable example, this tells the LLM that a calculate method, implemented here as a native Python function, is available for tool calling, then iterates with the LLM in a loop until there is nothing left to do on the agent harness’s side, then returns the final reply.

import inspect
import json
import os
import urllib.request

MODEL = "claude-opus-5"
API_URL = "https://api.anthropic.com/v1/messages"

def main():
    import math

    def calculate(expression: str):
        """Evaluate a Python arithmetic expression. The `math` module is in scope."""
        return eval(expression, {"__builtins__": {}, "math": math})

    messages = run([system("You are terse. Use tools rather than arithmetic."),
                    user("What is the square root of 12345, times pi?")],
                   tools={"calculate": calculate})
    print(answer(messages))