Featured image of post What Are Agent and Harness? — Video Notes

What Are Agent and Harness? — Video Notes

One-Sentence Summary

An Agent solves how AI gets work done; a Harness solves how AI does work reliably. Together, they form the foundation of products like Claude Code, Codex, Open Cloud, and Qwen.


1. From ChatGPT to AI Agent

The Original Problem

Asking ChatGPT to create an animation with HTML + SVG (e.g., an Apple logo drawn with lines) often produces poor results — even in 2026, AI struggles with even a minimalist logo.

Manual Improvement

  1. Search for Apple SVG assets on sites like iconfont
  2. Feed the SVG code to AI and ask it to recreate it
  3. Results improve significantly, but manually finding assets every time is tedious

Automated Improvement — Wrapper Sites (Earliest Agent Prototype)

Build a website that wraps ChatGPT via API calls, adding backend functions:

  1. User submits request → The site tells AI: “Here’s the user’s prompt. I also have a logo search function you can call when needed.”
  2. AI analyzes → Determines it needs an Apple logo, responds: “Call the asset search tool with parameter: Apple logo”
  3. Site executes → Backend runs the search function and finds SVG assets
  4. Final generation → The site sends assets + user prompt to AI to complete the animation


2. Two Core Working Modes of Agent

1. ReAct (Reasoning + Acting) — Step by Step

Three-step loop:

  • Think: AI analyzes the request, determines what’s needed and which tool to call
  • Act: AI tells the backend to call a tool (e.g., search for assets)
  • Observe: Backend returns results; AI sees them and continues

The loop repeats: Think → Act → Observe, until the task is complete.

This is the most fundamental working pattern of almost all AI Agents, including Claude Code, Codex, and Open Cloud.

2. Plan & Execute — Plan First, Then Execute

  • Plan first: Generate a work list / step sequence upon receiving the task
  • Execute: Follow the checklist step by step
Mode Analogy Characteristic
ReAct Improvise as you go Flexible, adjust on the fly
Plan & Execute Research before traveling Structured, suited for complex tasks

Real agents combine both. For example, Qwen’s “Task Assistant” mode: first analyzes requirements and creates steps (Plan), then proactively searches, writes code, and adjusts (Act).


3. Advanced Agent Capabilities

Tool Calling / Function Calling

The Agent determines which tools to call based on the user prompt; the backend executes them and returns results.

Context Management and Compression

  • Large models have no memory — each conversation starts fresh
  • The Agent must send the entire conversation history to the AI each time, which grows longer over time
  • Context window = AI’s workspace, limited in size
  • Context compression: When the conversation exceeds the window, earlier content is summarized into a condensed version
    • Trade-off: compression loses information, potentially causing the AI to forget previous instructions

Multi-Agent Collaboration

One AI acts as the project manager (understanding requirements, breaking down tasks, assigning work), while other AIs execute subtasks.

  • Each sub-AI has its own independent context window
  • The project manager only sees final results, not intermediate steps
  • Improves efficiency while mitigating context explosion


4. Harness — The Safety Net for Reliable AI

When Agents run in production, they encounter various issues that need a Harness to solve.

Engineering Checkpoints in a Harness

Checkpoint Problem Solution
Format Sanitization AI adds “okay”, markdown code blocks, or extra newlines when returning JSON Clean: remove fluff, wrapping symbols, extra newlines; send errors back for regeneration if still failing
Parameter Validation Tool call parameters are invalid (e.g., city field contains a non-city name) Validate format/range before calling; reject and request refill if invalid
Input Filtering Prompt injection attacks (“ignore all previous instructions”) / malicious SVG uploads Scan user input for suspicious instructions and asset safety before processing
Output Filtering AI-generated content may contain malicious code Scan output content for threats
Hard-Coded Validation AI repeatedly makes the same error (e.g., always using pure white backgrounds) Enforce with code: auto-detect SVG background color and replace white with dark

Core Principle of Harness Engineering

If you can enforce it with code, never rely on prompting alone.


5. Complete Architecture Summary

One Sentence to Elevate

Agent is the horse; Harness is the tack. A horse without tack runs into problems; tack without a horse does nothing. Together, they transform AI from a chat-only chatbot into a worker that gets things done in the real world.

comments powered by Disqus