What Are Agent and Harness? — Video Notes

Mon, 25 May 2026 02:00:00 +0800

One-Sentence Summary

An Agent solves how AI gets work done; a Harness solves how AI does work reliably. Together, they form the foundation of products like Claude Code, Codex, Open Cloud, and Qwen.

1. From ChatGPT to AI Agent

The Original Problem

Asking ChatGPT to create an animation with HTML + SVG (e.g., an Apple logo drawn with lines) often produces poor results — even in 2026, AI struggles with even a minimalist logo.

Manual Improvement

Search for Apple SVG assets on sites like iconfont
Feed the SVG code to AI and ask it to recreate it
Results improve significantly, but manually finding assets every time is tedious

Automated Improvement — Wrapper Sites (Earliest Agent Prototype)

Build a website that wraps ChatGPT via API calls, adding backend functions:

User submits request → The site tells AI: “Here’s the user’s prompt. I also have a logo search function you can call when needed.”
AI analyzes → Determines it needs an Apple logo, responds: “Call the asset search tool with parameter: Apple logo”
Site executes → Backend runs the search function and finds SVG assets
Final generation → The site sends assets + user prompt to AI to complete the animation

2. Two Core Working Modes of Agent

1. ReAct (Reasoning + Acting) — Step by Step

Three-step loop:

Think: AI analyzes the request, determines what’s needed and which tool to call
Act: AI tells the backend to call a tool (e.g., search for assets)
Observe: Backend returns results; AI sees them and continues

The loop repeats: Think → Act → Observe, until the task is complete.

This is the most fundamental working pattern of almost all AI Agents, including Claude Code, Codex, and Open Cloud.

2. Plan & Execute — Plan First, Then Execute

Plan first: Generate a work list / step sequence upon receiving the task
Execute: Follow the checklist step by step

Mode	Analogy	Characteristic
ReAct	Improvise as you go	Flexible, adjust on the fly
Plan & Execute	Research before traveling	Structured, suited for complex tasks

Real agents combine both. For example, Qwen’s “Task Assistant” mode: first analyzes requirements and creates steps (Plan), then proactively searches, writes code, and adjusts (Act).

3. Advanced Agent Capabilities

Tool Calling / Function Calling

The Agent determines which tools to call based on the user prompt; the backend executes them and returns results.

Context Management and Compression

Large models have no memory — each conversation starts fresh
The Agent must send the entire conversation history to the AI each time, which grows longer over time
Context window = AI’s workspace, limited in size
Context compression: When the conversation exceeds the window, earlier content is summarized into a condensed version
- Trade-off: compression loses information, potentially causing the AI to forget previous instructions

Multi-Agent Collaboration

One AI acts as the project manager (understanding requirements, breaking down tasks, assigning work), while other AIs execute subtasks.

Each sub-AI has its own independent context window
The project manager only sees final results, not intermediate steps
Improves efficiency while mitigating context explosion

4. Harness — The Safety Net for Reliable AI

When Agents run in production, they encounter various issues that need a Harness to solve.

Engineering Checkpoints in a Harness

Checkpoint	Problem	Solution
Format Sanitization	AI adds “okay”, markdown code blocks, or extra newlines when returning JSON	Clean: remove fluff, wrapping symbols, extra newlines; send errors back for regeneration if still failing
Parameter Validation	Tool call parameters are invalid (e.g., city field contains a non-city name)	Validate format/range before calling; reject and request refill if invalid
Input Filtering	Prompt injection attacks (“ignore all previous instructions”) / malicious SVG uploads	Scan user input for suspicious instructions and asset safety before processing
Output Filtering	AI-generated content may contain malicious code	Scan output content for threats
Hard-Coded Validation	AI repeatedly makes the same error (e.g., always using pure white backgrounds)	Enforce with code: auto-detect SVG background color and replace white with dark

Core Principle of Harness Engineering

If you can enforce it with code, never rely on prompting alone.

5. Complete Architecture Summary

One Sentence to Elevate

Agent is the horse; Harness is the tack. A horse without tack runs into problems; tack without a horse does nothing. Together, they transform AI from a chat-only chatbot into a worker that gets things done in the real world.

AI Engineering on Yanmu