AI Self-Improvement Starts Outside the Model: Lilian Weng on Harness Engineering
- Former OpenAI Safety Systems lead Lilian Weng coined "harness engineering": the runtime layer wrapped around an AI model — handling tool calls, context management, memory, and evaluation — is a key, often-overlooked piece of the path to recursive self-improvement (RSI).
- She frames what gets optimized inside a harness as a climbing ladder: prompts, structured context, workflows, the harness code itself, and finally the optimizer's own code — the higher you climb, the larger the searchable design space.
- The piece surveys over 20 papers from 2023 to 2026 — ACE, MCE, Meta-Harness, STOP, Self-Harness, AlphaEvolve, DGM, and more — showing that AI can already propose changes to its own "shell," run tests, and keep the stronger version.
- Empirical evidence shows self-improvement actually works: DGM evolved a coding agent from 20% to 50% on SWE-bench Verified. But STOP's experiments also show this mechanism only works when the underlying model is strong enough — weaker models actually regress.
- The piece closes with 7 open problems this path has hit so far — weak and fuzzy evaluation criteria, reward hacking, diversity collapse, and more — and stresses that humans need to stay in the loop at key checkpoints, not be pushed out of it.
Why the Layer Outside the Model Suddenly Matters
Former OpenAI Safety Systems lead Lilian Weng published a long essay on her personal blog Lil'Log proposing "harness engineering" as a thesis, systematically connecting it to research on AI recursive self-improvement (RSI).
Her core argument is simple: the system sitting between the model and the real world matters just as much as the model's raw intelligence. People habitually fixate on how smart a model is, while routinely underrating the shell wrapped around it that orchestrates everything.
Why it's worth reading: The idea of RSI traces back to I.J. Good's 1965 "ultraintelligent machine" (a machine that can design machines even more capable than itself), which Yudkowsky named "recursive self-improvement" in 2008. The modern version could mean a model directly rewriting its own weights, or a model improving its training pipeline and deployment system — the more indirect but more realistic path. Weng notes that both Anthropic and OpenAI have observed a clear acceleration in research velocity at frontier labs.
Weng's judgment: near-term RSI probably won't start with a model directly rewriting its own weights. It's more likely to land first at the "deployment system" layer — and the most critical component of that layer is the harness, the system behind successful coding agent products like Claude Code and Codex. This piece focuses precisely on the research around harness engineering, and how it leads step by step toward self-improvement.
What Exactly Is a Harness
A harness is the entire system wrapped around a base model. It orchestrates the whole execution process — how the model thinks and plans, how it calls tools to act, how it perceives and manages context, where it stores its outputs, and how it checks whether results are correct.
Early framings of agents used the formula "agent = model + memory + tools + planning + action." Harness engineering wraps a few more layers around that: workflow design (how you design that loop), evaluation, permission control, persistent state management. It's already close to runtime and software system design — it's about how the model observes, acts, remembers, self-checks, and improves, as one whole system.
The relationship between a harness and a model is a lot like an OS and a CPU: the model is the CPU supplying raw compute, and the harness is the operating system running around it, handling the "dirty work" — scheduling, permission management, file storage. Like a good OS, a good harness hides its complex logic inside while keeping its external interface simple. Weng also notes that as the industry matures, protocols like configuration and tool interfaces will gradually standardize.
Letting the Model Plan, Remember, and Even Split Itself to Work
Weng starts with three concrete, actionable harness design patterns. They all answer the same question: what are the foundational pieces a system needs to work autonomously over long stretches of time?
Pattern One: Give the Model a Workflow It Can Iterate On
The key is first defining a process where the model can "act, test, revise." A common approach is a goal-directed loop: plan, execute, observe or test the results, improve, execute again — round and round until the goal is reached. Along the way it can also proactively ask the user to clarify the task. Karpathy's autoresearch repo is a clean example of this kind of workflow. One thing this loop especially emphasizes: the model should analyze its own past trajectories and failure cases and iterate based on those, rather than clinging to a fixed prompt template.
Pattern Two: Treat the File System as the Model's Long-Term Memory
For an agent working over long stretches, experiment logs, code diffs, paper summaries, error records, and past execution traces quickly grow longer than any context window the model saw during training. So a harness shouldn't cram the entire process and all logs into context — it should write this persistent state to files and read it back when needed. Reading, writing, and editing files (usually via the bash command) is already a basic skill for AI models, and managing long-term memory through this simplest of forms naturally improves as the model's own capability improves.
Pattern Three: Dispatch Sub-Agents to Work in Parallel — and Keep Them Under Control
A harness can spawn multiple sub-agents to execute in parallel, then watch over those background tasks. This is useful when the main agent needs to test several hypotheses at once, run experiments in parallel, or offload independent sub-tasks without polluting the main-thread context. Here, the main agent needs a lightweight process manager: something that can launch tasks, inspect logs, cancel failed runs, and merge results back into the main thread.
The most critical design choice here is making parallelism inspectable and traceable. If a sub-agent's output only lives in volatile conversation context, it quickly goes stale and gets buried. Only by saving it as files, logs, and state records can the model recover after an interruption and go back to review its own execution history.
Put these three pieces together and you get what mainstream coding agents look like today. Weng notes that the core interfaces of products like Claude Code, Codex, OpenCode, and Cursor have already converged — they all run on a similar set of tools:
| Tool Group | Representative tools, what each does |
|---|---|
| File system | Find files: glob grep ls; read files: read; edit files: write (write an entirely new file), edit (precise string replacement), multi_edit, apply_patch (apply a structured patch) |
| Shell execution | Run commands: bash, PowerShell |
| IO and git | lsp, plus git tools like git_status, git_diff, git_commit |
| External context | MCP tools, Skills |
| Web search | web_search, web_fetch, browser tools |
| Artifact generation | Read documents and images; generate HTML, images |
| Background processes | e.g. CronCreate, CronDelete, CronList |
| Sub-agent dispatch | e.g. spawn_agent, resume_agent, wait_agent, list_agents, close_agent, interrupt_agent |