Agent Code Evolution
An agent that regenerates an entire file whenever it edits it needs long outputs even for small changes and risks breaking untouched sections. Instead of merely adding an instruction to “edit accurately,” could we replace its editing function with one that replaces only a uniquely matching string? This chapter examines methods that update the functions an agent uses, or the agent’s own implementation, through execution-based evaluation. The updated code is not an answer to one problem. It is used on subsequent problems as well.
Being able to rewrite code does not, however, establish that rewriting improves capability. We need to distinguish what can change, which previous candidate becomes the next starting point, and which resulting implementation is evaluated by what measure. We examine these distinctions through AgentOptimizer, which updates a function set, Automated Design of Agentic Systems (ADAS), which designs agents in code, and the Darwin Gödel Machine (DGM), which combines self-modification with candidate lineages. These are accepted papers at ICML 2024, ICLR 2025, and ICLR 2026, respectively. (S. Zhang et al. 2024; Hu et al. 2025; J. Zhang et al. 2026)
Changing Code Where Instructions Are Insufficient
To make the edit surface concrete, consider an editing capability actually generated by DGM. The initial agent’s editing tool overwrote an existing file with the supplied full text. The diff leading to the final best candidate in Appendix F.1 replaces this with str_replace, which accepts a search string and a replacement string. The following is a short extract from the added function body, not an invented illustrative patch. (J. Zhang et al. 2026, Appendix F.1, pp. 42–43)
content = read_file(path)
occurrences = content.count(old_str)
if occurrences == 0:
return f"Error: Could not find the exact text to replace in {path}"
elif occurrences > 1:
return f"Error: Found multiple ({occurrences}) occurrences of the text in {path}. Must be unique."
else:
new_content = content.replace(old_str, new_str)
write_file(path, new_content)This implements more than a textual description of string replacement. It implements conditions that prevent an edit when there are zero or multiple matches. The actual change also covers the tool argument schema, command dispatch, and tests. The evaluation target is therefore the complete agent incorporating these changes, not these few lines in isolation.
DGM’s results section and Figure 3 show capabilities discovered during search, including line-based viewing, string-replacement editing, summarization at the context limit, and generation and ranking of multiple proposed fixes. The final score improvement cannot, however, be attributed solely to the replacement conditions above. The appendix diff documents a real implementation change. It is not a causal measurement obtained by removing each feature individually. (J. Zhang et al. 2026, sec. 4.4 and Appendix G)
This example shows that code optimization has different scopes. Changing callable functions opens a different search space from changing when to use them, how often to try, and which result to accept.
Expanding from Function Sets to Agent Implementations
Even a narrow edit surface can update behavioral capabilities without changing model weights. AgentOptimizer treats functions as an agent’s learnable parameters. Its inputs are the current function set, execution histories on training problems, generated and reference answers, and training performance. A large language model (LLM) acting as the optimizer reads this information and incrementally adds, modifies, or deletes functions. Evaluation measures not the quality of the function code itself, but the problem-solving performance of the agent using the updated set. (S. Zhang et al. 2024, sec. 2)
This is not a mechanism that generates a useful function once for each problem and then discards it. The function set persists through training and is used on unseen problems. The method can undo changes that worsen training performance and stop after repeated updates without improvement. It does not, however, maintain a DGM-style candidate collection containing diverse past implementations as parents for branching search. Its main update target is the function set, not an unrestricted redesign of the agent’s entire control logic at every step.
Experiments use MATH, TabMWP’s tabular arithmetic problems, and a subset of GAIA’s real-world questions. On MATH and GAIA, both the optimizer and execution agent use GPT-4-1106-preview. On TabMWP, the optimizer uses the same GPT-4 model, but execution uses GPT-3.5-turbo-1106. The three tasks should therefore not be summarized as a uniform improvement rate for one model. (S. Zhang et al. 2024, sec. 3.1)
More importantly, a comparison within the paper shows that generating functions is not enough. On MATH intermediate algebra, the GPT-4+ agent achieves 30.0% accuracy without training, 28.8% when incremental function updates are replaced by one-shot generation, and 38.8% with the full method. Removing both rollback and early stopping gives 36.3%. These results concern a limited subset and do not isolate rollback from early stopping. Even so, they show that update granularity and acceptance criteria matter alongside the content of a change. (S. Zhang et al. 2024, Table 3)
Figure 1 shows that improvements on training problems do not always coincide with improvements on unseen problems. The paper selected intermediate algebra and algebra as contrasting examples. These are not average learning curves across all categories. The implementation from the final iteration is not necessarily the best on the test set.
Removing the restriction to function sets also allows changes to implementations containing LLM calls, reflection, voting, and information transfer. ADAS introduces Meta Agent Search, in which a fixed meta-agent writes code for new agents, evaluates them, and adds them to an archive of discovered candidates. Descriptions, code, and evaluation results from past candidates inform subsequent designs. Earlier attempts thus become information conditioning the next generation, not merely experiment logs. (Hu et al. 2025, sec. 3)
For example, an execution function can generate several reasoning outputs, critique and revise each, and then combine them. The paper’s use of code as a general representation does not mean its experiments searched every possible implementation. Experiments use the supplied LLM-calling and evaluation framework and express new control logic within it. The discovered implementations in the appendix distinguish this from merely generating design suggestions.
In Figure 2, the meta-agent on the left writes the next implementation and is distinct from the new candidate that executes. The bottom row shows three discovered designs. A meta-agent consulting archived candidate code is not the same as a candidate modifying itself.
In the ADAS experiment on the Abstraction and Reasoning Corpus (ARC), problems with grids no larger than 5×5 are split into 20 search problems and 60 test problems. A GPT-4 meta-agent runs for 25 iterations. Execution agents and comparison methods use GPT-3.5, and each evaluation is repeated five times to reduce stochastic variation. The curves in Figure 3 report performance on unused test problems under this setup. Separate experiments examine transfer from the Multilingual Grade School Math benchmark (MGSM) to other mathematical and nonmathematical tasks, and changes to the execution model on ARC. Evaluation during search, unused problems from the same task, and transfer to another task must not be conflated. (Hu et al. 2025, sec. 4.1 and 4.3)
Using Low-Scoring Candidates for Later Improvement
The current best implementation is not necessarily the best starting point for a new candidate. A change that initially reduces performance may provide a basis for combining another change later. This calls for an archive of past implementations and parent selection that decides which candidate to branch from next.
DGM combines both with self-modification. A selected parent agent proposes its next feature from its own evaluation logs and edits its own codebase. The resulting child is evaluated on coding tasks, and candidates that compile and retain basic editing capabilities enter the archive. Unlike design by ADAS’s fixed meta-agent, the improved execution agent itself is reused for subsequent self-modification. Model weights remain unchanged, however, and the outer mechanisms managing the archive and selecting parents are not themselves subject to self-modification. (J. Zhang et al. 2026, sec. 3)
Broadly, parent selection favors candidates with higher performance and fewer children that retain basic editing capability. This exploits strong implementations while avoiding excessive concentration on lineages already explored many times. Each eligible parent in the archive retains a nonzero selection probability, allowing improvement attempts from starting points other than the most recent best candidate.
On the left of Figure 3, nodes are agents and edges are parent–child relationships created by self-modification. Interior color indicates SWE-bench performance, while border color indicates the number of evaluated problems. Red borders include attempts that failed to retain basic functionality, so not every depicted node was available as a later parent. The black line on the right shows the scores of ancestors leading to the final best agent, not the best score found so far during search. Two intermediate declines show that the final candidate emerged from a lineage containing temporary regressions. (J. Zhang et al. 2026, fig. 3)
This experiment runs for 80 iterations and uses Claude 3.5 Sonnet (New) for self-modification. Evaluation uses the same model on SWE-bench and o3-mini on Polyglot. To limit cost, evaluation begins with a small set checking basic functionality and allocates additional evaluation to promising candidates. High-performing SWE-bench candidates are evaluated on up to 200 problems, while the Polyglot search curve uses a 50-problem subset. (J. Zhang et al. 2026, secs. 4.1–4.2)
The population underlying each number also matters. On Polyglot’s 50 search problems, performance rises from 14.0% initially to 38.0% for the best candidate. Evaluating the same initial candidate and selected best candidate on all problems instead gives 14.2% and 30.7%. Because the full evaluation includes problems used in search, it is not a completely independent unused test. The paper also evaluates pass@1 without intermediate feedback from correctness tests, unlike Aider’s pass@2 with one round of feedback from those tests. (J. Zhang et al. 2026, sec. 4.2 and 4.4)
Controlled comparisons examine the search mechanisms. Keeping the initial agent fixed as the source of new changes removes the reuse of improved agents for later modifications. Advancing only from the latest implementation removes branching into other archived lineages. Full DGM has better search curves than these conditions. This supports the usefulness of reusing self-modification capabilities and maintaining an archive, but it does not establish a universal ranking across methods with different models, initial implementations, and budgets. (J. Zhang et al. 2026, secs. 4.3–4.4)
Separating Current Performance from the Ability to Produce Descendants
Even with an archive, parent selection based only on current scores may undersample promising future lineages. Does current problem-solving capability coincide with the ability to produce good descendants? The Huxley-Gödel Machine (HGM), accepted at ICLR 2026, addresses this question directly. (Wang et al. 2026)
HGM calls a candidate and its descendants a clade and measures its ability to produce strong later candidates through clade-level metaproductivity (CMP). Search cannot observe every future descendant. Instead, it aggregates observed descendant successes and failures by lineage, updates estimates, and uses Thompson sampling with that uncertainty to select parents for expansion. What is learned here is not a new neural network’s weights, but a lineage’s promise as estimated from observed evaluations.
HGM also separates candidate generation from candidate evaluation. Rather than solving a fixed number of problems whenever a new implementation is created, it chooses whether to attempt another self-modification or evaluate an existing candidate on one more problem. Parent selection uses statistics including descendants, whereas evaluation selection uses statistics for the candidate itself. The contribution is therefore not merely preserving history. It is combining estimates of parents’ capacity for future improvement with sequential allocation of evaluation budgets. (Wang et al. 2026, secs. 2–3)
The paper’s controlled comparison uses the same initial agent and a maximum of 800 problem-level evaluations. On 60 SWE-bench Verified problems, expansion uses GPT-5 and evaluation uses GPT-5 mini. The final selected candidates achieve 53.3% for DGM and 56.7% for HGM, with 1,231 and 517 allocated CPU hours, respectively, for 800 evaluations. This is separate from the original DGM paper’s 80-iteration experiment, and the scores must not be joined into one series. HGM changes asynchronous execution and evaluation allocation as well as estimation, so the difference cannot be attributed to CMP estimation alone. (Wang et al. 2026, sec. 4.2 and Table 2)
The left of Figure 4 shows a stronger correlation between HGM’s metric and long-term improvement capability than for conventional metrics based on current performance. This is a correlation assessing parent-selection proxies, not a causal effect of individual modifications. The right shows the curve labeled “Best-found accuracy” in the original. The DGM and Self-Improving Coding Agent (SICA) endpoints do not match the Table 2 values discussed above and should not be compared as one numerical series. The text does not establish the reason for this discrepancy.
A separate experiment transfers candidates searched on the full SWE-bench Verified set to SWE-bench Lite. Of Lite’s 300 problems, 93 overlap with Verified, so the paper reports both the usual set and a set excluding overlaps. With GPT-5 mini, HGM scores 49.0% and 40.1%, respectively. Unlike improvement during search, the latter assesses transfer to unused problems. Reporting both lets us separate candidate selection from candidate generalization. (Wang et al. 2026, sec. 4.3.2 and Table 3)
Searching for Role Combinations Rather Than Evolving One Implementation
Even after improving code or prompts, an agent retaining the full history of localization, editing, and verification may be hindered by irrelevant context. Another approach delegates work to specialized subagents and combines them with an orchestrator. Here, evaluation must ask which partners make a candidate useful, rather than only which candidate should be a parent.
Bandit Optimization for Agent Design (BOAD), accepted at ICLR 2026, searches this hierarchy. It presents subagents as callable tools and generates candidates containing roles, input/output descriptions, and instructions. Each trial selects several subagents from a pool, constructs an orchestrator tailored to them, and executes a software repair task. The final evaluation target is the combination of the orchestrator and selected subagents, not an isolated subagent. (Xu et al. 2026, sec. 4)
Selection uses the Upper Confidence Bound (UCB) rule for multi-armed bandits. It exploits candidates with high observed utility while allocating evaluation to those not yet tried enough. The pool also expands during search. Unlike modifying implementations along DGM’s parent–child lineages, this searches a reusable subagent pool and combinations of its members.
Giving every member the team’s overall success score can let an unhelpful candidate benefit from strong teammates. BOAD passes execution traces to an LLM judge, which assigns binary utility according to whether a subagent advanced the solution. Useful intermediate work can receive credit even in a trace that ultimately fails. This value is an estimated contribution assigned by a judge, not a success rate measured by correctness tests. (Xu et al. 2026, sec. 4.2)
Table 1 tests this distinction. Design uses 12 SWE-bench Verified problems, while the table evaluates on 300 SWE-bench Live problems with Seed-OSS-36B-Instruct as the execution model. The condition without subagents also receives 10 iterations of prompt optimization, so this is not merely a comparison against an untuned initial agent. (Xu et al. 2026, sec. 5.1 and 5.3)
| Factor Compared | Control Condition | BOAD Condition |
|---|---|---|
| Subagent use | Prompt optimization only: 16.3% | With subagents: 20.0% |
| Orchestrator adaptation to the same subagents | Generic orchestration instructions: 16.7% | Candidate-specific orchestration instructions: 20.0% |
| Candidate pool expansion | Fixed pool: 17.0% | Expanded during search: 20.0% |
| Metric for selecting the top 2 candidates | Team success rate: 15.3% | Utility in execution traces: 20.0% |
More subagents are not necessarily better. In the same paper, the top 2 by utility achieve 20.0%, whereas the top 5 achieve 13.7%. The benefit of hierarchy depends on matching roles and orchestration, not headcount. These results come from one design setting and do not imply that two is optimal for every model. (Xu et al. 2026, Table 3)
Harness History Access, Action Constraints, and Local Patches
Recent work also explicitly studies editing the surrounding code collectively called a harness. We treat Meta-Harness, AutoHarness, and Self-Harness as preprints supplementing accepted research. All change code between the model and its environment without updating model weights, but their entry points differ.
In Meta-Harness, a proposer agent accesses previous candidates’ source code, scores, and execution traces through the filesystem. Rather than compressing all past experience into short reflections, it inspects relevant records to produce the next change. Relative to ADAS’s archive, the comparison concerns not just whether past candidates persist, but how much detailed evidence is accessible and how it can be accessed. (Lee et al. 2026)
AutoHarness uses feedback from game environments to synthesize code preventing prohibited actions. It also considers coding the entire action policy rather than only auxiliary checks. The former constrains LLM-proposed actions through surrounding code, whereas the latter no longer calls an LLM at decision time. Both generate code, but the LLM’s remaining runtime role should be distinguished rather than grouping both as implementation improvements to an LLM agent. (Lou et al. 2026)
Self-Harness extracts model-specific weaknesses from execution traces, generates small changes addressing them, and checks those changes through regression tests. It emphasizes improving surrounding implementation with the target agent itself, without requiring a stronger external design agent. This overlaps with DGM’s broad implementation search but explicitly pairs failure patterns with minimal edits. (H. Zhang et al. 2026)
Ranking these results by maximum improvement on different benchmarks does not establish which search method is better. As discussed in Evaluation, comparisons need matched editing permissions and budgets, evaluation not used for candidate selection, and separate accounting for search and execution costs.
The central question left by this research is how to balance selecting an implementation that solves problems well now with selecting a starting point that can produce better implementations later. Incremental function updates, archives of past implementations, estimates of descendant promise, and role-specific contribution scores offer different answers. Turning automatic code generation into verifiable improvement requires not only expanding the edit surface, but matching search and evaluation units to that surface.
Figure Credits
Figure 2: (S. Zhang et al. 2024). Original published version, Figure 2. © The authors. CC BY 4.0. Cropped and rasterized.
Figure 1: (Hu et al. 2025). Original published version, Figure 1. © The authors. CC BY 4.0. Cropped and rasterized.
Figure 3: (J. Zhang et al. 2026). Original published version, Figure 3. CC BY 4.0. Two original panels combined and rasterized, with margins cropped.
Figure 1: (Wang et al. 2026). Original published version, Figure 1. © The authors. CC BY 4.0. Cropped and rasterized.
Figure 2: (Lee et al. 2026). Original preprint, Figure 2 (arXiv 2603.28052v1). CC BY 4.0. Cropped and rasterized.
Figure 1: (Lou et al. 2026). Original preprint, Figure 1 (arXiv 2603.03329v1). CC BY 4.0. Cropped and rasterized.
Figure 2: (H. Zhang et al. 2026). Original preprint, Figure 2 (arXiv 2606.09498v3). CC BY 4.0. Cropped and rasterized.