Optimization Loops

Suppose the incumbent harness answers without checking a tool result. Producing one revision does not reveal whether the system has improved. Multiple revisions must be executed, compared with the original version, accepted or rejected, and saved if accepted. This complete procedure is the optimization loop.

Part I presented the basic form of this process. This chapter calls the harness currently in use the incumbent, a harness under evaluation a candidate, and the mechanism that produces candidates the optimizer, then separates their roles. Broad surveys of self-evolving agents and workflow optimization provide taxonomies based on what changes and when (Gao et al. 2026; Yue et al. 2026). This book considers only changes to an external harness that persist into future executions.

Separating the Five Roles

Harness optimization involves at least five roles.

Role Input Output Change Authority
Target agent Task, harness, environment Trajectory, answer, action Changes only within the scope authorized in advance by the harness
Optimizer Incumbent, trajectory, score, search history Candidate harness Changes only within the edit surface
Evaluator Candidate execution results Evaluation records for quality, cost, safety, and other dimensions Cannot be modified by the optimizer
Acceptance rule Evaluation records for the incumbent and candidate Accept, reject, inconclusive Fixed before the experiment
Artifact store Version, diff, evidence, evaluation records Reusable and rollback-capable history Stores accepted harnesses and search history

The evaluator and the acceptance rule are not the same. The evaluator measures how many tests passed, how many tool calls occurred, and whether any permission violations occurred. The acceptance rule decides from those records that a candidate should be accepted if quality improves, the cost cap is respected, and no regression occurs. If measurement and acceptance are combined in a judge that returns a scalar score, it becomes impossible to trace the conditions under which an update was made.

Update Loop

At round \(t\), the optimizer produces a candidate set from the incumbent \(H_t\), search history \(S_t\), and execution evidence \(Z_t\).

\[ \mathcal{C}_t = O(H_t, S_t, Z_t; B_t), \]

Here, \(B_t\) is the search budget, including the number of candidates, target-model calls, tokens, and wall time. Each candidate \(c\in\mathcal{C}_t\) is executed in a fixed environment, and the evaluator returns an evaluation record.

\[ r_{\mathrm{dev}}(c)=E(c;\mathcal{D}_{\mathrm{dev}},\xi), \]

The variable \(\xi\) represents stochasticity from model sampling, environmental variation, and seeds. Ranking candidates from a single score mistakes this variation for a search signal. The required number of repetitions and aggregation method should be determined in advance. The optimizer uses \(r_{\mathrm{dev}}(c)\) for provisional ranking and search-state updates.

Version promotion instead uses a separate record \(g(c)\) that combines validation, a preservation set, cost, and safety.

\[ H_{t+1}= \begin{cases} c^*_t & V\bigl(g(c^*_t),g(H_t)\bigr)=\mathrm{accept},\\ H_t & \text{otherwise}. \end{cases} \]

It is important to separate candidate generation, selection on development data, promotion on validation data, and final reporting on the sealed test. Do not feed the sealed test back into the promotion loop; use it once after closing the optimization campaign.

What the Optimizer Decides

Even with the same edit surface, different choices along the following axes constitute different optimizers.

Axis Main Options Question to Ask
Search state Incumbent only, beam, tree, population, archive How much of the candidate history should be retained?
Proposal operator Random, template, LLM edit, mutation, learned policy What information should produce the next change?
Evidence Scalar score, incorrect answers, critiques, trajectories, runtime diagnostics Which structure of the failure should be exposed to the optimizer?
Candidate evaluator Full rollout, subset, surrogate, static check Where should expensive executions be allocated?
Selection Argmax, bandit, Pareto, parent retention How should capabilities beyond the mean be preserved?
Stopping rule Budget, convergence, validation plateau, risk gate When should the search stop?
Promotion Score threshold, multi-objective contract, human review Which version should be distributed for future use?

Placing major method families along these axes shows that even methods described as “LLM optimization” operate on different units of update.

Method Family Main Persisted Artifact Main Feedback Search State Candidate-Generator Weight Updates
APE, OPRO, ProTeGi (Zhou et al. 2023; C. Yang et al. 2024; Pryzant et al. 2023) Instruction Score, incorrect answers, critiques History, beam No
DSPy, MIPRO, GEPA (Khattab et al. 2024; Opsahl-Ong et al. 2024; Agrawal et al. 2026) Text parameters of an LM program End-to-end score, trajectory Bayesian search, Pareto archive No
GPTSwarm, AFlow (Zhuge et al. 2024; Jiayi Zhang et al. 2025) Workflow graph or code Workflow score Graph, tree No
ADAS, DGM, Meta-Harness (Hu et al. 2025; Jenny Zhang et al. 2026; Lee et al. 2026) Agent or harness code Score, trace, implementation diagnostics Code archive No
ExpeL, SkillOpt (Zhao et al. 2024; Y. Yang et al. 2026) Experience, skill document Success/failure trajectories, held-out score Insight set, edit history No
AutoHarness, Self-Harness (Lou et al. 2026; H. Zhang et al. 2026) Verifier, code policy, runtime harness Environment errors, failure traces Refinement tree, version history No
Harness-R1 (Shao et al. 2026) Runtime-hook patch Paired-rerun reward Training episode Yes

This table describes what each method optimizes, not which method is superior. AutoHarness, for example, searches from code harnesses that prevent prohibited actions to code policies that do not use an LLM. The latter can achieve strong results, but it should be reported separately from settings in which a harness assists a model. A SkillOpt skill document is more structured than a memory item and more persistent as a procedural artifact than a prompt. Later chapters examine these boundaries.

A scalar score may be useful during search, but quality, cost, latency, and safety should be recorded separately at promotion time. An average score allows an improvement on one task to cancel a degradation on another. GEPA’s Pareto archive preserves candidates with instance-specific strengths during search, but does not guarantee that they should be promoted to deployment.

Defining the Search Budget

“10 iterations” does not have a consistent meaning across methods. In some prompt optimizers, one candidate requires only one LLM call. In others, evaluating one workflow runs multiple agents and tools. At minimum, distinguish the following.

  • Optimizer-model tokens and calls
  • Target-agent tokens and calls
  • Environment and tool executions
  • Number of candidates and case runs
  • Number of static checks, compilations, and sandbox executions
  • Wall time and parallelism
  • Per-task execution cost after promotion

Search Records to Preserve

The artifact store should retain more than the best score. It should also retain the following.

  • Parent version and candidate diff
  • Trajectories, feedback, and scores used for proposals
  • Versions of the optimizer, target model, tools, and environment
  • Whether the candidate was executable and why it was rejected
  • Number of queries to development and validation
  • Results from the promotion regression suite and the post-campaign sealed test
  • Promotion and rollback destinations

DGM does not monotonically retain only the best candidate. It maintains a branching archive of evaluated code (Jenny Zhang et al. 2026). This design preserves stepping stones for search, but it does not guarantee that candidates in the archive will generalize to future tasks. Preserving history and justifying acceptance must be evaluated separately.

Key Points

A harness optimizer consists not only of a proposal model, but also of search state, execution feedback, a candidate evaluator, selection, a budget, and promotion. Separate the optimizer, evaluator, acceptance rule, and artifact store, and do not treat the best candidate during search as the next version by default.

References

Agrawal, Lakshya A., Shangyin Tan, Dilara Soylu, et al. 2026. “GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning.” International Conference on Learning Representations. https://arxiv.org/abs/2507.19457.
Gao, Huan-ang, Jiayi Geng, Wenyue Hua, et al. 2026. “A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence.” Transactions on Machine Learning Research. https://arxiv.org/abs/2507.21046.
Hu, Shengran, Cong Lu, and Jeff Clune. 2025. “Automated Design of Agentic Systems.” International Conference on Learning Representations. https://arxiv.org/abs/2408.08435.
Khattab, Omar, Arnav Singhvi, Paridhi Maheshwari, et al. 2024. “DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines.” International Conference on Learning Representations. https://openreview.net/forum?id=sY5N0zY5Od.
Lee, Yoonho, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. 2026. Meta-Harness: End-to-End Optimization of Model Harnesses. https://arxiv.org/abs/2603.28052.
Lou, Xinghua, Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, and Kevin P. Murphy. 2026. AutoHarness: Improving LLM Agents by Automatically Synthesizing a Code Harness. https://arxiv.org/abs/2603.03329.
Opsahl-Ong, Krista, Michael J. Ryan, Josh Purtell, et al. 2024. “Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs.” Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. https://aclanthology.org/2024.emnlp-main.525/.
Pryzant, Reid, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. “Automatic Prompt Optimization with ‘Gradient Descent’ and Beam Search.” Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 7957–68. https://doi.org/10.18653/v1/2023.emnlp-main.494.
Shao, Shuai, Kangning Zhang, Qingyao Li, et al. 2026. Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories. https://arxiv.org/abs/2608.02276.
Yang, Chengrun, Xuezhi Wang, Yifeng Lu, et al. 2024. “Large Language Models as Optimizers.” International Conference on Learning Representations. https://arxiv.org/abs/2309.03409.
Yang, Yifan, Ziyang Gong, Weiquan Huang, et al. 2026. SkillOpt: Executive Strategy for Self-Evolving Agent Skills. https://arxiv.org/abs/2605.23904.
Yue, Ling, Kushal Raj Bhandari, Ching-Yun Ko, et al. 2026. From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents. https://arxiv.org/abs/2603.22386.
Zhang, Hangfan, Shao Zhang, Kangcong Li, et al. 2026. Self-Harness: Harnesses That Improve Themselves. https://arxiv.org/abs/2606.09498.
Zhang, Jenny, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. 2026. “Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents.” International Conference on Learning Representations. https://arxiv.org/abs/2505.22954.
Zhang, Jiayi, Jinyu Xiang, Zhaoyang Yu, et al. 2025. “AFlow: Automating Agentic Workflow Generation.” International Conference on Learning Representations. https://arxiv.org/abs/2410.10762.
Zhao, Andrew, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. “ExpeL: LLM Agents Are Experiential Learners.” AAAI Conference on Artificial Intelligence. https://arxiv.org/abs/2308.10144.
Zhou, Yongchao, Andrei Ioan Muresanu, Ziwen Han, et al. 2023. “Large Language Models Are Human-Level Prompt Engineers.” International Conference on Learning Representations. https://arxiv.org/abs/2211.01910.
Zhuge, Mingchen, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. 2024. “GPTSwarm: Language Agents as Optimizable Graphs.” Proceedings of the 41st International Conference on Machine Learning, 62743–67. https://proceedings.mlr.press/v235/zhuge24a.html.