What to Edit
An agent that answers without checking a tool result can be fixed in three ways: write a verification procedure in the prompt, make the workflow permit an answer only after verification, or add a validator to the runtime. These revisions address the same failure, but differ in both effect and risk.
The previous chapter used failure attribution to form hypotheses about fixes. This chapter asks how to express those hypotheses as changes to artifacts. The set of components an optimizer may rewrite is the edit surface. We divide it by persistence unit into prompts, language-model programs, memory and skills, workflows, agent code, and runtime hooks. These scopes do not form a strict linear hierarchy, and one fix may span several of them.
What Can Change and What Must Be Preserved
An edit surface cannot be defined by a list of filenames alone. Describe the following two boundaries.
- Syntactic boundary: Fields, modules, hooks, and files that may be changed
- Semantic boundary: Interfaces, permissions, and invariants that must remain intact after a change
Even when only the system prompt may be changed, the search space depends on whether the output JSON schema, tool names, and system policy may be deleted. When runtime hooks may be changed, the input type, output type, timeout, callable tools, and persistent state must be fixed.
Let \(\mathcal{H}\) be the edit surface and \(\mathcal{F}\) the set of verifiable harnesses. The optimizer should search
\[ H'\in\mathcal{H}\cap\mathcal{F} \]
Candidates that can be generated syntactically but do not compile, or that violate the permission policy, are not included in \(\mathcal{F}\).
Prompts and Demonstrations
The narrowest surface is a single instruction or demonstration. The target model, tools, and control flow remain fixed. Only the text placed before the input changes.
APE generates instruction candidates from task examples and selects them by task score (Zhou et al. 2023). OPRO places the history of previous instructions and scores in a meta-prompt to generate the next instruction (C. Yang et al. 2024). ProTeGi creates prompt edits from natural-language critiques of incorrect answers (Pryzant et al. 2023). RLPrompt learns a policy that generates discrete-token prompts while keeping the target model fixed (Deng et al. 2022).
A prompt-only surface has three advantages.
- Diffs are short and human-readable
- Execution structure and tool permissions are easy to align across candidates
- Reverting to the original prompt is easy
A prompt, however, is a soft constraint. Writing “always verify tool results” does not make the runtime enforce that order. In addition, ProTeGi’s main evaluation includes a setting that combines multiple prompts from the final beam at test time. Improvements to a single persistent prompt must be distinguished from a test-time ensemble.
LM Program Parameters
In a program that connects multiple language-model calls, the instructions and demonstrations for each module can be optimized jointly. This surface changes only text parameters while keeping the program topology and Python code fixed.
DSPy compiles language-model (LM) programs with declarative modules against a task metric (Khattab et al. 2024). MIPRO creates task-grounded instruction candidates and bootstrapped demonstrations, then uses Bayesian optimization to search over the score of the entire multi-stage program (Opsahl-Ong et al. 2024). TextGrad propagates natural-language feedback backward from the outputs of multiple nodes to update upstream text variables (Yuksekgonul et al. 2025). GEPA reflectively mutates multiple text components of a compound system from trajectories and evaluator feedback (Agrawal et al. 2026).
In a language-model program, optimizing only local scores for individual modules can damage end-to-end performance. Having a retriever return more context may improve retrieval recall while consuming the downstream generator’s context budget. Candidates should therefore be evaluated by executing the entire program.
Memory, Experience, and Skills
A memory surface includes two broad categories: stored content and read/write policies.
| Category | Example | Persisted Artifact |
|---|---|---|
| Content update | Successful trajectories, insights extracted from failures | Memory item |
| Retrieval update | Query, ranking, deduplication, forgetting | Read policy |
| Write-policy update | What to record, merge, and delete | Write policy |
| Schema update | Experience fields, provenance, confidence | Memory structure |
| Skill update | Procedures, examples, applicability conditions | Versioned skill document |
ExpeL compares successful and failed trajectories from training tasks and carries natural-language insights and retrievable experiences into unseen tasks (Zhao et al. 2024). This is an example of cross-task persistent memory. However, adding or deleting an insight does not require passing an external regression suite.
SkillOpt updates a single skill document read by a frozen agent through bounded edits and accepts only changes that improve held-out validation performance (Y. Yang et al. 2026). This held-out split is a selection set used for round-by-round acceptance, not the sealed test for final reporting. A skill is not a one-off memory item but a procedural artifact reused on subsequent tasks. Prompts, memory, and skills are distinguished not by filenames, but by who reads them when and by the unit at which they are versioned and accepted.
Simply adding a log is not optimization. It becomes harness optimization when memory items or policies are compared as candidates and the accepted content is distributed to subsequent tasks.
Workflow Graphs and Modular Agents
A workflow surface represents language-model calls, tools, verification steps, branches, and iterations as nodes and edges that may be changed. It can use the execution structure to enforce ordering constraints that are difficult to express in a prompt.
GPTSwarm represents language agents as a graph and optimizes edge connections and node prompts from task loss (Zhuge et al. 2024). AgentSquare divides planning, reasoning, tool use, and memory into modules and explores configurations through module evolution and recombination (Shang et al. 2025). AFlow represents workflows as executable code and generates and evaluates candidates with Monte Carlo tree search (Jiayi Zhang et al. 2025).
A graph representation has the advantage of making node types and edge semantics explicit. By contrast, a method that represents a graph in code can freely change exception handling and state sharing even with an apparently identical node set. Even when a paper uses “workflow” in its title, it is necessary to determine whether the actual surface is a finite graph or arbitrary code.
Agent System Code
Using agent system code as the surface allows simultaneous changes to prompts, tool wrappers, memory, control flow, and error handling. The Meta Agent Search in ADAS generates new Python implementations from an archive of evaluated agent programs, while DGM changes the coding agent’s own codebase and stores its lineage in a branching archive (Hu et al. 2025; Jenny Zhang et al. 2026). AutoHarness iteratively generates verifier code that excludes prohibited actions, using environment errors as feedback (Lou et al. 2026).
This surface is highly expressive, but candidate validity must be checked before score-based evaluation.
- Does it parse or compile?
- Does it implement the required interface?
- Does it respect timeout and resource limits?
- Does it avoid accessing prohibited files and credentials?
- Has it left the evaluator, tests, and permission policy unchanged?
- Can the version be reconstructed deterministically?
AutoHarness also reports a setting that replaces the entire decision process with a code policy. In that case, the optimization target shifts from a harness that assists a model to the task policy itself. Record not only the breadth of the code surface, but also whether the model or the code makes the final decision.
Runtime Harness Implementation
A runtime harness is an implementation that assembles context around model calls, mediates tools, validates actions, and recovers from failures. Meta-Harness uses a coding proposer to search harness source that includes not only prompts but also memory, retrieval, context presentation, and orchestration (Lee et al. 2026). The proposer consults previous candidates, scores, and traces on the filesystem and generates new code.
Harness-R1 divides the runtime into lifecycle hooks and learns a specialized editor that generates hook patches from failed trajectories (Shao et al. 2026). Patches pass compilation and sandbox validation before they are applied. This is a direct example of runtime editing, but it is a non-peer-reviewed preprint released in August 2026 and should not be considered an established example of a general-purpose learned editor.
For a runtime surface, constraining editable regions through hook contracts is easier to evaluate than describing the target as the “entire repository.”
| Hook | Readable State | Editable Output | Boundary to Fix |
|---|---|---|---|
| Pre-context | Task, authorized memory | Context | Secrets, system policy |
| Pre-action | Trajectory, tool schema | Action proposal | Tool allowlist, argument schema |
| Post-observation | Tool result | Normalized observation | Preservation of raw evidence |
| Validation | Candidate action | Allow / reject / repair | Permission policy |
| Recovery | Error, retry state | Retry / fallback | Retry limit, cost budget |
When Multiple Edit Surfaces Are Involved
Real optimizers span multiple surfaces. In ADAS and Meta-Harness, code edits can also change prompts and workflows, while GEPA jointly changes multiple text components. Compare prompt-only, workflow-only, runtime-hook-only, and joint changes under the same evaluation budget. Even if a broader surface wins, degrees of freedom, candidate count, and optimizer-model context are confounders, so do not attribute the result to a single cause such as “prompt improvement.”
Expanding from the Minimum Surface
The principle for choosing a surface is not always to use the narrowest one. It is to use the smallest surface that can fix the failure and can be verified adequately.
- Use a prompt for insufficient instructions
- Use language-model program parameters for coordination across multiple modules
- Use a workflow for tool ordering and branching
- Use a read/write policy for the memory lifecycle
- Use a runtime hook for interface enforcement and recovery
- Use agent code when the existing representation cannot express the fix
Each time the editable scope expands, expand the validity checks, regression suite, permission boundary, and rollback artifacts as well.
Key Points
Edit surfaces can be described as the overlapping scopes of prompts, language-model programs, memory, workflows, agent code, and runtime hooks. Examine the artifacts that actually change, the fixed interfaces, and the persistence unit rather than relying on labels. A broader surface is not necessarily stronger. Determine its scope from both fixability and verifiability.