Search Space

The preceding chapter identified the first divergence in an execution record and formed a hypothesis for fixing it. The next decision is how much the optimizer may change. The search space in agent harness optimization is the set of harnesses that an optimizer can generate as update candidates, and the edit surface (the scope within which changes are allowed) determines this set.

For example, changing only the system prompt produces different candidates than changing the order of tool calls and retry behavior. The latter can fix more failures, but also makes search and evaluation harder.

A Coarse Map of the Edit Surface

Edit surface	What can be changed	Failures it mainly fixes	Examples
Prompt	instructions, demonstrations	insufficient instructions, insufficient examples, output format mismatches	DSPy, GEPA
Workflow (execution procedure)	tool order, branches, iteration, retries	decision order, missed checks, premature termination	AFlow
Harness implementation	tool interfaces, state management, memory, recovery	control that workflow editing alone cannot express	ADAS, DGM

This table is a coarse map for understanding the initial design freedom, not a claim that the three levels are exclusive. Real artifacts overlap across prompts, memory, workflows, runtime hooks, and other storage units. Part II’s What to Edit divides them into six potentially overlapping surfaces and defines the interfaces that must remain fixed.

How Far to Expand the Edit Surface

Expanding the edit surface also increases risks such as breaking interfaces on rare paths, embedding shortcuts specific to validation tasks, and changing tool permissions. The principle is therefore to start with the smallest edit surface that can fix the failure and expand it only when necessary.

Key Points

A larger search space is not inherently better. Use the three levels for orientation, then define actual permissions in terms of artifacts and interfaces.