Multi-Module Optimization
A system generates a search query, retrieves documents, and writes an answer from them. Suppose this system calls a large language model (LLM) multiple times, but correct labels are available only for the final answer. When that answer is wrong, should we revise the query-generation instruction or the answer-generation instruction? Could each be reasonable on its own but ineffective in combination?
Joint optimization of multiple modules uses system-level evaluation to adjust instructions, demonstrations, and other components rather than supplying independent correct labels for each intermediate step. Generating candidates is only part of the challenge. There is also a credit-assignment problem: which changes contributed to the final result? Even if rewriting every instruction at once raises the score, the next component to change is not obvious.
This chapter separates the representation of editable components, candidate generation, and evaluation and selection. DSPy and MIPRO select instructions and demonstrations for declarative modules using overall scores. TextGrad and Trace/OptoPrime turn computational relationships or execution traces into revision proposals. GEPA combines reflection on traces with input-specific candidate selection. Optimas, MASPO, and TRUCE further specify how locally good outputs or rules can be aligned with outputs or rules that help the entire system.
Can Successful Executions Supply Intermediate Labels?
Even when labels exist only for final answers, intermediate behavior can be recorded. Inputs and outputs such as search queries, retrieval results, and answer drafts can become demonstration candidates for subsequent prompts when stored with a final evaluation. This differs from adding human-written intermediate labels.
DSPy represents language-model (LM) calls through signatures declaring input and output roles and modules that use those signatures. For example, it separates generating a search query from a question and known context from answering using context, connecting them with ordinary program code. Instead of embedding fixed, lengthy prompt strings at every stage, a compiler instantiates the modules for the data and evaluation metric. Compilation here means adjusting instructions, demonstrations, and related elements of the given program, not translating it into machine code (Khattab et al. 2024).
A representative approach runs the program on training inputs and extracts each module’s inputs and outputs from executions judged successful by the final evaluation. These become bootstrapped demonstration candidates, and search determines which sets to supply at each stage. Correct labels for search queries themselves are unnecessary. However, an execution with a correct final answer can contain redundant searches or succeed by chance. Extracted traces are therefore candidates, not proof of correct intermediate processing (Khattab et al. 2024; Opsahl-Ong et al. 2024, sec. 3.1).
Figure 1 shows both a simplified question and a keyword query as demonstrations in the same prompt, illustrating how intermediate executions become examples for later inputs.
DSPy is consequently not the name of a single candidate-selection algorithm. It separates module representation from optimization procedures, allowing different optimizers to operate on the same program. The original DSPy paper also includes fine-tuning, but we first consider selecting instructions and demonstrations while keeping the target model’s weights fixed.
Searching Instruction–Demonstration Combinations by Overall Score
Selecting demonstrations alone may fail to communicate conditions or output rules that are difficult to illustrate through data. Conversely, improving only instructions can lose behavior communicated by concrete examples. This creates a joint selection problem over each stage’s instruction and demonstration set.
MIPRO (Multi-prompt Instruction PRoposal Optimizer, EMNLP 2024) jointly optimizes module instructions and demonstrations while retaining the supplied program structure. It first builds demonstration candidates from successful end-to-end executions. It then conditions a candidate-generating LM on data characteristics, program flow, successful examples, and related information to generate instruction candidates for each module (Opsahl-Ong et al. 2024, secs. 3–4).
In the opening example, this means choosing search instruction A or B, answer instruction C or D, and demonstration sets for both stages. In this illustrative configuration, a candidate is not one instruction but the whole program with several choices instantiated. A search instruction’s effectiveness also depends on the answer module’s instruction and examples.
The bottom of Figure 2 shows that each candidate includes instructions and demonstrations for multiple modules. Finishing the candidate-generation stage at the top still leaves a separate search over combinations.
MIPRO uses previously tried combinations and their final scores to select promising next combinations. The original paper’s surrogate-based search is Bayesian optimization with Optuna’s multivariate Tree-structured Parzen Estimator. It uses evaluation history to account for the joint contributions of choices and combines minibatch evaluation with periodic broader evaluation to reduce search cost. Credit assignment here means learning the relationship between combinations and scores, not conclusively explaining error causes in prose (Opsahl-Ong et al. 2024, sec. 3.2, §4.3).
Grounding instruction proposals in data and executions is separate from searching their combinations. If a solution lies outside the fixed candidate set, the surrogate alone cannot invent that instruction. The paper separates proposal design from credit-assignment design and compares demonstrations alone, instructions alone, and both. Its result of exceeding baselines on five of seven tasks with Llama3-8B supports the complete design, not instruction generation alone (Opsahl-Ong et al. 2024, secs. 5–6).
The 2024 MIPRO paper must also be distinguished from MIPROv2 in later DSPy software. GEPA’s final version compares against MIPROv2 and separates instruction-only optimization from joint instruction–demonstration search for GPT-4.1 Mini. Not all conditions of the original paper carry over unchanged to later implementation comparisons (Agrawal et al. 2026, Table 2, Appendix G.4).
Turning Critiques and Execution Traces into Revisions
Scores compare candidates but do not directly explain failure. Recording format violations, missing documents, or tool exceptions in language supplies more material for generating instructions. The question is which editable variables should receive that feedback and in what form.
TextGrad (Nature 2025) treats instructions, outputs, and related elements as variables and passes natural-language critiques through a computation graph connecting model and tool calls. It combines downstream critiques with call inputs and outputs and the variable’s role, then asks an LM to generate improvement feedback for upstream variables. It aggregates information arriving through multiple paths and rewrites designated variables as text (Yuksekgonul et al. 2025, figs. 1–2).
In the retrieval-and-answer example, a critique that “the answer lacks evidence” could become feedback for both the answer instruction and the query-generation instruction. This example explains the mechanism. A computation graph does not establish that retrieval actually caused the failure.
TextGrad’s “gradients” are improvement information expressed in natural language, not derivatives of differentiable functions. Chaining locally generated critiques does not compute exact numerical chain-rule gradients or necessarily assign credit to the correct cause. This is more than a terminology warning. The design depends on the feedback-generating LM’s reasoning about what to communicate to each variable.
Trace/OptoPrime (NeurIPS 2024) uses computational graphs differently. Trace records execution and extracts the relevant subgraph between an editable component and an output receiving feedback. OptoPrime converts that structure, operation descriptions, intermediate values, and output feedback into a representation resembling a code-execution report for an LM. The LM reasons from the report and proposes new values for editable components. Previous parameter–feedback pairs can also be used as in-context history (Cheng et al. 2024, secs. 2–4).
Figure 4 is a report from a small computational example in the paper. It supplies the editable variable x, intermediate values, operation descriptions, and feedback asking for a larger output together. It illustrates what the optimizer sees when proposing a revision, not a measured success after applying a new value.
TextGrad generates operation-specific local feedback upstream, whereas OptoPrime presents the relevant execution subgraph as a revision problem. Trace does not record the entire actual program. It represents computational relationships at user-specified operation granularity, so that granularity changes the information visible to the optimizer.
Trace can also update more than prompts. Its Big-Bench Hard experiments change prompt-construction and answer-extraction functions as well as instructions, comparing against DSPy’s instruction-tuning COPRO. Even with the same initial program, the allowed changes differ. This tests joint optimization including code, not the isolated superiority of an instruction-only optimizer (Cheng et al. 2024, sec. 5.3).
TextGrad can also directly improve a particular problem’s answer, code, or molecule. Updating instructions shared across future inputs and improving one solution are different evaluation targets even when they use the same update mechanism. For evidence about joint optimization, this chapter focuses on settings that update a shared configuration and measure it on other inputs.
Editing from Traces and Retaining Input-Specific Strengths
Even if rich feedback produces good edits, always starting from the candidate with the highest mean score can discard candidates strong on different inputs. Choosing the next search parent is a separate issue from generating edits.
GEPA (Genetic-Pareto, ICLR 2026) evolves sets of module prompts while keeping the target model’s weights fixed. A candidate instantiates the entire system’s prompts. It selects a parent, runs it on a training minibatch, and gives a reflection LM the selected module’s execution traces and the evaluator’s textual feedback. It generates a new instruction from that reflection. Candidates that improve on the minibatch receive broader evaluation and enter the candidate pool (Agrawal et al. 2026, sec. 3, Figure 3).
The score matrix at the lower left of Figure 5 explains the selection. Rows are individual evaluation inputs and columns are candidates. P3 solves Task 1, P1 and P2 solve Task 2, and P2 solves Task 3. P1 only ties P2 on Task 2 and has no unique strength relative to P2 in this example. Filtering leaves P2 and P3 on the right, preserving success on different inputs for subsequent edits.
Pareto selection here uses per-input scores as its axes. It does not directly optimize trade-offs such as cost against accuracy or safety against speed. It filters candidates using input-level best scores and dominance, then probabilistically selects parents according to how many inputs each candidate leads on. This differs both from retaining only the best overall mean and from uniformly selecting all nondominated candidates (Agrawal et al. 2026, sec. 3.1).
The figure’s right side shows two candidate-generation mechanisms. Reflective mutation revises instructions from P2’s traces and textual feedback. System Aware Merge takes two candidates whose different modules improved from a common ancestor and selects which module instructions to inherit. The upper decision discards a new candidate without minibatch improvement and sends an improving candidate to broader evaluation. Parent diversity, instruction editing, module merging, and staged evaluation are distinct design elements.
In the final version, training contents and labels are available for optimization, while direct access to validation contents is restricted and validation scores support candidate selection and related decisions. Thus, showing failure details to the reflection LM and selecting candidates using validation scores are different information pathways (Agrawal et al. 2026, sec. 4).
Aligning Local Quality with Downstream Success
A fluent search query is unhelpful if it retrieves no necessary documents. Similarly, an individually reasonable multi-agent utterance need not help subsequent agents succeed. Research addresses this through learned rewards, joint candidate evaluation, and rule-level credit.
Optimas (ICLR 2026) provides a local reward function (LRF) that scores each component’s input and output. It creates preference data from the overall rewards obtained by passing candidate local outputs through downstream processing and aligns local reward rankings with overall rewards. Since useful local outputs change as the system changes, the reward functions also adapt using small amounts of new data. Component-appropriate optimizers then maximize these local rewards (S. Wu et al. 2026, sec. 4).
Figure 6 separates learning a local reward aligned with global rewards from optimizing a component against that local reward.
Instead of obtaining a final score from a full execution for every update, this design uses learned local rewards as update objectives. The reward functions use a shared LLM with component-specific output heads. Unlike TextGrad’s textual critiques, these are learned numerical ranking functions.
Training the reward-estimation model must also be separated from updating the target system’s model weights. Optimas can target prompts, hyperparameters, model selection, and model weights. In its Amazon recommendation system, trainable local models are updated with proximal policy optimization (PPO), while the DSPy and TextGrad comparison settings adjust the final component’s prompt. Differences in this recommendation experiment therefore cannot be read as differences solely in fixed-weight prompt search. Its theoretical convergence target is also a component-wise maximum under assumptions, not a guaranteed global optimum (S. Wu et al. 2026, sec. 4.4, Appendix G.4).
MASPO (ICML 2026) jointly improves agents’ role-specific prompts. Its central joint-evaluation mechanism assesses not only whether outputs are locally reasonable but also whether they support downstream agents’ success. It uses evaluation independent of correct labels and data-driven evolutionary beam search to improve candidates (Wang et al. 2026).
MIPRO relates overall scores to instruction–demonstration combinations, whereas MASPO includes the quality of inter-agent handoffs in candidate evaluation. Searching role-specific prompts with joint evaluation also differs from Optimas’s learned local rewards for heterogeneous settings. The authors’ six-task experiments test the former. Not requiring intermediate correct labels must not be expanded into not requiring evaluation signals.
TRUCE (Trajectory-based Rule Credit Estimation, ICML 2026) uses behavioral rules within prompts as its update unit. Rather than relating final feedback indiscriminately to the complete interaction history, it links feedback to informative partial trajectories and turns their contributions into rule-level edits. It aggregates local updates from multiple tasks while preserving agent roles and interaction structure (B. Wu et al. 2026).
TRUCE therefore does not simply append raw traces to a long critique. The design specifies which partial trajectory corresponds to which rule. MASPO emphasizes candidate evaluation that includes downstream success, whereas TRUCE makes the mapping from outcomes to edit units more granular. Since it fixes roles and connections, its results can be distinguished from searching agent counts or communication structure.
Separating Whole-Method Results from Mechanism Effects
Whole-method comparisons show final performance, while comparisons within a fixed editing procedure reveal effects of candidate selection. GEPA’s ICLR 2026 final version includes both. Table 1 extracts three Qwen3 8B tasks from its Table 1 (Agrawal et al. 2026, Table 1).
| Method | HotpotQA | IFBench | AIME-2025 |
|---|---|---|---|
| Initial configuration | 42.33 | 36.90 | 27.33 |
| MIPROv2 | 55.33 | 36.22 | 20.00 |
| GEPA | 62.33 | 38.61 | 32.00 |
| GEPA+Merge | 64.33 | 28.23 | 32.00 |
| GRPO | 43.33 | 35.88 | 38.00 |
Columns contain different task metrics and should not be treated as the same quantity across tasks. GEPA’s total optimization budgets are 6,871, 3,593, and 1,839 rollouts, respectively, versus 24,000 for GRPO on each task. A rollout is one input’s system execution and evaluation, not an equal unit of tokens, monetary cost, or wall time. Group relative policy optimization (GRPO) updates the target model’s weights, so its editable components also differ.
Under these conditions, GEPA exceeds MIPROv2 in all three columns. Adding Merge raises HotpotQA but lowers IFBench. On AIME-2025, GRPO exceeds GEPA. These differences concretely show why module merging is not always beneficial and prompt optimization cannot generally be said to replace weight training.
The same final version’s Table 2 directly compares TextGrad and Trace/OptoPrime. On HotpotQA with GPT-4.1 Mini, Trace/OptoPrime scores 60.33, MIPROv2 58.00, TextGrad 62.33, and GEPA 69.00. This compares complete optimizers with the task and target model aligned. It does not replace only TextGrad’s local critique or OptoPrime’s subgraph representation (Agrawal et al. 2026, Table 2).
By contrast, Table 2 fixes the editing mechanism with Qwen3 8B and varies parent selection. It compares using the single best candidate, maintaining a four-candidate beam, and GEPA’s input-specific selection (Agrawal et al. 2026, Table 3).
| Parent Selection | HotpotQA | IFBench | HoVer | PUPA |
|---|---|---|---|---|
| SelectBestCandidate | 58.33 | 30.44 | 45.33 | 85.45 |
| BeamSearch (4 candidates) | 57.33 | 36.39 | 41.00 | 81.08 |
| GEPA input-specific selection | 62.33 | 38.61 | 52.33 | 91.85 |
This supports the finding that parent selection changes results even with the same reflective editing. SelectBestCandidate should not be treated as a reproduction of all of TextGrad, nor BeamSearch as a reproduction of another complete optimizer. These are mechanism comparisons within GEPA’s editing procedure. All are experiments by GEPA’s authors.
Intermediate instructions and demonstrations can be optimized even with final labels alone. However, selecting combinations by overall score, propagating critiques, editing from traces, learning local rewards, and assigning credit to rules are not equivalent. The remaining question is which intermediate information and credit assignment contribute most to joint improvement on unseen inputs under the same edit scope and evaluation cost. Workflow Search addresses changing the fixed roles and call structure themselves.
Figure Credits
Figure 4: (Opsahl-Ong et al. 2024). Original figure: published version, Figure 4. CC BY 4.0. Original figure cropped and rasterized.
Figure 1: (Yuksekgonul et al. 2025). Published version. Figure from arXiv 2406.07496v1, Figure 1 (not the Nature version). CC BY 4.0. Original figure cropped and rasterized.
Figure 4: (Cheng et al. 2024). Original figure: published version, Figure 4. CC BY 4.0. Original figure cropped and rasterized.
Figure 3: (Agrawal et al. 2026). Original figure: ICLR 2026 public version, Figure 3. CC BY 4.0. Original figure cropped and rasterized.
Figure 1: (Wang et al. 2026). Original figure: published version, Figure 1. CC BY 4.0. Original figure cropped and rasterized.
Figure 2: (B. Wu et al. 2026). Original figure: published version, Figure 2. CC BY 4.0. Original figure cropped and rasterized.
Figure 11: (Khattab et al. 2024). Source: paper PDF, Figure 11. Original figure region cropped and rasterized to PNG.
Figure 3: (S. Wu et al. 2026). Source: paper PDF, Figure 3. Original figure region cropped and rasterized to PNG.