Markdown

GEPA: reflective prompt evolution¶

Scope: optimizing the text of a system (prompt instructions, tool descriptions, config, policy code) against a metric, using an LLM to read execution traces and propose edits, with an instance-wise Pareto front deciding which candidate to mutate next. This page covers why a scalar reward throws away most of what a failed rollout tells you, the Pareto selection rule that keeps specialists alive (ported from the reference implementation and validated here), the two-stage acceptance gate that makes the search affordable, the adapter contract you implement to plug in your own system, and an audit of the published results including where the method makes things worse. It is the concrete method behind the prompt-optimization line in self-improving harnesses, a sibling of automated harness optimization, and the non-weight-updating counterpart to GRPO.

The Python block below is executed and asserted in this page (Python 3.11, stdlib only). Its Pareto selection is a faithful port of remove_dominated_programs and select_program_candidate_from_pareto_front from gepa-ai/gepa at commit 0310bb7b4952 (2026-07-28), and it audits Table 1 of arXiv:2507.19457 against that paper's own aggregate row. The API snippets are reference templates against gepa v0.1.4 (released 2026-07-15), not executed here. Benchmark numbers are the paper's; results quoted from the project blog or vendor posts are marked as such and are not peer-reviewed.

What it is¶

GEPA (Genetic-Pareto) is a prompt optimizer for compound AI systems, published at ICLR 2026 (Oral). Formally it treats a system as a set of language modules, each with its own prompt, plus control flow. It optimizes the prompts and leaves the weights alone.

The loop is four steps:

Select a candidate to mutate, sampled from the instance-wise Pareto front rather than always taking the current best.
Execute the candidate on a stochastically sampled minibatch from the feedback split, tracing the whole program (reasoning, tool calls, tool outputs).
Reflect and mutate. A feedback function returns diagnostic text, an LLM reads the traces plus that text, diagnoses what went wrong, and rewrites the instruction of one selected module.
Accept. Score the child on the same minibatch. If it improved, add it to the pool with an ancestry record and only then evaluate it on the full Pareto validation set.

There is an optional fifth operator, system-aware merge, which combines two Pareto-optimal candidates that excel on different tasks.

The concept the implementation names as central is Actionable Side Information (ASI): diagnostic text your evaluator returns alongside the score. The project describes it as the text-optimization analogue of a gradient, and it is the part most people leave out.

Why use it¶

A scalar reward discards almost everything a rollout produced. The paper's premise is that "the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards." A failed run produced a stack trace, a wrong intermediate answer, and a tool call with bad arguments. Policy gradient compresses all of that into one number.
Sample efficiency. Across six tasks with Qwen3 8B, GEPA beat GRPO on the aggregate while GRPO used 24,000 rollouts per task. On IFBench specifically, the paper reports GEPA "found optimal prompts after just 678 rollouts achieving 38.61%, outperforming GRPO's test set score of 35.88% with 24,000 rollouts."
It touches no weights. No trainer, no optimizer state, no GPU fleet for the policy. The artifact is text, which means it works on closed models, deploys as a config change, and is reviewable in a pull request.
The output is inspectable and portable. The paper reports that prompts optimized entirely on Qwen3 8B transferred to GPT-4.1 Mini for a +9% gain without modification, beating optimizers that were run directly against GPT-4.1 Mini.
It generalizes past prompts. The reference implementation's optimize_anything API optimizes any text artifact against any evaluator. That includes artifacts an infrastructure team owns: scheduling policies, configs, kernels, agent scaffolds.

When to use it (and when not)¶

Use it when:

You have a metric and, critically, a way to produce diagnostic text about failures. Compiler errors, assertion messages, profiler output, judge rationales, validation failures.
Your system is a prompt pipeline you cannot or will not fine-tune (closed model, no GPUs for training, weights owned by someone else).
Your evaluation is expensive per rollout, so a method that converges in hundreds of evaluations rather than tens of thousands is the difference between feasible and not.

Do not use it when:

Your metric is a bare scalar with no explicable failure signal. Then reflection has nothing to read and you have an expensive random search. This is the single most common way to waste a GEPA run.
You need a capability the base model does not have. Prompt optimization redistributes existing capability; it does not create new skills the way weight updates can. The published AIME-2025 column is the honest illustration: GRPO reached 38.00 where GEPA reached 32.00.
Your eval set is tiny. Pareto selection needs per-instance scores across a validation set to have any front to work with. With a handful of instances it degenerates toward greedy selection.
You cannot afford a strong reflection model. The reflection LM is usually a larger model than the task LM (the quickstart pairs a GPT-4.1-mini task model with a GPT-5 reflection model). That is a real cost line.

Architecture¶

flowchart TB
  SEED["seed candidate<br/>(module prompts as text)"] --> POOL
  subgraph SEARCH["GEPA loop, until budget exhausted"]
    POOL[("candidate pool P<br/>+ ancestry A<br/>+ per-instance scores S")]
    POOL --> SEL["Pareto candidate selection<br/>sample ∝ instances led"]
    SEL --> MOD["select one module j to update<br/>(round-robin / all-component)"]
    MOD --> RUN["execute on minibatch of size b<br/>capture full traces"]
    RUN --> ASI["feedback fn returns score<br/>+ Actionable Side Information"]
    ASI --> REFL["reflection LM reads traces + ASI<br/>proposes new instruction for module j"]
    REFL --> GATE{"minibatch score improved?"}
    GATE -->|"no"| POOL
    GATE -->|"yes"| FULL["evaluate on full Pareto valset<br/>(the expensive step, gated)"]
    FULL --> POOL
  end
  POOL --> MERGE["optional: system-aware merge<br/>of two Pareto-optimal candidates"]
  MERGE --> POOL
  POOL --> OUT["return candidate maximizing<br/>mean score on Dpareto"]

Two things in that diagram carry most of the design. The gate before the full evaluation is what keeps the budget small. The selection step is what keeps the search from collapsing into a single lineage.

How it works: selection and budget, validated¶

"""GEPA's Pareto candidate selection, ported faithfully from gepa-ai/gepa
src/gepa/gepa_utils.py @ 0310bb7b4952, and the budget arithmetic behind its
sample-efficiency claim."""
import random
from collections import Counter


# ---------------------------------------------------------------- 1. the front
def pareto_fronts(scores: dict[int, list[float]]) -> dict[int, set[int]]:
    """P*[i] = the set of candidates achieving the best score on instance i."""
    n = len(next(iter(scores.values())))
    fronts = {}
    for i in range(n):
        best = max(s[i] for s in scores.values())
        fronts[i] = {k for k, s in scores.items() if s[i] == best}
    return fronts


def is_dominated(y, programs, fronts) -> bool:
    """GEPA's definition: y is dominated when every front containing y also
    contains some other surviving program. Equivalently, y survives only if it
    is the SOLE leader on at least one instance."""
    for front in fronts.values():
        if y not in front:
            continue
        if not any(other in programs for other in front):
            return False
    return True


def remove_dominated(fronts, agg_scores) -> dict[int, set[int]]:
    freq = Counter(p for front in fronts.values() for p in front)
    programs = sorted(freq, key=lambda p: agg_scores[p])   # ascending: drop weakest first
    dominated, changed = set(), True
    while changed:
        changed = False
        for y in programs:
            if y in dominated:
                continue
            others = set(programs) - {y} - dominated
            if is_dominated(y, others, fronts):
                dominated.add(y)
                changed = True
                break
    return {i: {p for p in front if p not in dominated} for i, front in fronts.items()}


def select(fronts, agg_scores, rng) -> int:
    pruned = remove_dominated(fronts, agg_scores)
    freq = Counter(p for front in pruned.values() for p in front)
    pool = [p for p, f in freq.items() for _ in range(f)]
    assert pool
    return rng.choice(pool)


def aggregate(scores):
    return {k: sum(v) / len(v) for k, v in scores.items()}


# --- a specialist that is worst on average but sole leader on one instance ---
S = {
    0: [0.9, 0.9, 0.9, 0.9, 0.0],   # strong generalist, best average
    1: [0.8, 0.8, 0.8, 0.8, 0.0],   # strictly worse than 0 everywhere
    2: [0.1, 0.1, 0.1, 0.1, 1.0],   # WORST average, the only one solving instance 4
}
agg = aggregate(S)
assert agg[0] == max(agg.values())
assert agg[2] == min(agg.values())

fronts = pareto_fronts(S)
assert fronts[0] == {0} and fronts[4] == {2}
pruned = remove_dominated(fronts, agg)
survivors = {p for f in pruned.values() for p in f}
# Candidate 1 never leads any instance, so it is pruned. Candidate 2 survives
# despite the worst average, because it is the sole leader on instance 4.
assert survivors == {0, 2}, survivors
assert 1 not in survivors
# Greedy "always mutate the current best" would never look at candidate 2.
best_avg = max(agg, key=agg.get)
assert best_avg == 0 and best_avg != 2

# --- the sampling weight is exactly the number of instances led -------------
rng = random.Random(0)
draws = Counter(select(fronts, agg, rng) for _ in range(20_000))
freq = Counter(p for f in pruned.values() for p in f)
total = sum(freq.values())
for p, f in freq.items():
    assert abs(draws[p] / 20_000 - f / total) < 0.02, (p, draws[p] / 20_000, f / total)
assert freq[0] == 4 and freq[2] == 1                 # 4 instances vs 1
assert abs(draws[2] / 20_000 - 0.2) < 0.02           # the specialist still gets ~20%

# --- a high-average candidate that is never a SOLE leader is still pruned ---
T = {
    0: [1.0, 1.0, 0.2],
    1: [1.0, 1.0, 0.2],   # ties with 0 everywhere: never sole leader
    2: [0.3, 0.3, 1.0],
}
t_agg = aggregate(T)
t_pruned = remove_dominated(pareto_fronts(T), t_agg)
t_surv = {p for f in t_pruned.values() for p in f}
assert len(t_surv & {0, 1}) == 1                     # exactly one of the tied pair survives
assert 2 in t_surv
assert t_agg[0] == t_agg[1]                          # equal averages, one is still dropped
# Pruning walks candidates from LOWEST aggregate score up, so ties are broken
# toward keeping the higher-scoring duplicate. It is order-dependent by design.
U = {0: [1.0, 1.0, 0.2], 1: [1.0, 1.0, 0.9], 2: [0.3, 0.3, 1.0]}
u_surv = {p for f in remove_dominated(pareto_fronts(U), aggregate(U)).values() for p in f}
assert 1 in u_surv and 0 not in u_surv               # the stronger twin is kept

# --- every instance keeps at least one leader after pruning ----------------
for front in pruned.values():
    assert front
for front in t_pruned.values():
    assert front


# ------------------------------------------------- 2. the two-stage accept gate
def iteration_cost(minibatch_b, n_pareto, accepted):
    """Algorithm 1: always pay the minibatch; pay the full valset ONLY on a
    minibatch improvement."""
    return minibatch_b + (n_pareto if accepted else 0)


B, NPARETO = 5, 100
assert iteration_cost(B, NPARETO, False) == 5
assert iteration_cost(B, NPARETO, True) == 105


def budget_for(iters, accept_rate, b=B, n=NPARETO):
    return iters * (b + accept_rate * n)


# At a realistic acceptance rate the gate is what makes the search affordable.
gated = budget_for(200, 0.15)
always_full = budget_for(200, 1.0)
assert gated == 4000.0 and always_full == 21000.0
assert always_full / gated > 5.2
# The gate only helps while acceptance is low; at 100% acceptance it costs the
# minibatch as pure overhead.
assert budget_for(200, 1.0) > 200 * NPARETO
assert budget_for(200, 0.0) == 200 * B
# Sensitivity: the full-valset term dominates once acceptance exceeds b/n.
crossover = B / NPARETO
assert crossover == 0.05
assert budget_for(1, 0.05) == 2 * B


# ------------------------------------------- 3. the published rollout numbers
GRPO_ROLLOUTS = 24_000            # per task, every benchmark
IFBENCH_GEPA_TO_BEST = 678        # rollouts at which GEPA found its best prompt
IFBENCH_GEPA_TOTAL = 3_593        # total optimization budget consumed
assert abs(GRPO_ROLLOUTS / IFBENCH_GEPA_TO_BEST - 35.4) < 0.05     # the "35x" claim
assert abs(GRPO_ROLLOUTS / IFBENCH_GEPA_TOTAL - 6.68) < 0.01       # total-budget ratio
# So "35x fewer rollouts" is time-to-best-candidate, not total spend. Both are
# real, they answer different questions, and they differ by more than 5x.
assert (GRPO_ROLLOUTS / IFBENCH_GEPA_TO_BEST) / (GRPO_ROLLOUTS / IFBENCH_GEPA_TOTAL) > 5


# ------------------------------------------------- 4. Table 1 (Qwen3 8B) audit
TASKS = ["HotpotQA", "IFBench", "Hover", "PUPA", "AIME-2025", "LiveBench-Math"]
T1 = {
    "Baseline":   [42.33, 36.90, 35.33, 80.82, 27.33, 48.70],
    "MIPROv2":    [55.33, 36.22, 47.33, 81.55, 20.00, 46.60],
    "GEPA":       [62.33, 38.61, 52.33, 91.85, 32.00, 51.95],
    "GEPA+Merge": [64.33, 28.23, 51.67, 86.26, 32.00, 51.95],
}
PUBLISHED_AGG = {"Baseline": 45.23, "MIPROv2": 47.84, "GEPA": 54.85, "GEPA+Merge": 52.40}
for name, row in T1.items():
    assert abs(sum(row) / 6 - PUBLISHED_AGG[name]) < 0.008, (name, sum(row) / 6)

# GEPA beats the baseline on every task; MIPROv2 does not.
assert all(g > b for g, b in zip(T1["GEPA"], T1["Baseline"]))
mipro_regress = [t for t, m, b in zip(TASKS, T1["MIPROv2"], T1["Baseline"]) if m < b]
assert mipro_regress == ["IFBench", "AIME-2025", "LiveBench-Math"]
assert T1["Baseline"][4] - T1["MIPROv2"][4] > 7        # AIME: prompt optimization HURT by 7.3 pts

# Merge is not a free win: it loses to plain GEPA on 3 of 6 tasks, and on
# IFBench it falls below the unoptimized baseline.
merge_worse = [t for t, m, g in zip(TASKS, T1["GEPA+Merge"], T1["GEPA"]) if m < g]
assert set(merge_worse) == {"IFBench", "Hover", "PUPA"}
assert T1["GEPA+Merge"][1] < T1["Baseline"][1]
assert abs(T1["GEPA"][1] - T1["GEPA+Merge"][1] - 10.38) < 0.01
GRPO_IFBENCH = 35.88
assert T1["GEPA+Merge"][1] < GRPO_IFBENCH              # ...and below GRPO too
assert PUBLISHED_AGG["GEPA+Merge"] < PUBLISHED_AGG["GEPA"]

print("all GEPA assertions passed")
print("  survivors of Pareto pruning:", sorted(survivors), "(candidate 1 dominated)")
print("  sampling share of the worst-average specialist:", f"{draws[2] / 20_000:.1%}")
print("  budget, gated vs always-full:", gated, "vs", always_full)
print("  IFBench rollout ratios  to-best / total:",
      f"{GRPO_ROLLOUTS / IFBENCH_GEPA_TO_BEST:.1f}x", "/",
      f"{GRPO_ROLLOUTS / IFBENCH_GEPA_TOTAL:.1f}x")
print("  MIPROv2 regressions vs baseline:", mipro_regress)
print("  GEPA+Merge worse than GEPA on:", merge_worse)

Executed output:

all GEPA assertions passed
  survivors of Pareto pruning: [0, 2] (candidate 1 dominated)
  sampling share of the worst-average specialist: 20.1%
  budget, gated vs always-full: 4000.0 vs 21000.0
  IFBench rollout ratios  to-best / total: 35.4x / 6.7x
  MIPROv2 regressions vs baseline: ['IFBench', 'AIME-2025', 'LiveBench-Math']
  GEPA+Merge worse than GEPA on: ['IFBench', 'Hover', 'PUPA']

"Dominated" does not mean what you probably think¶

The most important detail in the selection rule is the domination test, and it is not classical Pareto dominance. A candidate survives pruning only if it is the sole leader on at least one validation instance. Block 1 makes both consequences concrete:

A candidate with the worst average score survives, because it is the only one that solves instance 4. That is the whole point: greedy "mutate the current best" would never look at it, and the paper's Figure 6a describes exactly that trap, where the search "repeatedly attempts to refine it, fails to improve, and ultimately depletes the budget."
A candidate with a strong average is pruned when it merely ties on every instance it leads. In the second example two candidates have identical scores everywhere, and one is dropped. That is correct (the duplicate adds nothing to exploration) but surprising if you expected average score to protect a candidate.

The pruning is also order-dependent by construction: the loop walks candidates from lowest aggregate score upward and removes the first dominated one it finds, so among tied duplicates the higher-scoring one is kept. Block 1 asserts that too. If you reimplement this and iterate in a different order, you will get a different surviving set.

Selection probability is then exactly proportional to the number of instances a surviving candidate leads. In the example the generalist leads 4 instances and the specialist leads 1, so the specialist still receives about 20% of the mutation budget. That number is your exploration rate, and it is not a hyperparameter you set; it emerges from how diverse your validation set is. A validation set whose instances all reward the same behaviour will produce a single leader and collapse GEPA into greedy hill climbing without any warning.

The gate is where the sample efficiency comes from¶

Algorithm 1 evaluates a child on the minibatch first and only pays for the full validation set if the minibatch improved. Block 2 prices that: at a minibatch of 5, a 100-instance validation set, and a 15% acceptance rate, 200 iterations cost 4,000 rollouts against 21,000 if you validated every child. The crossover where the full-validation term starts to dominate is b / n_pareto, which is 5% here. So the gate pays for itself the moment your acceptance rate exceeds a few percent, and its value grows as the search matures and acceptance falls.

This is a paradigm worth stealing whether or not you use GEPA: make your expensive evaluation conditional on a cheap one, and size the cheap one so its cost is a small fraction of the expensive one.

Read the sample-efficiency headline precisely¶

The abstract claims "up to 35x fewer rollouts". Block 3 shows where that comes from: on IFBench, 24,000 GRPO rollouts against the 678 rollouts at which GEPA found its best prompt, which is 35.4x. GEPA's total optimization budget on that task was 3,593 rollouts, which is 6.7x fewer than GRPO, not 35x. Both numbers are real and both are defensible, but they answer different questions ("how fast did it find the winner" against "what did the whole search cost") and they differ by more than a factor of 5. When you budget a GEPA run, budget the total.

What the results table actually shows¶

Block 4 reproduces all four published aggregates from the per-task cells, which confirms the cell values, then reads them adversarially.

Method (Qwen3 8B)	HotpotQA	IFBench	Hover	PUPA	AIME-2025	LiveBench-Math	Aggregate
Baseline	42.33	36.90	35.33	80.82	27.33	48.70	45.23
MIPROv2	55.33	36.22	47.33	81.55	20.00	46.60	47.84
GEPA	62.33	38.61	52.33	91.85	32.00	51.95	54.85
GEPA+Merge	64.33	28.23	51.67	86.26	32.00	51.95	52.40

Three findings:

GEPA improves on the baseline on all six tasks. MIPROv2 does not: it regresses on IFBench, LiveBench-Math, and most sharply on AIME-2025, where it drops 7.33 points below the unoptimized prompt. Prompt optimization is not automatically safe, and a stronger optimizer is the reason to prefer GEPA rather than evidence that the category is risk-free.
Merge is not a free win. GEPA+Merge is worse than plain GEPA on three of six tasks and lower on the aggregate (52.40 against 54.85). On IFBench it collapses to 28.23, which is below the unoptimized baseline of 36.90 and below GRPO's 35.88. The table caption states that "GEPA and GEPA+Merge achieve better performance than GRPO with far fewer rollouts on all benchmarks except AIME"; for GEPA+Merge on IFBench that is contradicted by the table's own numbers. Treat merge as an ablation to run, not a default to enable.
RL still wins where new capability is needed. On AIME-2025, GRPO reaches 38.00 against GEPA's 32.00. That is the honest boundary of the method.

How to use it¶

Reference templates against gepa v0.1.4 (PyPI, MIT). Pin the version; the API is young and moving.

Direct prompt optimization. The minimum viable run:

import gepa

trainset, valset, _ = gepa.examples.aime.init_dataset()

seed_prompt = {
    "system_prompt": "You are a helpful assistant. Answer the question. "
                     "Put your final answer in the format '### <answer>'"
}

result = gepa.optimize(
    seed_candidate=seed_prompt,
    trainset=trainset,
    valset=valset,
    task_lm="openai/gpt-4.1-mini",
    max_metric_calls=150,          # the WHOLE budget, not per iteration
    reflection_lm="openai/gpt-5",  # deliberately stronger than the task model
)
print(result.best_candidate["system_prompt"])

Note the two-model split: a cheap task model executes rollouts, a stronger reflection model reads traces and rewrites instructions. That asymmetry is the cost structure of the method, and it is why GEPA can optimize a small open model economically.

Inside a DSPy pipeline, which is where multi-module systems belong:

import dspy

optimizer = dspy.GEPA(
    metric=your_metric,
    max_metric_calls=150,
    reflection_lm="openai/gpt-5",
)
optimized = optimizer.compile(student=MyProgram(), trainset=trainset, valset=valset)

Optimizing something that is not a prompt. This is the API that matters for infrastructure work, because the artifact can be a config, a policy, or code:

import gepa.optimize_anything as oa
from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig

def evaluate(candidate: str) -> float:
    result = run_my_system(candidate)
    oa.log(f"Output: {result.output}")   # Actionable Side Information
    oa.log(f"Error: {result.error}")     # ...this is what reflection actually reads
    return result.score

result = optimize_anything(
    seed_candidate="<your initial artifact>",
    evaluator=evaluate,
    objective="Describe what you want to optimize for.",
    config=GEPAConfig(engine=EngineConfig(max_metric_calls=100)),
)

The two oa.log lines are not instrumentation. They are the gradient. An evaluator that returns only result.score gives reflection nothing to reason about, and the run degrades to random text mutation.

How to develop with it: the adapter contract¶

For anything beyond a single prompt you implement GEPAAdapter, which is the single integration point. It has two required responsibilities and two optional ones:

Method	Required	Responsibility
`evaluate(batch, candidate, capture_traces)`	yes	Run the candidate, return `EvaluationBatch` with per-example `outputs`, `scores`, and (when `capture_traces=True`) `trajectories`
`make_reflective_dataset(candidate, eval_batch, components)`	yes	Turn traces into a small JSON-serializable dataset per component, which is what the reflection LM sees
`propose_new_texts(...)`	no	Override the default instruction proposal, for example to update coupled components together
`get_adapter_state` / `set_adapter_state`	no	Persist adapter state across checkpoint and resume boundaries

Engineering details from the interface that change how you write it:

Scores are summed for minibatch acceptance and averaged over the validation set for Pareto tracking. If your metric is not comparable across instances, the acceptance decision and the front will disagree with your intuition.
trajectories must align one-to-one with outputs and scores. The reflective dataset is built by index.
Keep the reflective dataset small. It goes into the reflection LM's context on every proposal, so it is a recurring cost, and a dataset stuffed with raw logs will crowd out the signal.
make_reflective_dataset is where you decide what the optimizer can learn. This is the highest-leverage function you will write. Include the error message, the failing intermediate value, and the expected value. Exclude anything that is noise or that leaks the answer.

Practical selection knobs the implementation exposes as strategies: ParetoCandidateSelector (the default described above), CurrentBestCandidateSelector (greedy, the local-optimum trap), EpsilonGreedyCandidateSelector, and TopKParetoCandidateSelector (Pareto restricted to the top K by aggregate score, a useful middle ground when your pool grows large and the front fills with weak specialists).

How to run it in production¶

Optimization is a build step, not a serving step. Run it in CI or a scheduled job, commit the resulting text, and deploy it like any other config. Nothing about GEPA belongs on the request path.
Pin everything that defines the substrate. Task model, reflection model, eval set hash, seed, and the gepa version. An optimized prompt is a measurement against a specific substrate, and the fixed-substrate rule from evaluation integrity applies exactly as it does to harness search.
Hold out a test split the optimizer never sees. The Pareto validation set is used for selection and is therefore fit to; a prompt that improves on it and not on held-out data is overfitting.
Budget the reflection model separately. It is called once per proposal with a context containing traces. On a long run this can exceed the task model's spend even though it makes far fewer calls.
Re-run on model upgrades. A prompt tuned against one model version is not guaranteed to transfer, though the paper's cross-model result is encouraging. Treat it as a regression test on every model bump.
Version prompts as code. Ancestry is tracked during the search; keep the same discipline afterward so you can attribute a quality regression to a specific prompt change.

Where this touches infrastructure work¶

The paper's benchmarks are QA and math, but the artifact-agnostic framing is what makes it relevant to a GPU platform team. Three uses reported by the project and its adopters, all of which optimize text that an infrastructure team owns rather than a chatbot prompt:

Scheduling policy. The project blog reports a cloud scheduling policy discovered by GEPA delivering 40.2% cost savings against expert heuristics. If you own a scheduler or a routing policy expressed as text or code, it is an optimizable artifact with a metric you already collect.
Data-pipeline judge prompts. Microsoft reports using GEPA and DSPy to optimize the Qwen3-30B LLM-judge prompt that filters code pages in the MAI-Thinking-1 pre-training pipeline, over roughly 233B tokens of curated data. A judge prompt sitting in front of a large data-curation run is a high-leverage target: small quality changes multiply across the whole corpus.
Agent scaffolds and skills. The project reports an ARC-AGI agent improving from 32% to 89% through architecture discovery, and a coding agent's resolve rate on Jinja moving from 55% to 82% via auto-learned skills.

These are vendor and project-blog claims, not peer-reviewed results, and none of them was reproduced here. The transferable point is structural: anything you can score, and about which you can emit a diagnostic string, is a candidate artifact, which includes far more of a platform than most teams treat as tunable.

Failure modes¶

A scalar-only evaluator. No Actionable Side Information means reflection is guessing. This is the difference between GEPA and expensive random search, and it is the most common implementation mistake.
A homogeneous validation set. If one candidate leads every instance, the Pareto front has a single member, exploration collapses, and you have reimplemented greedy hill climbing.
Enabling merge by default. It regressed on three of six published tasks and fell below the unoptimized baseline on one.
Reading "35x fewer rollouts" as total spend. It is rollouts-to-best-candidate; total budget on that task was 6.7x fewer.
Overfitting the Pareto valset. It is a selection set, so it is fit to. Hold out a separate test split.
Expecting new capability. Prompt optimization redistributes what the model can already do. Where a genuinely new skill is needed, the AIME column says RL still wins.
Unbounded prompt growth. Reflective edits accumulate. Watch the token length of the optimized artifact, since it becomes a per-request cost forever and interacts with your prompt cache prefix boundaries.
Reimplementing the domination test as classical Pareto dominance. GEPA's rule is sole-leadership on at least one instance, and it is order-dependent. A different rule gives a different surviving set and different exploration behaviour.
Letting the optimizer see the metric definition. If the reflection LM can read how it is scored, it will optimize the score rather than the task. This is the standard reward-hacking boundary from evaluation integrity.

References¶

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (ICLR 2026 Oral): https://arxiv.org/abs/2507.19457
Reference implementation, gepa-ai/gepa (MIT): https://github.com/gepa-ai/gepa
GEPA documentation and guides: https://gepa-ai.github.io/gepa/
dspy.GEPA tutorials: https://dspy.ai/tutorials/gepa_ai_program/
MIPROv2 and the DSPy optimizer lineage: https://arxiv.org/abs/2406.11695
DSPy (the compound-AI-system framing GEPA optimizes): https://arxiv.org/abs/2310.03714
MAP-Elites illumination, the source of GEPA's Pareto strategy: https://arxiv.org/abs/1504.04909
TextGrad (a compared text optimizer): https://arxiv.org/abs/2406.07496
Databricks, automated prompt optimization for enterprise agents: https://www.databricks.com/blog/building-state-art-enterprise-agents-90x-cheaper-automated-prompt-optimization