GEPA (Genetic-Pareto) is a prompt optimization algorithm that uses natural language reflection to improve LLM prompts. Learn how it analyzes failures, proposes fixes, and evolves better prompts automatically.
César Miguelañez
Feb 10, 2026
What is the GEPA algorithm?
GEPA (Genetic-Pareto) is a prompt optimization algorithm that uses natural language reflection to automatically improve LLM prompts. Rather than relying on sparse numerical rewards like traditional reinforcement learning methods, GEPA learns high-level rules from trial and error by analyzing what went wrong in plain language and proposing targeted fixes.
The algorithm works by sampling system-level trajectories—including reasoning steps, tool calls, and outputs—then reflecting on them to diagnose problems, propose prompt updates, and combine the best lessons from multiple optimization attempts.
Why GEPA matters for prompt optimization
Most prompt optimization approaches require thousands of rollouts to learn new tasks. This makes them expensive, slow, and impractical for teams iterating on production AI systems.
GEPA takes a fundamentally different approach. It treats language itself as the learning medium. Instead of deriving policy gradients from scalar rewards, GEPA uses natural language reflection to understand why a prompt failed and how to fix it.
This design allows GEPA to turn even a handful of rollouts into meaningful quality improvements. Teams can optimize prompts without burning through massive compute budgets or waiting days for results.
How GEPA works
GEPA operates through a cycle of sampling, reflection, and refinement.
Sampling trajectories: The algorithm runs prompts through your AI system and captures the full execution path—reasoning chains, tool invocations, intermediate outputs, and final results.
Natural language reflection: Instead of reducing outcomes to a single score, GEPA analyzes trajectories in natural language. It identifies specific failure patterns, diagnoses root causes, and articulates what went wrong in terms a human would understand.
Proposing updates: Based on its diagnosis, GEPA generates candidate prompt modifications designed to address the identified problems.
Testing and selection: New prompt variants are tested against evaluations. GEPA tracks performance across multiple objectives simultaneously.
Pareto frontier combination: Here's where the "Genetic-Pareto" name comes from. GEPA maintains a Pareto frontier of its best attempts—prompts that represent different tradeoffs between objectives. It then combines complementary lessons from these frontier solutions to produce even better candidates.
This process repeats, with each cycle building on the insights from previous iterations.
GEPA vs traditional optimization approaches
Traditional reinforcement learning methods like Group Relative Policy Optimization (GRPO) treat prompt optimization as a black-box problem. They adjust prompts based on reward signals without understanding why changes help or hurt.
GEPA's reflection-based approach produces measurably better results with far less data.
Compared to GRPO: GEPA outperforms GRPO by 10% on average across benchmark tasks, with improvements reaching 20% in some cases. More importantly, GEPA achieves these gains using up to 35x fewer rollouts. This efficiency difference matters enormously for production teams where each rollout has real cost.
Compared to MIPROv2: GEPA outperforms MIPROv2, a leading prompt optimizer, by over 10% across multiple LLMs. This advantage holds across different model architectures, suggesting GEPA's reflection-based learning transfers well.
The key difference is interpretability. GRPO and similar methods optimize through gradient signals that don't explain themselves. GEPA's natural language reflections produce human-readable explanations of what's changing and why.
When GEPA makes sense
GEPA works best when you have a working prompt that handles real production traffic. At that stage, the goal shifts from invention to continuous improvement based on actual usage data.
Strong fit for GEPA:
Prompts already deployed in production
Structured output tasks like classification
Systems where success is measurable through evaluations
Scenarios where you need efficiency (limited rollout budget)
Less suited for GEPA:
Brand new prompts with no baseline performance
Tasks where evaluation criteria aren't well-defined
Exploratory prompt design where you're still finding the right approach
The algorithm thrives on clear feedback signals. If your evaluations don't reliably capture what "good" means, GEPA will optimize toward the wrong target.
The role of evaluations in GEPA optimization
GEPA is evaluation-driven. The quality of your evaluations directly determines the quality of your optimized prompts.
For most real-world tasks, a composite evaluation works best. This combines multiple signals—correctness, safety, format compliance, relevance—into a balanced score that prevents overfitting to any single criterion.
Single-metric evaluations risk producing prompts that excel on one dimension while regressing on others. A prompt optimized purely for accuracy might become verbose. One optimized for brevity might lose nuance.
Composite evaluations force GEPA to find prompts that balance competing objectives, which typically produces more robust results in production.
Practical considerations
Before running GEPA optimization, you need two things:
Inputs: Either production logs from real usage or a curated golden dataset that represents the range of inputs your system handles.
Evaluations: Stable, well-calibrated evaluations that reflect what success actually means for your use case. Weak evaluations amplify the wrong behaviors.
The optimization process itself is abstracted away. You don't reason about individual edits. Instead, you focus on whether your evaluations reflect your goals and whether improvements are consistent across runs.
This makes GEPA accessible to teams without deep ML expertise. The complexity lives in the algorithm. Your job is defining what good looks like.
GEPA and the reliability loop
GEPA fits naturally into the broader workflow of building reliable AI systems. After you've instrumented your application, captured production traces, annotated failures, and built automated evaluations, GEPA becomes the tool that closes the loop.
It takes the patterns you've discovered and systematically searches for prompt improvements that address them. Each optimization cycle feeds back into your observability data, revealing new edge cases and opportunities for further refinement.
This continuous improvement process—observe, evaluate, optimize, repeat—is what separates AI products that degrade over time from those that get better with use.
