FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents
Abstract
FC-SWE improves reinforcement learning for software engineering agents by conditioning recovery attempts on failed patch feedback, boosting resolution rates across multiple attempts.
Repository-level software engineering (SWE) is a challenging long-horizon setting: agents must reason over extended interactions, use tools, and adapt to stateful environments. Recent work trains SWE agents with reinforcement learning methods such as Group Relative Policy Optimization (GRPO), which independently sample multiple trajectories per issue, test the resulting patches, and compare terminal rewards within a fixed group. However, this training setup does not reuse verifier feedback from failed patches as context for subsequent attempts, even though this feedback contains valuable diagnostic information about what went wrong. Training on recovery trajectories is challenging because the preceding outcome determines whether the next trajectory is generated, while the failed execution determines its conditioning context. We introduce FC-SWE, a failure-conditioned RL framework that incorporates recovery attempts into policy training. After a patch fails verification, FC-SWE restores the repository to its original task state and uses the failed patch and verifier feedback as context for a recovery trajectory. FC-SWE adapts GRPO to these chains of complete, multi-turn tool-use trajectories through two mechanisms. Trajectory-local rewards preserve each attempt's verifier outcome, preventing recovery success from rewarding an earlier failed patch. Active-set advantage estimation forms a comparison group from all initial and recovery trajectories actually executed for the same issue, so failed attempts remain in the group while unexecuted attempts are excluded. On all 500 SWE-bench Verified tasks under a verifier-assisted protocol, FC-SWE with Qwen3.5-4B and SWE-agent achieves 41.7% Resolved@1 and 52.8% Resolved@2, compared with 38.9% and 48.5% for GRPO. Although trained with at most two attempts per chain, FC-SWE reaches 70.7% Resolved@11 under an eleven-attempt test-time budget.
Community
Excited to share FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents!
How can we train coding agents to learn from failed attempts?
FC-SWE makes recovery part of reinforcement learning. After a patch fails verification, we restore the repository to its original state and condition a fresh attempt on the original issue, failed patch, and verifier feedback.
Two mechanisms support this training: trajectory-local rewards preserve each attempt’s own outcome, while active-set advantage estimation compares only trajectories actually executed for the same issue.
Using Qwen3.5-4B + SWE-agent on all SWE-bench Verified tasks, averaged over three decoding seeds:
- 52.8% Resolved@2, compared with 48.5% for our GRPO baseline.
- 70.7% Resolved@11, despite training with at most two attempts per chain.
These are verifier-assisted results: the evaluation controller uses benchmark-test feedback between attempts. Resolved@K measures cumulative success within K attempts, not independent-sampling Pass@K or standard single-attempt leaderboard performance.
📄 Paper
💻 Code coming soon. We plan to release the code in a dedicated branch of NVIDIA NeMo RL and will update this page when it is available.
We welcome feedback and discussion from the agent RL and software engineering communities!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- SPROUT: Scaling Agentic Reinforcement Learning for Long-Horizon Software Engineering (2026)
- ExecCritic: Learn to Test, Test to Improve for Coding Agents (2026)
- One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents (2026)
- Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents (2026)
- LatentSift: Policy-State Filtering for Token-Efficient Verification of Software Engineering Agents (2026)
- AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents (2026)
- Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.07898 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper