🏗️ Building on HF

Sergio Paniego PRO

sergiopaniego

232 202 127

https://sergiopaniego.github.io/

AI & ML interests

None yet

Recent Activity

updated a Space 2 minutes ago

sergiopaniego/huggingface-static-ab21bb

published a Space 2 minutes ago

sergiopaniego/huggingface-static-ab21bb

updated a bucket 2 minutes ago

sergiopaniego/huggingface-static-ab21bb-bucket

View all activity

Organizations

buckets 104

super interesting new paper from Microsoft "Agent Lightning v1.0: Towards Harnessed Agentic RL" by Zhiyuan He et al.

same idea we've seen already several times: you train the agent inside the real harness it ships with, instead of a reimplementation of it

now that recipe has a name → harnessed agentic RL

paper: huggingface.co/papers/2608.17528

the tricky bit they nail down: one rollout is not one training sample

the harness calls the model many times, so a single episode → a variable number of (prompt, response) rows

you don't even know the batch size until the episode finishes running

its real contribution is being first to systematically map the four problems that fall out of that:

> retokenization + sample merging
> advantage calculation over a variable sample count
> loss normalization at the rollout level, not per sample
> backend scheduling when the batch size is dynamic

and it actually works → plain RL inside the real harness, no reimplementation

Qwen3.5-9B on SWE-bench Verified 41.8 → 56.4 (+14.6), with only ~6k examples

the whole thing is ~3,500 lines, any harness, self-hosted k8s

from our side, we've shared some materials on the same line you may want to check out :)

> Agentic RL: Token-In, Token-Out Done Right: https://huggingface.co/blog/huggingface/tito
> a full worked example, opencode owning its loop trained with GRPO: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
> Harness, Scaffold, and the AI Agent Terms Worth Getting Right: https://huggingface.co/blog/agent-glossary

on a similar line:

https://x.com/SergioPaniego/status/2062911580564496576