Ā·
AI & ML interests
None yet
Recent Activity
reacted to savoji's post with š„ about 19 hours ago One Qwen3-VL-8B checkpoint improves all three streaming spatial benchmarks at once: OST-Bench 50.52 to 63.93 (+13.41), UCS-Bench +7.44, OVO-S-Bench +4.91. The change is in the supervision, with no new memory module or annotation.
The ChronoReplay paper calls the failure mode interface lock-in: post-train on one answer format and that readout improves while others stall. A categorical-only control hits 58.03 on OVO-S but drops OST from 50.52 to 47.48.
š The recipe
ā¢ Keep targets in native formats (option, metric value with units, timestamp)
ā¢ Replay episodes in source order, each target attached to its causal prefix
ā¢ Loss mass: 2/3 on 33,180 single-turn examples, 1/3 on 6,381 episodes (alpha = 0.5)
š The numbers
ā¢ Three-benchmark mean 56.86, against 50.74 (categorical-only) and 50.76 (trajectory-only)
ā¢ Chronological order beats a shifted-order control by about 2.41 OST points on two seeds; the 13.41 belongs to the whole recipe
ā¢ Adaptive frame selection adds +4.09 on OVO-S (57.98) with 80.50 frames instead of 128
ā ļø Caveats
ā¢ One backbone, indoor scenes only; the single-family comparison is confounded by data coverage
ā¢ The repo has a CPU smoke test and a --dry-run training command, but no license and no fine-tuned weights
https://huggingface.co/papers/2610.34283
https://arxiv.org/abs/2610.34283
https://github.com/yihaohu0118/ChronoReplay
If you fine-tune VLMs for embodied work, do you score categorical, metric and temporal readouts together?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/8d2fb1bf832d
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/chronoreplay-lifts-qwen3-vl-8b-by
ā¢ The thread on X: https://x.com/saavoji/status/2116229682844877156
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#chronoreplay Organizations
