Article
Horizon RunMap: An Integrated Architecture for MoE Inference
Zaptrapper
• • 3Lucas Ribeiro PRO
Zaptrapper
AI & ML interests
None yet
Recent Activity
posted an update 3 days ago
I’ve published an article about Horizon RunMap, an MoE inference architecture I’m developing independently to explore more memory-efficient execution.
In my RTX 5070 benchmark, OLMoE-1B-7B-0924-Instruct reached 61.26 tokens/s during decode after the first token, with 5.99 GiB peak VRAM usage, keeping all compact experts on the GPU.
The architecture combines compact expert representations for both storage and execution, custom GPU kernels, and full or partial GPU residency.
The article explains how these pieces work together, with results across three models and an instruction-following evaluation using IFEval.
If you’re working on MoE inference or GPU memory efficiency, I’d welcome your thoughts.
Read the article: https://huggingface.co/blog/Zaptrapper/horizon-runmap-moe-inference published an article 3 days ago
Horizon RunMap: An Integrated Architecture for MoE InferenceOrganizations
None yet
published an article 3 days ago