In partnership with Bench Labs

What is "FWKV"???
FWKV (call it Feed-forward Weighted Key Value, or Floored Weighted Key Value, its up to you) is a efficient language model architecture proposed on May 13, 2026 by FlameF0X.

It started as a fun side project inspired by RWKV. Then I (We, FlameF0X) realised that for a 50 million parameter language model is fast, like, faster than any Transformers base Dense or MoE 50m model without needing a costume kernel to make it run faster.

Now, this architecture was not intended to be similar with RWKV intentionally, it was accidentally because of the idea of keeping it simple. That lead to its RWKV like design. (The name is intentional -ish, since it uses FFNN and WKV and I had no idea how to name it, so FWKV was the name)

Why should I use FWKV?
You should not use it, yet, since its still in its early stages and its very unstable. But its fast.

About this organisation
We do not self proclaim as a lab as the rest of the small or "inovative" ai/ml organisation. This is a small side project on the idea to make language models very simple while trying to achive RWKV like language modeling quality.

Uh, I made it so I can keep track of this project much more easier.

Models

How do we name the models?
Very simple actually!
FWKV-[model size or type or something uniquie]
They must stay simple so you can search easily.

Models:

• Language models:
  • FWKV-100M (being planned to enter in development in 2026-2027)
  • FWKV-Translation (being planned to enter in development in late 2026)
  • FWKV-ROSA
  • FWKV-50M
  • FWKV-29M
  • FKWV-Tinystories
• Vision models:
  • FWKV-Image
  • FWKV-CLIP (NAP - Not A Prioriy, you will see this very frequently)
• Other modalities:
  • FWKV-TTS (NAP)
  • FWKV-Bert (NAP)