FromZero

We build stuff and share it with the community.

That's pretty much it. And yes, "stuff" was the right word to use.

Why "FromZero"?

FromZero is inspired by Re:Zero - Starting Life in Another World. Just like the main character, all of our models begin from zero.

Thankfully, they don't spend their time dying repeatedly or having mental breakdowns.

I mean, training is already painful enough.

Our goal

Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the greatest. But so does everyone else.

We're just here to build weird models, share them with the community, and grow side-by-side with everyone.

Current Models

• Zero-v0.1-150M: Our highest-performing and most recent model, boasting a total of 151.6M parameters and a 2048-token context window.
• Er-Large: A 31M-parameter model trained on 34B tokens, achieving near state-of-the-art performance across multiple benchmarks.
• MrPong: Our SOTA Ping Pong reinforcement learning (RL) agent, trained for over 10M steps against nine different opponents.

Partners

If you like us, then you might also like:

AxiomicLabs | BenchLabs | QyrouLabs | GODELEV

Discord

Join our Discord community: FromZero

YouTube

Visit our YouTube channel at https://www.youtube.com/@fromziro-ia.

If You Want to Join Us

We are always looking for more members. If you'd like to join, open a discussion under this README and tell us why. We'll review your request and either approve or reject it.