FromZero

We build stuff and share it with the community.

That's pretty much it. And yes, "stuff" was the right word to use.

Why "FromZero"?

FromZero is inspired by Re:Zero - Starting Life in Another World. Just like the main character, all of our models begin from zero.

Thankfully, they don't spend their time dying repeatedly or having mental breakdowns.

Training is already painful enough.

Our goal

Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the greatest. But so does everyone else.

We're just here to build weird models, share them with the community, and grow side-by-side with everyone.

Future Releases

• Orez - Tiny (1.2M), Small (5M), Medium (12.4M), Large (30M): A family of Small Language Models ranging from 1.2M to 30M parameters. Each of them will be massively overtrained with a minimum of 1000+ tokens per parameter.
• Syn-2.6M: A 2.6M-parameter Tiny Language Model (TLM) trained only on synthetic data (tiny-textbooks, tiny-stories-v2, longpage, and finephrase).
• AskJASP.com: A website where you can test all of our models for free.

Discord

Join our Discord community: FromZero.