Squeal Studio builds Russian-first language models small enough to run on the device already in your pocket — no API key, no data center, no bill that scales with usage.
Most labs answer every problem with more parameters. We think that solves "can it do everything" while ignoring "can it run where you need it."
drakon_28m uses a 24,001-token custom BPE vocabulary built around Cyrillic — Russian first, the language most "multilingual" models treat as a rounding error.
A hybrid of depthwise conv, MinGRU and sparse attention keeps inference cheap enough for phones, Raspberry Pis, and browser tabs — not just racks in a data center.
Raw pretraining on ~3B tokens of Russian text — education corpora, FineWeb2, Wikipedia, Habr, news and more — no instruction tuning.
Fine-tuned from base on ~59K Russian instruction examples to follow instructions and hold a conversation. Experimental — expect mistakes.
The in-browser demo no longer lives on this page. Chat with drakon_28m on its own home instead.