Janas streams a large MoE model from SSD instead of fitting it all in RAM

Janas is an experimental C inference engine that loads the needed experts of a mixture-of-experts model from an NVMe SSD into a RAM cache. Its author reports 24 tokens per second for a 48.4 GB Qwen3-Next model on a 32 GB laptop, without a draft model. That is one hardware-specific result; the project warns of memory pressure and GPU resets.

Janas

Why it matters

Selective expert loading could make larger local models usable on memory-constrained machines.

More context

Watch or try on a nonessential machine. The README documents Linux and Windows x86-64 support; the benchmark uses an Intel Core Ultra 9 185H laptop and fast NVMe storage. Discovery date is not a verified launch date.

Keep in mind

The author calls this an experiment rather than a product; interfaces change frequently and broader hardware reliability is unproven.

Original read