Nakshatra: Vendor-Agnostic Distributed Inference on Heterogeneous Consumer Hardware · May 11, 2026
If the paper doesn’t render here, click open pdf above.
by Bishwa Bastola · @fthrvi
· connect wallet to react
A distributed inference engine that splits one large transformer model across mismatched consumer machines (Mac, Linux, AMD, NVIDIA, CPU) by patching llama.cpp to load partial models and ship 16 KB hidden states between layers per token. v0.1 is validated against a single-machine reference. The larger goal is a personal compute realm spanning inference (Nakshatra), coordination (Sthambha), and a long-running agent (Prithvi) — surviving any one device dying.
The code
Read the README
The pulse
