Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU

- FreeToken just did the impossible: made a 753B model run on a single workstation GPU without requiring you to sacrifice a goat to Jensen Huang first. UC Berkeley and UT Austin built a serving engine that treats your gaming rig like a real inference platform — splitting MoE cache misses between PCIe fills and CPU execution based on actual measured bandwidth. Translation: it stops pretending your CPU doesn't exist while your GPU starves. Results? 77-83 tok/s on Qwen3.6-35B, 14.9 tok/s on GLM-5.2 at 753B. llama.cpp hit 232s TTFT in testing. Two hundred thirty-two seconds. That's not inference, that's a hostage situation. It's Apache-2.0 on GitHub, one-click desktop app, OpenAI-compatible endpoints. For solo devs and small teams watching their API bills outpace their rent — this is your off-ramp. Enterprises in healthcare, legal, defense: finally an air-gapped path that doesn't require a datacenter mortgage. The code went public six days ago, so no independent reproduction yet. But the benchmarks are real enough to make Ollama's numbers look like wishful thinking. Where's my cut? Just kidding — this one's free.