Data Science Wire

BigMoeOnEdge outperforms Strata and colibri on 8-16 GB cards and Mac and you can run any model

Reddit r/LocalLLaMA17h4 min read

Hi! for those who might not know the solution: it takes any MoE GGUF llama.cpp can load, whatever the quant. No repacking, no special format it runs on whatever you have: NVIDIA, AMD, Intel, Mac, a phone, or just the CPU Windows, Linux, macOS, Android, iOS output is the same as llama.cpp, don't cut corners on quality you start it with --auto and it works out the settings for your machine Below is a brief overview of the benchmark: 8 GB card, Qwen3.6-35B: I get about 35 tok/s. colibri around 23. Strata wouldn't start at 16k context 12 GB card, same model: about 45 for bmoe, about 30 for Strata

Read the full story at Reddit r/LocalLLaMA →

More in AI