Data Science Wire

Unsloth UD-quants - Qwen 3.8 27b for example - worth using 8-bit or stick with faster 6 bit for coding?

Reddit r/LocalLLaMA3d4 min read

For those using these models for coding in larger projects where things can get complex, do you find yourself using the 8-bit quants if you have enough memory? Or do you stick with UD-Q6_K_XL? The 6-bit is faster, noticeably so on my setup. And I keep seeing people say it's imperceptible. I've been doing tests myself, and well, I can't tell, but maybe that's just because I'm an idiot. That said, can you tell? Have you ever done some tests to see? submitted by /u/Jorlen [link] [comments]

Read the full story at Reddit r/LocalLLaMA

More in AI