@Migueldeicaza@mastodon.social
2026-06-20 23:23 UTC
@matt@toot.cafe @glyph@mastodon.social they quantized it, and you can run various smaller versions.
UnslothAI shrunk it to 2-bit quantization. Retains 82% accuracy - can run on a Mac with 256 megs of ram
Model: https://huggingface.co/unsloth/GLM-5.2-GGUF
Docs: https://unsloth.ai/docs/models/glm-5.2
Replies (2)
-
@Migueldeicaza@mastodon.social 2026-06-20 23:24
@matt@toot.cafe @glyph@mastodon.social and hopefully, the next step is to fold in the new research from Apple that can run models from an SSD (that was a big surprise from wwdc this year)
-
@jssfr@zombofant.net 2026-06-21 08:29
@Migueldeicaza@mastodon.social @glyph@mastodon.social If I'm reading that correctly, that's 256 *gigs*, not *megs* of VRAM. And in which world is that a "consumer grade GPU"? My favourite IT retailer's filter ends at 96 GB, which is still more than any non-server machine I ever owned had in total RAM.