Elektrine lite

← Feed

@Migueldeicaza@mastodon.social

2026-06-20 23:23 UTC

@matt@toot.cafe @glyph@mastodon.social they quantized it, and you can run various smaller versions. UnslothAI shrunk it to 2-bit quantization. Retains 82% accuracy - can run on a Mac with 256 megs of ram Model: https://huggingface.co/unsloth/GLM-5.2-GGUF Docs: https://unsloth.ai/docs/models/glm-5.2

Replies (2)

  • @matt@toot.cafe @glyph@mastodon.social and hopefully, the next step is to fold in the new research from Apple that can run models from an SSD (that was a big surprise from wwdc this year)

    Open ##5282375

  • @jssfr@zombofant.net 2026-06-21 08:29

    @Migueldeicaza@mastodon.social @glyph@mastodon.social If I'm reading that correctly, that's 256 *gigs*, not *megs* of VRAM. And in which world is that a "consumer grade GPU"? My favourite IT retailer's filter ends at 96 GB, which is still more than any non-server machine I ever owned had in total RAM.

    Open ##5282378