2026-04-06 07:54 UTC
@MichaelTBacon @alienghic @cwebber The subscription products are moving to multi-model hybrids under the heading of "model routing" and "sub agents" and related schemes. I think primarily motivated by cost although I don't know if they admit it in public. This already exists in the current products in small ways but they'll likely push it a lot farther.
Replies (1)
-
@mirth@mastodon.sdf.org 2026-04-06 07:57
@MichaelTBacon @alienghic @cwebber Re: chips... That post about Azure is quite interesting but not related to AI accelerators. The core workload for these neural nets is almost totally unrelated to what a general purpose CPU does, and for large models the majority of energy consumption is DRAM and interconnect (e.g. not the arithmetic). So the answer to how much more customized they can get is "a lot" if you frame the problem as how to avoid interconnect and DRAM usage. Wafer scale etc.