AI & Compute
Small models, distillation and the case against always using the biggest
Capability per parameter has improved faster than parameter counts have grown, which changes the economics of what to deploy.
Topic
cost
Every Muskeology article tagged cost — 2 pieces across 1 section.
AI & Compute
How models are actually deployed
Serving a model in production is a systems problem — batching, caching, routing and cost — and it looks nothing like running one on a laptop.