Writing
Inference Lab
Field reports on inference performance, from HTTP buffering and Metal traces to memory limits, benchmark lineage, and vLLM compilation.
- Your agent's clock is not the model's clock 7 min read
- The cluster scaled to zero before its first request 9 min read
- Leave your coding agent for 5 minutes and most of its cache is gone 11 min read
- Your coding agent loses its cache while it waits for you 19 min read
Understanding GPU-Level Bottlenecks in Large Language Model Inference
Peer-reviewed conference paper Springer · 2026
Pinned
More writing