Kubernetes has the Wrong Abstractions for LLM Inference
Kubernetes is a strong deployment scheduler, but LLM inference needs continuous control of live state, KV cache, HBM, compute, bandwidth, and request placement below the pod boundary.
Read post →