KV cache and PagedAttention: how vLLM uses GPU memory more efficiently
PagedAttention manages the KV cache in fixed-size blocks, reducing allocation waste and enabling larger batches. What that changes, where continuous batching helps, and what the original vLLM results actually show.