Gigatoken: Fast Tokenization Without Changing the Tokens
A source-based look at Gigatoken's claimed tokenizer speedup, its SIMD and cache design, the benchmark conditions, and its current limits.
Back to Blog
CHANNEL / ACTIVE
#infrastructure
KV cache and PagedAttention: how vLLM uses GPU memory more efficiently
PagedAttention manages the KV cache in fixed-size blocks, reducing allocation waste and enabling larger batches. What that changes, where continuous batching helps, and what the original vLLM results actually show.
LLM model routing: choosing a model for each request
Model routing selects among models according to task, quality, budget, latency, availability, and data-residency requirements. A practical guide to routing policies, evaluation, cache locality, and their tradeoffs.