Posts tagged #kv-cache

2 posts

All agent-architecture agent-harness agent-orchestration agent-safety agentic-ai agentic-coding agentic-engineering agentic-memory agentic-systems agents ai ai-agents ai-arbitrage ai-compiler ai-cost-optimization ai-economics ai-engineering ai-reasoning architecture automation cctoggle claude-code clother cloudflare compute-graph context-engineering dns email embodied-ai gonka gpu graph-rag graphify harness hermes-agent inference infrastructure karpathy kimi kimi-k3 knowledge-compilation knowledge-graph knowledge-graphs kv-cache langchain langgraph life-harness llm llm-inference llm-math mcp mixture-of-experts model-routing moonshot-ai multi-agent-systems muxy neurosymbolic-ai notebooklm open-source open-weights openclaude opencode openrouter optimization opus-4.7 orchestration paged-attention plugins productivity providers proxy resend self-improving-ai software-3 terminal tokens tool-calling tooling vibe-graphing vibecoding video vllm youtube

Jul 19, 2026

KV-Cache and PagedAttention: How vLLM Gets 4x More Throughput From the Same GPU

LLM serving runs out of GPU memory long before it runs out of compute. PagedAttention borrows a 1970s operating-system idea — virtual memory paging — to kill the KV-cache fragmentation that wastes up to 80% of your VRAM, and continuous batching keeps the cores busy.

kv-cachepaged-attentionvllm

Jul 19, 2026

LLM Model Routing: How to Cut AI API Spend by 85% Without Losing Quality

Firing your most expensive frontier model at every request is a structural defect. Model routing sends each query to the right model instead — cascade, classifier, and semantic routing, the traps that quietly kill your savings, and why orchestration is the new moat.

llmmodel-routingai-cost-optimization