#GPU Memory

Tokencake: Multi-Agent KV Cache Scheduling That Cuts vLLM Latency by Half

Beihang/Peking/Alibaba introduce Tokencake, a KV-cache-centric serving framework for multi-agent apps. With …