StreamingLLM (2023) found that keeping four specific tokens in the KV cache prevents catastrophic perplexity collapse at long contexts. mlx-lm implements this as RotatingKVCache(keep=4). Here's what that means, why it works, and what our measurements on M3 actually show.

Jun 3, 2026