VideoMLA’s low‑rank latent KV cache cuts KV‑cache demand by roughly 90 % and LongLive‑RAG’s retrieval‑augmented memory helps mitigate the temporal drift introduced by sliding‑window attention. The KV‑cache reduction comes from replacing per‑head keys and values with a shared low‑rank latent, shaving 92.7 % off per‑token cache size; separately, the retrieval module lets the generator attend to non‑local history instead of a stale recent window, helping prevent error accumulation across thousands of frames.
Before these advances, long‑horizon video diffusion relied on a fixed‑size sliding window that constantly overwrites the KV cache. As the window slides, any appearance error that slips in becomes permanent, and because the model can only look at the most recent tokens, identity drift compounds unchecked. Researchers tried shuffling token order or tweaking positional encodings, but the fundamental bottleneck—a growing KV cache that forces either truncation or out‑of‑memory failures—remained.
VideoMLA achieves a 92.7 % reduction in per‑token KV cache memory while preserving compatibility with standard chunk‑causal generation. The paper shows that “VideoMLA reduces per-token KV cache memory by 92.7 % while preserving compatibility with standard chunk‑causal generation” and Figure 4 confirms that subject identity, scene structure, and visual fidelity stay intact over 30‑second rollouts despite the compact latent cache [1].
LongLive‑RAG adds a lightweight retrieval step that draws from the entire self‑generated latent history, so the generator can condition on truly relevant frames instead of a narrow window. The authors note that “this lightweight retrieval step adds only a small overhead relative to generation and lets the generator condition on non‑local context instead of only the recent window,” and experiments across multiple AR backbones report “improved long‑video quality and the best average VBench‑Long rank” [2].
Both methods leave open questions about scaling to truly minute‑scale generation without additional latency. Retrieval introduces a query‑embedding cost and a memory‑search latency that can become noticeable as the history grows, a limitation the authors acknowledge but do not quantify. VideoMLA’s low‑rank cache has only been validated up to 30‑second rollouts; whether the same rank budget suffices for several‑minute sequences remains speculative.
If these techniques hold, the default architecture for autoregressive video diffusion should drop the sliding‑window KV cache in favor of a shared low‑rank latent cache plus a history‑retrieval module, enabling minute‑scale synthesis on a single high‑end GPU without exceeding memory limits.
References
For further actions, you may consider blocking this person and/or reporting abuse
