Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models

Strengths:

1. The paper addresses a highly practical and important challenge, enabling video diffusion generation with very large temporal context while preventing drifting. The proposed FramePack method, along with its extensive exploration of design choices and variants, is both novel and insightful.
2. I especially appreciate the discussion on the fundamental dilemma between forgetting and drifting. The paper clearly articulates how existing approaches often trade one for the other, and how the observation of substantial duplication of visual features across nearby frames naturally motivates the efficient packing strategy.
3. The experimental study is exceptionally thorough. It not only covers the key innovations, such as time proximity-based, feature similarity-based, and hybrid packing strategies, endpoint planning, sampling order adjustments, and history discretization, but also rigorously investigates many design variants, including packing schedules, patchification levels, and anti-drifting sampling orders. The method is tested on two popular video diffusion backbones (Wan and Hunyuan), across diverse datasets (VBench, TECO DMLab, MineCraft, UCF-101), with quantitative metrics and human assessments.
4. The paper compares carefully against multiple relevant methods from recent literature, including anchor-frame approaches (StreamingT2V), causal attention (CausVid), Diffusion Forcing, and HistoryGuidance. These comparisons convincingly demonstrate the advantages of the proposed techniques.
5. The proposed framework is highly computationally efficient, making it feasible for personal or academic-scale experimentation. The clear implementation details and open-sourced code will likely have a significant impact on the community, lowering barriers for future work in this important area.
6. The video demos provided in the supplemental materials are of exceptional quality, strongly validating the effectiveness of the approach and highlighting its potential for real-world applications.
7. Despite introducing many intertwined ideas and technical components, the paper is very well organized. All methods are clearly described, justified, and discussed, making this a very solid and convincing contribution.

I did not find any critical weaknesses in this paper. The only minor concern is that the proposed packing and anti-drifting strategies are largely hand-crafted and manually selected. However, this limitation is acknowledged by the authors in Section K, and the effectiveness of their choices is well demonstrated by the video results.

Strengths

• Presents a novel method for effectively reducing temporal redundancy in videos through a binary pyramid approach.
• The paper is well-written, with a clear problem definition, informative figures, and sound formulations.
• Provides solid and comprehensive experimental analysis to support the proposed method.

Weakness

• The practical usage appears limited. The supplementary videos are mainly repetitive or periodic scenes with near-static background. It remains unclear whether the method can handle more dynamic scenes as suggested in L166-170.
• The packing strategy merely based on temporal order and similarity might introduce redundancy into $F$. There is concern that the model may learn to copy $F_{i\leq k}$ by attending only to them, rather than making use of $F_{i>k}$ for small $k$ (e.g. 2~5). Visualization of the activated attention would help confirm that all historical frames are effectively utilized. Furthermore, I think this kind of strategy cannot extract important information from previous frames which are visually different with the current frame.
• The method relies on 3D attention models, which makes it challenging to integrate with approaches that use decoupled spatial and temporal attention. Most of the initial approaches for video diffusion models [1] and a SOTA approach recently exhibited [2] tried to train the video models efficiently using decoupled attention. FramePack's artificial packing may misalign temporal attention in such cases. However, this is likely a minor issue since many models still employ 3D attention.

[1] Blattmann et al., Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models, CVPR 2023. [2] ByteDance Seed, Seedance 1.0: Exploring the Boundaries of Video Generation Models, arXiv 2025

Strengths

1. The idea of compressing frame context based on frame importance (time proximity and feature similarity) is intuitive and elegant, enabling practical scaling to thousands of frames within a fixed context size.
2. The method achieves high efficiency and can run on laptops with limited GPU memory (e.g., 6–8GB), making it appealing for both academia and industry.
3. The method systematically explores a wide range of compression and anti-drifting sampling strategies, with thorough empirical validation that offers valuable insights into their strengths. Weaknesses
4. The use of heuristic-based importance measures (e.g., time proximity or latent similarity) may not generalize well to dynamic scenes, such as fast human actions, rapid camera movements, or sudden scene transitions. Aggressive compression or discretization could discard fine-grained temporal cues, which is not adequately analyzed in the paper.
5. While strategies like endpoint planning or inverted sampling handle repetitive motions well, it remains unclear whether they can support storylines with foreshadowing, twists, or scene-level progression beyond local coherence.

Strengths

1. The FramePack architecture serves as a robust memory by combining both temporal-proximity and feature-level permutations for packing, ensuring reliable context selection. Furthermore, its time-proximity based packing is shown to be independent of context length, allowing fixed-length compression and removing the need for an attention KV cache—resulting in a highly efficient design.

2. Novel inverted anti-drifting: By first predicting the first and last frames and then reconstructing intermediate frames in reverse temporal order, the method effectively mitigates drift and error accumulation. As shown in Table 1, this inverted prediction scheme yields the best performance in reducing long-term prediction errors.

3. History discretization: Replacing continuous frame histories with discrete integer tokens offers a compact, loss-aware representation of past contexts. Supplementary experiments demonstrate that this discretization retains essential information and further prevents error accumulation.

Weaknesses

1. The ablation names in Table 1 are overly complex; using simpler labels such as "vanilla sampling" and "inverted anti-drifting sampling" would make the study easier for readers to follow.

2. Drift is measured only on the first and last 15% of frames, which—while efficient—may miss errors accumulating in the middle of sequences; quantitative drift measurement itself remains a challenging open problem.

3. It is unclear whether history discretization could be combined with the inverted anti-drifting scheme for additional gains, as this potential synergy is not investigated.

4. The abstract claims that the anti-drifting methods also work in single-direction video streaming, but no concrete experiments or results are provided to support this.

5. The current FramePack relies on manually designed feature selection; a dynamic, content-adaptive context compression strategy could enable multi-shot videos with richer narrative variation rather than repetitive motion.

6. Overall, the paper more like an extensive empirical report and would benefit from more direct, high-level conclusions, i.e., clear conclusion which support by the rich ablation results.