SEED-X: Multimodal Models in Real World

1. The authors claim their results in Figure 1 and Figure 5 come from a "unified and versatile foundation model," but it seems these results are from different instruction-tuned models (SEED-X-Edit, SEED-X-PPT, etc.). This could mislead readers into thinking a single model handles all functionalities.
2. The authors only compare their approach with a few related works.
3. Results are only provided for MMB, SEED-Bench-2, and MME. Other benchmarks like VQA, MM-Vet, MMMUv, and MathVista are missing.
4. The model is pre-trained on LLaMA2, which is outdated. This raises concerns about the results being state-of-the-art, especially compared to newer models like LLaMA 3.2-Vision.

1） Lack of quantitative ablation study, in Sec4.3 ablation study, the authors perform ablation studies on the training of visual de-tokenizer and the pre-training of SEED-X and visualize the results of the ablation study. However, they lack the quantitative ablation study of their framework, e.g.: the image gridding operation which is claimed to support arbitrary size and aspect ratios. This operation contributes to one of their motivations, but lacks necessary quantitative analysis.

2. As shown in Tab.2 the experiment results compared to other MLLM methods on MLLM benchmarks are not competitive.

1. Grammar Mistake: This paper is filled with numerous grammatical mistakes, which severely compromise its quality. Here are some examples.Ep1: The use of singular and plural forms of “work” is in chaos. In Line 81, it has “Some pioneering work” while in Line 99, it owns “none of the previous works”.Ep2: The form of verb has not been used correctly. Such as in Line 209, “which effectively incorporate the aforementioned characteristics for real-world applications ...” should be changed as “which effectively incorporates the aforementioned characteristics for real-world applications ...”

2. Missing Component: I couldn’t find the summary of the authors’ contributions, which usually can be found in the end of the Introduction section. It makes me confused about the authors’ contributions.

3. Unknown Structure: In this paper, the authors only depict the independent modules in the method, while lacking of the comprehensive and total processing pipeline of the method. For example, the authors may provide with an architecture figure about their method and introduce the processing pipeline about image comprehension and generation step by step, which can be better for the readers to understand their method.

4. Extra Experiment: In the ablation study, 1) the authors only provide the visualization results without numerical metrics which could better reflect the comprehensive performance. 2). the authors only conduct the experiments belong to the image generation, while image comprehension is also significant in their settings. 3). the authors only ablate the number of learnable queries, which can not reflect the effectiveness of their proposed modules: visual tokenization and de-tokenization, dynamic resolution image encoding.

5. Insufficient Innovation: I concern about the innovations about this paper from the following perspectives: 1). The two proposed innovations are disconnected from each other, and there is no strong correlation in the paper. 2). The authors propose a grand blueprint, while their innovations look like a bit ordinary, especially considering about their moderate performance compared with GPT.

1. The Dynamic Resolution Image Encoding section is very interesting. However, the current processing way still inherits some drawbacks, for example we still need to concat the features of all larger patches and resized global image, which will inevitably result in the increased sequence length and contain redundant information. It is also important to show more ablation studies here to show its more significance.

2. What is the motivation of the visual de-tokenizer training in the second stage and how will it help? As shown in Figure 3, SEED-X uses the image editing data to further finetune the visual detokenizer, hence the conditional image has some differences with the reconstruction target, for example, the removed dog shown in the example. I partially agree with the claim of Section 3.1 that the given reference could help recover more fine-grained details, but somehow the editing capability which should be fully provided by the large multimodal model actually came from the UNET of SD-XL to some extent, since most of the time we only want the tokenizer to compress the signal rather than modify the signal. Could the authors share any ablation study to see how it will impact the large model capability?

3. During inference, when the model wants to generate a new image I wonder if the previous image in this sequence is also required to send to the de-tokenizer as a condition?

4. In Figure 4, may I ask what is the white token (between IMG and the regressed image features)?

5. The writing of the paper could be improved as some parts are very confusing and high-level, I would recommend to include more details regarding the implementation and the training .

• Novelty. The proposed method looks like a combination of Emu which regresses continuous visual tokens and DreamLLM which uses continuous queries as image generation conditions. However, I am not saying the efforts of such exploration should not be encouraged.
• Writing. In the method part, it is quite unclear about every design's motivation. There is not a single reference in this part, which may lead to confusion on method detail and differences from previous works. The authors should carefully discuss the technical motivation and the literature comparison.
• Evaluation. Current experiments only consist of MMBench and MME. More classical benchmarks like typical VQA including VQAv2 and advanced benchmarks like MM-Vet should be used.