A Super-Aligned Driving Generalist Is Your Cockpit

1、The paper shows limited novelty, like RAG, are pre-existing approaches. 2、The "Expert Knowledge Fusion" section isn’t clearly explained. Adding pseudocode or a flowchart could make it easier to follow. 3、The paper lacks ablation studies to verify the effectiveness of individual modules, such as physiological indicators and expert knowledge fusion.

Overall, I regret to say that this submission appears of low quality, with numerous errors suggesting it was submitted without thorough proofreading—a potential disservice to the reviewers.

1. There are numerous typographical and formatting errors, including typos, incorrect notations, capitalization issues, and incomplete sentences. Examples include:

   • L050: "technologies, s possess"
   • L053: '"with s,"'
   • L208: "supplement the captain"
   • L261, L265: "the language spaceemface ∈ C × L.", "emrgb ∈ C × L, < RGB bos > emrgb < RGB cos >."
   • L271, L277: "emfront ∈ C × L,"
   • L288: "framework ash shown in Fig. 3"
   • L307: "The Relationship Between Physiological State and Emotion: Classical"
   • L315-L317: "other tasks, including: The Relationship Between Physiological State and Behavior…" (repeated thrice)

2. The proposed method lacks novelty, as it is essentially a multimodal LLM with RAG, without any specific design tailored for the target task. Additionally, key methodological details, such as training strategies, specific model architectures, and hyperparameters, are missing.

3. Experimental analysis is limited. In-depth experimentation and analysis are needed to substantiate the claimed benefits of using a multimodal approach.

4. The dataset setup is unclear. Since the captions are generated by open-source VLMs, please clarify the measures taken to ensure their quality.

5. The related work citations do not consistently support the claims made. For instance, L308 references "Classical studies in psychophysiology (e.g., James-Lange, Schachter-Singer)…” without sufficient context.

6. The appendix section is empty. Please remove the placeholder text: "You may include other additional sections here."

7. Finally, as the dataset includes human subjects, please provide an ethics statement to address concerns regarding its use.

1. How are the various inputs (e.g., visual, physiological) integrated to influence real-time driving decisions?
2. Could the paper delve deeper into how user interactions are managed, especially in complex scenarios? Are there any limitations to the system’s ability to interpret nuanced or less common user behaviors?
3. There are a few errors: for example, the purpose of "s" in lines 50 and 53 is unclear, “Out View Caption” is duplicated in Figure 2, and “Accurate labele” contains a spelling error.

Weakness:

• The authors should provide more comprehensive information about the model architecture, including specifics such as the choice of LLM with its size and so on.

• In Figure 3, a "Pre-trained Video Encoder" is depicted, whereas Section 4.2 mentions the use of an "ImageNet pre-trained ResNet18." Are these referring to the same component? Additionally, how does this encoder handle other modalities? Lastly, how many tokens does the encoder output? Providing more detailed explanations would enhance understanding.

• In Section 4.1, the author introduces specific start and end symbols to denote different modalities. Are these symbols newly added special tokens to the LLM's vocabulary? If so, how are these tokens initialized? Since the LLM remains frozen and is not further trained, how does the pretrained model recognize these new tokens?

• In Section 5.2, the maximum sentence length is set to 64. How was this value determined? Since text sentences are processed by a tokenizer, why not base this parameter on the number of tokens instead? Were any experiments conducted to evaluate the impact of this choice on performance or the training and inference computational budget?

• The sequence of tables and figures should be adjusted for consistency. For instance, Table 2 is only mentioned in Section 5.5, while Tables 3 and 4 are referenced earlier in the document before Table 2.

• The manuscript requires improved writing quality, as numerous typographical errors are present. For example, on line 414, "model" should be corrected to "figure," and on line 261, a space is needed between the text and the equation.

The manuscript currently contains several typographical and writing errors, as well as some missing details, which is not ready for submission. I believe it would benefit from further revisions to address these issues and ensure it meets the standards required for submission to ICLR.

The paper needs improvement in writing. There are mistakes and citation errors. See at the end of this message.

The main issue is the novelty. It seems to combine multiple models to improve intelligent driving. This contribution is not good enough for a conference like ICLR.

In (Section 4.3), the authors mentioned that the model relies on visual tokenization of physiological data, such as heart rate and blood oxygen levels, to infer emotional or behavioural states. However, this approach assumes direct correlations with emotions, potentially leading to inaccuracies. While the authors have cited studies in psychophysiology suggesting links between physiological signals and emotions, the real-world application requires greater nuance. Authors should discuss impact on signals due to factors like individual baselines, environmental conditions, and physical activity.

The model’s use of a pre-trained ResNet18 for tokenizing RGB, NIR, and depth inputs may lack the capacity to capture the complex nuances needed for an intelligent cockpit system. To address this, the authors should conduct ablation studies comparing ResNet18 with advanced models like ResNet50, EfficientNet, ViT, and Swin Transformer to assess improvements in accuracy and robustness. Additionally, the current concatenation-based fusion strategy may underutilize the complementary data from multi-modal inputs. Testing different fusion techniques, such as attention-based and cross-attention methods, could identify more effective integration approaches. Further analysis of each modality’s impact would clarify the significance of RGB, NIR, and depth data, while transformer-based models could improve temporal understanding for tasks like fatigue tracking.

The reliance on a language model for contextual understanding may oversimplify dynamic driving scenarios, missing essential non-verbal cues for real-time safety. Ablation studies could address this by comparing language-only input to multi-modal input (e.g., visual, physiological, behavioural data) to assess non-verbal contributions to accuracy in safety-critical tasks. Testing each modality individually would highlight their impact while comparing the language model with and without RAG would clarify RAG’s role in context accuracy.

Writing: Line 37: He -> it Line 50: s possess? Line 53: with s? Line 61: reference a? Line 66: repeat of Sima et al. Line 188: Beachmarking -> Benchmarking

• The dataset construction part is extremely lacking in details, including data curation, GPT4 labeling, etc. The paper states in many places that the details are in supplementary materials, but the supplementary materials have not been submitted. Besides, the contribution of "An intelligent driving cockpit super alignment evaluation protocol involving generalization ability for different needs was established" cannot be well-established in the paper.
• The qualitative results are very limited. Only in Fig. 4, some conversations are provided, and from this figure, we cannot know the full ability of the model.
• The writing of the paper was very hasty. many sentences are not clear and typos are everywhere, e.g., "serves as the interface for human interaction with s" in L053, and "Tokenizing Multi-Model" in L257.