Liquid Dino: A Multi-Task Neural Network towards Autonomous Driving

The authors in this work attempt to classify specific driver behaviors using a complex framework. However, the paper requires substantial improvement to be considered for acceptance. Below are some of the main weaknesses:

1. The related work section needs to be expanded to include relevant studies. In the third paragraph, fusion techniques are discussed, but this seems irrelevant as no data fusion is performed in this study.

2. In Figure 1, the driver’s face is partially obscured by wires, and the eyes are not visible. How can meaningful features be learned with such images?

3. Why are all images combined into a single frame? Wouldn’t using weight-sharing in the encoder allow for a better representation of learning from the images?

4. What role do the external camera images play in the classification task? Does using three external cameras improve the model’s performance, or would a single forward-facing camera suffice?

5. The methodology section does not present a cohesive description of the framework. The parts are divided into unrelated sections, and the motivation for using the CFC module is unclear.

6. What is the rationale for including a CNN backbone after DiNOv2?

7. Table 1 is not discussed in the text, and its purpose is unclear.

8. The authors only report accuracy as the evaluation metric. F1 score and AUC should be included to provide a more comprehensive assessment of the framework's performance.

9. There is no ablation study to support their design choices.

The main weakness is the novelty. The approach is the integration of the existing approaches and looks like a technical paper.

The second weakness is the evaluation of the proposed approach on a single dataset does not justify its suitability. There are many datasets available for driver behaviour analysis. The list of datasets can be found in the AIDE paper (Yang et al., ICCV 2023). At least an evaluation on another one or two datasets would have made this paper stronger.

In the Introduction (Section 1), the term "Liquid" is introduced, suggesting the use of Liquid Neural Networks (LNNs). However, the methodology primarily combines CNNs, DINOv2, and CFCs, with minimal discussion on the specific role or implementation of LNNs. For a more accurate portrayal of the architecture’s functionality, further elaboration on LNN integration would be beneficial. This could be addressed by referencing the study, "Liquid Neural Networks: A Novel Approach to Dynamic Information Processing" (https://ieeexplore.ieee.org/document/10466162), which explores LNNs' capabilities in handling dynamic data.

In the Experiments (Section 4), the model's performance is evaluated across tasks that have varying temporal dynamics. However, there is limited discussion on how the model adapts to these differences across tasks, which is crucial in applications where time-dependent accuracy is essential. Explaining the confusion matrix scores (Figure 3) for each task in detail would increase the explainability of the model.

The Results (Section 5) provide inference times, which indicate the model’s performance in real-time scenarios, but omit details on the computational resources required during training and deployment. This information is critical for understanding the model's feasibility in resource-constrained environments.

In the Discussion (Section 6), the model's performance is evaluated using the AIDE dataset. However, the paper lacks exploration into how well the model generalizes to other environmental conditions, such as different weather patterns or diverse geographic settings. Testing across various datasets or environments could address potential generalization issues.

The diagram (Figure 2) lacks detail in DINOv2 and CFC modules, with unlabelled arrows between them and CNN, making data transformations unclear. The CNN module omits critical parameters (e.g., kernel size, stride), while multi-view inputs are positioned too far from DINOv2. Missing data dimensions hinder understanding of transformations, and inconsistent module details (e.g., CFC as a single block) disrupt coherence. The absence of loss function indicators and missing representation of any feature fusion or attention mechanisms further limit completeness.

Lastly, in Future Work (Section 8), potential model enhancements are mentioned, but there is a lack of specific details on scalability. Addressing how the architecture can expand to include additional tasks would clarify its future applicability.

1. In the introduction section, the authors mentioned that we need to achieve a more advanced architecture to deal with the complex demands of modern driver monitor system. This motivation is not very convincing. The authors should introduce why Liquid Dino can well overcome these complex challenges.

2. The approach is only verified on one dataset. The generalizability of the model is doubtful. Could the authors extend this approach for video based driver distracted behavior recognition dataset, e.g., DriveACT and DAD?

a. Martin, M., Roitberg, A., Haurilet, M., Horne, M., Reiß, S., Voit, M., & Stiefelhagen, R. (2019). Drive&act: A multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2801-2810).

b. Kopuklu, O., Zheng, J., Xu, H., & Rigoll, G. (2021). Driver anomaly detection: A dataset and contrastive learning approach. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 91-100).

3. The performance of Liquid Dino is not very promising compared with DINOv2 baseline. Thereby the conttibution of this proposed new method is doubtful.

4. Lack of implementation details. The authors are suggested to add a separate section to introduce the implementation details.

5. The proposed approach is not novel enough. It seems that the performance gain mainly comes from adding more layers for feature learning after DINOv2 framework.

6. Lack of failure case analysis. The authors are suggested to add some qualitative samples with failure cases to illustrate the limitation of the proposed approach.