Is Your Video Language Model a Reliable Judge?

Thanks for addressing all my concerns. The detailed explanations and additional analyses provided valuable insights. Based on this, I raise my score to 8. However, I feel the paper writing still needs improvement, I hope the authors can address this in future versions.

Additional Experiments and Insights

To further strengthen our findings, we conducted additional experiments to explore the relationship between a model's understanding ability and its reliability as a judge. These results including figures can be found in last section in Appendix.

1. Relationship Between Model Performance and Reliability as a Judge

We compared several judge models to see if higher performance correlates with greater reliability in evaluation tasks:

[1] Results from the CVRR-ES benchmark paper (Zhang et al., 2024).

The data shows that GPT-4o, with the highest understanding ability, also has the highest reliability score as a judge. This suggests that a model's understanding ability contributes to its reliability in as judge for evaluation.

2. Effect of Fine-Tuning on Underperforming Models

We fine-tuned the underperforming model Video-LLAVA using QLoRA on the CVRR-ES dataset to see if enhancing its understanding ability would improve its reliability as a judge.

Ratings Distribution:

Average Reliability Scores (Kappa Coefficient):

Despite fine-tuning, Video-LLAVA's rating distribution remained skewed towards higher ratings, and its reliability as a judge improved only slightly. The agreement scores with the benchmark did not approach those of GPT-4o.

Conclusion:

These findings indicate that simply improving a model's understanding ability is not sufficient to enhance its reliability as a judge. A reliable judge must possess both strong comprehension skills and specific capabilities in assessment and critical analysis. In our new submission, we discuss speculative methods to improve VLMs as judges, such as incorporating specialized training focused on evaluation skills and utilizing robust training to reduce biases and hallucinations.

We believe these additional experiments provide valuable insights and further address your concerns. Thank you for your constructive feedback, which has helped us improve our work.

Questions:

1. Processing by the Advanced Judge

Yes, we have considered the processing mechanism of the advanced judge:

• Processing Method: The advanced judge takes both original video input and VLM reviews as inputs. It analyzes the video-question-answer pair independently and also considers the assessments provided by the initial VLM judges. See samples in Appendix.
• Final Review Composition: The final review incorporates the advanced judge's own evaluation of the content and a critical analysis of the VLM assessments.
• Rationale: By leveraging its superior understanding ability, the advanced judge can mitigate the influence of less reliable assessments and enhance the overall evaluation quality.

2. Real-Time Selection of Reliable Judges Without Pre-Computed Scores

This is an insightful question. We acknowledge that relying on pre-computed scores and reference responses is not always practical. Some future research

• Possible Alternative Methods:
  • Consensus-Based Filtering: Identifying outlier judgments by measuring the deviation of each judge's assessment from the median or mean of all assessments.
  • Confidence Scores: Utilizing the VLMs' own confidence estimates in their evaluations to weigh their contributions.
  • Anomaly Detection Algorithms: Applying statistical models to detect and exclude anomalous judgments without the need for reference responses.

3. VLMs' High Scores Due to Misunderstanding vs. Bias

See previous reply in 3. Expanding on "Hallucination" as an Explanation

4. Optimality of Weighted Cohen's Kappa for Selecting Reliable Judges

We agree that the choice of metric is crucial:

• Justification for Kappa: We initially selected Weighted Cohen's Kappa due to its ability to measure agreement while accounting for chance agreement and the degree of disagreement.
• Limitations Acknowledged: We recognize that Kappa may not fully capture the nuances of judge reliability, especially in complex, open-ended tasks. We could to use other metrics such as Krippendorff's Alpha.

5. Rationale for Metric Choice and Collective Thought Implementation

Here is a clearer rationale:

• Metric Choice:
  • Rationale: Weighted Cohen's Kappa was chosen because it accounts for partial agreement and is suitable for ordinal ratings.
  • Application: It allows us to quantify the degree of agreement between judges and the benchmark, aiding in the selection of more reliable judges.
• Collective Thought Implementation:
  • Design Explanation: Our collective thought approach involves aggregating assessments from multiple VLMs to enhance evaluation reliability. We added some samples in Appendix.
  • Justification: The implementation is based on principles of collective thoughts, aiming to leverage diverse perspectives for better outcomes.

Conclusion

We are grateful for your constructive feedback, which has significantly helped us improve our paper!

We sincerely appreciate your thorough review and valuable feedback on our paper. We have revised the submission. Below, we provide detailed responses to each of your points.

Weaknesses:

1. Typographical Errors and Inconsistencies in VLM Names

Thank you for pointing out these errors. We have revised the manuscript and corrected typographical errors and inconsistencies in the naming of VLMs.

2. Convincing Conclusion on VLMs' Reliability and Potential Solutions

We appreciate this insight and agree that our conclusion could be strengthened. In the revised manuscript, we have:

• Added Additional Experiments: We conducted further experiments to investigate the relationship between a model's understanding ability and its reliability as a judge. Specifically, we fine-tuned an underperforming model, Video-LLVA, on the CVRR-ES dataset to enhance its understanding ability and evaluated its reliability as a judge. The details can be found in discussions section in paper.
• Presented New Findings: Despite the improved performance of Video-LLAVA after fine-tuning, the model's reliability as a judge showed only a marginal increase. The rating distribution remained skewed toward higher scores, and agreement with the benchmark (measured by the Kappa coefficient) improved minimally.
• Provided Insights: These results suggest that simply enhancing a model's understanding ability is insufficient to make it a reliable judge. Effective evaluation requires both strong comprehension and specific capabilities in critical assessment and reasoning.
• Proposed Potential Solutions: We discuss speculative methods to improve VLMs as judges, such as incorporating specialized training focused on evaluation skills and utilizing robust training to reduce biases and hallucinations.

3. Expanding on "Hallucination" as an Explanation

We appreciate your suggestion to delve deeper into the factors contributing to the unreliable judgments by VLMs.

• Inability to Fully Understand Complex Content: VLMs may struggle with comprehending video content that involves nuanced social cues, emotional contexts, or abstract concepts. This limitation leads to superficial understanding and inaccurate evaluations.
• Lack of Critical Reasoning Capabilities: Some VLMs lack the ability to critically analyze and assess the content they process. Unlike humans, they may not question the validity or relevance of certain information.
• Inherent Biases in Training Data: The training data for VLMs may contain biases that skew the evaluations. For example, if the data overrepresent positive feedback, the model might be predisposed to assign higher ratings.

4. Thorough Explanation of VLMs' Unreliability

• Failure to Fully Understand Complex Content:
  • Limited Comprehension: VLMs judges struggle with videos understanding. Without a deep understanding, their evaluations can be superficial or inaccurate.
  • Evidence from Experiments: We observed instances where VLMs misinterpreted the main events or missed critical details in the videos, leading to incorrect assessments. These examples can be found in Appendix.
  • Uniformly High Ratings: Our analysis shows that VLMs disproportionately assign higher scores, indicating a tendency to overrate responses regardless of their actual quality. This pattern suggests an inability to discern nuances in performance.
• Quantitative Evidence Supporting Unreliability:
  • Low Agreement Scores: The Weighted Cohen's Kappa scores between VLM judges and the benchmark evaluations are significantly low, indicating poor agreement and reliability. For instance, models like Video-LLAVA and LLAMA-VID have Kappa scores close to zero.
  • Rating Distribution Analysis: We included charts showing that VLMs' ratings are skewed towards higher scores, while more reliable methods like GPT-4o and the agent debate exhibit a more balanced distribution.

Thank you for your constructive feedback! We appreciate your recognition of the importance of our work. We agree that providing more detail on our methodology will enhance the clarity and transparency of our study. We also update most charts in our paper for better visualization!

• Expanded Explanation of Model Reliability Determination:
  • Reliability Metrics: We have elaborated on how we used the Weighted Cohen's Kappa coefficient to assess the reliability of each Video Language Model (VLM) as a judge. This metric measures the agreement between each VLM's evaluations and the baseline established by the LLM agent debate method.
  • Calculation Details: Additional information on how the kappa scores were calculated, including formulas and interpretation guidelines can be found in the our Methodology.
• Detailed Description of Judgment Aggregation:
  • Aggregation Process: We have provided a step-by-step explanation of how judgments from multiple VLMs were aggregated. As shown in figure 2, this includes how the advanced model (GPT-4o) processes the original video input along with the individual VLM reviews to produce a final assessment. We also added some samples in Appendix.
  • Collective Thought Method: See the samples we added in Appendix.
• Specifics on Metrics and Statistical Techniques:
  • Statistics: We have listed all the metrics and statistical techniques employed in our analysis, such as agreement scores, rating distributions. These results can be found in figure 4.
  • Transparency in Data Presentation: We added more detailed tables, charts, and graphs illustrating the results in the main paper and the Appendix. These visuals are accompanied by explanations to aid understanding.

By providing these additional details, we aim to make our methodology more transparent and accessible to readers. Thank you again for your valuable feedback!

Thank you for your valuable feedback. We have revised the submission and addressed your concerns as follows:

1. Structure and Clarity

We have reorganized the paper for better clarity:

• Rewritten Sections: We have changed the title for each subsections in the section of methodology to make it more clear. We also have a brief description of layout at the beginning of each section.
• Improved Writing: We have revised the writing throughout the paper to enhance readability and coherence. We also updated most of the images for better visualization.

2. Use of a Single Dataset

• Dataset Justification: We researched multiple VLM benchmarks before made the selection. We chose the CVRR-ES dataset because it is open-ended and covers multiple real-world scenarios like social context, which is not addressed by commonly used, closed-ended datasets, such as Video-MME.
• Acknowledged Limitations: Due to the constraints of limited computation resources, we have moderated our conclusions and acknowledged that findings may not be universally applicable without further validation.
• Future Work: We plan to explore additional datasets in future research to generalize our results.

3. Data Presentation

• Improved Visualizations: We have updated our figures to include diverse chart types (e.g., bar charts, line plots) for better clarity.
• Focused Analysis: Detailed results are now highlighted.
• Enhanced Clarity: We have ensured that all figures effectively illustrate the key findings.

4. Quality of Plots

• High-Quality Figures: Most figures have been regenerated in high-resolution formats with improved design for better visualization.
• Consistent Formatting: We have ensured consistency in style and formatting across all figures.

Additional Comments

We have made significant revisions to enhance the paper's quality, including restructuring, rewriting, and improving visual presentations. We also conducted ablation study to explore how to improve the reliability of VLM judges.

• Dataset Limitations: We found that most existing datasets are closed-ended and not suitable for evaluating open-ended, real-world scenarios.
• Focused Scope: Our chosen dataset aligns with our research objectives, and we have discussed this limitation in the paper.
• Future Research: We aim to include more datasets as they become available to validate and extend our findings.
• Methods Section: The Methodology section has been reorganized with clear subsections for each method, enhancing clarity and readability.

We appreciate your constructive feedback, which has helped us improve our manuscript significantly. We hope that the revised paper addresses your concerns.

Additional Experiments and Insights

To further strengthen our findings, we conducted additional experiments to explore the relationship between a model's understanding ability and its reliability as a judge. These results including the figures that can be found in the last section in Appendix.

1. Relationship Between Model Performance and Reliability as a Judge

We compared several judge models to see if higher performance correlates with greater reliability in evaluation tasks:

[1] Results from the CVRR-ES benchmark paper (Zhang et al., 2024).

The data shows that GPT-4o, with the highest understanding ability, also has the highest reliability score as a judge. This suggests that a model's understanding ability contributes to its reliability in as judge for evaluation.

2. Effect of Fine-Tuning on Underperforming Models

We fine-tuned the underperforming model Video-LLAVA using QLoRA on the CVRR-ES dataset to see if enhancing its understanding ability would improve its reliability as a judge.

Ratings Distribution:

Average Reliability Scores (Kappa Coefficient):

Despite being fine-tuned, Video-LLAVA's rating distribution remained skewed towards higher ratings, and its reliability as a judge was improved only slightly. The agreement scores with the benchmark did not approach those of GPT-4o.

Conclusion:

These findings indicate that simply improving a model's understanding ability is not sufficient to enhance its reliability as a judge. A reliable judge must possess both strong comprehension skills and specific capabilities in assessment and critical analysis. In our new submission, we discuss speculative methods to improve VLMs as judges, such as incorporating specialized training focused on evaluation skills and utilizing robust training to reduce biases and hallucinations.

We believe these additional experiments provide valuable insights and further address your concerns. Thank you for your constructive feedback, which has helped us improve our work.

Thank you for your positive feedback on the improvements we have made to the paper. We understand your concern about drawing conclusions based on a single dataset. To address this, we have extended our experiments to include an additional dataset, the VideoChatGPT dataset (https://huggingface.co/datasets/lmms-lab/VideoChatGPT).

Due to limited time and computational resources, we used half of the data from the generic section of this dataset, totaling 1,000 samples. Running experiments over four candidate models and four judges required significant computational effort, including extensive GPU hours and costs associated with running the GPT-4o API.

Our main results are as follows:

Candidates Received Scores

• Findings:
  • VLMs like Video-LLaVA tend to give higher scores to all candidates, mirroring the pattern observed in the CVRR dataset.
  • GPT-4o and InternVL2, provide more varied scores, reflecting a more nuanced evaluation.
  • LLM (text-only reference-guided judge) assigns lower scores across candidates.

Scores Distribution Given by Judges

• Findings:
  • Video-LLaVA predominantly assigns Score 4, showing a tendency to rate candidates highly regardless of performance**.**
  • GPT-4o and LLM exhibit a more balanced score distribution, suggesting a critical evaluation approach.
  • This distribution aligns with our findings from the CVRR dataset, reinforcing the observation that less capable VLMs tend to overrate candidates.

Weighted Cohen's Kappa Coefficient(%) of Judges on Candidates

• Findings:
  • GPT-4o has the highest average Kappa score (44.42%), indicating moderate agreement with the LLM reference-guided judge.
  • Video-LLaVA has minimal agreement beyond chance, with average Kappa scores close to zero.
  • These results are consistent with those from the CVRR dataset, demonstrating that less capable VLMs are unreliable as judges.

Conclusion:

• The consistency of results across both the CVRR and VideoChatGPT datasets strengthens our conclusions about the limitations of using less capable VLMs as judges.
• Our findings suggest that the observed patterns are not specific to a single dataset but are indicative of a broader issue in the evaluation of VLMs.
• Despite computational constraints, extending our experiments to another dataset enhances the generalizability and robustness of our work.

We hope that these additional experiments address your concern regarding the reliance on a single dataset. Thank you again for your constructive feedback, which has helped us improve the rigor and scope of our paper!

Additional Experiments and Insights

To further strengthen our findings, we conducted additional experiments to explore the relationship between a model's understanding ability and its reliability as a judge. These results including the figures that can be found in the last section in Appendix.

1. Relationship Between Model Performance and Reliability as a Judge

We compared several judge models to see if higher performance correlates with greater reliability in evaluation tasks:

[1] Results from the CVRR-ES benchmark paper ([Zhang et al., 2024] (https://arxiv.org/abs/2405.03690)).

The data shows that GPT-4o, with the highest understanding ability, also has the highest reliability score as a judge. This suggests that a model's understanding ability contributes to its reliability as judge for evaluation.

2. Effect of Fine-Tuning on Underperforming Models

We fine-tuned the underperforming model Video-LLAVA using QLoRA on the CVRR-ES dataset to see if enhancing its understanding ability could improve its reliability as a judge.

Ratings Distribution:

Average Reliability Scores (Kappa Coefficient):

Despite being fine-tuned, Video-LLAVA's rating distribution remained skewed towards higher ratings, and its reliability as a judge was improved only slightly. The agreement scores with the benchmark did not approach those of GPT-4o.

Conclusion:

These findings indicate that simply improving a model's understanding ability is not sufficient to enhance its reliability as a judge. A reliable judge must possess both strong comprehension skills and specific capabilities in assessment and critical analysis. In our new submission, we discuss speculative methods to improve VLMs as judges, such as incorporating specialized training focused on evaluation skills and utilizing robust training to reduce biases and hallucinations.

We believe these additional experiments provide valuable insights and further address your concerns. Thank you for your constructive feedback, which has helped us improve our work.

Thank you for your thoughtful review. Below, we address your concerns point by point. We also have updated most of the charts in our paper for better visualization.

1. Recent Benchmarks for Evaluating VLMs

Our research prioritized selecting a dataset suitable for assessing nuanced, open-ended reasoning abilities. While recent benchmarks like Video-MME ([Wu et al., 2024] (https://arxiv.org/abs/2405.21075)) offer insights through short-answer, multiple-choice questions, they are less suited for open-ended evaluation. Similarly, HourVideo ([Zhang et al., 2024] (https://arxiv.org/abs/2411.04998)) focuses exclusively on long video evaluation, limiting its applicability.

The CVRR-ES dataset was chosen for its emphasis on open-ended questions across diverse real-world scenarios, such as social context, emotional understanding, and complex visual reasoning. It challenges VLMs to generate contextually rich responses, making it a robust benchmark for comprehensive evaluation.

2. Need for an Evaluation Benchmark/System

Our work emphasizes improving VLM reliability through:

• A Structured Evaluation Framework: Assessing VLMs across 11 diverse dimensions to highlight strengths and weaknesses.
• Identifying Methodological Gaps: Addressing limitations in collective intelligence methods that include unreliable models.
• Enhancing Judge Reliability: Exploring key factors such as understanding and critical reasoning to improve VLM evaluation methods.

We view our contributions as a step toward standardized evaluation systems, providing insights that can guide future benchmark development.

3. Contribution to Evaluating VLMs

We expanded our analysis to detail VLMs' unreliability, identifying key issues:

• Limited Understanding Ability: VLM judges have misinterpreted the events in videos and severe hallucinations (examples in Appendix).
• Lack of Critical Review Skills: VLM judges tend to assign uniformly high ratings, lacking the reasoning to assess response quality critically.
• Empirical Evidence of Unreliability: Low Weighted Cohen’s Kappa scores and skewed rating distributions highlight poor reliability compared to human evaluators.
• Contributing Factors: Limited training diversity and inherited biases hinder VLMs' ability to generalize and assess accurately.

Key Contributions:

• Systematic identification of the limitations of VLMs as judges.
• Empirical evidences supporting the need for improved evaluation methods.
• Guidance for enhancing VLMs' evaluation capabilities.
• Proposals for robust evaluation frameworks for complex video understanding tasks.

We hope this detailed response clarifies our contributions and addresses your concerns effectively.

Thank you for your time! Please feel free to reach out if you need any further clarification to help increase your score.

Meanwhile, we have extended our experiments to include an additional dataset, the VideoChatGPT dataset (https://huggingface.co/datasets/lmms-lab/VideoChatGPT).

Due to limited time and computational resources, we used half of the data from the generic section of this dataset, totaling 1,000 samples. Running experiments over four candidate models and four judges required significant computational effort, including extensive GPU hours and costs associated with running the GPT-4o API.

Our main results are as follows:

Candidates Received Scores

• Findings:
  • VLMs like Video-LLaVA tend to give higher scores to all candidates, mirroring the pattern observed in the CVRR dataset.
  • GPT-4o and InternVL2, provide more varied scores, reflecting a more nuanced evaluation.
  • LLM (text-only reference-guided judge) assigns lower scores across candidates.

Scores Distribution Given by Judges

• Findings:
  • Video-LLaVA predominantly assigns Score 4, showing a tendency to rate candidates highly regardless of performance**.**
  • GPT-4o and LLM exhibit a more balanced score distribution, suggesting a critical evaluation approach.
  • This distribution aligns with our findings from the CVRR dataset, reinforcing the observation that less capable VLMs tend to overrate candidates.

Weighted Cohen's Kappa Coefficient(%) of Judges on Candidates

• Findings:
  • GPT-4o has the highest average Kappa score (44.42%), indicating moderate agreement with the LLM reference-guided judge.
  • Video-LLaVA has minimal agreement beyond chance, with average Kappa scores close to zero.
  • These results are consistent with those from the CVRR dataset, demonstrating that less capable VLMs are unreliable as judges.

Conclusion:

• The consistency of results across both the CVRR and VideoChatGPT datasets strengthens our conclusions about the limitations of using less capable VLMs as judges.
• Our findings suggest that the observed patterns are not specific to a single dataset but are indicative of a broader issue in the evaluation of VLMs.
• Despite computational constraints, extending our experiments to another dataset enhances the generalizability and robustness of our work.

We hope that these additional experiments provide more insights! Thank you again for your constructive feedback, which has helped us improve the rigor and scope of our paper!