Shafiq Joty

Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains

SWERank: Software Issue Localization with Code Ranking

LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild

References Improve LLM Alignment in Non-Verifiable Domains

Variation in Verification: Understanding Verification Dynamics in Large Language Models

On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization

Gradually Compacting Large Language Models for Reasoning Like a Boiling Frog

Enabling Tool Use of Reasoning Models Without Verifiable Reward via SFT-RL Loop

WAFER-QA: Evaluating Vulnerabilities of Agentic Workflows with Agent-as-Judge

Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math

Synthesizing Agentic Data for Web Agent Training with Progressive Difficulty Enhancement

MAS-Zero: Designing Multi-Agent Systems with Zero Supervision

Preference Optimization for Reasoning with Pseudo Feedback

CodeXEmbed: A Generalist Embedding Model Family for Multilingual and Multi-task Code Retrieval

Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning

The Emergence of Abstract Thought in Large Language Models Beyond Any Language

Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators

Beyond In-Context Learning: Enhancing Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines

FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction

Direct Judgement Preference Optimization

Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown

Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks

Expanding the Web, Smaller Is Better: A Comprehensive Study in Post-training