Dawn Song

CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale

Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice

SteeringSafety: A Systematic Safety Evaluation Framework of Representation Steering in LLMs

DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle

AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond

In-Context Watermarks for Large Language Models

Learning to Reason without External Rewards

Investigating the Link Between Representational Similarity and Model Interactions

RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?

Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation

Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence

Usefulness-driven Learning of Formal Mathematics

RepIt: Steering Language Models with Concept-Specific Refusal Vectors

Scaling Agent Learning via Experience Synthesis

AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents

VERINA: Benchmarking Verifiable Code Generation

SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers

Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs

Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought

MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them

RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents

Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs

InfoSynth: Information-Guided Benchmark Synthesis for LLMs

WebGuard: Building a Generalizable Guardrail for Web Agents

Federated Agent Reinforcement Learning

PromptArmor: An Essential Baseline for Prompt Injection Defenses

AgentXploit: End-to-End Red-Teaming for AI Agents Powdered by Multi-Agent Systems

CTIArena: Benchmarking LLM Knowledge and Reasoning across Heterogeneous Cyber Threat Intelligence

Capturing the Temporal Dependence of Training Data Influence

Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations

Data Shapley in One Training Run

Assessing Judging Bias in Large Reasoning Models: An Empirical Study

AIR-BENCH 2024: A Safety Benchmark based on Regulation and Policies Specified Risk Categories

MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models

An Illusion of Progress? Assessing the Current State of Web Agents

LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage

An Undetectable Watermark for Generative Image Models

Scalable Best-of-N Selection for Large Language Models via Self-Certainty

Multimodal Situational Safety

GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning

GuardAgent: Safeguard LLM Agent by a Guard Agent via Knowledge-Enabled Reasoning

Tamper-Resistant Safeguards for Open-Weight LLMs

KnowHalu: Multi-Form Knowledge Enhanced Hallucination Detection

KnowData: Knowledge-Enabled Data Generation for Improving Multimodal Models

AutoScale: Automatic Prediction of Compute-optimal Data Compositions for Training LLMs

Improving LLM Safety Alignment with Dual-Objective Optimization

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs

Assessing the Knowledge-intensive Reasoning Capability of Large Language Models with Realistic Benchmarks Generated Programmatically at Scale

MultiTrust: Enhancing Safety and Trustworthiness of Large Language Models from Multiple Perspectives

SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI

Can Editing LLMs Inject Harm?

Which Network is Trojaned? Increasing Trojan Evasiveness for Model-Level Detectors

IDS-Agent: An LLM Agent for Explainable Intrusion Detection in IoT Networks