Haitham Bou Ammar

Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning

Contextual Causal Bayesian Optimisation

Paying Attention to Hybrid Attention: Untangling the Issues with Conversion Methods

SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning

ShortCircuit: AlphaZero-Driven Generative Circuit Design

Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective

Subjective Depth and Timescale Transformers: Learning Where and When to Compute

Mixture of Attentions For Speculative Decoding

Efficient Reinforcement Learning with Large Language Model Priors

Human-inspired Episodic Memory for Infinite Context LLMs

SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks

ShortCircuit: AlphaZero-Driven Generative Circuit Design