07-21

Sayash Kapoor will present his FPO "The Missing Science of AI Evaluation"

Sayash Kapoor will present his FPO "The Missing Science of AI Evaluation"

 

Date: Jul 21, 2026, 12:00 pm – 2:00 pm

Location: 306 Sherrerd Hall

 

Examiners: Arvind Narayanan (Adviser), Peter Henderson, and Brandon Stewart

Readers: Arvind Narayanan, Karthik Narasimhan, and Jessica Hullman (Northwestern Univ)

 

AI evaluations inform critical decisions, from the valuations of trillion-dollar companies to policies on regulating AI. Yet, evaluation methods have failed to keep pace with deployment, creating an evaluation crisis where performance in the lab fails to predict real-world utility. In this talk, Kapoor discusses the evaluations crisis in a high-stakes domain: AI-based science. Across dozens of fields, from medicine to political science, he finds that flawed evaluation practices have led to overoptimistic claims about AI’s accuracy, affecting hundreds of published papers.

 

To address these evaluation failures, Kapoor presents a consensus-based checklist that identifies common pitfalls and consolidates best practices for researchers adopting AI, and a benchmark to foster the development of AI agents that can verify scientific reproducibility. AI evaluation failures affect several other applications. Beyond science, he examines how AI agent benchmarks miss many failure modes, and present HAL, a system to address evaluation shortcomings and conduct large-scale agent evaluations. 

 

The talk examines inference scaling, a recent technique to boost AI capabilities, and show that many claims of improvement fail to hold under realistic conditions. Finally, Kapoor discusses how better AI evaluation can inform policymaking, drawing on work on open foundation models and engagement with state and federal agencies.

 

Why does the evaluation crisis persist? The AI community has poured enormous resources into building evaluations for models, but not into investigating how models impact the world. To address the crisis we need to build a systematic science of AI evaluation to bridge the gap between benchmark performance and real-world impact.