Alexander Wettig

I'm a PhD student in the Princeton NLP group, advised by Danqi Chen, and affiliated with Princeton Language and Intelligence (PLI). Previously, I graduated with a BA & MEng from the University of Cambridge, where I worked with Adrian Weller.

I am interested in improving our understanding of language models in terms of their training data, architecture and training objectives.

Publications

(* indicates equal contribution)

• How to Train Long-Context Language Models (Effectively) Tianyu Gao*, Alexander Wettig*, Howard Yen, Danqi Chen Pre-print 2024

• OLMoE: Open Mixture-of-Experts Language Models Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi Pre-print 2024

• Finding Transformer Circuits with Edge Pruning Adithya Bhaskar, Alexander Wettig, Dan Friedman, Danqi Chen NeurIPS 2024 (Spotlight)

• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering John Yang*, Carlos E. Jimenez*, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press NeurIPS 2024

• QuRating: Selecting High-Quality Data for Training Language Models Alexander Wettig, Aatmik Gupta, Saumya Malik, Danqi Chen ICML 2024 (Spotlight)

• Language Models as Science Tutors Alexis Chevalier, Jiayi Geng, Alexander Wettig, Howard Chen, Sebastian Mizera, Toni Annala, Max Jameson Aragon, Arturo Rodríguez Fanlo, Simon Frieder, Simon Machado, Akshara Prabhakar, Ellie Thieu, Jiachen T. Wang, Zirui Wang, Xindi Wu, Mengzhou Xia, Wenhan Jia, Jiatong Yu, Jun-Jie Zhu, Zhiyong Jason Ren, Sanjeev Arora, Danqi Chen ICML 2024

• SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez*, John Yang*, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan ICLR 2024 (Oral)

• Learning Transformer Programs Dan Friedman, Alexander Wettig, Danqi Chen NeurIPS 2023 (Oral)

• Adapting Language Models to Compress Contexts Alexis Chevalier*, Alexander Wettig*, Anirudh Ajith, Danqi Chen EMNLP 2023

• Poisoning Retrieval Corpora by Injecting Adversarial Passages Zexuan Zhong*, Ziqing Huang*, Alexander Wettig, Danqi Chen EMNLP 2023

• A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora ICML 2023 ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models (Spotlight)

• Should You Mask 15% in Masked Language Modeling? Alexander Wettig*, Tianyu Gao*, Zexuan Zhong, Danqi Chen EACL 2023

• Finding Dataset Shortcuts with Grammar Induction Dan Friedman, Alexander Wettig, Danqi Chen EMNLP 2022

• Phrase Retrieval Learns Passage Retrieval, Too Jinhyuk Lee, Alexander Wettig, Danqi Chen EMNLP 2021