Ameet Deshpande will present
The members of Ameet’s committee are as follows:
Examiners: Karthik Narasimhan (Adviser), Danqi Chen, Sanjeev Arora
Readers: Andrés Monroy-Hernández, Suma Bhat
A copy of
Everyone is invited to attend
Abstract follows below:
Large language models (LLMs) have transformed natural language understanding and generation, yet their development remains a fragmented process. Each stage in the LLM lifecycle—pre-training, post-training, evaluation, and improvement—poses distinct challenges that are rarely studied together. This thesis pushes the envelope on all phases while also tying them together for continuous adaptation of models.
We begin with the pre-training phase, analyzing what enables cross-lingual generalization in multilingual models, where pre-training on one language bestows skills in another. In the posttraining phase, we show that instruction-tuned LLMs exhibit biases related to the persona they are simulating, highlighting the need for socially aware alignment methods.
Turning to evaluation, we demonstrate that existing semantic similarity benchmarks fail to capture contextual reasoning. We propose Conditional STS, a framework that measures similarity relative to explicit conditions, producing fine-grained feedback. Finally, we introduce QUALEVAL, a system that closes the loop between evaluation and improvement by automatically surfacing qualitative insights and guiding targeted model refinement.
Together, these contributions reimagine the LLM lifecycle as a continuous feedback process—one where insights from each stage inform the next. By integrating pre-training, alignment, evaluation, and improvement, this thesis moves toward the vision of adaptive, self-improving language models.