Student Recruitment

Research Philosophy

Nearly every project produces both rigorous theoretical insights and practical algorithms that are immediately useful to practitioners (and accordingly, often adopted across industry). My research bridges deep learning theory and practice by combining (1) new mathematical tools that move beyond the restrictive assumptions typical in traditional deep learning theory with (2) rigorous experiments to verify the applicability of the theory to real-world settings. This two-pronged approach advances fundamental understanding of current models while identifying lasting insights and algorithms that extend beyond specific settings and models. This is especially useful as the field evolves rapidly. Here are some examples:

  1. Using optimization theory to select hyperparameters for large-scale training runs (blog). Although this research was developed between 2020 and 2022, it remains useful and applicable to training large-scale models today and is widely adopted across industry.
  2. Current ways of making models output less toxic content fail (paper) actually make them output more undesirable text (paper). Theory lets us characterize how and why this happens and mitigate it via efficient and practical data filtering strategies.
  3. Applying gradient-free optimization techniques to fine-tune large language models without the need for backpropagation, thereby substantially reducing memory consumption (paper).

Each of these works began with novel theoretical analyses and concluded with practical insights and algorithms that would have otherwise been hard to identify. I am interested in applying this philosophy to a wide variety of topics within machine learning!

Mentorship Philosophy

I am deeply invested in your development as a researcher and communicator, and I am formally trained in state-of-the-art pedagogical practices. I will work closely with you to identify topics that match your interests, help you get "unstuck" when you need it, and give you room to explore. Beyond research, I will facilitate opportunities for you to collaborate and disseminate your work externally across academia and industry. I am knowledgeable about and sympathetic to the challenges international students face, and I will do my best to support you. If you have specific concerns, I would be happy to chat about them.

Joining the Group

If you are applying to PhD programs, please apply to the CSE department at UCSD and mention me in your application. Feel free to send me an email with "[PhD Interest]" in the subject title if you want to specifically highlight your application. I expect students to have both mathematical maturity and programming familiarity, though many ML-specific theoretical and empirical skills can be picked up over the course of research. If you are not sure if you will be a good fit, please apply anyways or reach out to chat.

If you are interested in collaborating or want to chat about research ideas, please send me an email any time with "[Collaboration Interest]" in the subject line.