Preference Learning Algorithms Do Not Learn Preference Rankings

Angelica Chen, Sadhika Malladi, Lily H. Zhang, Xinyi Chen, Qiuyi Zhang, Rajesh Ranganath, Kyunghyun Cho.

In submission.

LESS: Selecting Influential Data for Targeted Instruction Tuning

Mengzhou Xia*, Sadhika Malladi*, Suchin Gururangan, Sanjeev Arora, Danqi Chen.

To appear in ICML 2024.
Data Problems for Foundation Models (DPFM) Workshop at ICLR 2024.

Trainable Transformer in Transformer

Abhishek Panigrahi*, Sadhika Malladi*, Mengzhou Xia, Sanjeev Arora.

To appear in ICML 2024.
Robustness of Few-Shot and Zero-Shot Learning in Foundation Models (R0-FoMo) Workshop at NeurIPS 2023.

The Marginal Value of Momentum for Small Learning Rate SGD

Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li.

ICLR 2024 (OpenReview).
High-dimensional Learning Dynamics Workshop at ICML 2023.

Fine-Tuning Language Models with Just Forward Passes

Sadhika Malladi*, Tianyu Gao*, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, Sanjeev Arora.

Oral presentation at NeurIPS 2023 (OpenReview).
Oral presentation at Efficient Systems for Foundation Models (ES-FoMo) workshop at ICML 2023.
Differentiable Everything Workshop at ICML 2023.

A Kernel-Based View of Language Model Fine-Tuning

Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora.

ICML 2023 (OpenReview).
Oral presentation at ICLR 2023 Foundation Models Workshop (ME-FoMo).

On the SDEs and Scaling Rules for Adaptive Gradient Algorithms

Sadhika Malladi*, Kaifeng Lyu*, Abhishek Panigrahi, Sanjeev Arora.

NeurIPS 2022 (OpenReview).
Oral presentation at ICML 2022 Workshop on Continuous Time Methods.

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)

Zhiyuan Li, Sadhika Malladi, Sanjeev Arora.

NeurIPS 2021 (OpenReview).

A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora.

ICLR 2021 (OpenReview).
Oral presentation at NeurIPS 2020 Women in Machine Learning Workshop (Video).
Self-Supervised Learning Workshop at NeurIPS 2020.
Nikunj's Talk at the Vector Institute.

Prediction Propagation for Domain Adaptation in NLP

Bjarke Felbo, Michiel Bakker, Abhimanyu Dubey, Sadhika Malladi, Alex "Sandy" Pentland, Iyad Rahwan.

Towards learning with limited labels: Equivariance, Invariance, and Beyond at ICML 2018.

FastNorm: Improving Numerical Stability of Deep Network Training with Efficient Normalization

Sadhika Malladi, Ilya Sharapov

Women in Machine Learning Workshop at NeurIPS 2017.

Systematic Analysis of Sex-Linked Molecular Alterations and Therapies in Cancer

Sadhika Malladi*, Jonathan Ma*, Andrew H. Beck

Nature Scientific Reports, 2016.

EMDomics: a robust and powerful method for the identification of genes differentially expressed between heterogeneous classes

Daniel Schmolze, Mayineur Matituoheti, Sadhika Malladi, Andrew H. Beck.

Bioinformatics, 2016.

Assessing treatment response in triple-negative breast cancer from quantitative image analysis in perfusion magnetic resonance imaging

Imon Banerjee, Sadhika Malladi, Daniela Lee, Adrien Depeursinge, Melinda Telli, Jafi Lipson, Daniel Golden, Daniel L. Rubin.

Journal of Medical Imaging, 2017.