Publications

For a full list of up-to-date publications, please refer to Google Scholar.

2024

• SimPO: Simple Preference Optimization with a Reference-Free Reward
  Yu Meng*, Mengzhou Xia*, Danqi Chen
  NeurIPS 2024;
  [arXiv] [Code]
  


• CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
  Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, Danqi Chen
  NeurIPS 2024 Datasets and Benchmarks Track;
  [arXiv] [Code] [Project Page]


• LitSearch: A Retrieval Benchmark for Scientific Literature Search
  Anirudh Ajith, Mengzhou Xia, Alexis Chevalier, Tanya Goyal, Danqi Chen, Tianyu Gao
  EMNLP 2024;
  [arXiv]
  


• BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
  Hongjin Su*, Howard Yen*, Mengzhou Xia*, Weijia Shi, Niklas Muennighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S. Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O. Arik, Danqi Chen, Tao Yu
  preprint;
  [arXiv] [Code]
  


• What is in Your Safe Data? Identifying Benign Data that Breaks Safety
  Luxi He*, Mengzhou Xia*, Peter Henderson
  COLM 2024;
  DPFM Workshop@ICLR 2024 (Best Paper);
  [arXiv] [Code]


• Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-trainin
  Zexuan Zhong, Mengzhou Xia, Danqi Chen, Mike Lewis
  COLM 2024;
  [arXiv] [Code]


• LESS: Selecting Influential Data for Targeted Instruction Tuning
  Mengzhou Xia*, Sadhika Malladi*, Suchin Gururangan, Sanjeev Arora, Danqi Chen
  ICML 2024;
  [arXiv] [Code] [Blog]


• Trainable Transformer in Transformer
  Abhishek Panigrahi, Sadhika Malladi, Mengzhou Xia, Sanjeev Arora
  ICML 2024
  [arXiv] [Code]


• Language Models as Science Tutors
  Alexis Chevalier, Jiayi Geng, Alexander Wettig, Howard Chen, Sebastian Mizera, Toni Annala, Max Jameson Aragon, Arturo Rodríguez Fanlo, Simon Frieder, Simon Machado, Akshara Prabhakar, Ellie Thieu, Jiachen T Wang, Zirui Wang, Xindi Wu, Mengzhou Xia, Wenhan Jia, Jiatong Yu, Jun-Jie Zhu, Zhiyong Jason Ren, Sanjeev Arora, Danqi Chen
  ICML 2024
  [arXiv] [Code]


• Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
  Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, Peter Henderson
  ICML 2024
  [arXiv] [Code] [Project Page]



2023

• Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
  Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen
  ICLR 2024
  [arXiv] [Code] [Blog]


• Detecting Pretraining Data from Large Language Models
  Weijia Shi*, Anirudh Ajith*, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, Luke Zettlemoyer
  ICLR 2024
  [arXiv] [Code] [Project Page]


• Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation
  Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, Danqi Chen
  ICLR 2024 (Spotlight);
  [arXiv] [Code] [Project Page]


• InstructEval: Systematic Evaluation of Instruction Selection Methods
  Anirudh Ajith, Chris Pan, Mengzhou Xia, Ameet Deshpande, Karthik Narasimhan
  NAACL 2024
  [arXiv] [Code]


• Training Trajectories of Language Models Across Scales
  Mengzhou Xia, Mikel Artetxe, Chunting Zhou, Xi Victoria Lin, Ramakanth Pasunuru, Danqi Chen, Luke Zettlemoyer, Ves Stoyanov
  ACL 2023
  [arXiv] [Code]



2022

• Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models
  Mengzhou Xia, Mikel Artetxe, Jingfei Du, Danqi Chen, Ves Stoyanov
  EMNLP 2022
  [arXiv] [code]


• MABEL: Attenuating Gender Bias using Textual Entailment Data
  Jacqueline He, Mengzhou Xia, Christiane Fellbaum, Danqi Chen
  EMNLP 2022
  [arXiv] [code]


• Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Models
  Mozes van de Kar, Mengzhou Xia, Danqi Chen, Mikel Artetxe
  EMNLP 2022
  [arXiv]


• Structured Pruning Learns Compact and Accurate Models
  Mengzhou Xia, Zexuan Zhong, Danqi Chen
  ACL 2022
  [arXiv] [code]] [talk]



2021

• Meta Feature Transformations for Cross-lingual Transfer Learning
  Mengzhou Xia, Guoqing Zheng, Subho Mukherjee, Graham Neubig, Ahmed Hassan Awadallah
  NAACL 2021
  [arXiv][code]


• Non-Parametric Few-Shot Learning for Word Sense Disambiguation
  Howard Chen, Mengzhou Xia, and Danqi Chen
  NAACL 2021
  [arXiv][code]



2020

• Predicting Performance for Natural Language Processing Tasks
  Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, Graham Neubig
  ACL 2020
  [arXiv] [code] [blog]


• Demoting Racial Bias in Hate Speech Detection
  Mengzhou Xia, Anjalie Field, Yulia Tsvetkov
  SocialNLP Workshop @ACL 2020
  [arXiv]



2019

• Generalized Data Augmentation for Low-Resource Translation
  Mengzhou Xia, Xiang Kong, Antonios Anastasopoulos, Graham Neubig
  ACL 2019
  [arXiv]] [code]


• Domain Adaptation of Neural Machine Translation by Lexicon Induction
  Junjie Hu, Mengzhou Xia, Graham Neubig, Jaime Carbonell
  ACL 2019
  [arXiv] [code]


• Choosing Transfer Languages for Cross-Lingual Learning
  Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig
  ACL 2019
  [arXiv] [code]


• Graph based Translation Memory for Neural Machine Translation
  Mengzhou Xia, Guoping Huang, Lemao Liu, Shuming Shi
  AAAI 2019
  [paper]