Yujie Lu

Yujie Lu (卢雨洁) is a Research Scientist at Meta GenAI, working on multimodal post-training for Llama. She earned her Ph.D. from UC Santa Barbara, where she was advised by William Wang. She did research internships at Meta AI (FAIR), Microsoft Research (MSR), and AWS AI. She obtained bachelor's degree from Chu Kochen Honors College, Zhejiang University.

Email: yujielu10@gmail.com

Email / CV / Google Scholar / Github / LinkedIn / X

Research

Yujie's research focuses on vision language models (VLMs) and large language models (LLMs), with an emphasis on robust and efficient post-training (text, image, video), interpretable and faithful evaluation, task planning with LLM agents, and alignment with human preferences.

News

News! 02/26/2025 VITED: Video Temporal Evidence Distillation is accepted at CVPR 2025!

News! 01/24/2025 I successfully complete my PhD defense and will join Meta GenAI Llama Research Team!

News! 01/22/2025 MMWorld is accepted at ICLR 2025!

News! 09/26/2024 WildVision is accepted at NeurIPS 2024 D&B Track! T2IScoreScore is accepted at NeurIPS 2024 Main Track!

News! 09/20/2024 Multimodal Procedural Planning is accepted at EMNLP 2024!

News! 08/27/2024 We release WildVision datasets: WV-Chat, WV Battle, and WV Bench.

News! 06/24/2024 I started working on video large language models at Meta (FAIR Embodied AI) in NYC this Summer!

News! 03/01/2024 Check out our WildVision Arena demo on HuggingFace for live benchmark VLMs!

Selected Publications

VITED: Video Temporal Evidence Distillation

Yujie Lu, Yale Song, William Yang Wang, Lorenzo Torresani, Tushar Nagarajan

Conference on Computer Vision and Pattern Recognition (CVPR), 2025

WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences

Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, Bill Yuchen Lin

Conference on Neural Information Processing Systems (NeurIPS), Dataset and Benchmark, 2024

Multimodal Procedural Planning via Dual Text-Image Prompting

Yujie Lu, Pan Lu, Zhiyu Chen, Wanrong Zhu, Xin Eric Wang, William Yang Wang

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

From Text to Pixel: Advancing Long-Context Understanding in MLLMs

Yujie Lu, Xiujun Li, Tsu-Jui Fu, Miguel Eckstein, William Yang Wang

ICML 2024 Workshop LCFM, 2024

Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?

Yujie Lu*, Xiujun Li*, Zhe Gan, Jianfeng Gao, William Yang Wang, Yejin Choi

NeurIPS Workshop on Adaptive Foundation Models, 2024

Visual grounding for user interfaces

Yijun Qian, Yujie Lu, Alexander G Hauptmann, Oriana Riva

Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024

Imagenhub: Standardizing the evaluation of conditional image generation models

Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, Wenhu Chen

International Conference on Learning Representations (ICLR), 2024

Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought

Vaishnavi Himakunthala, Andy Ouyang, Daniel Philip Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon, William Yang Wang

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, William Yang Wang

Conference on Neural Information Processing Systems (NeurIPS), 2023

Neuro-Symbolic Causal Procedural Planning with Commonsense Prompting

Yujie Lu, Weixi Feng, Wanrong Zhu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

International Conference on Learning Representations (ICLR), Spotlight, 2023

ULN: Towards Underspecified Vision-and-Language Navigation

Weixi Feng, Tsu-Jui Fu, Yujie Lu, William Yang Wang

The Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022

Imagination-Augmented Natural Language Understanding

Yujie Lu, Wanrong Zhu, Xin Eric Wang, Miguel Eckstein, William Yang Wang,

North American Chapter of the Association for Computational Linguistics (NAACL), Oral Presentation, 2022