Yujie Lu

Yujie Lu (卢雨洁) is a final year CS PhD at UC Santa Barbara, advised by William Wang at Natural Language Processing Group. She did research internship in Meta AI (FAIR), Microsoft Research (MSR), AWS AI. She obtained bachelor's degree from Chu Kochen Honors College, Zhejiang University.

Email: yujielu10@gmail.com

Email / CV / Google Scholar / Github / LinkedIn / X
Yujie Lu
Yujie is actively looking for full time industry positions! Feel free to drop me an email at yujielu10@gmail.com if you're interested!

Research

Yujie's research focuses on vision language models (VLMs) and large language models (LLMs), with an emphasis on robust and efficient post-training (text, image, video), interpretable and faithful evaluation, task planning with LLM agents, and alignment with human preferences.

News

News! 09/26/2024 WildVision is accepted at NeurIPS 2024 D&B Track! T2IScoreScore is accepted at NeurIPS 2024 Main Track!

News! 09/20/2024 Multimodal Procedural Planning is accepted at EMNLP 2024!

News! 08/27/2024 We release WildVision datasets: WV-Chat, WV Battle, and WV Bench.

News! 06/24/2024 I started working on video large language models at Meta (FAIR Embodied AI) in NYC this Summer!

News! 03/01/2024 Check out our WildVision Arena demo on HuggingFace for live benchmark VLMs!

Industry Experience

  • Meta (FAIR Embodied AI) Jun, 2024 - present

    Research Intern, Mentor: Tushar Nagarajan, Yale Song, Lorenzo Torresani

  • Amazon (AWS AI Labs) Jun, 2022 - Nov, 2023

    Research Intern, Mentor: Zhaowei Cai, Hao Yang, Stefano Soatto

  • Microsoft Research May, 2022 - Sept, 2022

    Research Intern, Mentor: Oriana Riva

  • Tencent Aug, 2019 - Aug, 2021

    Applied Researcher
Selected Publications

WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences

Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, Bill Yuchen Lin

Conference on Neural Information Processing Systems (NeurIPS), Dataset and Benchmark, 2024

Multimodal Procedural Planning via Dual Text-Image Prompting

Yujie Lu, Pan Lu, Zhiyu Chen, Wanrong Zhu, Xin Eric Wang, William Yang Wang

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

From Text to Pixel: Advancing Long-Context Understanding in MLLMs

Yujie Lu, Xiujun Li, Tsu-Jui Fu, Miguel Eckstein, William Yang Wang

ICML 2024 Workshop LCFM, 2024

Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?

Yujie Lu*, Xiujun Li*, Zhe Gan, Jianfeng Gao, William Yang Wang, Yejin Choi

NeurIPS Workshop on Adaptive Foundation Models, 2024

Visual grounding for user interfaces

Yijun Qian, Yujie Lu, Alexander G Hauptmann, Oriana Riva

Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024

Imagenhub: Standardizing the evaluation of conditional image generation models

Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, Wenhu Chen

International Conference on Learning Representations (ICLR), 2024

Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought

Vaishnavi Himakunthala, Andy Ouyang, Daniel Philip Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon, William Yang Wang

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, William Yang Wang

Conference on Neural Information Processing Systems (NeurIPS), 2023

Neuro-Symbolic Causal Procedural Planning with Commonsense Prompting

Yujie Lu, Weixi Feng, Wanrong Zhu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, William Yang Wang

International Conference on Learning Representations (ICLR), Spotlight, 2023

ULN: Towards Underspecified Vision-and-Language Navigation

Weixi Feng, Tsu-Jui Fu, Yujie Lu, William Yang Wang

The Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022

Imagination-Augmented Natural Language Understanding

Yujie Lu, Wanrong Zhu, Xin Eric Wang, Miguel Eckstein, William Yang Wang,

North American Chapter of the Association for Computational Linguistics (NAACL), Oral Presentation, 2022

Dancer

Soccer Player

Drummer